QA » openQA Project

Setting due date based on mean cycle time of SUSE QE Tools

Worked fine in https://github.com/os-autoinst/openQA/pull/5748 and locally.
Unable to reproduce or even find out what actually failed.

Just some recent examples:

• In https://github.com/os-autoinst/openQA/pull/5733
  • https://github.com/os-autoinst/openQA/actions/runs/9759302448/job/26935676249?pr=5733#step:3:1999 failed with

#6 [nginx 2/5] RUN zypper ar -p 95 -f http://download.opensuse.org/repositories/devel:openQA/15.6 devel_openQA &&     zypper ar -p 90 -f http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6 devel_openQA_Leap &&     zypper --gpg-auto-import-keys ref &&     zypper in -y openQA nginx &&     zypper clean
#6 0.300 Adding repository 'devel_openQA' [...done]
#6 0.301 Repository 'devel_openQA' successfully added
#6 0.301 
#6 0.301 URI         : http://download.opensuse.org/repositories/devel:openQA/15.6
#6 0.301 Enabled     : Yes
#6 0.301 GPG Check   : Yes
#6 0.301 Autorefresh : Yes
#6 0.301 Priority    : 95 (raised priority)
#6 0.301 
#6 0.301 Repository priorities in effect: (See 'zypper lr -P' for details)
#6 0.301       95 (raised priority)  :  1 repository
#6 0.301       99 (default priority) :  7 repositories
#6 0.388 Adding repository 'devel_openQA_Leap' [...done]
#6 0.390 Repository 'devel_openQA_Leap' successfully added
#6 0.390 
#6 0.390 URI         : http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6
#6 0.390 Enabled     : Yes
#6 0.390 GPG Check   : Yes
#6 0.390 Autorefresh : Yes
#6 0.390 Priority    : 90 (raised priority)
#6 0.390 
#6 0.390 Repository priorities in effect: (See 'zypper lr -P' for details)
#6 0.390       90 (raised priority)  :  1 repository
#6 0.390       95 (raised priority)  :  1 repository
#6 0.390       99 (default priority) :  7 repositories
#6 0.994 Retrieving repository 'devel_openQA' metadata [...........
#6 3.154 
#6 3.154 Automatically importing the following key:
#6 3.154 
#6 3.154   Repository:       devel_openQA
#6 3.154   Key Fingerprint:  A99A 72E3 06F2 0929 E6DE E378 5B12 1667 CBDF 5E8F
#6 3.154   Key Name:         devel:openQA OBS Project <devel:openQA@build.opensuse.org>
#6 3.154   Key Algorithm:    RSA 2048
#6 3.154   Key Created:      Tue Dec 26 20:20:51 2023
#6 3.154   Key Expires:      Thu Mar  5 20:20:51 2026
#6 3.154   Rpm Name:         gpg-pubkey-cbdf5e8f-658b35a3
#6 3.154 
#6 3.154 
#6 3.154 
#6 3.154     Note: A GPG pubkey is clearly identified by its fingerprint. Do not rely on the key's name. If
#6 3.154     you are not sure whether the presented key is authentic, ask the repository provider or check
#6 3.154     their web site. Many providers maintain a web page showing the fingerprints of the GPG keys they
#6 3.154     are using.
#6 3.246 Signature verification failed for file 'repomd.xml' from repository 'devel_openQA'.
#6 3.246 
#6 3.246     Note: Signing data enables the recipient to verify that no modifications occurred after the data
#6 3.246     were signed. Accepting data with no, wrong or unknown signature can lead to a corrupted system
#6 3.246     and in extreme cases even to a system compromise.
#6 3.246 
#6 3.246     Note: File 'repomd.xml' is the repositories master index file. It ensures the integrity of the
#6 3.246     whole repo.
#6 3.246 
#6 3.246     Warning: This file was modified after it has been signed. This may have been a malicious change,
#6 3.246     so it might not be trustworthy anymore! You should not continue unless you know it's safe.
#6 3.246 
#6 3.246     Note: This might be a transient issue if the server is in the midst of receiving new data. The
#6 3.246     data file and its signature are two files which must fit together. In case the request hit the
#6 3.246     server in the midst of updating them, the signature verification might fail. After a few
#6 3.246     minutes, when the server has updated its data, it should work again.
#6 3.247 
#6 3.247 Signature verification failed for file 'repomd.xml' from repository 'devel_openQA'. Continue? [yes/no] (no): Cannot read input: bad stream or EOF.
#6 3.247 If you run zypper without a terminal, use '--non-interactive' global
#6 3.247 option to make zypper use default answers to prompts.
#6 3.249 error]
#6 3.249 Repository 'devel_openQA' is invalid.
#6 3.249 [devel_openQA|http://download.opensuse.org/repositories/devel:openQA/15.6] Valid metadata not found at specified URL
#6 3.249 History:
#6 3.249  - Cannot read input. Bad stream or EOF.
#6 3.249 
#6 3.249 Please check if the URIs defined for this repository are pointing to a valid repository.

• https://github.com/os-autoinst/openQA/actions/runs/9759301860/job/26935675076?pr=5733#step:3:1935 failed with

File './repodata/828b3f26d27acb6f47c0dc35507d51659ab327c4387efbd04939363fdbee35cb-primary.xml.gz' not found on medium 'http://download.opensuse.org/repositories/devel:openQA/15.6'

other similar cases:

• https://github.com/os-autoinst/openQA/actions/runs/9663317626/job/26655178398?pr=5691
• https://github.com/os-autoinst/openQA/actions/runs/9286020067/job/25720017945?pr=5647#step:3:609

let's block on #163112

And it happened again: https://github.com/os-autoinst/openQA/actions/runs/9856943908/job/27214994886?pr=5752

You said in the daily that you block this ticket but it didn't make much sense to me so I thought I misunderstood what ticket you are talking about.

I don't think adding a retry in openQA-in-openQA tests (that's what #163112 is about) will help here. We need to add a retry on for the docker-compose check. I'm not sure whether it makes sense to add a retry directly in the line in question (as it is a RUN line within a Dockerfile). So maybe we should just retry the whole thing.

Unassigning for now then as I'm working on another ticket atm

• Retry the test (e.g. with https://github.com/marketplace/actions/retry-step)

I'm preparing a change to use this action.

https://github.com/os-autoinst/openQA/pull/5768

livdywan wrote in #note-20:

https://github.com/os-autoinst/openQA/pull/5768

Went with this one.

Looks like the default 3 retries are not enough, and we need more:

https://github.com/os-autoinst/openQA/actions/runs/10040705287/job/27747275454?pr=5784

failed to solve: process "/bin/sh -c zypper ar -p 95 -f http://download.opensuse.org/repositories/devel:openQA/15.6 devel_openQA &&     zypper ar -p 90 -f http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6 devel_openQA_Leap &&     zypper --gpg-auto-import-keys ref &&     zypper in -yl ca-certificates-mozilla curl gzip rsync xz &&     zypper in -yl openQA-worker os-autoinst-s390-deps os-autoinst-ipmi-deps &&     zypper in -yl qemu-arm qemu-ppc qemu-x86 qemu-tools &&     zypper in -yl qemu-hw-display-virtio-gpu qemu-hw-display-virtio-gpu-pci qemu-hw-display-virtio-vga &&     zypper in -yl kmod &&     (zypper in -yl qemu-ovmf-x86_64 ||:) &&     (zypper in -yl qemu-uefi-aarch64 ||:) &&     zypper clean &&     mkdir -p /root/qemu" did not complete successfully: exit code: 4
 > [worker 2/5] RUN zypper ar -p 95 -f http://download.opensuse.org/repositories/devel:openQA/15.6 devel_openQA &&     zypper ar -p 90 -f http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6 devel_openQA_Leap &&     zypper --gpg-auto-import-keys ref &&     zypper in -yl ca-certificates-mozilla curl gzip rsync xz &&     zypper in -yl openQA-worker os-autoinst-s390-deps os-autoinst-ipmi-deps &&     zypper in -yl qemu-arm qemu-ppc qemu-x86 qemu-tools &&     zypper in -yl qemu-hw-display-virtio-gpu qemu-hw-display-virtio-gpu-pci qemu-hw-display-virtio-vga &&     zypper in -yl kmod &&     (zypper in -yl qemu-ovmf-x86_64 ||:) &&     (zypper in -yl qemu-uefi-aarch64 ||:) &&     zypper clean &&     mkdir -p /root/qemu:
0.367 Repository priorities in effect: (See 'zypper lr -P' for details)
0.367       90 (raised priority)  :  1 repository
0.367       95 (raised priority)  :  1 repository
0.367       99 (default priority) :  7 repositories
done]
..done]
..done]
done]
42.88 Building repository 'Update Repository (Non-Oss)' cache [...done]
42.89 Some of the repositories have not been refreshed because of an error.
------
docker compose exited with non-zero code 1, showing logs:
time="2024-07-22T12:30:40Z" level=warning msg="/tmp/tmp.A9yc0PKuun/worker/docker-compose.yaml: `version` is obsolete"
time="2024-07-22T12:30:40Z" level=warning msg="/tmp/tmp.A9yc0PKuun/worker/docker-compose.yaml: `version` is obsolete"

Setting due date based on mean cycle time of SUSE QE Tools

I guess @okurz is taking over from here

https://github.com/os-autoinst/openQA/pull/5789 merged. Awaiting verification from production.

Past days no problems showed up in CI tests.