action #162848
closedwebui-docker-compose tests failing on GitHub PR's size:S
Description
Observation¶
#6 ERROR: process "/bin/sh -c zypper ar -p 95 -f http://download.opensuse.org/repositories/devel:openQA/15.6 devel_openQA && zypper ar -p 90 -f http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6 devel_openQA_Leap && zypper --gpg-auto-import-keys ref && zypper in -yl ca-certificates-mozilla curl gzip rsync xz && zypper in -yl openQA-worker os-autoinst-s390-deps os-autoinst-ipmi-deps && zypper in -yl qemu-arm qemu-ppc qemu-x86 qemu-tools && zypper in -yl qemu-hw-display-virtio-gpu qemu-hw-display-virtio-gpu-pci qemu-hw-display-virtio-vga && zypper in -yl kmod && (zypper in -yl qemu-ovmf-x86_64 ||:) && (zypper in -yl qemu-uefi-aarch64 ||:) && zypper clean && mkdir -p /root/qemu" did not complete successfully: exit code: 4
Acceptance criteria¶
- AC1: webui-docker-compose test passes consistently
Suggestions¶
- Retry the test (e.g. with https://github.com/marketplace/actions/retry-step)
- Add retry to zypper command within the container
- Move the test into a different environment e.g. rely on openQA tests instead
Out of scope¶
- Only run the test depending on certain modified files
Updated by openqa_review 3 months ago
- Due date set to 2024-07-19
Setting due date based on mean cycle time of SUSE QE Tools
Updated by dheidler 3 months ago
- Due date deleted (
2024-07-19) - Status changed from In Progress to Rejected
Worked fine in https://github.com/os-autoinst/openQA/pull/5748 and locally.
Unable to reproduce or even find out what actually failed.
Updated by okurz 3 months ago
- Status changed from Rejected to Workable
Just some recent examples:
#6 [nginx 2/5] RUN zypper ar -p 95 -f http://download.opensuse.org/repositories/devel:openQA/15.6 devel_openQA && zypper ar -p 90 -f http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6 devel_openQA_Leap && zypper --gpg-auto-import-keys ref && zypper in -y openQA nginx && zypper clean
#6 0.300 Adding repository 'devel_openQA' [...done]
#6 0.301 Repository 'devel_openQA' successfully added
#6 0.301
#6 0.301 URI : http://download.opensuse.org/repositories/devel:openQA/15.6
#6 0.301 Enabled : Yes
#6 0.301 GPG Check : Yes
#6 0.301 Autorefresh : Yes
#6 0.301 Priority : 95 (raised priority)
#6 0.301
#6 0.301 Repository priorities in effect: (See 'zypper lr -P' for details)
#6 0.301 95 (raised priority) : 1 repository
#6 0.301 99 (default priority) : 7 repositories
#6 0.388 Adding repository 'devel_openQA_Leap' [...done]
#6 0.390 Repository 'devel_openQA_Leap' successfully added
#6 0.390
#6 0.390 URI : http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6
#6 0.390 Enabled : Yes
#6 0.390 GPG Check : Yes
#6 0.390 Autorefresh : Yes
#6 0.390 Priority : 90 (raised priority)
#6 0.390
#6 0.390 Repository priorities in effect: (See 'zypper lr -P' for details)
#6 0.390 90 (raised priority) : 1 repository
#6 0.390 95 (raised priority) : 1 repository
#6 0.390 99 (default priority) : 7 repositories
#6 0.994 Retrieving repository 'devel_openQA' metadata [...........
#6 3.154
#6 3.154 Automatically importing the following key:
#6 3.154
#6 3.154 Repository: devel_openQA
#6 3.154 Key Fingerprint: A99A 72E3 06F2 0929 E6DE E378 5B12 1667 CBDF 5E8F
#6 3.154 Key Name: devel:openQA OBS Project <devel:openQA@build.opensuse.org>
#6 3.154 Key Algorithm: RSA 2048
#6 3.154 Key Created: Tue Dec 26 20:20:51 2023
#6 3.154 Key Expires: Thu Mar 5 20:20:51 2026
#6 3.154 Rpm Name: gpg-pubkey-cbdf5e8f-658b35a3
#6 3.154
#6 3.154
#6 3.154
#6 3.154 Note: A GPG pubkey is clearly identified by its fingerprint. Do not rely on the key's name. If
#6 3.154 you are not sure whether the presented key is authentic, ask the repository provider or check
#6 3.154 their web site. Many providers maintain a web page showing the fingerprints of the GPG keys they
#6 3.154 are using.
#6 3.246 Signature verification failed for file 'repomd.xml' from repository 'devel_openQA'.
#6 3.246
#6 3.246 Note: Signing data enables the recipient to verify that no modifications occurred after the data
#6 3.246 were signed. Accepting data with no, wrong or unknown signature can lead to a corrupted system
#6 3.246 and in extreme cases even to a system compromise.
#6 3.246
#6 3.246 Note: File 'repomd.xml' is the repositories master index file. It ensures the integrity of the
#6 3.246 whole repo.
#6 3.246
#6 3.246 Warning: This file was modified after it has been signed. This may have been a malicious change,
#6 3.246 so it might not be trustworthy anymore! You should not continue unless you know it's safe.
#6 3.246
#6 3.246 Note: This might be a transient issue if the server is in the midst of receiving new data. The
#6 3.246 data file and its signature are two files which must fit together. In case the request hit the
#6 3.246 server in the midst of updating them, the signature verification might fail. After a few
#6 3.246 minutes, when the server has updated its data, it should work again.
#6 3.247
#6 3.247 Signature verification failed for file 'repomd.xml' from repository 'devel_openQA'. Continue? [yes/no] (no): Cannot read input: bad stream or EOF.
#6 3.247 If you run zypper without a terminal, use '--non-interactive' global
#6 3.247 option to make zypper use default answers to prompts.
#6 3.249 error]
#6 3.249 Repository 'devel_openQA' is invalid.
#6 3.249 [devel_openQA|http://download.opensuse.org/repositories/devel:openQA/15.6] Valid metadata not found at specified URL
#6 3.249 History:
#6 3.249 - Cannot read input. Bad stream or EOF.
#6 3.249
#6 3.249 Please check if the URIs defined for this repository are pointing to a valid repository.
- https://github.com/os-autoinst/openQA/actions/runs/9759301860/job/26935675076?pr=5733#step:3:1935 failed with
File './repodata/828b3f26d27acb6f47c0dc35507d51659ab327c4387efbd04939363fdbee35cb-primary.xml.gz' not found on medium 'http://download.opensuse.org/repositories/devel:openQA/15.6'
other similar cases:
Updated by okurz 3 months ago
- Related to action #163112: test fails in openqa_webui due to repeated and reproducible errors in reading from the devel:openQA repository "repodata…filelists-ext.xml.gz not found on medium" size:S added
Updated by dheidler 3 months ago
- Related to deleted (action #163112: test fails in openqa_webui due to repeated and reproducible errors in reading from the devel:openQA repository "repodata…filelists-ext.xml.gz not found on medium" size:S)
Updated by dheidler 3 months ago
- Blocked by action #163112: test fails in openqa_webui due to repeated and reproducible errors in reading from the devel:openQA repository "repodata…filelists-ext.xml.gz not found on medium" size:S added
Updated by mkittler 3 months ago · Edited
- Status changed from Blocked to Workable
And it happened again: https://github.com/os-autoinst/openQA/actions/runs/9856943908/job/27214994886?pr=5752
You said in the daily that you block this ticket but it didn't make much sense to me so I thought I misunderstood what ticket you are talking about.
I don't think adding a retry in openQA-in-openQA tests (that's what #163112 is about) will help here. We need to add a retry on for the docker-compose check. I'm not sure whether it makes sense to add a retry directly in the line in question (as it is a RUN
line within a Dockerfile
). So maybe we should just retry the whole thing.
Updated by mkittler 3 months ago
- Blocked by deleted (action #163112: test fails in openqa_webui due to repeated and reproducible errors in reading from the devel:openQA repository "repodata…filelists-ext.xml.gz not found on medium" size:S)
Updated by okurz 3 months ago
- Related to action #163112: test fails in openqa_webui due to repeated and reproducible errors in reading from the devel:openQA repository "repodata…filelists-ext.xml.gz not found on medium" size:S added
Updated by okurz 3 months ago
- Related to action #161729: [sporadic] test fails in containers/build of openqa-in-openqa probably due to temporary download.opensuse.org and zypper issues added
Updated by livdywan 3 months ago
- Status changed from Workable to In Progress
- Assignee set to livdywan
- Retry the test (e.g. with https://github.com/marketplace/actions/retry-step)
I'm preparing a change to use this action.
Updated by livdywan 3 months ago
livdywan wrote in #note-20:
https://github.com/os-autoinst/openQA/pull/5772 is another approach retrying docker compose
from within the test script itself
Updated by livdywan 3 months ago
- Status changed from Resolved to In Progress
Looks like the default 3 retries are not enough, and we need more:
https://github.com/os-autoinst/openQA/actions/runs/10040705287/job/27747275454?pr=5784
failed to solve: process "/bin/sh -c zypper ar -p 95 -f http://download.opensuse.org/repositories/devel:openQA/15.6 devel_openQA && zypper ar -p 90 -f http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6 devel_openQA_Leap && zypper --gpg-auto-import-keys ref && zypper in -yl ca-certificates-mozilla curl gzip rsync xz && zypper in -yl openQA-worker os-autoinst-s390-deps os-autoinst-ipmi-deps && zypper in -yl qemu-arm qemu-ppc qemu-x86 qemu-tools && zypper in -yl qemu-hw-display-virtio-gpu qemu-hw-display-virtio-gpu-pci qemu-hw-display-virtio-vga && zypper in -yl kmod && (zypper in -yl qemu-ovmf-x86_64 ||:) && (zypper in -yl qemu-uefi-aarch64 ||:) && zypper clean && mkdir -p /root/qemu" did not complete successfully: exit code: 4
> [worker 2/5] RUN zypper ar -p 95 -f http://download.opensuse.org/repositories/devel:openQA/15.6 devel_openQA && zypper ar -p 90 -f http://download.opensuse.org/repositories/devel:openQA:Leap:15.6/15.6 devel_openQA_Leap && zypper --gpg-auto-import-keys ref && zypper in -yl ca-certificates-mozilla curl gzip rsync xz && zypper in -yl openQA-worker os-autoinst-s390-deps os-autoinst-ipmi-deps && zypper in -yl qemu-arm qemu-ppc qemu-x86 qemu-tools && zypper in -yl qemu-hw-display-virtio-gpu qemu-hw-display-virtio-gpu-pci qemu-hw-display-virtio-vga && zypper in -yl kmod && (zypper in -yl qemu-ovmf-x86_64 ||:) && (zypper in -yl qemu-uefi-aarch64 ||:) && zypper clean && mkdir -p /root/qemu:
0.367 Repository priorities in effect: (See 'zypper lr -P' for details)
0.367 90 (raised priority) : 1 repository
0.367 95 (raised priority) : 1 repository
0.367 99 (default priority) : 7 repositories
done]
..done]
..done]
done]
42.88 Building repository 'Update Repository (Non-Oss)' cache [...done]
42.89 Some of the repositories have not been refreshed because of an error.
------
docker compose exited with non-zero code 1, showing logs:
time="2024-07-22T12:30:40Z" level=warning msg="/tmp/tmp.A9yc0PKuun/worker/docker-compose.yaml: `version` is obsolete"
time="2024-07-22T12:30:40Z" level=warning msg="/tmp/tmp.A9yc0PKuun/worker/docker-compose.yaml: `version` is obsolete"
Updated by openqa_review 3 months ago
- Due date set to 2024-08-06
Setting due date based on mean cycle time of SUSE QE Tools
Updated by okurz 2 months ago
- Priority changed from High to Normal
https://github.com/os-autoinst/openQA/pull/5789 merged. Awaiting verification from production.