Project

General

Profile

Actions

action #109319

closed

[qe-core] aarch64 tests failing in qemu-img due to broken image (was: "with cache error") size:S

Added by martinsmac about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2022-03-31
Due date:
% Done:

0%

Estimated time:

Description

Observation

openQA test show cache error in aarch64 tests.

https://openqa.suse.de/tests/8445867

Acceptance criteria

  • AC1: behavior of conflicting assets is documented

Test suite description

Reproducible

Fails since (at least) Build 20220331-1

Further details

Tests failing when finish and try copy results/disk:
https://openqa.suse.de/tests/8445867

Likely error from autoinst-log.txt:
[2022-03-31T13:31:38.644100+02:00] [info] [pid:76743] +++ setup notes +++
[2022-03-31T13:31:38.644688+02:00] [info] [pid:76743] Running on openqaworker-arm-3:16 (Linux 5.3.18-150300.59.60-default #1 SMP Fri Mar 18 18:37:08 UTC 2022 (79e1683) aarch64)
[2022-03-31T13:31:38.661507+02:00] [debug] [pid:76743] Found HDD_1, caching SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio.qcow2
[2022-03-31T13:31:38.672284+02:00] [info] [pid:76743] Downloading SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio.qcow2, request #211 sent to Cache Service
[2022-03-31T13:31:48.914663+02:00] [info] [pid:76743] Download of SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio.qcow2 processed:
[info] [#211]
Cache size of "/var/lib/openqa/cache" is 46 GiB, with limit 50 GiB
[info] [#211]
Downloading "SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio.qcow2" from "http://openqa.suse.de/tests/8445867/asset/hdd/SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio.qcow2"
[info] [#211]
Content of "/var/lib/openqa/cache/openqa.suse.de/SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio.qcow2" has not changed, updating last use

[2022-03-31T13:31:48.930541+02:00] [debug] [pid:76743] Found ISO, caching SLE-15-SP2-Installer-DVD-aarch64-GM-DVD1.iso
[2022-03-31T13:31:48.942152+02:00] [info] [pid:76743] Downloading SLE-15-SP2-Installer-DVD-aarch64-GM-DVD1.iso, request #212 sent to Cache Service
[2022-03-31T13:32:29.661289+02:00] [info] [pid:76743] Download of SLE-15-SP2-Installer-DVD-aarch64-GM-DVD1.iso processed:
[info] [#207]
Cache size of "/var/lib/openqa/cache" is 45 GiB, with limit 50 GiB
[info] [#207]
Downloading "SLE-15-SP2-Installer-DVD-aarch64-GM-DVD1.iso" from "http://openqa.suse.de/tests/8445832/asset/iso/SLE-15-SP2-Installer-DVD-aarch64-GM-DVD1.iso"
[info] [#207]
Cache size 46 GiB + needed 7.8 GiB exceeds limit of 50 GiB, purging least used assets
[info] [#207]
Purging "/var/lib/openqa/cache/openqa.suse.de/SLES-15-SP1-aarch64-GM-gnome-allpatterns.qcow2" because we need space for new assets, reclaiming 3.4 GiB
[info] [#207]
Purging "/var/lib/openqa/cache/openqa.suse.de/SLES-15-SP1-aarch64-GM-SDK-gnome-allpatterns.qcow2" because we need space for new assets, reclaiming 2.8 GiB
[info] [#207]
Size of "/var/lib/openqa/cache/openqa.suse.de/SLE-15-SP2-Installer-DVD-aarch64-GM-DVD1.iso" is 7.8 GiB, with ETag ""1f0fbf000-5a7d39678f200""
[info] [#207]
Download of "/var/lib/openqa/cache/openqa.suse.de/SLE-15-SP2-Installer-DVD-aarch64-GM-DVD1.iso" successful (55 MiB/s), new cache size is 47 GiB

[2022-03-31T13:32:29.678158+02:00] [debug] [pid:76743] Found UEFI_PFLASH_VARS, caching SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio-uefi-vars.qcow2
[2022-03-31T13:32:29.695262+02:00] [info] [pid:76743] Downloading SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio-uefi-vars.qcow2, request #213 sent to Cache Service
[2022-03-31T13:32:34.862055+02:00] [info] [pid:76743] Download of SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio-uefi-vars.qcow2 processed:
[info] [#213]
Cache size of "/var/lib/openqa/cache" is 47 GiB, with limit 50 GiB
[info] [#213]
Downloading "SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio-uefi-vars.qcow2" from "http://openqa.suse.de/tests/8445867/asset/hdd/SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio-uefi-vars.qcow2"
[info] [#213]
Size of "/var/lib/openqa/cache/openqa.suse.de/SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio-uefi-vars.qcow2" is 323 KiB, with ETag ""50c00-5db7fbe069dde""
[info] [#213]
Download of "/var/lib/openqa/cache/openqa.suse.de/SLES-15-SP2-aarch64-mru-install-minimal-with-addons-Build20220331-1-Server-DVD-Updates-aarch64-virtio-uefi-vars.qcow2" successful (2.3 MiB/s), new cache size is 47 GiB

[2022-03-31T13:32:34.875713+02:00] [info] [pid:76743] Rsync from 'rsync://openqa.suse.de/tests' to '/var/lib/openqa/cache/openqa.suse.de', request #215 sent to Cache Service
[2022-03-31T13:32:55.245782+02:00] [info] [pid:76743] Output of rsync:
[info] [#215] Calling: rsync -avHP --timeout 1800 rsync://openqa.suse.de/tests/ --delete /var/lib/openqa/cache/openqa.suse.de/tests/
Try 1:
receiving incremental file list

Suggestions

  • put into our documentation that multiple jobs that try to publish the same image are not supported
    • use a more specific PUBLISH_HDD variable
  • have openQA detect that two assets are conflicting
    • guestimating to be a task of 2-3 days, involving a new database column

Related issues 2 (0 open2 closed)

Related to openQA Project - action #106898: Protection against asset clobberingResolvedokurz2022-02-16

Actions
Related to openQA Project - action #112742: [tools] aarch64 - qemu-img: /var/lib/openqa/pool/14/raid/hd0-overlay0: Image is not in qcow2 formatRejected2022-06-20

Actions
Actions

Also available in: Atom PDF