action #105690
Updated by livdywan almost 3 years ago
## Observation
jobs incomplete with `qemu-img: Could not open '/var/lib/libvirt/images/a.img': Could not open '/var/lib/libvirt/images/a.img': No such file or directory`, for instance https://openqa.suse.de/tests/8047803:
```
[2022-01-28T09:34:25.924322+01:00] [debug] running `nice ionice qemu-img convert -p -O qcow2 /var/lib/libvirt/images/a.img assets_public/sle-12-SP4-s390x-4.12.14-41.1.g82b276a-Server-DVD-Incidents-Kernel-KOTD@s390x-kvm-sle12-with-ltp.qcow2 -c`
[2022-01-28T09:34:25.930323+01:00] [info] ::: backend::driver::__ANON__: Driver backend collected unknown process with pid 398978 and exit status: 0
[2022-01-28T09:34:25.930519+01:00] [info] ::: backend::driver::__ANON__: Driver backend collected unknown process with pid 399590 and exit status: 0
[2022-01-28T09:34:25.934036+01:00] [debug] (0.00/100%)
qemu-img: Could not open '/var/lib/libvirt/images/a.img': Could not open '/var/lib/libvirt/images/a.img': No such file or directory
[2022-01-28T09:34:25.934973+01:00] [warn] !!! bmwqemu::serialize_state: unable to extract assets: runcmd 'nice ionice qemu-img convert -p -O qcow2 /var/lib/libvirt/images/a.img assets_public/sle-12-SP4-s390x-4.12.14-41.1.g82b276a-Server-DVD-Incidents-Kernel-KOTD@s390x-kvm-sle12-with-ltp.qcow2 -c' failed with exit code 1: ' (0.00/100%)
qemu-img: Could not open '/var/lib/libvirt/images/a.img': Could not open '/var/lib/libvirt/images/a.img': No such file or directory
' at /usr/lib/os-autoinst/osutils.pm line 109.
osutils::runcmd("nice", "ionice", "qemu-img", "convert", "-p", "-O", "qcow2", "/var/lib/libvirt/images/a.img", ...) called at /usr/lib/os-autoinst/backend/svirt.pm line 189
backend::svirt::do_extract_assets(backend::svirt=HASH(0x100359af2d8), HASH(0x100349611d8)) called at /usr/lib/os-autoinst/backend/driver.pm line 80
backend::driver::extract_assets(backend::driver=HASH(0x1003621a9f8), HASH(0x100349611d8)) called at /usr/lib/os-autoinst/OpenQA/Isotovideo/Utils.pm line 163
eval {...} called at /usr/lib/os-autoinst/OpenQA/Isotovideo/Utils.pm line 163
OpenQA::Isotovideo::Utils::handle_generated_assets(OpenQA::Isotovideo::CommandHandler=HASH(0x100359c7600), 1) called at /usr/bin/isotovideo line 409
```
seems like https://github.com/os-autoinst/os-autoinst/pull/1936 is the culprit. Complete changelog since last good:
> * Thu Jan 27 2022 okurz@suse.com
> - Update to version 4.6.1643299616.01abba344:
> * Add more perl signatures
> * Add more perl signatures
> * Simplify code for assigning job settings in `create_from_settings`
> * Use constant for referring to a job's main settings
> * Allow changing job settings via restart API
> * Check whether clones created by the restart API take over the group
> * Prevent error when restarting jobs with `skip_parents=1`
> * Use tidyall instead of custom implementation
>
> * Tue Jan 25 2022 okurz@suse.com
> - Update to version 4.6.1643089984.09669586b:
> * Dependency cron 2022-01-22
> * Add signatures to OpenQA::App and OpenQA::BuildResults
> * templates: Fix wording for the "VNC display number"
> * Show number of restarts of a job within info box on details page
>
> os-autoinst changes:
> * Thu Jan 27 2022 okurz@suse.com
> - Update to version 4.6.1643273407.65ca16b7:
> * svirt: Store vmname early for use after test run
>
> * Tue Jan 25 2022 okurz@suse.com
> - Update to version 4.6.1643061641.d319802b:
> * Continue further checks in fullstack test after one fails
> * Add OBS workflow
> * Simplify base inheritance statement with Mojo::Base everywhere
> * Exclude 29-backend-driver.t from OBS checks
> * Simplify string concatenation in log.pm
> * Extract all log functions into new module "log"
> * Add test for defining/starting VM via VMware in svirt backend
> * Add test for generating XML file with UEFI loader in svirt backend
> * Use tidyall for faster tidying
> * Fix single, unnecessary UTF8 character in consoles::VNC
> * git subrepo pull (merge) external/os-autoinst-common
## Reproducible
Always reproducible on s390x svirt jobs since we have that change in.
## Expected result
Last good https://openqa.suse.de/tests/8019621/logfile?filename=autoinst-log.txt
showing:
```
[2022-01-23T01:41:54.907320+01:00] [debug] <<< testapi::type_string(string="nice ionice qemu-img convert -p -O qcow2 /var/lib/libvirt/images/openQA-SUT-3a.img /var/lib/libvirt/images/sle-12-SP4-s390x-4.12.14-40.1.g1475601-Server-DVD-Incidents-Kernel-KOTD\@s390x-kvm-sle12-with-ltp.qcow2 && echo OK", max_interval=250, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)
[2022-01-23T01:42:02.749003+01:00] [debug] tests/kernel/../shutdown/svirt_upload_assets.pm:60 called svirt_upload_assets::extract_assets -> tests/kernel/../shutdown/svirt_upload_assets.pm:32 called testapi::assert_screen
```
## Workaround
Revert https://github.com/os-autoinst/os-autoinst/pull/1936 and retrigger tests. OSD deployment was rolled back with https://gitlab.suse.de/openqa/osd-deployment/-/jobs/813165