action #88217
closed[qe-core] test fails in bootloader_svirt - libxenlight failed to create new domain: leftover qemu process
0%
Description
Observation¶
error: Failed to start domain openQA-SUT-1
error: internal error: libxenlight failed to create new domain 'openQA-SUT-1'
Reproducible¶
- In all xen machines
- Fails since Build 132.4
- Current occurrence for scenario [sle-15-SP3-Online-x86_64-Build132.4-memtest@svirt-xen-hvm](bootloader_svirt)
- Last good: 130.3
- latest
Updated by SLindoMansilla about 3 years ago
- Subject changed from [qe-core] test fails in bootloader_svirt to [qe-core] test fails in bootloader_svirt - libxenlight failed to create new domain
Updated by szarate about 3 years ago
- Related to action #54863: [functional][u] test fails in bootloader_svirt - Missing domains in libvirt but still runnning in XEN. added
Updated by szarate about 3 years ago
- Status changed from New to In Progress
- Assignee set to szarate
Smells like: poo#54863
Updated by szarate about 3 years ago
- Subject changed from [qe-core] test fails in bootloader_svirt - libxenlight failed to create new domain to [qe-core] test fails in bootloader_svirt - libxenlight failed to create new domain: leftover qemu process
- Status changed from In Progress to Resolved
I modified the script added before for a similar problem.
What happened here is:
At some point due to possibly a bug in libvirt, when a domain is destroyed, there could be leftovers in xen (as previously mentioned), and sometimes, the leftovers could be left in a worse state, like leaving a qemu process roaming around, pgrep -f qemu.*openQA-SUT- comes particularly handy here.
I remember something similar happening in the past, so for the time being I updated the cleanup script to also kill the qemu process (https://github.com/foursixnine/stunning-octo-chainsaw/commit/be43227d76bfd55bfd99e2311e61d8faa8c8ed36)
For now, https://openqa.suse.de/tests/05347291#live is running on the virsh instance that was previously failing.
Also the cron job has been changed to only run once a day, but I'll be getting emails if failures ocurr (later to be moved to one of the monitoring mailing lists??)
Updated by szarate about 3 years ago
- Related to action #88299: [virtualization] Worker openqaw5-xen-1.qa.suse.de is not reachable (xen-hvm/xen-pv failing) added
Updated by okurz about 3 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: extra_tests_gnome@svirt-xen-pv
https://openqa.suse.de/tests/5478993
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Updated by okurz about 3 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: extra_tests_gnome@svirt-xen-pv
https://openqa.suse.de/tests/5585765
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Updated by okurz about 3 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: extra_tests_gnome@svirt-xen-pv
https://openqa.suse.de/tests/5585765
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Updated by szarate over 2 years ago
- Related to action #97532: [qe-core][sporadic] s390x jobs are failing to boot auto_review:"error: Cannot set interface flags on 'macvtap.*': Address already in use":retry added