QA » openQA Project » openQA Tests

which architectures are being covered ? I got a missing dependency error installing the package in aarch64

Problem: nothing provides 'qemu-kvm' needed by the to be installed systemd-testsuite-254.5-150600.1.9.aarch64
 Solution 1: do not install systemd-testsuite-254.5-150600.1.9.aarch64
 Solution 2: break systemd-testsuite-254.5-150600.1.9.aarch64 by ignoring some of its dependencies

while on ppc64le

Activating sle-module-python3 15.6 ppc64le ...
Error: Registration server returned 'The product you are attempting to activate (Python 3 Module 15 SP6 ppc64le) is not available on your system's base product (SUSE Linux Enterprise Server 15 SP6 ppc64le). Python 3 Module 15 SP6 ppc64le is available on SUSE Linux Enterprise Server 15 SP5 ppc64le, SUSE Linux Enterprise Server for SAP Applications 15 SP5 ppc64le.' (422)
eTLB5-67-

amanzini wrote in #note-4:

which architectures are being covered ? I got a missing dependency error installing the package in aarch64

Problem: nothing provides 'qemu-kvm' needed by the to be installed systemd-testsuite-254.5-150600.1.9.aarch64
 Solution 1: do not install systemd-testsuite-254.5-150600.1.9.aarch64
 Solution 2: break systemd-testsuite-254.5-150600.1.9.aarch64 by ignoring some of its dependencies

while on ppc64le

Activating sle-module-python3 15.6 ppc64le ...
Error: Registration server returned 'The product you are attempting to activate (Python 3 Module 15 SP6 ppc64le) is not available on your system's base product (SUSE Linux Enterprise Server 15 SP6 ppc64le). Python 3 Module 15 SP6 ppc64le is available on SUSE Linux Enterprise Server 15 SP5 ppc64le, SUSE Linux Enterprise Server for SAP Applications 15 SP5 ppc64le.' (422)
eTLB5-67-

Seems a product bug on aarch64 and ppc64le, We can double check it with developer :)

I'd ask your opinion on how to handle failures on this kind of systemd test: https://openqa.suse.de/tests/13225554
basically the test cases run in openQA but they are not developed from us, as they come from upstream testsuite and dinamically scheduled from the initial module:

    foreach my $test (@schedule) {
        my $args = OpenQA::Test::RunArgs->new(test => $test, dir => $testdir, make_opts => $test_opts);
        autotest::loadtest('tests/systemd_testsuite/runner.pm', name => $test, run_args => $args);
    }   

should we open a bugzilla for each failing test case ? Contact the developers, try to fix ourself ? Most of them just end due to timeouts like

[   67.521547] testsuite-17.sh[520]: + echo 'Subtest /usr/lib/systemd/tests/testdata/units/testsuite-17.02.sh failed'
[   67.541776] testsuite-17.sh[520]: Subtest /usr/lib/systemd/tests/testdata/units/testsuite-17.02.sh failed
[   67.552700] testsuite-17.sh[520]: + return 1
[   79.177547][    T1] systemd-shutdown[1]: Waiting for process: 491 (systemd-udevd), 772 ((udev-worker))
[  101.240673][  T491] systemd-udevd[491]: foobar: Worker [772] processing SEQNUM=1993 is taking a long time
[  132.281233][  T772] (udev-worker)[772]: hoge: Failed to rename network interface 4 from 'foobar' to 'hoge': File exists
[  132.298053][  T772] (udev-worker)[772]: foobar: Failed to process device, ignoring: File exists

seems they are running a qemu VM for each test case (so at the end it's 3 layers of nested virtualization) ...

draft PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18462

proposed a way to tackle the sub-tests failure; either excluding them or via softfail. After merge, we need only to schedule this module

MR: https://gitlab.suse.de/qe-core/qa-sle-functional-userspace/-/merge_requests/206

Nice, thanks https://openqa.suse.de/tests/13365482

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upstream_systemd_tests
https://openqa.suse.de/tests/13647391

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upstream_systemd_tests
https://openqa.suse.de/tests/13958836

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 68 days if nothing changes in this ticket.