action #151738
open[qe-core] Enable systemd-testsuite in 15-SP6
Added by jlausuch 5 months ago. Updated 28 days ago.
40%
Description
Background¶
There is an important update for systemd in 15-SP6 to v254: https://jira.suse.com/browse/PED-4846
Some important partners are a bit worried about this update and have asked us to please make sure this doesn't break any application (e.g. SAP applications).
We can't ensure 3rd party applications will be ok with this update, but what we can do is to provide good coverage for systemd testing (of course, SAP part will be covered by SAP squad).
systemd-testsuite has been integrated in openQA and enabled in TW by Martin Loviska when he was on rotation. e.g. https://openqa.opensuse.org/tests/3760221
It was also enabled in 15-SP4 and SP5 after GA, but the package is already in 15-SP6 channel http://download.suse.de/download/ibs/SUSE:/SLE-15-SP6:/GA/standard/x86_64/systemd-testsuite-254.5-150600.1.2.x86_64.rpm
Acceptance criteria¶
systemd-testsuiteis scheduled in 15-SP6 functional group.
Related slack thead: https://suse.slack.com/archives/C02CSAZLAR4/p1701087663691749
Updated by amanzini 5 months ago · Edited
which architectures are being covered ? I got a missing dependency error installing the package in aarch64
Problem: nothing provides 'qemu-kvm' needed by the to be installed systemd-testsuite-254.5-150600.1.9.aarch64
Solution 1: do not install systemd-testsuite-254.5-150600.1.9.aarch64
Solution 2: break systemd-testsuite-254.5-150600.1.9.aarch64 by ignoring some of its dependencies
while on ppc64le
Activating sle-module-python3 15.6 ppc64le ...
Error: Registration server returned 'The product you are attempting to activate (Python 3 Module 15 SP6 ppc64le) is not available on your system's base product (SUSE Linux Enterprise Server 15 SP6 ppc64le). Python 3 Module 15 SP6 ppc64le is available on SUSE Linux Enterprise Server 15 SP5 ppc64le, SUSE Linux Enterprise Server for SAP Applications 15 SP5 ppc64le.' (422)
eTLB5-67-
Updated by rfan1 5 months ago
amanzini wrote in #note-4:
which architectures are being covered ? I got a missing dependency error installing the package in aarch64
Problem: nothing provides 'qemu-kvm' needed by the to be installed systemd-testsuite-254.5-150600.1.9.aarch64 Solution 1: do not install systemd-testsuite-254.5-150600.1.9.aarch64 Solution 2: break systemd-testsuite-254.5-150600.1.9.aarch64 by ignoring some of its dependencieswhile on ppc64le
Activating sle-module-python3 15.6 ppc64le ... Error: Registration server returned 'The product you are attempting to activate (Python 3 Module 15 SP6 ppc64le) is not available on your system's base product (SUSE Linux Enterprise Server 15 SP6 ppc64le). Python 3 Module 15 SP6 ppc64le is available on SUSE Linux Enterprise Server 15 SP5 ppc64le, SUSE Linux Enterprise Server for SAP Applications 15 SP5 ppc64le.' (422) eTLB5-67-Seems a product bug on aarch64 and ppc64le, We can double check it with developer :)
Updated by amanzini 5 months ago · Edited
- Status changed from In Progress to Feedback
- % Done changed from 0 to 30
on ppc64le platform, Python3 module can not be activated, error message refers to 15 SP5 ?
https://openqa.suse.de/tests/13223290
Activating sle-module-python3 15.6 ppc64le ...
Error: Registration server returned 'The product you are attempting to activate (Python 3 Module 15 SP6 ppc64le) is not available on your system's base product (SUSE Linux Enterprise Server 15 SP6 ppc64le). Python 3 Module 15 SP6 ppc64le is available on SUSE Linux Enterprise Server 15 SP5 ppc64le, SUSE Linux Enterprise Server for SAP Applications 15 SP5 ppc64le.' (422)
eTLB5-67-
opened https://bugzilla.suse.com/show_bug.cgi?id=1218683
on aarch64, https://openqa.suse.de/tests/13223289/logfile?filename=serial_terminal.txt
0| 2024-01-10 02:37:43 <1> susetest(4532) [zypp::solver] SATResolver.cc(problems):1358 ====================================
0| 2024-01-10 02:37:43 <1> susetest(4532) [zypp::solver] SATResolver.cc(problems):1362 nothing provides 'qemu-kvm' needed by the to be installed systemd-testsuite-254.5-150600.2.4.aarch64
0| 2024-01-10 02:37:43 <1> susetest(4532) [zypp::solver] SATResolver.cc(problems):1363 ------------------------------------
Updated by amanzini 4 months ago · Edited
I'd ask your opinion on how to handle failures on this kind of systemd test: https://openqa.suse.de/tests/13225554
basically the test cases run in openQA but they are not developed from us, as they come from upstream testsuite and dinamically scheduled from the initial module:
foreach my $test (@schedule) {
my $args = OpenQA::Test::RunArgs->new(test => $test, dir => $testdir, make_opts => $test_opts);
autotest::loadtest('tests/systemd_testsuite/runner.pm', name => $test, run_args => $args);
}
should we open a bugzilla for each failing test case ? Contact the developers, try to fix ourself ? Most of them just end due to timeouts like
[ 67.521547] testsuite-17.sh[520]: + echo 'Subtest /usr/lib/systemd/tests/testdata/units/testsuite-17.02.sh failed'
[ 67.541776] testsuite-17.sh[520]: Subtest /usr/lib/systemd/tests/testdata/units/testsuite-17.02.sh failed
[ 67.552700] testsuite-17.sh[520]: + return 1
[ 79.177547][ T1] systemd-shutdown[1]: Waiting for process: 491 (systemd-udevd), 772 ((udev-worker))
[ 101.240673][ T491] systemd-udevd[491]: foobar: Worker [772] processing SEQNUM=1993 is taking a long time
[ 132.281233][ T772] (udev-worker)[772]: hoge: Failed to rename network interface 4 from 'foobar' to 'hoge': File exists
[ 132.298053][ T772] (udev-worker)[772]: foobar: Failed to process device, ignoring: File exists
seems they are running a qemu VM for each test case (so at the end it's 3 layers of nested virtualization) ...
Updated by amanzini 4 months ago · Edited
- % Done changed from 30 to 40
seems like some tests are always failing on SLE, because package needs some modifications. Example run: https://openqa.suse.de/tests/13259003
After discussing with Thomas Blume, we came up with the idea to softfail some specific tests.
sample VR: https://openqa.suse.de/tests/13264222
the one that are safe to softfail are controlled via the variable SYSTEMD_SOFTFAIL in the testsuite settings.
Currently it's
SYSTEMD_SOFTFAIL=(13-NSPAWN|16-EXTEND-TIMEOUT|17-UDEV|24-CRYPTSETUP|29-PORTABLE|43-PRIVATEUSER-UNPRIV|50-DISSECT|55-OOMD|62-RESTRICT-IFACES|65-ANALYZE|67-INTEGRITY|70-TPM2|82-SOFTREBOOT)
Updated by jlausuch 3 months ago
Nice, thanks https://openqa.suse.de/tests/13365482
Updated by openqa_review 3 months ago
- Status changed from Resolved to Feedback
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: upstream_systemd_tests
https://openqa.suse.de/tests/13493808
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released" or "EOL" (End-of-Life)
- The bugref in the openQA scenario is removed or replaced, e.g.
label:wontfix:boo1234
Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.
Updated by openqa_review 2 months ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: upstream_systemd_tests
https://openqa.suse.de/tests/13647391
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released" or "EOL" (End-of-Life)
- The bugref in the openQA scenario is removed or replaced, e.g.
label:wontfix:boo1234
Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.
Updated by openqa_review 28 days ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: upstream_systemd_tests
https://openqa.suse.de/tests/13958836
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released" or "EOL" (End-of-Life)
- The bugref in the openQA scenario is removed or replaced, e.g.
label:wontfix:boo1234
Expect the next reminder at the earliest in 68 days if nothing changes in this ticket.