Actions
action #45158
closed[systemd] Implement systemd testsuite as openQA perl module
Status:
Resolved
Priority:
Low
Assignee:
Category:
Enhancement to existing tests
Target version:
Start date:
2019-01-18
Due date:
% Done:
0%
Estimated time:
Difficulty:
Description
Motivation¶
We are using the systemd's upstream test suite, but it is designed to run at build time. With some patches done by tsaupe, we are able to run the test suite in a running system.
Find the custom systemd-testsuite here:
- https://build.opensuse.org/package/show/devel:openSUSE:QA:Tumbleweed/systemd-v237-testsuite
- https://build.suse.de/package/show/QA:SLE15/systemd-v234-testsuite
In our effort to run automatic tests (openQA) on all 4 supported architectures (aarch64, ppc64le, s390x and x86_64), we have faced several issues.
suse_patches-systemd_testsuite:¶
- Tumbleweed#latest
- SLE15-SP1-aarch64#latest
- SLE15-SP1-ppc64le#latest
- SLE15-SP1-s390x#latest
- SLE15-SP1-x86_64#latest
Known problems¶
- Nested virtualization (systemd-qemu's testsuite running on top of openQA qemu [even running on top of a VM s390x]) causes a lot of hangs on tests. In order to avoid this nested virtualization, we would need implement those test as an openQA perl module.
- Test suite maintenance: The maintenance of the test suite used in openQA became very difficult.
- Factory currently using v239: Starting from systemd-v237 (Tumbleweed), some test of the upstream test suite stopped working because they need a seed generated at build time for the systemd package. Since the custom systemd-testsuite by tsaupe is build in a separate OBS package, it doesn't have the same seed and the test fails to run. Find the OBS package here: https://build.opensuse.org/package/show/openSUSE:Factory/systemd
- On development versions (Leap 15.1 and SLE 15-SP1 using systemd-v234) hangs on the test are reproducible as shown in previous jobs. Find the OBS package here: https://build.suse.de/package/show/home:fbui:systemd:SLE-15/systemd
- On released versions, also maintenance (SLE 12-SP3, Leap 42.3) there is no OBS package, so this test suite is not executed on QAM for regressions qa_userspace_systemd is used instead)
- See also related tickets, which could be resolved by this new approach.
Acceptance criteria¶
- AC1: The upstream test suite is implemented as an openQA perl module to be executed as a normal test, not depending on any special OBS package.
- AC2: Factory, development and maintenance openQA tests are able to use this new implemented module.
Actions