As #111258 describes right now the testsuites running in Leap 15.3 Updates are not covering many scenarios: https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.3&build=20220518-1&groupid=80.
The purpose of this ticket is to gather candidate testsuites, to be scheduled in another ticket.
- AC.1 Gather the testsuites that can be added in Leap maintenance in order to have broader testing.
- AC.2 Contact Lubos and gather his feedback on said list
#5 Updated by mloviska about 1 month ago
- Status changed from In Progress to Feedback
According to discussion with Lubos, the highest priority is put on migration test scenarios, as of now we are executing
For Leap15.4 it lists below test suites.
zdup-Leap-15.2-gnome zdup-Leap-15.2-kde zdup-Leap-15.3-gnome zdup-Leap-15.3-kde
Other than that Lubos mentioned that we should cover
security_* test suites.
As a QA I would also like to have
upgrade_Leap_15.2+? scenarios, however we need to add online repos along with
Untested Updates repository that is omitted by default.
Along with this, I would suggest following list of test suites that should be added
apparmor autoyast_gnome security_audit transactional_server extra_tests_textmode_podman_containers extra_tests_textmode_docker_containers upgrade_Leap_15.2+ + enable untested package repo extra_tests_transactional_server
Also, we should add
#7 Updated by mloviska about 1 month ago
Thanks a lot Martin! could you give on top of this a look at what from SLES maintenance could be added too?, for instance the extratests aren't complete from what I can see (there's no textmode for instance)
I have checked that as well, maybe for leap makes sense to test
clamav. We do test
textmode in Leap Updates, but indeed you are right that more extratests can be added there as well.
Let's follow up walking over eggshells:
- Prioritize anything that doesn't depend on a desktop environment (consoletests)
- with extratests_*_console
- Incindents and Backports haven't been toched now
- Review coverage of X11 tests before adding more stuff (specially on aarch64)