QA » openQA Project

Motivation¶

Originally brought up in #97118#note-10 by okurz and mgrifalconi.
https://github.com/os-autoinst/scripts/blob/master/openqa-trigger-bisect-jobs originally removed individual incidents from a list to conduct tests excluding specific incidents to be able to bisect with/without individual incidents. Seems like something was changed in the test architecture to install all updates in the image generation jobs meaning that the repos and packages from the repos are not removed by just changing the test variable content within bisection jobs. The test code needs to be changed to account for that. Maybe as simple as removing the according zypper repo and calling zypper dup or zypper rm or something.

Impact¶

All results of "openqa-trigger-bisect-jobs" are invalid as the mentioned incidents are not actually excluded. Without correct results also "openqa-trigger-bisect-jobs" can not be used for any automatic decisions which would help in the daily work of openQA reviewers concerning SLE maintenance update testing.

Acceptance criteria¶

• AC1: openQA jobs triggered by https://github.com/os-autoinst/scripts/blob/master/openqa-trigger-bisect-jobs ensure that the updates from incidents that are not to be included are not present in the SUT

Suggestions¶

• Take a look at any jobs from https://openqa.suse.de/tests?match=bisect. E.g. https://openqa.suse.de/tests/10217887 is a job "sle-15-SP4-Server-DVD-Updates-aarch64-Build20221220-20221221-qam-minimal+base:investigate:bisect_without_27235@aarch64-virtio" so judging from the name of the job it should not include incident "27235". https://openqa.suse.de/tests/10217887#settings shows that while 27235 is mentioned in the variable "BASE_TEST_ISSUES" it is not included in the test variable "BASE_TEST_REPOS". But https://openqa.suse.de/tests/10217887/logfile?filename=serial_terminal.txt mentions a zypper repo with incident 27235