QA » openQA Project » openQA Tests

Motivation¶

In build.suse.de the review role "qam-openqa" is assigned regardless if there are actual openQA tests scheduled for a release request effectively blocking requests for which there are no openQA tests.
https://gitlab.suse.de/tools/smelt/-/issues/924 'Only assign "qam-openqa" for incidents which are covered by tested channels or products' originally describes the problem which apparently can not be easily solved in SMELT. So instead a better and simpler way is to ensure that there is just any openQA test for a release request even if it's just a "fake" openQA test but I guess in many cases better tests can be found.

Further details¶

From https://suse.slack.com/archives/C02CCRM8946/p1677573667946989 for more details

(Oliver Kurz) Hi, who can help me understand why there is still an open review for both "qam-sle" and "qam-openqa" in https://build.suse.de/request/show/288587 from about 1 month ago? 1. In that page I don't see any comment or reference that I would be able to follow. I am asking to understand the process and how to improve in general, only seeing that request as one example. http://dashboard.qam.suse.de/incident/27473 does not show any related openQA test results (anymore?). 2. I assume for "Update Products SLERT Update" there are no openQA tests enabled in general, is it? 3. I assume a link to https://smelt.suse.de/incident/27473/ would be helpful although that also does not tell me more
(Ioannis Bonatakis) because there is no configuration on rt to run incident test against rt-tests. in this case you need to approve the update manually.
(Jozef Pupava) There is no test coverage in openQA for RT_15-SP4 AFAIK, there are also many others like this waiting for qam-openqa without test coverage
(Oliver Kurz) so it's again https://gitlab.suse.de/tools/smelt/-/issues/924 'Only assign "qam-openqa" for incidents which are covered by tested channels or products'. This leaves points 1. and 3. and a new question 4. Who will take care that eventually https://gitlab.suse.de/tools/smelt/-/issues/924 is solved?
(Jozef Pupava) From openQA point of view I would make sure, if there can be test, to add some simple test to "solve" this issue. Not sure if e.g. SLE-Module-Development-Tools-OBS does make sense to cover in openQA, if yes let's do it if not then don't add qam-openqa, as simple as it sounds IDK how difficult is to do in SMELT.

As an alternative I wonder how qem-bot could simply approve a release request without any assigned openQA jobs. I guess the tricky part is how to distinguish between "there are no tests and never will be any" and "well, there should be tests but right now there are none (yet?) due to a scheduling problem with openQA"