action #124473
open
coordination #99303: [saga][epic] Future improvements for SUSE Maintenance QA workflows with fully automated testing, approval and release
[tools] Automatic regression tests export from openQA
Added by vpelcak about 1 year ago.
Updated about 1 year ago.
Description
Motivation¶
With our progression in automation of regression tests we reached the state that many updates just need the results to be looked up in openQA and linked in our testreports.
That is pretty repetitive manual work which begs for the automation.
Acceptance criteria¶
- AC1: Automation fills the links to the appropriate regression tests in openQA and their result into the testreport of the update.
Suggestions¶
- This will perhaps require knowledge of our test coverage
- Maybe extension of openQA API will be needed
Related issues
2 (2 open — 0 closed)
- Target version set to future
Discussed with vpelcak. The test report needs to include information about the test coverage which allows a 2nd level reviewer to crosscheck the test coverage reviewed by test reviewers. The generated test reports already contain openQA references but here we need additional job informations. This is about aggregate tests only. vpelcak showed me an example looking for "multipath" tests. For that we needed to look around on the webUI and look primarily for test modules, maybe other scenarios as well. We can try to use the openQA built-in search functionality.
One idea: An openQA search link like https://openqa.suse.de/search?q=multipath but then need to filter the thousands of search results to match only something like "Maintenance". We could include such search term automatically in reports but often the exact search term is not obvious.
Also there is the "metadata" field in test modules.
There should be more related tickets that we should link here.
- Related to action #94600: [tools][mtui] Communicate reduced visibility of openQA incident related coverage in log added
- Parent task set to #99303
- Related to coordination #110016: [epic][teregen] teregen (maintenance test report template generator) improvements added
Also available in: Atom
PDF