action #98820


coordination #99303: [saga][epic] Future improvements for SUSE Maintenance QA workflows with fully automated testing, approval and release

Various requirements for qem-dashboard (was: Design document for openQA CI dashboard)

Added by vpelcak over 2 years ago. Updated over 2 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


The dashboard itself consists of 2 components. The dashboard itself openQA bot performing the operations underneath.


  • Test state overview - The openQA reviewer can easily see what tests passed/failed... for the particular update to see commonalities, e.g. the same test modules failing in all codestreams and versions
    • Alternative: A test module centric openQA view
  • Have a section for each squad (need to be able to assign tests/job groups to squads before)
    • It should be sufficient or better suitable to just have that in openQA itself unless there is benefit in knowing which updates are blocked by tests maintained by particular squads
  • History - be able to see which tests passed/failed for the updates in recent history to not need to wait for all jobs for incidents to complete
    • alternative: seen as less helpful than automatic actions by any approving automation that looks into the history
    • further details: openQA knows the history for each scenario
      • Is it possible some example how verify this easily, as an old history from smelt?
        • yes, openQA can provide test overviews for specific parameters, e.g. a certain incident, for a specific time
        • remark: Right now maintenance openQA test results on are mostly stored for only about 2 weeks, would need storage investment to store results going back longer into the past


  • Ensure that all to-be-scheduled tests are considered for any approval decision
    • rejection, especially manual rejection, can be done on individual already finished test results
    • alternative: Only approve if there are at least as many passed results as in previous incident releases
      • Would need possibility to mark the decrease of test numbers or test coverage as acceptable

Also available in: Atom PDF