QA

The dashboard itself consists of 2 components. The dashboard itself openQA bot performing the operations underneath.

Dashboard

• Test state overview - The openQA reviewer can easily see what tests passed/failed... for the particular update to see commonalities, e.g. the same test modules failing in all codestreams and versions
  • Alternative: A test module centric openQA view
• Have a section for each squad (need to be able to assign tests/job groups to squads before)
  • It should be sufficient or better suitable to just have that in openQA itself unless there is benefit in knowing which updates are blocked by tests maintained by particular squads
• History - be able to see which tests passed/failed for the updates in recent history to not need to wait for all jobs for incidents to complete
  • alternative: seen as less helpful than automatic actions by any approving automation that looks into the history
  • further details: openQA knows the history for each scenario
    • Is it possible some example how verify this easily, as an old history from smelt?
      • yes, openQA can provide test overviews for specific parameters, e.g. a certain incident, for a specific time
      • remark: Right now maintenance openQA test results on openqa.suse.de are mostly stored for only about 2 weeks, would need storage investment to store results going back longer into the past

Bot

• Ensure that all to-be-scheduled tests are considered for any approval decision
  • rejection, especially manual rejection, can be done on individual already finished test results
  • alternative: Only approve if there are at least as many passed results as in previous incident releases
    • Would need possibility to mark the decrease of test numbers or test coverage as acceptable