Project

General

Profile

Actions
Copy link

action #124473

open

coordination #99303: [saga][epic] Improvements for SUSE Maintenance QA workflows with fully automated testing, approval and release

[tools] Automatic regression tests export from openQA

Added by vpelcak over 2 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
future
Start date:
2023-02-14
Due date:
% Done:

0%

Estimated time:

Description

Motivation

With our progression in automation of regression tests we reached the state that many updates just need the results to be looked up in openQA and linked in our testreports.
That is pretty repetitive manual work which begs for the automation.

Acceptance criteria

  • AC1: Automation fills the links to the appropriate regression tests in openQA and their result into the testreport of the update.

Suggestions

  • This will perhaps require knowledge of our test coverage
  • Maybe extension of openQA API will be needed

Related issues 2 (2 open — 0 closed)

Related to QA (public) - action #94600: [tools][mtui] Communicate reduced visibility of openQA incident related coverage in logNew2021-06-23

Actions
Related to QA (public) - coordination #110016: [epic][teregen] teregen (maintenance test report template generator) improvementsNew2021-01-14

Actions
Actions
Copy link
 #1

Updated by okurz over 2 years ago

  • Target version set to future

Discussed with vpelcak. The test report needs to include information about the test coverage which allows a 2nd level reviewer to crosscheck the test coverage reviewed by test reviewers. The generated test reports already contain openQA references but here we need additional job informations. This is about aggregate tests only. vpelcak showed me an example looking for "multipath" tests. For that we needed to look around on the webUI and look primarily for test modules, maybe other scenarios as well. We can try to use the openQA built-in search functionality.

One idea: An openQA search link like https://openqa.suse.de/search?q=multipath but then need to filter the thousands of search results to match only something like "Maintenance". We could include such search term automatically in reports but often the exact search term is not obvious.

Also there is the "metadata" field in test modules.

There should be more related tickets that we should link here.

Actions
Copy link
 #2

Updated by okurz over 2 years ago

  • Related to action #94600: [tools][mtui] Communicate reduced visibility of openQA incident related coverage in log added
Actions
Copy link
 #4

Updated by okurz over 2 years ago

  • Parent task set to #99303
Actions
Copy link
 #7

Updated by okurz over 2 years ago

https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Definition-of-DONE mentions already Test modules that have been touched have updated metadata, e.g. "Maintainer" and "Summary" which is also ensured by CI so we could extend that for additional metadata as described in e.g. #77002

Actions
Copy link
 #10

Updated by okurz over 2 years ago

  • Related to coordination #110016: [epic][teregen] teregen (maintenance test report template generator) improvements added
Actions
Copy link

Also available in: Atom PDF