action #88127
Updated by hurhaj over 3 years ago
## User story Problem statement As a member During regression testing of QEM maintenance update squad, I want to know the extent of automation available and running for the package with automated tests in openQA, the maintenance update I'm testing, to decide engineer take a brief look at latest runs, but are rarely evaluating whether additional manual testing coverage is needed. ## Acceptance criteria * **AC1:** output about the maximum coverage needs enough. In case when some test was unscheduled due to take long term failing, the tested product/code-stream into account * **AC2:** URL to engineer is in no way notified about the test suite where tests are running needs to be provided in test report * **AC3:** list decrease of tests that are testing coverage and therefore does not running needs to be provided in test report * **AC4:** entries in DB needs to be updatable make up for both adding new available coverage but also removing tests from the list it with manual testing. ## Tasks Solution * to There should be added later ## Further details To illustrate rough idea a database with maximum automated coverage for specific package. During generation of expected outcome in test report, lets assume we have MU containing packages ABC diff between max and XYZ for SLE 15 SP0,1,2 current coverage would be created and filled into test report, so that that both packages have two dedicated tests that have always been running only on SP1,2 and there's also one indirect test. The testers can be notified about possible output in test report might look something like: ``` regression tests: -------------------------- Regression tests are covered in openQA: SLE 15 - indirect_test in <URL_to_latest> SLE 15 SP1 - abc_first, abc_second, xyz_first in <URL_to_latest_SERVER>, <URL_to_latest_DESKTOP>,... - indirect_test in <URL_to_latest> SLE 15 SP2 - abc_first, abc_second, xyz_first, xyz_second in <URL_to_latest_SERVER>, <URL_to_latest_DESKTOP>,... - indirect_test in <URL_to_latest> WARNING: Decreased coverage, replace with manual testing! SLE 15 SP1 - xyz_second ``` lowering of the coverage.