Project

General

Profile

coordination #76966

Trigger different scope of test suites depending on the product development phase

Added by riafarov 9 months ago. Updated about 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
Start date:
Due date:
2021-06-03
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

As a QE engineer, I would like to trigger different set of test suites depending on the product development phase.
For instance, I do not want to trigger release notes checks for alpha builds.

Motivated by the discussion about package hub module, see #76960

We will need 2 parts here, as need to parse or even introduce some meta data for the builds, so that openQA receives this information. And then have a mechanism to label scenarios for matching phase.

This will allow us to run reduced set of tests for builds created daily and run extended set of tests for candidate builds.

As number of the executed scenarios grows rapidly, this feature would be more notable in near future.


Subtasks

openQA Tests - action #89996: [qe-core] Revert changes introduced by https://gitlab.suse.de/qsf-u/qa-sle-functional-userspace/-/merge_requests/125Resolvedszarate


Related issues

Related to openQA Tests - coordination #76960: Test modules that depends on PackageHub for software provided in supported SLE modulesNew2020-11-04

History

#1 Updated by riafarov 9 months ago

  • Related to coordination #76960: Test modules that depends on PackageHub for software provided in supported SLE modules added

#2 Updated by okurz 9 months ago

  • Priority changed from Normal to Low
  • Target version set to future

Could you please structure the feature request according to https://progress.opensuse.org/projects/openqav3/wiki/#Feature-requests so that we understand better what is required. Strictly speaking I think changing the test scope based on a product development phase is very "enterprisy" and also not long-term sustainable, complex, hard to maintain. What I can think of you could do already is simply solve that based on job templates which many teams have in a git repo. So you could have different documents that you push at different teams or branches as well.

#3 Updated by maritawerner 2 months ago

Since Rodion has left the company is there anybody else who would be interested in that topic?

Actually the migration team has solved this problem manually: they have a dedicated job goup for migration tests that are only trigggered manually for a released milestone build, background is here that this tests run against SCC and not proxy SCC. And real SCC does only work when the milestone is released.

The virtualization team has also split there testcases into two job groups. The two job groups are triggered for each build but for a milestone validation ("24 hours") only the first job group "virtualization acceptance" with the Prio 1 testcases is reviewed, the other job groups is reviewed later. The Prio 1 testcases are reviewed an confirmed by the VT PJM.

Also available in: Atom PDF