Project

General

Profile

Actions

action #122953

closed

[qe-core] Extend openqa-cli to trigger multiple jobs (Add support for statistical investigation)

Added by szarate almost 2 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Refactor/Code Improvements
Target version:
QA - QE-Core: Ready
Start date:
2023-01-11
Due date:
% Done:

100%

Estimated time:
Difficulty:
Sprint:
QE-Core: October Sprint 23 (Oct 11 - Nov 08)

Description

Too often, we need to either trigger multiple jobs to know if an intermittent test failure is gone or to try and catch a sporadic product bug or an issue somewhere in the infrastructure, so the story would go like this:

  • As a developer, I would like the openQA client to trigger a list of jobs N times so I can catch a flaky testcase or reproduce a sporadic issue that doesn't affect other builds or JobGroups.

I'm looking for an extension of the openQA Client code; knowing how to extend the openQA Client will allow us to do even more in the future to support other squads.

Acceptance Criteria

  1. Solution is documented in https://open.qa/docs/ in a Section dedicated to statistical investigation
  2. Solution is packaged and distributed with openQA-client
  3. Job count is added to the name of the test
  4. Allow for a PR (CASEDIR) also to be triggered

Notes

  1. See https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Statistical-investigation and Oliver's comment
  2. TL;DR https://github.com/okurz/scripts/blob/master/openqa-clone-set or https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Statistical-investigation already achieve but built-in into openqa-cli?
  3. Initially we want a sensible amount of jobs, so we don't end up DoS'ing ourselves, so things like 1M jobs or even 1K, should be taken into consideration on how to approach that.

openqa-statistics-whatever $JOB $TIMES $JOB_SETTINGS
openqa-clone-job ... --job-count 15 ...

Actions

Also available in: Atom PDF