Project

General

Profile

action #122953

[qe-core] Extend openqa-cli to trigger multiple jobs (Add support for statistical investigation)

Added by szarate 3 months ago. Updated 23 days ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Refactor/Code Improvements
Target version:
Start date:
2023-01-11
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Too often, we need to either trigger multiple jobs to know if an intermittent test failure is gone or to try and catch a sporadic product bug or an issue somewhere in the infrastructure, so the story would go like this:

  • As a developer, I would like the openQA client to trigger a list of jobs N times so I can catch a flaky testcase or reproduce a sporadic issue that doesn't affect other builds or JobGroups.

I'm looking for an extension of the openQA Client code; knowing how to extend the openQA Client will allow us to do even more in the future to support other squads.

Acceptance Criteria

  1. Solution is documented in https://open.qa/docs/ in a Section dedicated to statistical investigation
  2. Solution is packaged and distributed with openQA-client
  3. Job count is added to the name of the test
  4. Allow for a PR (CASEDIR) also to be triggered

Notes

  1. See https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Statistical-investigation and Oliver's comment
  2. TL;DR https://github.com/okurz/scripts/blob/master/openqa-clone-set or https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Statistical-investigation already achieve but built-in into openqa-cli?

openqa-statistics-whatever $JOB $TIMES $JOB_SETTINGS
openqa-clone-job ... --job-count 15 ...

History

#1 Updated by szarate 3 months ago

  • Tags set to qe-core-january-sprint
  • Project changed from openQA Project to openQA Tests
  • Subject changed from [qe-core] Extend openqa-cli to trigger multiple jobs to [qe-core] Extend openqa-cli to trigger multiple jobs (Add support for statistical investigation)
  • Description updated (diff)
  • Category changed from Feature requests to Refactor/Code Improvements

#2 Updated by szarate 3 months ago

  • Sprint set to QE-Core: January Sprint (Jan 11 - Feb 08)

#3 Updated by szarate 3 months ago

  • The URL fragments can be cleaned up from the URL

#4 Updated by okurz about 2 months ago

  • As a developer, I would like the openQA client to trigger a list of jobs N times so I can catch a flaky testcase or reproduce a sporadic issue that doesn't affect other builds or JobGroups.

Isn't that what
https://github.com/okurz/scripts/blob/master/openqa-clone-set or https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Statistical-investigation already provide? We could consider moving those scripts to either https://github.com/os-autoinst/scripts/ or https://github.com/os-autoinst/openQA/ of course but I wouldn't start from script

#5 Updated by szarate about 2 months ago

  • Description updated (diff)

okurz wrote:

  • As a developer, I would like the openQA client to trigger a list of jobs N times so I can catch a flaky testcase or reproduce a sporadic issue that doesn't affect other builds or JobGroups.

Isn't that what
https://github.com/okurz/scripts/blob/master/openqa-clone-set or https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Statistical-investigation already provide? We could consider moving those scripts to either https://github.com/os-autoinst/scripts/ or https://github.com/os-autoinst/openQA/ of course but I wouldn't start from script

It is, just not quite; I'd prefer if there's actual code extending the openQA-cli, whether the perl or python clients, rather than a bash script (I'm here mostly pushing the boundary to gain knowledge as a team on how to enhance the clients directly)

#6 Updated by szarate about 2 months ago

  • Description updated (diff)

#7 Updated by okurz about 2 months ago

szarate wrote:

It is, just not quite; I'd prefer if there's actual code extending the openQA-cli, whether the perl or python clients, rather than a bash script (I'm here mostly pushing the boundary to gain knowledge as a team on how to enhance the clients directly)

So the actual requirement would be: Do what https://github.com/okurz/scripts/blob/master/openqa-clone-set or https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Statistical-investigation already achieve but built-in into openqa-cli, right?

#8 Updated by szarate about 2 months ago

  • Sprint changed from QE-Core: January Sprint (Jan 11 - Feb 08) to QE-Core: February Sprint (Feb 08 - Mar 08)

#9 Updated by szarate about 2 months ago

  • Tags changed from qe-core-january-sprint to qe-core-january-sprint, platform-team
  • Description updated (diff)

#10 Updated by szarate about 1 month ago

  • Status changed from New to Workable

#11 Updated by szarate 25 days ago

  • Sprint changed from QE-Core: February Sprint (Feb 08 - Mar 08) to QE-Core: March Sprint (Mar 08 - Apr 05)

#12 Updated by fgerling 23 days ago

  • Assignee set to fgerling

#13 Updated by fgerling 23 days ago

  • Status changed from Workable to In Progress

Also available in: Atom PDF