Project

General

Profile

Actions
Copy link

coordination #44360

open

openQA Tests (public) - coordination #15132: [saga][epic] Better structure of test plans in main.pm

[epic] Parameterize test suites within job groups

Added by coolo about 6 years ago. Updated over 3 years ago.

Status:
Blocked
Priority:
Low
Assignee:
okurz
Category:
Feature requests
Target version:
QA (public) - future
Start date:
2019-01-25
Due date:
% Done:

86%

Estimated time:
(Total: 0.00 h)

Description

We discussed how we can reduce the number of test suites and one solution is to move
the workflow parameters to job groups.

So instead of install-kde, install-gnome and install-text, we schedule
install 3 times with 3 different DESKTOP parameters.

This has several implicitions - hence the epic.

  • we need to provide a variable setting in the job group interface
  • we need to summarize the test suite differently in the review interfaces, possibly by requiring the user to give the paramterized test suite another name
  • we need to extend the DB interface of job groups
  • we need to provide a new API for this

Subtasks 21 (2 open — 19 closed)

action #46667: Define version-able and human readable format for job scheduling-related tablesResolvedlivdywan2019-01-25

Actions
action #46670: Create import for format defined in #46667Resolvedlivdywan2019-01-25

Actions
action #46673: Extend tables for job scheduling by another table for parametersResolvedlivdywan2019-01-25

Actions
action #50675: Commit changes to scheduling YAML to Git repositoryNew2019-04-24

Actions
action #54143: Support parameters in {load,dump}_templatesResolvedlivdywan2019-07-11

Actions
action #54146: Expose saving YAML and opt-in migration in the editorResolvedlivdywan2019-07-11

Actions
action #54149: Diffs for better error handling in the YAML editorResolvedlivdywan2019-07-11

Actions
action #54179: Re-use YAML betweens different groupsNew2019-07-12

Actions
action #55454: The job group YAML schedule should support multiple scenarios with different variables, not just settingsResolvedlivdywan2019-08-13

Actions
coordination #55730: [epic] Move parameters from test suites into job groupsResolvedokurz2019-09-06

Actions
action #56540: convert staging job groups to YAMLResolvedlivdywan2019-09-06

Actions
action #57845: Switch more job groups to YAML job templatesResolvedokurz2019-10-09

Actions
action #58652: Write a training file about how to use YAML in job groupResolvedXiaojing_liu2019-10-24

Actions
action #60329: Use more parameterized job templates for test suites only used onceResolvedtinita2019-11-27

Actions
action #60782: descriptions for parameterized job templates independant of test suite descriptionsResolvedtinita2019-12-06

Actions
openQA Tests (public) - action #64967: job templates are duplicated as job template in job groups as well as test suitesResolvedtinita2020-03-29

Actions
action #60014: Show YAML editor for unmigrated groupsResolvedokurz2019-11-19

Actions
action #60020: Provide a script to migrate job groups to YAML automaticallyResolvedokurz2019-11-19

Actions
action #60017: Document process and potential issues with migration to YAMLResolvedlivdywan2019-11-19

Actions
coordination #66727: [epic] Define structure to define test suites not in openQA databaseResolvedokurz2020-05-13

Actions
action #66781: hidden keys in yaml job templatesResolvedlivdywan2020-05-13

Actions

Related issues 1 (0 open — 1 closed)

Related to openQA Project (public) - action #45413: Support markdown in the test suites descriptionRejectedokurz2018-12-19

Actions
Actions
Copy link
 #1

Updated by okurz about 6 years ago

  • Parent task set to #15132
Actions
Copy link
 #2

Updated by mkittler about 6 years ago

we need to extend the DB interface of job groups

Or, more specifically, of the job templates table.

In the last meeting we also discussed that having a config file in some Git repo could be preferable compared to saving the config in the database. So I suppose we save this idea for later?

Actions
Copy link
 #3

Updated by okurz about 6 years ago

mkittler wrote:

So I suppose we save this idea for later?

I think that was the idea, yes.

Btw, I realized that with the "inter-machine dependencies" we reference machines within testsuites even though they are not defined there. Maybe this could be streamlined along with this epic here as I see it related.

Actions
Copy link
 #4

Updated by JERiveraMoya almost 6 years ago

  • Blocked by action #45413: Support markdown in the test suites description added
Actions
Copy link
 #5

Updated by JERiveraMoya almost 6 years ago

  • Blocked by deleted (action #45413: Support markdown in the test suites description)
Actions
Copy link
 #6

Updated by JERiveraMoya almost 6 years ago

  • Blocks action #45413: Support markdown in the test suites description added
Actions
Copy link
 #7

Updated by mkittler almost 6 years ago

In the last meeting the idea to have the data in a version-able and human readable document (e.g. YAML file) was picked up again.

Actions
Copy link
 #8

Updated by livdywan over 5 years ago

  • Due date set to 2019-06-18

due to changes in a related task

Actions
Copy link
 #9

Updated by okurz over 5 years ago

  • Category changed from 122 to Feature requests
Actions
Copy link
 #10

Updated by sebchlad over 5 years ago

"We discussed how we can reduce the number of test suites and one solution is to move
the workflow parameters to job groups.

So instead of install-kde, install-gnome and install-text, we schedule
install 3 times with 3 different DESKTOP parameters."

The best example to try this out, which I can think of, would be hpc/mpi testing.

For SLE15 we might be supporting 5+ MPI implementations.
Testing of MPIs requires the multimachine set-up; currently I define following test suits for each MPI flavor: master, slaves*2, supportserver.
That means 5 MPIs * 4 test suits = 20 entries using the same test code beneath.
The matter is actually worse as the real bugs were found with different CPU count.
Currently I maintain 2 different CPU counts; QEMUCPUS=1 and QEMUCPUS=2.
So effectively = 20 test suits * 2.

We could consider parameterizing mpi_flavour to begin with.

@Oli: thx for pointing this poo to me.

Actions
Copy link
 #11

Updated by okurz over 5 years ago

  • Due date changed from 2019-06-18 to 2019-09-06

due to changes in a related task

Actions
Copy link
 #12

Updated by okurz about 5 years ago

  • Status changed from New to Blocked
  • Assignee set to okurz
  • Target version changed from Ready to Current Sprint

Discussed in QA tools meeting 2019-10-08, waiting for colleagues mainly from "QA SLE Migration" and "QA SLE Virtualization" to be able to follow on with migration, roughly mid of October. Blocked by subtask #55730

Actions
Copy link
 #13

Updated by okurz about 5 years ago

  • Due date changed from 2019-06-18 to 2019-10-22

due to changes in a related task

Actions
Copy link
 #15

Updated by okurz about 5 years ago

  • Status changed from Blocked to Feedback

Going further with a deprecation notice: https://github.com/os-autoinst/openQA/pull/2518

Actions
Copy link
 #16

Updated by okurz about 5 years ago

  • Due date changed from 2019-11-19 to 2020-02-29

due to changes in a related task

Actions
Copy link
 #17

Updated by okurz about 5 years ago

  • Status changed from Feedback to Blocked

PR merged and also #60014 done. Rest is again in subtickets.

Actions
Copy link
 #18

Updated by okurz about 5 years ago

  • Due date changed from 2020-02-29 to 2020-11-27

due to changes in a related task

Actions
Copy link
 #19

Updated by szarate about 4 years ago

  • Tracker changed from action to coordination
  • Status changed from Blocked to New
  • Difficulty deleted (hard)
Actions
Copy link
 #21

Updated by okurz about 4 years ago

  • Target version changed from Current Sprint to Ready
Actions
Copy link
 #22

Updated by okurz about 4 years ago

  • Blocks deleted (action #45413: Support markdown in the test suites description)
Actions
Copy link
 #23

Updated by okurz about 4 years ago

  • Related to action #45413: Support markdown in the test suites description added
Actions
Copy link
 #24

Updated by okurz about 4 years ago

  • Status changed from New to Blocked
Actions
Copy link
 #25

Updated by okurz over 3 years ago

  • Target version changed from Ready to future

saga not in current backlog as well as the only remaining subtask, moving to "future" for now again.

Actions
Copy link

Also available in: Atom PDF