Project

General

Profile

Actions

action #71758

closed

coordination #58184: [saga][epic][use case] full version control awareness within openQA

[spike][timeboxed:20h] complete test definition from a yaml schedule file in local test distribution folder

Added by okurz about 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Motivation

See #58184 . As a test maintainer using openQA I want openQA to be able to load the complete test definition from a yaml schedule in a local test distribution folder so that all the necessary configuration can come from a central place and no configuration needs to be added to the openQA database beforehand

Acceptance criteria

  • AC1: proof-of-concept exists for a complete test definition consisting of at least two scenarios (1) loadable from text files within the local test distribution folder

Suggestions

References

  1. Note 1 in AC1: A single job can be completely specified by test parameters on job creation time already. So the necessary challenge is that we have test definitions as commonly done by the combination of medium types, test suites, job templates, job groups (and machines and worker config) to form a test schedule consisting of multiple scenarios, i.e. at least two

Related issues 2 (0 open2 closed)

Copied to openQA Project - action #92311: Complete test definition from a single yaml schedule file in local test distribution folderResolvedmkittler

Actions
Copied to openQA Project - action #124502: [spike][timeboxed:20h] complete test definition from yaml schedule in git checked out test distributionResolvedmkittler2023-02-14

Actions
Actions #1

Updated by okurz about 4 years ago

  • Description updated (diff)
Actions #2

Updated by okurz about 4 years ago

  • Subject changed from [spike][timeboxed:20h] complete test definition from yaml schedule in local test distribution folder to [spike][timeboxed:20h] complete test definition from a yaml schedule in local test distribution folder
Actions #3

Updated by okurz about 4 years ago

  • Subject changed from [spike][timeboxed:20h] complete test definition from a yaml schedule in local test distribution folder to [spike][timeboxed:20h] complete test definition from a yaml schedule file in local test distribution folder
Actions #4

Updated by livdywan about 4 years ago

  • Assignee set to livdywan
Actions #5

Updated by livdywan almost 4 years ago

  • Assignee deleted (livdywan)

I'll unassign myself since I rather focus on flaky CI and issues with production workers atm. If someone beats me to it, go for it.

Actions #6

Updated by okurz almost 4 years ago

  • Description updated (diff)

Added suggestion instead of just "reference".

Actions #7

Updated by livdywan over 3 years ago

Some links for reference:

https://gitlab.suse.de/qsf-y/qa-sle-functional-y/-/blob/3ef279f9d4a536c3004c292442f8e50d48c024d1/JobGroups/test_suites.yaml

As discussed:

  • Let's assume we have nothing defined in the database
  • Think of how to do ISOs post from a file in a git repo
  • Keep in mind the user who just sets up a PR to run tests
  • Propose a format to load a test. We can change the format later, we have schemas for that and ways to communicate new versions supported by API routes.
  • Consider multi-machine tests - but the first version doesn't have to support all use cases
Actions #8

Updated by Xiaojing_liu over 3 years ago

  • Assignee set to Xiaojing_liu

I spent some time learning this ticket, so I would like to try.

Actions #9

Updated by openqa_review over 3 years ago

  • Due date set to 2021-04-23

Setting due date based on mean cycle time of SUSE QE Tools

Actions #10

Updated by okurz over 3 years ago

  • Status changed from Workable to In Progress
Actions #11

Updated by Xiaojing_liu over 3 years ago

  • Due date deleted (2021-04-23)
  • Status changed from In Progress to Workable
  • Assignee deleted (Xiaojing_liu)

I committed above pr to describe my thought.

  1. we could schedule more than two jobs at one time from a file (e.g. from a yaml file), and the dependency will be created.
  2. the format of the file needs to be confirmed with others (members of the tool team or users?)

I am not sure if the pr is in the right way.
This ticket is [spike] [timeboxed:20h], and I have spent more than 20 hours, I unassign it to wait for more feedback.

Actions #12

Updated by livdywan over 3 years ago

Xiaojing_liu wrote:

I committed above pr to describe my thought.

  1. we could schedule more than two jobs at one time from a file (e.g. from a yaml file), and the dependency will be created.
  2. the format of the file needs to be confirmed with others (members of the tool team or users?)

I am not sure if the pr is in the right way.
This ticket is [spike] [timeboxed:20h], and I have spent more than 20 hours, I unassign it to wait for more feedback.

For the record, it's a spike, so there is no correct way. Thank you for coming up with a very nice proof of concept here!

  • I like how this is very similar to our existing YAML and mirrors the definitions in the database. So even without new documentation I know what it does.
  • Maybe as a user I'd want separate files here, for instance a separate file for the job templates so that I can re-use the machine definitions easily.
  • The code has a comment about cloning from a git repo. Maybe we could even allow a PR or branch name here e.g. #branchname to support a review/CI workflow.
Actions #13

Updated by okurz over 3 years ago

  • Assignee set to okurz

as discussed in weekly I will take over to come up with follow-up plans accordingly.

Actions #14

Updated by openqa_review over 3 years ago

  • Due date set to 2021-05-08

Setting due date based on mean cycle time of SUSE QE Tools

Actions #15

Updated by okurz over 3 years ago

  • Copied to action #92311: Complete test definition from a single yaml schedule file in local test distribution folder added
Actions #16

Updated by okurz over 3 years ago

  • Status changed from Workable to Resolved
  • Assignee changed from okurz to Xiaojing_liu

Sorry about my delay. Covered followup in #92311

Actions #17

Updated by okurz over 3 years ago

  • Due date deleted (2021-05-08)
Actions #18

Updated by okurz over 1 year ago

  • Copied to action #124502: [spike][timeboxed:20h] complete test definition from yaml schedule in git checked out test distribution added
Actions

Also available in: Atom PDF