Project

General

Profile

action #71758

coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings

[spike][timeboxed:20h] complete test definition from a yaml schedule file in local test distribution folder

Added by okurz 10 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

See #58184 . As a test maintainer using openQA I want openQA to be able to load the complete test definition from a yaml schedule in a local test distribution folder so that all the necessary configuration can come from a central place and no configuration needs to be added to the openQA database beforehand

Acceptance criteria

  • AC1: proof-of-concept exists for a complete test definition consisting of at least two scenarios (1) loadable from text files within the local test distribution folder

Suggestions

References

  1. Note 1 in AC1: A single job can be completely specified by test parameters on job creation time already. So the necessary challenge is that we have test definitions as commonly done by the combination of medium types, test suites, job templates, job groups (and machines and worker config) to form a test schedule consisting of multiple scenarios, i.e. at least two

Related issues

Copied to openQA Project - action #92311: Complete test definition from a single yaml schedule file in local test distribution folderWorkable

History

#1 Updated by okurz 10 months ago

  • Description updated (diff)

#2 Updated by okurz 10 months ago

  • Subject changed from [spike][timeboxed:20h] complete test definition from yaml schedule in local test distribution folder to [spike][timeboxed:20h] complete test definition from a yaml schedule in local test distribution folder

#3 Updated by okurz 10 months ago

  • Subject changed from [spike][timeboxed:20h] complete test definition from a yaml schedule in local test distribution folder to [spike][timeboxed:20h] complete test definition from a yaml schedule file in local test distribution folder

#4 Updated by cdywan 10 months ago

  • Assignee set to cdywan

#5 Updated by cdywan 9 months ago

  • Assignee deleted (cdywan)

I'll unassign myself since I rather focus on flaky CI and issues with production workers atm. If someone beats me to it, go for it.

#6 Updated by okurz 8 months ago

  • Description updated (diff)

Added suggestion instead of just "reference".

#7 Updated by cdywan 6 months ago

Some links for reference:

https://gitlab.suse.de/qsf-y/qa-sle-functional-y/-/blob/3ef279f9d4a536c3004c292442f8e50d48c024d1/JobGroups/test_suites.yaml

As discussed:

  • Let's assume we have nothing defined in the database
  • Think of how to do ISOs post from a file in a git repo
  • Keep in mind the user who just sets up a PR to run tests
  • Propose a format to load a test. We can change the format later, we have schemas for that and ways to communicate new versions supported by API routes.
  • Consider multi-machine tests - but the first version doesn't have to support all use cases

#8 Updated by Xiaojing_liu 4 months ago

  • Assignee set to Xiaojing_liu

I spent some time learning this ticket, so I would like to try.

#9 Updated by openqa_review 4 months ago

  • Due date set to 2021-04-23

Setting due date based on mean cycle time of SUSE QE Tools

#10 Updated by okurz 3 months ago

  • Status changed from Workable to In Progress

#11 Updated by Xiaojing_liu 3 months ago

  • Due date deleted (2021-04-23)
  • Status changed from In Progress to Workable
  • Assignee deleted (Xiaojing_liu)

I committed above pr to describe my thought.

  1. we could schedule more than two jobs at one time from a file (e.g. from a yaml file), and the dependency will be created.
  2. the format of the file needs to be confirmed with others (members of the tool team or users?)

I am not sure if the pr is in the right way.
This ticket is [spike] [timeboxed:20h], and I have spent more than 20 hours, I unassign it to wait for more feedback.

#12 Updated by cdywan 3 months ago

Xiaojing_liu wrote:

I committed above pr to describe my thought.

  1. we could schedule more than two jobs at one time from a file (e.g. from a yaml file), and the dependency will be created.
  2. the format of the file needs to be confirmed with others (members of the tool team or users?)

I am not sure if the pr is in the right way.
This ticket is [spike] [timeboxed:20h], and I have spent more than 20 hours, I unassign it to wait for more feedback.

For the record, it's a spike, so there is no correct way. Thank you for coming up with a very nice proof of concept here!

  • I like how this is very similar to our existing YAML and mirrors the definitions in the database. So even without new documentation I know what it does.
  • Maybe as a user I'd want separate files here, for instance a separate file for the job templates so that I can re-use the machine definitions easily.
  • The code has a comment about cloning from a git repo. Maybe we could even allow a PR or branch name here e.g. #branchname to support a review/CI workflow.

#13 Updated by okurz 3 months ago

  • Assignee set to okurz

as discussed in weekly I will take over to come up with follow-up plans accordingly.

#14 Updated by openqa_review 3 months ago

  • Due date set to 2021-05-08

Setting due date based on mean cycle time of SUSE QE Tools

#15 Updated by okurz 3 months ago

  • Copied to action #92311: Complete test definition from a single yaml schedule file in local test distribution folder added

#16 Updated by okurz 3 months ago

  • Status changed from Workable to Resolved
  • Assignee changed from okurz to Xiaojing_liu

Sorry about my delay. Covered followup in #92311

#17 Updated by okurz 2 months ago

  • Due date deleted (2021-05-08)

Also available in: Atom PDF