action #71758
closedcoordination #58184: [saga][epic][use case] full version control awareness within openQA
[spike][timeboxed:20h] complete test definition from a yaml schedule file in local test distribution folder
0%
Description
Motivation¶
See #58184 . As a test maintainer using openQA I want openQA to be able to load the complete test definition from a yaml schedule in a local test distribution folder so that all the necessary configuration can come from a central place and no configuration needs to be added to the openQA database beforehand
Acceptance criteria¶
- AC1: proof-of-concept exists for a complete test definition consisting of at least two scenarios (1) loadable from text files within the local test distribution folder
Suggestions¶
- See https://github.com/os-autoinst/openQA/pull/2706 for a draft of the idea
References¶
- Note 1 in AC1: A single job can be completely specified by test parameters on job creation time already. So the necessary challenge is that we have test definitions as commonly done by the combination of medium types, test suites, job templates, job groups (and machines and worker config) to form a test schedule consisting of multiple scenarios, i.e. at least two
Updated by okurz about 4 years ago
- Subject changed from [spike][timeboxed:20h] complete test definition from yaml schedule in local test distribution folder to [spike][timeboxed:20h] complete test definition from a yaml schedule in local test distribution folder
Updated by okurz about 4 years ago
- Subject changed from [spike][timeboxed:20h] complete test definition from a yaml schedule in local test distribution folder to [spike][timeboxed:20h] complete test definition from a yaml schedule file in local test distribution folder
Updated by livdywan almost 4 years ago
- Assignee deleted (
livdywan)
I'll unassign myself since I rather focus on flaky CI and issues with production workers atm. If someone beats me to it, go for it.
Updated by okurz almost 4 years ago
- Description updated (diff)
Added suggestion instead of just "reference".
Updated by livdywan over 3 years ago
Some links for reference:
- https://progress.opensuse.org/issues/77071#note-5
- https://gitlab.suse.de/qa-maintenance/qam-openqa-yml/-/merge_requests/53
As discussed:
- Let's assume we have nothing defined in the database
- Think of how to do ISOs post from a file in a git repo
- Keep in mind the user who just sets up a PR to run tests
- Propose a format to load a test. We can change the format later, we have schemas for that and ways to communicate new versions supported by API routes.
- Consider multi-machine tests - but the first version doesn't have to support all use cases
Updated by Xiaojing_liu over 3 years ago
- Assignee set to Xiaojing_liu
I spent some time learning this ticket, so I would like to try.
Updated by openqa_review over 3 years ago
- Due date set to 2021-04-23
Setting due date based on mean cycle time of SUSE QE Tools
Updated by okurz over 3 years ago
- Status changed from Workable to In Progress
Updated by Xiaojing_liu over 3 years ago
- Due date deleted (
2021-04-23) - Status changed from In Progress to Workable
- Assignee deleted (
Xiaojing_liu)
I committed above pr to describe my thought.
- we could schedule more than two jobs at one time from a file (e.g. from a yaml file), and the dependency will be created.
- the format of the file needs to be confirmed with others (members of the tool team or users?)
I am not sure if the pr is in the right way.
This ticket is [spike] [timeboxed:20h], and I have spent more than 20 hours, I unassign it to wait for more feedback.
Updated by livdywan over 3 years ago
Xiaojing_liu wrote:
I committed above pr to describe my thought.
- we could schedule more than two jobs at one time from a file (e.g. from a yaml file), and the dependency will be created.
- the format of the file needs to be confirmed with others (members of the tool team or users?)
I am not sure if the pr is in the right way.
This ticket is [spike] [timeboxed:20h], and I have spent more than 20 hours, I unassign it to wait for more feedback.
For the record, it's a spike, so there is no correct way. Thank you for coming up with a very nice proof of concept here!
- I like how this is very similar to our existing YAML and mirrors the definitions in the database. So even without new documentation I know what it does.
- Maybe as a user I'd want separate files here, for instance a separate file for the job templates so that I can re-use the machine definitions easily.
- The code has a comment about cloning from a git repo. Maybe we could even allow a PR or branch name here e.g.
#branchname
to support a review/CI workflow.
Updated by okurz over 3 years ago
- Assignee set to okurz
as discussed in weekly I will take over to come up with follow-up plans accordingly.
Updated by openqa_review over 3 years ago
- Due date set to 2021-05-08
Setting due date based on mean cycle time of SUSE QE Tools
Updated by okurz over 3 years ago
- Copied to action #92311: Complete test definition from a single yaml schedule file in local test distribution folder added
Updated by okurz over 3 years ago
- Status changed from Workable to Resolved
- Assignee changed from okurz to Xiaojing_liu
Sorry about my delay. Covered followup in #92311
Updated by okurz over 1 year ago
- Copied to action #124502: [spike][timeboxed:20h] complete test definition from yaml schedule in git checked out test distribution added