coordination #66727
closedopenQA Tests - coordination #15132: [saga][epic] Better structure of test plans in main.pm
coordination #44360: [epic] Parameterize test suites within job groups
[epic] Define structure to define test suites not in openQA database
Description
Motivation¶
As an openQA operators, we would like to have audit of all the changes applied to the test suites and use md for description.
openQA test suites became too big and without structure search sometimes produces too many results, as well as only plain text can be used for the test suites description.
Moving test suites definitions to yaml and storing it in git solves both problems, whereas there are limitations.
https://gitlab.suse.de/qsf-y/qa-sle-functional-y/-/merge_requests/193/diffs here is an example how test suites could be defined and then using aliases in all places where we want to enable it.
As of now, we need custom script as we cannot define any additional node not allowed in schema. Oliver has proposed allowing nodes starting with a dot, which can be put anywhere and ignored during parsing on openQA side.
Also, storing everything in job group makes yaml quite big, which either requires more efforts for each team to build yaml from multiple files, or if we could make openQA support another structure, then none of these hacks will be needed on the storing side.
In our team we though of using test suites in openQA database to define environment specific things, like HDDSIZEGB, HDDMODEL, etc. and move testing related ones to different place. As of now we already use yaml schedule files for that.
Updated by riafarov over 4 years ago
- Subject changed from define structure to define test suites not in openQA database to Define structure to define test suites not in openQA database
Updated by tinita over 4 years ago
As of now, we need custom script as we cannot define any additional node not allowed in schema. Oliver has proposed allowing nodes starting with a dot, which can be put anywhere and ignored during parsing on openQA side.
It should be easy to make that possible (ignore things starting with dot)
Updated by okurz over 4 years ago
- Due date set to 2020-05-13
- Start date changed from 2020-05-12 to 2020-05-13
due to changes in a related task: #66781
Updated by okurz over 4 years ago
- Subject changed from Define structure to define test suites not in openQA database to [epic] Define structure to define test suites not in openQA database
- Category set to Feature requests
- Status changed from New to Blocked
- Assignee set to okurz
- Target version set to Current Sprint
ok, I created a specific user story in #66781 as a subtask of this ticket making this ticket an epic
Updated by okurz over 4 years ago
@riafarov with https://github.com/os-autoinst/openQA/pull/3092 we have support for hidden keys on the top level as visible in https://github.com/os-autoinst/openQA/blob/master/public/schema/JobTemplates-01.yaml#L10 . #66781#note-6 shows an example of what is possible now. As usual you can expect to be able to use that already tomorrow on o3 or next week Wednesday on osd. About the other points: To have more actionable items I prefer to have acceptance criteria but I do not want to change the ticket when it is not covering your intention anymore so I plan to leave it open, we follow up with plans trying to cover your intention and then we can check back on your requirements. My plan is to have #58184 first so that we can load schedule files dynamically from test distri first. Then we can cross-reference different schedule files within that repo.
Updated by szarate about 4 years ago
- Tracker changed from action to coordination
- Status changed from Blocked to New
Updated by szarate about 4 years ago
See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html
Updated by okurz about 4 years ago
- Target version changed from Current Sprint to Ready
Updated by okurz over 3 years ago
- Status changed from Blocked to Resolved
situation has changed regarding reporter so considering the original request done referring to #66727#note-5 while still keeping #58184