Project

General

Profile

coordination #66727

openQA Tests - coordination #15132: [saga][epic] Better structure of test plans in main.pm

coordination #44360: [epic] Parameterize test suites within job groups

[epic] Define structure to define test suites not in openQA database

Added by riafarov about 1 year ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-05-13
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Motivation

As an openQA operators, we would like to have audit of all the changes applied to the test suites and use md for description.
openQA test suites became too big and without structure search sometimes produces too many results, as well as only plain text can be used for the test suites description.

Moving test suites definitions to yaml and storing it in git solves both problems, whereas there are limitations.
https://gitlab.suse.de/qsf-y/qa-sle-functional-y/-/merge_requests/193/diffs here is an example how test suites could be defined and then using aliases in all places where we want to enable it.
As of now, we need custom script as we cannot define any additional node not allowed in schema. Oliver has proposed allowing nodes starting with a dot, which can be put anywhere and ignored during parsing on openQA side.
Also, storing everything in job group makes yaml quite big, which either requires more efforts for each team to build yaml from multiple files, or if we could make openQA support another structure, then none of these hacks will be needed on the storing side.

In our team we though of using test suites in openQA database to define environment specific things, like HDDSIZEGB, HDDMODEL, etc. and move testing related ones to different place. As of now we already use yaml schedule files for that.


Subtasks

action #66781: hidden keys in yaml job templatesResolvedcdywan

History

#1 Updated by riafarov about 1 year ago

  • Subject changed from define structure to define test suites not in openQA database to Define structure to define test suites not in openQA database

#2 Updated by tinita about 1 year ago

As of now, we need custom script as we cannot define any additional node not allowed in schema. Oliver has proposed allowing nodes starting with a dot, which can be put anywhere and ignored during parsing on openQA side.

It should be easy to make that possible (ignore things starting with dot)

#3 Updated by okurz about 1 year ago

  • Due date set to 2020-05-13
  • Start date changed from 2020-05-12 to 2020-05-13

due to changes in a related task: #66781

#4 Updated by okurz about 1 year ago

  • Subject changed from Define structure to define test suites not in openQA database to [epic] Define structure to define test suites not in openQA database
  • Category set to Feature requests
  • Status changed from New to Blocked
  • Assignee set to okurz
  • Target version set to Current Sprint

ok, I created a specific user story in #66781 as a subtask of this ticket making this ticket an epic

#5 Updated by okurz about 1 year ago

riafarov with https://github.com/os-autoinst/openQA/pull/3092 we have support for hidden keys on the top level as visible in https://github.com/os-autoinst/openQA/blob/master/public/schema/JobTemplates-01.yaml#L10 . #66781#note-6 shows an example of what is possible now. As usual you can expect to be able to use that already tomorrow on o3 or next week Wednesday on osd. About the other points: To have more actionable items I prefer to have acceptance criteria but I do not want to change the ticket when it is not covering your intention anymore so I plan to leave it open, we follow up with plans trying to cover your intention and then we can check back on your requirements. My plan is to have #58184 first so that we can load schedule files dynamically from test distri first. Then we can cross-reference different schedule files within that repo.

#6 Updated by szarate 10 months ago

  • Tracker changed from action to coordination
  • Status changed from Blocked to New

#8 Updated by okurz 9 months ago

  • Target version changed from Current Sprint to Ready

#9 Updated by okurz 9 months ago

  • Status changed from New to Blocked

still blocked by #58184

#10 Updated by okurz 3 months ago

  • Status changed from Blocked to Resolved

situation has changed regarding reporter so considering the original request done referring to #66727#note-5 while still keeping #58184

Also available in: Atom PDF