Project

General

Profile

Actions

coordination #66727

closed

openQA Tests - coordination #15132: [saga][epic] Better structure of test plans in main.pm

coordination #44360: [epic] Parameterize test suites within job groups

[epic] Define structure to define test suites not in openQA database

Added by riafarov almost 4 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-05-13
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)

Description

Motivation

As an openQA operators, we would like to have audit of all the changes applied to the test suites and use md for description.
openQA test suites became too big and without structure search sometimes produces too many results, as well as only plain text can be used for the test suites description.

Moving test suites definitions to yaml and storing it in git solves both problems, whereas there are limitations.
https://gitlab.suse.de/qsf-y/qa-sle-functional-y/-/merge_requests/193/diffs here is an example how test suites could be defined and then using aliases in all places where we want to enable it.
As of now, we need custom script as we cannot define any additional node not allowed in schema. Oliver has proposed allowing nodes starting with a dot, which can be put anywhere and ignored during parsing on openQA side.
Also, storing everything in job group makes yaml quite big, which either requires more efforts for each team to build yaml from multiple files, or if we could make openQA support another structure, then none of these hacks will be needed on the storing side.

In our team we though of using test suites in openQA database to define environment specific things, like HDDSIZEGB, HDDMODEL, etc. and move testing related ones to different place. As of now we already use yaml schedule files for that.


Subtasks 1 (0 open1 closed)

action #66781: hidden keys in yaml job templatesResolvedlivdywan2020-05-13

Actions
Actions #1

Updated by riafarov almost 4 years ago

  • Subject changed from define structure to define test suites not in openQA database to Define structure to define test suites not in openQA database
Actions #2

Updated by tinita almost 4 years ago

As of now, we need custom script as we cannot define any additional node not allowed in schema. Oliver has proposed allowing nodes starting with a dot, which can be put anywhere and ignored during parsing on openQA side.

It should be easy to make that possible (ignore things starting with dot)

Actions #3

Updated by okurz almost 4 years ago

  • Due date set to 2020-05-13
  • Start date changed from 2020-05-12 to 2020-05-13

due to changes in a related task: #66781

Actions #4

Updated by okurz almost 4 years ago

  • Subject changed from Define structure to define test suites not in openQA database to [epic] Define structure to define test suites not in openQA database
  • Category set to Feature requests
  • Status changed from New to Blocked
  • Assignee set to okurz
  • Target version set to Current Sprint

ok, I created a specific user story in #66781 as a subtask of this ticket making this ticket an epic

Actions #5

Updated by okurz almost 4 years ago

@riafarov with https://github.com/os-autoinst/openQA/pull/3092 we have support for hidden keys on the top level as visible in https://github.com/os-autoinst/openQA/blob/master/public/schema/JobTemplates-01.yaml#L10 . #66781#note-6 shows an example of what is possible now. As usual you can expect to be able to use that already tomorrow on o3 or next week Wednesday on osd. About the other points: To have more actionable items I prefer to have acceptance criteria but I do not want to change the ticket when it is not covering your intention anymore so I plan to leave it open, we follow up with plans trying to cover your intention and then we can check back on your requirements. My plan is to have #58184 first so that we can load schedule files dynamically from test distri first. Then we can cross-reference different schedule files within that repo.

Actions #6

Updated by szarate over 3 years ago

  • Tracker changed from action to coordination
  • Status changed from Blocked to New
Actions #8

Updated by okurz over 3 years ago

  • Target version changed from Current Sprint to Ready
Actions #9

Updated by okurz over 3 years ago

  • Status changed from New to Blocked

still blocked by #58184

Actions #10

Updated by okurz almost 3 years ago

  • Status changed from Blocked to Resolved

situation has changed regarding reporter so considering the original request done referring to #66727#note-5 while still keeping #58184

Actions

Also available in: Atom PDF