action #46673

openQA Tests - action #15132: [saga][epic] Better structure of test plans in main.pm

action #44360: [epic] Parameterize test suites within job groups

Extend tables for job scheduling by another table for parameters

Added by mkittler about 1 year ago. Updated 9 months ago.

Status:ResolvedStart date:25/01/2019
Priority:NormalDue date:18/06/2019
Assignee:cdywan% Done:

0%

Category:Feature requests
Target version:Current Sprint
Difficulty:
Duration: 103

Description

  • Add parameter table and its relationship with job_templates table.
  • Extend format defined in https://progress.opensuse.org/issues/46667 to support parameter table.
  • Add some kind of UI to support that feature (minimum would be a text area with the YAML representation).

History

#1 Updated by mkittler 11 months ago

I suppose adding the parameter table involves the following steps:

  1. Create the tables TestParameter and TestParameterSettings similar to the existing tables TestSuites and TestSuiteSettings. I'd say the parameters should also have the description field like the test suites have.
  2. Extend the JobTemplates table to store optionally a test_parameter_id which is a foreign key to the table TestParameter created in 1.
  3. Extend the YAML import/export to support the column added in 2. Also extend the example shown in the UI accordingly.
  4. Create a new "admin table page" in the web UI to allow editing tables created in 1. similar to the existing page for editing test suites. This should be easily done without duplicating much code because existing admin tables also share common code already.
  5. Make the algorithm for scheduling products use of the parameters. I assume it should do what it already does so far and if a job template has parameters it simply adds those settings, too.
  6. The test suite descriptions we have so far are shown as 'scenario description' in the test result pages in in the test overview pages. For this, the TEST setting of the job simply has to match the test suite name. It might be good if parameter descriptions would be shown in a similar way. However, I'm not sure how the mapping should be done.
  7. Add support for the new table in dump templates and load templates scripts.

@coolo Does this make sense for you?

@cdywan I could start with 1. and 2. and you could continue with 3. while I'll continue with 4. I would wait for feedback from @coolo before starting.

#2 Updated by coolo 11 months ago

Sorry for the delay, but this is the wrong direction (so thanks for waiting :)

The parameters shouldn't be top level admin entries, but belong to the scheduled entry in the job group. If the job group is done, the parameter is gone as well. I.e. the we schedule TEST install_desktop+test_application in a job group.

And you once schedule it with DESKTOP=kde,APPLICATION=firefox and once with DESKTOP=gnome,APPLICATION=evince (the example is of course completely nonsense, but I hope it explains my point).

As such the parameters are purely edited within yaml of the job group.

#3 Updated by mkittler 11 months ago

  • Assignee set to mkittler
  • Target version set to Current Sprint

#4 Updated by mkittler 11 months ago

  • Status changed from New to In Progress

#5 Updated by mkittler 11 months ago

So this does not add "another dimension" after all. It just extends an existing dimension with parameters. That means we could actually make this accessible in the current job template editor as well by inserting another column next to the priority.

#6 Updated by mkittler 11 months ago

PR for dump_templates/load_templates support: https://github.com/os-autoinst/openQA/pull/2074

#7 Updated by mkittler 11 months ago

  • Status changed from In Progress to Blocked

@coolo wants to delay merging the PR for dump_templates/load_templates until we actively use the YAML.

#8 Updated by cdywan 10 months ago

The next steps as of today's weekly meeting:

  • Save the YAML in the database
  • Disable legacy UI once YAML is present for a template
  • There is no support for parameters w/o YAML
  • Lock API calls that affect a YAMLized template
  • No git backend for now
  • Remove the preview, validation is mandatory when saving
  • Post an announcement to the mailing list

#9 Updated by cdywan 10 months ago

  • Due date set to 18/06/2019
  • Status changed from Blocked to In Progress
  • Assignee changed from mkittler to cdywan

#10 Updated by coolo 10 months ago

To avoid unintended overwriting of each other's changes, I suggest to put a revision (or timestamp) next to the yaml to be checked if it stayed the same between load and save.

#11 Updated by mkittler 10 months ago

  • Status changed from In Progress to Blocked
  • Assignee changed from cdywan to mkittler

@cdwywan In addition to your list there are also the following TODOs: https://github.com/os-autoinst/openQA/pull/2074#issuecomment-496431091 (point "There is no support for parameters w/o YAML" is covered by both lists)

Note that the TODOs I commented on the PR are actually the ones this issue is about. The points mentioned by @cdywan are different tasks and some of them block this issue.

#12 Updated by cdywan 10 months ago

  • Status changed from Blocked to In Progress
  • Assignee changed from mkittler to cdywan

One additional point as per today's meeting:

  • Remove the name from the YAML ie. lose the group name

We don't want to support editing the name or creating new groups right now, or any other settings.

#13 Updated by okurz 10 months ago

  • Category set to Feature requests

#14 Updated by cdywan 9 months ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF