Project

General

Profile

action #54806

[epic] Better user control over job group overview

Added by leli 12 months ago. Updated 3 months ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2019-07-30
Due date:
% Done:

0%

Estimated time:
Difficulty:
hard
Duration:

Description

Motivation

For example SLE migration has different migration paths that need to be covered. A logical grouping of according test results in the display of /tests/overview can help to understand aggregated results.

Acceptance criteria

  • AC1: Logical grouping of tests within the /tests/overview page can be controlled by test maintainers

Further details

It could looks like below:

-----------SLE11SP4-----------
test 1
test 2
test 3
-----------SLE11SP4-----------
-----------SLE12SP3-----------
test 4
test 5
test 6
-----------SLE12SP3-----------
-----------SLE12SP4-----------
test 7
test 8
test 9
-----------SLE12SP4-----------

Currently in SLES 12/Migration: SLES job group, we add three types media types(Migration-from-SLE11-SPx, Migration-from-SLE12-SP5-to-SLE15-SPx, Migration-from-SLE12-SPx) in the job group for different sub job groups.


Related issues

Related to openQA Project - action #55424: Error about "Use of uninitialized value in join or string at template branding/openSUSE/external_reporting.html.ep line 45" on every view of a screenshot of a job that has non-standard flavor, e.g. "Regression-on-migrated-system-from-SLE12-SPx"Resolved2019-08-13

History

#1 Updated by okurz 12 months ago

there are already the flavors as defined by the media types to distinguish as visible in https://openqa.suse.de/tests/overview?distri=sle&version=12-SP5&build=0248&groupid=245 . What would you need the sub groups for? Or what about a new feature about folding of flavors to fold away blocks by clicking on the table header?

#2 Updated by leli 12 months ago

I mean realize the feature with sub groups in same job group with the same flavor, with different flavors in a job group cause we need maintenance different flavors in media types and can't set load sequence with 'start_after' if in different flavor.
For SLE15SP2, there is no migration path from SLE-N-1 to SLE-N such as SLE12SP5 to SLE15SP1, so all test suites can be in the same flavor. With the feature of define sub groups in same job group just a UI change for openQA user, while these test suites should still be in the same job group.

#3 Updated by okurz 12 months ago

So are you looking for an alternative to the flavors then? Sorry that I still don't fully get it.

#4 Updated by leli 12 months ago

No, I hope we can use this feature to set sub group without using several flavors in a job group.

#5 Updated by okurz 12 months ago

isn't this what I tried to say: You want an alternative or a replacement for multiple flavors instead?

#6 Updated by coolo 12 months ago

This is only for display, right? So basically an annotated sort order within a job group?

#7 Updated by okurz 12 months ago

I think so

leli I guess it would be more clear if you could follow https://progress.opensuse.org/projects/openqav3/wiki#Feature-requests in the ticket description. That would be helpful. Thanks.

#8 Updated by leli 12 months ago

coolo wrote:

This is only for display, right? So basically an annotated sort order within a job group?

Yes.

#9 Updated by leli 12 months ago

okurz wrote:

I think so

leli I guess it would be more clear if you could follow https://progress.opensuse.org/projects/openqav3/wiki#Feature-requests in the ticket description. That would be helpful. Thanks.

Ok, I will follow the doc to update the ticket to make it more clear. Thanks.

#10 Updated by Xiaojing_liu 12 months ago

I discussed with Leli.
I understand his meaning is that: Now in 'Test result overview', the test suites are distinguished by flavor. such as

Flavor:Installer-DVD
testsuite_1 result
testsuite_2 result

He wants the test suites are distinguished by 'sub group' (not existed in OpenQA for now).
Such as:
Migration-from-SLE11-SPx
testsuite_1 result
testsuite_2 result

Migration-from-SLE12-SP5-to-SLE15-SPx
testsuite_1 result
testsuite_2 result
....

In my opinion, maybe the 'sub group' is just a name, a section of one job group. Users could add or modify 'sub group' when they create or modify job group. And a test suite has an attribute that which 'sub group' belong to.

For now, migration team uses flavors as the solution. The result is https://openqa.suse.de/tests/overview?distri=sle&version=12-SP5&build=0251&groupid=245

Leli thinks that it will take much time to create and maintain Flavors. If we provide sub group, they don't need to create many flavors.

#11 Updated by leli 11 months ago

Thanks Xiaojing help to clarify, in general I agree her description. I'd like to say the pain points of multi flavor in a job group is that we need to create and maintain much flavors and hard for us to control openQA job sequence if two test suites are not in same flavor(such as continuous migration test).
Besides, I file this ticket just hope it can make it easy for reviewers to review different sections of migration job group and do not use multi flavors any more. I think if it is so much effort to realize this feature then we can continue to use multi flavors, feel free to evaluate it. Thanks.

lemon

#12 Updated by okurz 11 months ago

  • Related to action #55424: Error about "Use of uninitialized value in join or string at template branding/openSUSE/external_reporting.html.ep line 45" on every view of a screenshot of a job that has non-standard flavor, e.g. "Regression-on-migrated-system-from-SLE12-SPx" added

#13 Updated by okurz 11 months ago

The new YAML based approach for job templates should allow to specify test settings on scenarios directly. This does not cover any UI feature requests though.

#14 Updated by coolo 9 months ago

  • Subject changed from Self-define sub groups in one job group to EPIC: Better user control over job group overview
  • Target version set to Ready
  • Difficulty set to hard

I'm not yet sure what a good approach is here, but I accept the user story

#15 Updated by okurz 7 months ago

  • Subject changed from EPIC: Better user control over job group overview to [epic] Better user control over job group overview
  • Description updated (diff)

I tried to refine the story but only ended up again with what we have: Defining flavors to reach the logical grouping. The only challenge I see is the bit "hacky" approach in ObsRsync or formerly rsync.pl due to the need to trigger individual "isos post" calls to set the different flavor values. How about just defining sub-flavors? E.g. a flavor selection in the job templates definition "Server-DVD:Migration-from-SLE12-SP5" which is triggered whenever an isos post FLAVOR=Server-DVD is called?

#16 Updated by okurz 3 months ago

  • Priority changed from Normal to Low

Also available in: Atom PDF