action #41852: [functional][y][sle] add more staging tests - openQA Tests (public) - openSUSE Project Management Tool

Actions

Copy link

action #41852

closed

[functional][y][sle] add more staging tests

Added by okurz about 6 years ago. Updated about 6 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

oorlov

Category:

Enhancement to existing tests

Target version:

SUSE QA (private) - Milestone 20

Start date:

2018-10-01

Due date:

2018-11-20

% Done:

Estimated time:

5.00 h

Difficulty:

Description

Motivation¶

from #opensuse-factory

[01/10/2018 10:19:02] <maxlin> okurz: is there a ticket for the 'addraid' shortcut change already? looks we have the same failure on SLE15 so I assume we had a one :) https://openqa.opensuse.org/tests/763397#step/partitioning_raid/222
[01/10/2018 10:20:06] <okurz> I assume it is one of https://v.gd/Lt3q2j
[01/10/2018 10:22:38] <maxlin> hmm,, can't find it
[01/10/2018 10:22:54] <okurz> https://openqa.suse.de/tests/2105139#step/partitioning_raid/222 is the same on 15-SP1. I wonder how that passed staging …
[01/10/2018 10:23:26] <okurz> ok, simple. No RAID in SLE staging :D
[01/10/2018 10:23:40] <maxlin> okurz: you know, 1 min ago lnussel just asked me the same question :)
[01/10/2018 10:23:47] <okurz> so good for pointing that out. I will see if we can add RAID to SLE staging

https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Staging%3AB&build=1378.2&groupid=2 shows 11 scenarios in staging for openSUSE, https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=B.48.3&groupid=125 only 5. To follow on with our plan to reduce the test maintenance effort as stated in the parent ticket #41036 we should add more tests to staging, e.g. a RAID test.

Acceptance criteria¶

AC1: At least any RAID test is covered in staging
AC2: Same set of test suites is used for SLE 12 and SLE 15
AC3: The review wiki page https://wiki.microfocus.net/index.php/RD-OPS_QA/openQA_review mentions how staging tests are extended or changed, e.g. as in the case above

Related issues 2 (0 open — 2 closed)

Actions

Copy link

Updated by okurz about 6 years ago

Status changed from New to In Progress
Assignee set to okurz

Discussed in the RM sync call. SLE12SP4 has more staging scenarios than SLE15. RAID1 is a good candidate to be added to SLE15 and probably also a migration test. I will look for a good example. For openSUSE the staging tests are considered more important for the release process than in SLE as in openSUSE the automatic release process based on TTM means that any failing tests within the validation tests which come after acceptance of stagings will halt a release and therefore should be prevented sooner. In SLE more stages in the release processes exist.

I added RAID1 to the staging test schedule of SLE15 for now

Actions

Copy link

Updated by okurz about 6 years ago

Blocked by action #41849: [functional][y] test fails in partitioning_raid - RAID toolbar has changed thus shortcut is different added

Actions

Copy link

Updated by okurz about 6 years ago

Status changed from In Progress to Blocked

As discussed with behlert I removed the RAID1 scenario from SLE15 staging again on request until #41849 is fixed.

Actions

Copy link

Updated by okurz about 6 years ago

Due date set to 2018-10-23

Actions

Copy link

Updated by riafarov about 6 years ago

Due date changed from 2018-10-23 to 2018-11-06
Status changed from Blocked to Workable
Assignee deleted (~~okurz~~)

Actions

Copy link

Updated by riafarov about 6 years ago

Description updated (diff)

As per discussion with okurz, we first sync staging tests for the both code flows and extending coverage will be clarified separately, as potentially triggering same tests on different architectures makes more sense as first step.

Actions

Copy link

Updated by okurz about 6 years ago

Related to coordination #40475: [functional][y][saga] Establish YaST team split added

Actions

Copy link

Updated by riafarov about 6 years ago

Estimated time set to 5.00 h

Actions

Copy link

Updated by riafarov about 6 years ago

Due date changed from 2018-11-06 to 2018-11-20

No capacity

Actions

Copy link

#10

Updated by oorlov about 6 years ago

Status changed from Workable to In Progress

Actions

Copy link

#11

Updated by oorlov about 6 years ago

Status changed from In Progress to Feedback

AC1: RAID1 and minimal+base test scenarios were added to 'Staging: Sle15' Job groups;
AC2:

installcheck from Sle12 cannot be applied to Sle15;
migration_zdup_offline_sle12sp2_staging analog is not added for Sle15 as all the migration tests are failing at the moment; AC3: is not clear for me.

@okurz, could you please provide more details upon AC3:

What this information should contain?
Where (in which section of wiki) it should be written?

Actions

Copy link

#12

Updated by oorlov about 6 years ago

Status changed from Feedback to Workable
Assignee set to okurz

Actions

Copy link

#13

Updated by okurz about 6 years ago

oorlov wrote:

AC2:

installcheck from Sle12 cannot be applied to Sle15;

migration_zdup_offline_sle12sp2_staging analog is not added for Sle15 as all the migration tests are failing at the moment;

Then I suggest to have the migration tests fixed first before an according test is added to staging.

AC3: is not clear for me.

@okurz, could you please provide more details upon AC3:

What this information should contain?

Where (in which section of wiki) it should be written?

Basically the wiki should provide the "lesson learned" from this very ticket: How do new staging tests end up in the staging projects. That should be documented on the wiki page. Where exactly I don't know, search for "staging", I suggest :)

Actions

Copy link

#14