Project

General

Profile

action #41852

[functional][y][sle] add more staging tests

Added by okurz over 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Enhancement to existing tests
Target version:
SUSE QA - Milestone 20
Start date:
2018-10-01
Due date:
2018-11-20
% Done:

0%

Estimated time:
5.00 h
Difficulty:

Description

Motivation

from #opensuse-factory

[01/10/2018 10:19:02] <maxlin> okurz: is there a ticket for the 'addraid' shortcut change already? looks we have the same failure on SLE15 so I assume we had a one :) https://openqa.opensuse.org/tests/763397#step/partitioning_raid/222
[01/10/2018 10:20:06] <okurz> I assume it is one of https://v.gd/Lt3q2j
[01/10/2018 10:22:38] <maxlin> hmm,, can't find it
[01/10/2018 10:22:54] <okurz> https://openqa.suse.de/tests/2105139#step/partitioning_raid/222 is the same on 15-SP1. I wonder how that passed staging …
[01/10/2018 10:23:26] <okurz> ok, simple. No RAID in SLE staging :D
[01/10/2018 10:23:40] <maxlin> okurz: you know, 1 min ago lnussel just asked me the same question :)
[01/10/2018 10:23:47] <okurz> so good for pointing that out. I will see if we can add RAID to SLE staging

https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Staging%3AB&build=1378.2&groupid=2 shows 11 scenarios in staging for openSUSE, https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=B.48.3&groupid=125 only 5. To follow on with our plan to reduce the test maintenance effort as stated in the parent ticket #41036 we should add more tests to staging, e.g. a RAID test.

Acceptance criteria


Related issues

Related to openQA Tests - coordination #40475: [functional][y][saga] Establish YaST team splitResolved2018-10-092019-11-19

Blocked by openQA Tests - action #41849: [functional][y] test fails in partitioning_raid - RAID toolbar has changed thus shortcut is differentResolved2018-10-012018-10-23

History

#1 Updated by okurz over 4 years ago

  • Status changed from New to In Progress
  • Assignee set to okurz

Discussed in the RM sync call. SLE12SP4 has more staging scenarios than SLE15. RAID1 is a good candidate to be added to SLE15 and probably also a migration test. I will look for a good example. For openSUSE the staging tests are considered more important for the release process than in SLE as in openSUSE the automatic release process based on TTM means that any failing tests within the validation tests which come after acceptance of stagings will halt a release and therefore should be prevented sooner. In SLE more stages in the release processes exist.

  • I added RAID1 to the staging test schedule of SLE15 for now

#2 Updated by okurz over 4 years ago

  • Blocked by action #41849: [functional][y] test fails in partitioning_raid - RAID toolbar has changed thus shortcut is different added

#3 Updated by okurz over 4 years ago

  • Status changed from In Progress to Blocked

As discussed with behlert I removed the RAID1 scenario from SLE15 staging again on request until #41849 is fixed.

#4 Updated by okurz over 4 years ago

  • Due date set to 2018-10-23

#5 Updated by riafarov over 4 years ago

  • Due date changed from 2018-10-23 to 2018-11-06
  • Status changed from Blocked to Workable
  • Assignee deleted (okurz)

#6 Updated by riafarov over 4 years ago

  • Description updated (diff)

As per discussion with okurz, we first sync staging tests for the both code flows and extending coverage will be clarified separately, as potentially triggering same tests on different architectures makes more sense as first step.

#7 Updated by okurz over 4 years ago

#8 Updated by riafarov over 4 years ago

  • Estimated time set to 5.00 h

#9 Updated by riafarov over 4 years ago

  • Due date changed from 2018-11-06 to 2018-11-20

No capacity

#10 Updated by oorlov about 4 years ago

  • Status changed from Workable to In Progress

#11 Updated by oorlov about 4 years ago

  • Status changed from In Progress to Feedback

AC1: RAID1 and minimal+base test scenarios were added to 'Staging: Sle15' Job groups;
AC2:

  • installcheck from Sle12 cannot be applied to Sle15;
  • migration_zdup_offline_sle12sp2_staging analog is not added for Sle15 as all the migration tests are failing at the moment; AC3: is not clear for me.

okurz, could you please provide more details upon AC3:

  1. What this information should contain?
  2. Where (in which section of wiki) it should be written?

#12 Updated by oorlov about 4 years ago

  • Status changed from Feedback to Workable
  • Assignee set to okurz

#13 Updated by okurz about 4 years ago

oorlov wrote:

AC2:

  • installcheck from Sle12 cannot be applied to Sle15;
  • migration_zdup_offline_sle12sp2_staging analog is not added for Sle15 as all the migration tests are failing at the moment;

Then I suggest to have the migration tests fixed first before an according test is added to staging.

AC3: is not clear for me.

okurz, could you please provide more details upon AC3:

  1. What this information should contain?
  2. Where (in which section of wiki) it should be written?

Basically the wiki should provide the "lesson learned" from this very ticket: How do new staging tests end up in the staging projects. That should be documented on the wiki page. Where exactly I don't know, search for "staging", I suggest :)

#14 Updated by okurz about 4 years ago

  • Assignee changed from okurz to oorlov

#15 Updated by okurz about 4 years ago

  • Status changed from Workable to Resolved

Discussed with oorlov in person, we are ok with no changes on the wiki.

Also available in: Atom PDF