action #41852

[functional][y][sle] add more staging tests

Added by okurz over 1 year ago. Updated over 1 year ago.

Status:ResolvedStart date:01/10/2018
Priority:NormalDue date:20/11/2018
Assignee:oorlov% Done:

0%

Category:Enhancement to existing testsEstimated time:5.00 hours
Target version:SUSE QA tests - Milestone 20
Difficulty:
Duration: 37

Description

Motivation

from #opensuse-factory

[01/10/2018 10:19:02] <maxlin> okurz: is there a ticket for the 'addraid' shortcut change already? looks we have the same failure on SLE15 so I assume we had a one :) https://openqa.opensuse.org/tests/763397#step/partitioning_raid/222
[01/10/2018 10:20:06] <okurz> I assume it is one of https://v.gd/Lt3q2j
[01/10/2018 10:22:38] <maxlin> hmm,, can't find it
[01/10/2018 10:22:54] <okurz> https://openqa.suse.de/tests/2105139#step/partitioning_raid/222 is the same on 15-SP1. I wonder how that passed staging …
[01/10/2018 10:23:26] <okurz> ok, simple. No RAID in SLE staging :D
[01/10/2018 10:23:40] <maxlin> okurz: you know, 1 min ago lnussel just asked me the same question :)
[01/10/2018 10:23:47] <okurz> so good for pointing that out. I will see if we can add RAID to SLE staging

https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Staging%3AB&build=1378.2&groupid=2 shows 11 scenarios in staging for openSUSE, https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=B.48.3&groupid=125 only 5. To follow on with our plan to reduce the test maintenance effort as stated in the parent ticket #41036 we should add more tests to staging, e.g. a RAID test.

Acceptance criteria


Related issues

Related to openQA Tests - action #40475: [functional][y][saga] Establish YaST team split Blocked 31/08/2018 21/04/2020
Blocked by openQA Tests - action #41849: [functional][y] test fails in partitioning_raid - RAID to... Resolved 01/10/2018 23/10/2018

History

#1 Updated by okurz over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to okurz

Discussed in the RM sync call. SLE12SP4 has more staging scenarios than SLE15. RAID1 is a good candidate to be added to SLE15 and probably also a migration test. I will look for a good example. For openSUSE the staging tests are considered more important for the release process than in SLE as in openSUSE the automatic release process based on TTM means that any failing tests within the validation tests which come after acceptance of stagings will halt a release and therefore should be prevented sooner. In SLE more stages in the release processes exist.

  • I added RAID1 to the staging test schedule of SLE15 for now

#2 Updated by okurz over 1 year ago

  • Blocked by action #41849: [functional][y] test fails in partitioning_raid - RAID toolbar has changed thus shortcut is different added

#3 Updated by okurz over 1 year ago

  • Status changed from In Progress to Blocked

As discussed with behlert I removed the RAID1 scenario from SLE15 staging again on request until #41849 is fixed.

#4 Updated by okurz over 1 year ago

  • Due date set to 23/10/2018

#5 Updated by riafarov over 1 year ago

  • Due date changed from 23/10/2018 to 06/11/2018
  • Status changed from Blocked to Workable
  • Assignee deleted (okurz)

#6 Updated by riafarov over 1 year ago

  • Description updated (diff)

As per discussion with okurz, we first sync staging tests for the both code flows and extending coverage will be clarified separately, as potentially triggering same tests on different architectures makes more sense as first step.

#7 Updated by okurz over 1 year ago

  • Related to action #40475: [functional][y][saga] Establish YaST team split added

#8 Updated by riafarov over 1 year ago

  • Estimated time set to 5.00

#9 Updated by riafarov over 1 year ago

  • Due date changed from 06/11/2018 to 20/11/2018

No capacity

#10 Updated by oorlov over 1 year ago

  • Status changed from Workable to In Progress

#11 Updated by oorlov over 1 year ago

  • Status changed from In Progress to Feedback

AC1: RAID1 and minimal+base test scenarios were added to 'Staging: Sle15' Job groups;
AC2:
* installcheck from Sle12 cannot be applied to Sle15;
* migration_zdup_offline_sle12sp2_staging analog is not added for Sle15 as all the migration tests are failing at the moment;
AC3: is not clear for me.

@okurz, could you please provide more details upon AC3:
1. What this information should contain?
2. Where (in which section of wiki) it should be written?

#12 Updated by oorlov over 1 year ago

  • Status changed from Feedback to Workable
  • Assignee set to okurz

#13 Updated by okurz over 1 year ago

oorlov wrote:

AC2:

* installcheck from Sle12 cannot be applied to Sle15;

* migration_zdup_offline_sle12sp2_staging analog is not added for Sle15 as all the migration tests are failing at the moment;

Then I suggest to have the migration tests fixed first before an according test is added to staging.

AC3: is not clear for me.


@okurz, could you please provide more details upon AC3:

1. What this information should contain?

2. Where (in which section of wiki) it should be written?

Basically the wiki should provide the "lesson learned" from this very ticket: How do new staging tests end up in the staging projects. That should be documented on the wiki page. Where exactly I don't know, search for "staging", I suggest :)

#14 Updated by okurz over 1 year ago

  • Assignee changed from okurz to oorlov

#15 Updated by okurz over 1 year ago

  • Status changed from Workable to Resolved

Discussed with oorlov in person, we are ok with no changes on the wiki.

Also available in: Atom PDF