Project

General

Profile

coordination #39302

[qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets)

Added by lnussel over 2 years ago. Updated 3 months ago.

Status:
New
Priority:
Normal
Category:
Bugs in existing tests
Target version:
SUSE QA tests - Milestone 30
Start date:
2018-08-08
Due date:
% Done:

70%

Estimated time:
(Total: 210.00 h)
Difficulty:

Description

Suggestions

  • Ensure TW is a superset of Leap, i.e. add missing upgrade tests only on Leap to TW
  • If necessary according to lnussel add 42.1 upgrade tests by creating the assets somehow, manually or automatically

Further details

Original observation

This problem affects O3 only (openqa.opensuse.org)
When upgrading from any Leap version to TW.
So, the assets are disk image with previous Leap installation.

In https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1

[2018-08-08T00:55:39.0872 CEST] [info] result: setup failure: Can't download opensuse-42.1-x86_64-Updates-20170213-1-cryptlvm@uefi.qcow2
[2018-08-08T00:48:01.0851 CEST] [info] result: setup failure: Can't download opensuse-42.2-x86_64-Updates-20180206-1-cryptlvm@uefi.qcow2


Subtasks

action #55874: [qe-core][functional] Create matrix about possible upgrade scenarios in combination with machine types distribution versionWorkable

action #55877: [qe-core][functional] Describe the process on how we get a Leap scenario to a TumbleweedWorkable

action #51857: [qe-core][functional] create scenario upgrade_tumbleweed which uses released ISO instead of tested ISO and publishes the qcow2 for extra testsWorkable

action #48725: [functional][u] Add upgrade tests from Leap 15.1 to openSUSE TumbleweedResolveddheidler

action #48731: [functional][u] Add a single upgrade test from Leap 15.1 to openSUSE TumbleweedResolvedSLindoMansilla

action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G)ResolvedSLindoMansilla

action #40052: [functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2ResolvedSLindoMansilla

action #39305: [functional][u] 42.3 upgrade tests use GM image instead of updated oneResolvedSLindoMansilla

action #56531: [functional][u][migration] please create 15.[01] zdup testsResolveddheidler

action #51860: [functional][u] Add update from Leap 15.1 testsRejected


Related issues

Related to openQA Tests - action #39713: [functional][y] add openqa upgrade tests from 15.0Resolved2018-08-132018-08-28

Related to openQA Tests - action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assetsResolved2018-11-29

Related to openQA Tests - action #46868: [aarch64][functional][y][migration] enable migration tests on aarch64Resolved2019-01-302019-03-12

Related to openQA Tests - action #40349: [sle][functional][u][migration][sle15sp1] Jobs incomplete due to missing assetsResolved2018-08-28

History

#1 Updated by okurz over 2 years ago

  • Project changed from openQA Project to openQA Tests
  • Category set to Bugs in existing tests
  • Status changed from New to In Progress
  • Assignee set to okurz

#2 Updated by okurz over 2 years ago

  • Assignee changed from okurz to lnussel

The assets are gone. Recreating them using openQA clone_job should be feasible if necessary. However, the problem I see is that you as RM for openSUSE Leap defined a custom testing scope (uefi upgrade scenarios) which certainly make sense but they are for Leap only, hence the last successful jobs marking the openQA assets as used are 15.0 GM. Since then – 3 months ago – the assets have been unused and therefore deleted as such. I suggest to primarily extend the test scope on openSUSE Tumbleweed first as this will mark the assets as always used and only define a sub-set for openSUSE Leap. Back to lnussel, WDYT?

#3 Updated by lnussel over 2 years ago

  • Assignee changed from lnussel to okurz

feel free to put the same test cases also on TW. It should be possible to recreate the assets by restarting the jobs that originally created them

#4 Updated by lnussel over 2 years ago

note there's also the hdd/fixed directory which contains some images. not sure what the rule is to put stuff there

#5 Updated by okurz over 2 years ago

  • Subject changed from missing assets to [functional][u] uefi upgrade tests on TW+Leap (was: missing assets)
  • Description updated (diff)
  • Due date set to 2018-09-11
  • Status changed from In Progress to Workable
  • Assignee deleted (okurz)
  • Priority changed from High to Normal
  • Target version set to Milestone 19

lnussel wrote:

It should be possible to recreate the assets by restarting the jobs that originally created them

True but I doubt we still have these jobs on o3, most likely pruned already.

lnussel wrote:

note there's also the hdd/fixed directory which contains some images. not sure what the rule is to put stuff there

I am fully aware of the fixed assets. IMHO the "rule" is: Just do it or run into problems like the one described in this ticket ;)

So for a start I removed the following test scenarios from the Leap job group as they are incomplete and have the missing assets assigned as well as are for now unsupported Leap versions but also we still have the non-uefi, non-cryptlvm upgrade scenarios still as well as the uefi,cryptlvm scenario doing upgrade from 42.1 to current:

  • DVD-x86_64-update_Leap_42.1_cryptlvm@uefi
  • DVD-x86_64-update_Leap_42.2_cryptlvm@uefi
  • NET-x86_64-update_Leap_42.1_cryptlvm@uefi
  • NET-x86_64-update_Leap_42.2_cryptlvm@uefi

This should remove the urgency.

Next steps:

  • Ensure TW is a superset of Leap, i.e. add missing upgrade tests only on Leap to TW
  • If necessary according to lnussel add 42.1 upgrade tests by creating the assets somehow, manually or automatically

#6 Updated by SLindoMansilla over 2 years ago

  • Description updated (diff)

#7 Updated by SLindoMansilla over 2 years ago

  • Subject changed from [functional][u] uefi upgrade tests on TW+Leap (was: missing assets) to [functional][opensuse][u] uefi upgrade tests on TW+Leap (was: missing assets)
  • Description updated (diff)

#8 Updated by SLindoMansilla over 2 years ago

  • Subject changed from [functional][opensuse][u] uefi upgrade tests on TW+Leap (was: missing assets) to [functional][opensuse][u][medium] uefi upgrade tests on TW+Leap (was: missing assets)
  • Estimated time set to 5.00 h
  • Difficulty set to medium

#9 Updated by okurz over 2 years ago

  • Copied to action #40052: [functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2 added

#10 Updated by SLindoMansilla over 2 years ago

  • Related to action #39713: [functional][y] add openqa upgrade tests from 15.0 added

#11 Updated by SLindoMansilla over 2 years ago

  • Assignee set to SLindoMansilla

#12 Updated by SLindoMansilla over 2 years ago

  • Status changed from Workable to In Progress

#14 Updated by mgriessmeier over 2 years ago

  • Due date changed from 2018-09-11 to 2018-09-25

try to reproduce on o3 directly by e.g. a jobs post or fallback to manual verification

#15 Updated by SLindoMansilla over 2 years ago

Running clone of 42.1 to generate qcow2 for cryptlvm on O3: https://openqa.opensuse.org/tests/752499

#16 Updated by okurz over 2 years ago

  • Due date deleted (2018-09-25)
  • Priority changed from Normal to Low
  • Target version changed from Milestone 19 to Milestone 22

failed because the ISO is not there anymore: https://openqa.opensuse.org/tests/752499/file/autoinst-log.txt shows "Download of /var/lib/openqa/cache/openSUSE-Leap-42.1-DVD-x86_64-Build0265-Media.iso failed with: 404 - Not Found" . You could try to reference the ISO from download.o.o . As this becomes more effort now and with limited value I suggest to drop the work on it and revisit the situation later. We have a good coverage spanning different versions, desktops, flavors, etc., still and no currently incomplete or failing tests linked with this ticket here.

@sergio unassign ok?

#17 Updated by okurz over 2 years ago

  • Status changed from In Progress to Workable
  • Assignee deleted (SLindoMansilla)

SLindoMansilla is fine with unassigning

#18 Updated by okurz about 2 years ago

  • Related to action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assets added

#19 Updated by SLindoMansilla about 2 years ago

  • Subject changed from [functional][opensuse][u][medium] uefi upgrade tests on TW+Leap (was: missing assets) to [functional][opensuse][epic][u][medium] uefi upgrade tests on TW+Leap (was: missing assets)

Since it was copied to the ticket I am working on, and my ticket is called "part 2", I consider this an epic ticket

#20 Updated by okurz about 2 years ago

  • Related to action #46868: [aarch64][functional][y][migration] enable migration tests on aarch64 added

#21 Updated by okurz about 2 years ago

  • Target version changed from Milestone 22 to Milestone 24

#22 Updated by okurz almost 2 years ago

  • Status changed from Workable to Blocked
  • Assignee set to SLindoMansilla

@SLindoMansilla please update after #40052 is resolved

#23 Updated by SLindoMansilla almost 2 years ago

  • Status changed from Blocked to Feedback

Blocker resolved.

I asked lnussel and DimStar in #opensuse-factory @freenode if the test coverage for each upgrade test should have uefi and legacy scenarios. Waiting for their answer. If the answer is yes, I would resolve this epic. If not, I would transform it into a normal task to complete the missing scenarios.

#24 Updated by okurz almost 2 years ago

  • Related to action #48725: [functional][u] Add upgrade tests from Leap 15.1 to openSUSE Tumbleweed added

#25 Updated by okurz almost 2 years ago

"each upgrade test" should not have UEFI and legacy but we want a "sparse test matrix", i.e. cover all variants in a least one combination

#26 Updated by SLindoMansilla almost 2 years ago

  • Status changed from Feedback to Blocked

Blocked by #48725

#27 Updated by mgriessmeier almost 2 years ago

  • Target version changed from Milestone 24 to Milestone 25

#28 Updated by okurz almost 2 years ago

  • Status changed from Blocked to Workable

@SLindoMansilla it seems like something went wrong here. You set the ticket to "Blocked" by #48725 which in turn is blockey by this one :D
Considering that Leap 15.1 is about to be released like … today that is a bit unfortunate in case we we missed some serious issue, let's just cross fingers that we not ;)

#29 Updated by okurz almost 2 years ago

  • Related to action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G) added

#30 Updated by mgriessmeier over 1 year ago

  • Target version changed from Milestone 25 to Milestone 26

#31 Updated by mgriessmeier over 1 year ago

  • Target version changed from Milestone 26 to Milestone 27

we should revisit for leap 15.2
@Sergio wdyt?

#32 Updated by SLindoMansilla over 1 year ago

  • Assignee deleted (SLindoMansilla)

mgriessmeier wrote:

we should revisit for leap 15.2
@Sergio wdyt?

Sure. This needs more love.
Let's take this to be properly refined today.

#33 Updated by SLindoMansilla over 1 year ago

  • Assignee set to mgriessmeier

To be tracked

#34 Updated by SLindoMansilla over 1 year ago

  • Due date set to 2019-08-24

due to changes in a related task

#35 Updated by SLindoMansilla over 1 year ago

  • Related to deleted (action #48725: [functional][u] Add upgrade tests from Leap 15.1 to openSUSE Tumbleweed)

#36 Updated by SLindoMansilla over 1 year ago

  • Due date set to 2019-08-24

due to changes in a related task

#37 Updated by SLindoMansilla over 1 year ago

  • Related to deleted (action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G))

#38 Updated by SLindoMansilla over 1 year ago

  • Copied to deleted (action #40052: [functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2)

#39 Updated by SLindoMansilla over 1 year ago

  • Related to action #40349: [sle][functional][u][migration][sle15sp1] Jobs incomplete due to missing assets added

#40 Updated by mgriessmeier over 1 year ago

  • Target version changed from Milestone 27 to Milestone 28

#41 Updated by mgriessmeier about 1 year ago

  • Target version changed from Milestone 28 to Milestone 30

needs to be discussed offline

#42 Updated by SLindoMansilla 10 months ago

  • Assignee changed from mgriessmeier to SLindoMansilla

#43 Updated by szarate 5 months ago

  • Tracker changed from action to coordination
  • Status changed from Workable to New
  • Difficulty deleted (medium)

#45 Updated by tjyrinki_suse 4 months ago

  • Subject changed from [functional][opensuse][epic][u][medium] uefi upgrade tests on TW+Leap (was: missing assets) to [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets)

Also available in: Atom PDF