Project

General

Profile

Actions

coordination #39302

open

[qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets)

Added by lnussel almost 6 years ago. Updated 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Bugs in existing tests
Target version:
SUSE QA - Milestone 30
Start date:
2018-08-08
Due date:
% Done:

90%

Estimated time:
(Total: 210.00 h)
Difficulty:

Description

Suggestions

  • Ensure TW is a superset of Leap, i.e. add missing upgrade tests only on Leap to TW
  • If necessary according to lnussel add 42.1 upgrade tests by creating the assets somehow, manually or automatically

Further details

Original observation

This problem affects O3 only (openqa.opensuse.org)
When upgrading from any Leap version to TW.
So, the assets are disk image with previous Leap installation.

In https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1

[2018-08-08T00:55:39.0872 CEST] [info] result: setup failure: Can't download opensuse-42.1-x86_64-Updates-20170213-1-cryptlvm@uefi.qcow2
[2018-08-08T00:48:01.0851 CEST] [info] result: setup failure: Can't download opensuse-42.2-x86_64-Updates-20180206-1-cryptlvm@uefi.qcow2


Subtasks 10 (1 open9 closed)

action #55874: [qe-core][functional] Create matrix about possible upgrade scenarios in combination with machine types distribution versionRejected2019-08-23

Actions
action #55877: [qe-core][functional] Describe the process on how we get a Leap scenario to a TumbleweedFeedback2019-08-24

Actions
action #51857: [qe-core][functional] create scenario upgrade_tumbleweed which uses released ISO instead of tested ISO and publishes the qcow2 for extra testsRejected2019-05-22

Actions
action #48725: [functional][u] Add upgrade tests from Leap 15.1 to openSUSE TumbleweedResolveddheidler2019-03-06

Actions
action #48731: [functional][u] Add a single upgrade test from Leap 15.1 to openSUSE TumbleweedResolvedSLindoMansilla2019-03-06

Actions
action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G)ResolvedSLindoMansilla2019-05-22

Actions
action #40052: [functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2ResolvedSLindoMansilla2018-08-08

Actions
action #39305: [functional][u] 42.3 upgrade tests use GM image instead of updated oneResolvedSLindoMansilla2018-08-08

Actions
action #56531: [functional][u][migration] please create 15.[01] zdup testsResolveddheidler2019-09-06

Actions
action #51860: [functional][u] Add update from Leap 15.1 testsRejected2019-05-22

Actions

Related issues 4 (0 open4 closed)

Related to openQA Tests - action #39713: [functional][y] add openqa upgrade tests from 15.0Resolvedriafarov2018-08-132018-08-28

Actions
Related to openQA Tests - action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assetsResolvedokurz2018-11-29

Actions
Related to openQA Tests - action #46868: [aarch64][functional][y][migration] enable migration tests on aarch64Resolvedggardet_arm2019-01-302019-03-12

Actions
Related to openQA Tests - action #40349: [sle][functional][u][migration][sle15sp1] Jobs incomplete due to missing assetsResolvedszarate2018-08-28

Actions
Actions #1

Updated by okurz almost 6 years ago

  • Project changed from openQA Project to openQA Tests
  • Category set to Bugs in existing tests
  • Status changed from New to In Progress
  • Assignee set to okurz
Actions #2

Updated by okurz almost 6 years ago

  • Assignee changed from okurz to lnussel

The assets are gone. Recreating them using openQA clone_job should be feasible if necessary. However, the problem I see is that you as RM for openSUSE Leap defined a custom testing scope (uefi upgrade scenarios) which certainly make sense but they are for Leap only, hence the last successful jobs marking the openQA assets as used are 15.0 GM. Since then – 3 months ago – the assets have been unused and therefore deleted as such. I suggest to primarily extend the test scope on openSUSE Tumbleweed first as this will mark the assets as always used and only define a sub-set for openSUSE Leap. Back to lnussel, WDYT?

Actions #3

Updated by lnussel almost 6 years ago

  • Assignee changed from lnussel to okurz

feel free to put the same test cases also on TW. It should be possible to recreate the assets by restarting the jobs that originally created them

Actions #4

Updated by lnussel almost 6 years ago

note there's also the hdd/fixed directory which contains some images. not sure what the rule is to put stuff there

Actions #5

Updated by okurz almost 6 years ago

  • Subject changed from missing assets to [functional][u] uefi upgrade tests on TW+Leap (was: missing assets)
  • Description updated (diff)
  • Due date set to 2018-09-11
  • Status changed from In Progress to Workable
  • Assignee deleted (okurz)
  • Priority changed from High to Normal
  • Target version set to Milestone 19

lnussel wrote:

It should be possible to recreate the assets by restarting the jobs that originally created them

True but I doubt we still have these jobs on o3, most likely pruned already.

lnussel wrote:

note there's also the hdd/fixed directory which contains some images. not sure what the rule is to put stuff there

I am fully aware of the fixed assets. IMHO the "rule" is: Just do it or run into problems like the one described in this ticket ;)

So for a start I removed the following test scenarios from the Leap job group as they are incomplete and have the missing assets assigned as well as are for now unsupported Leap versions but also we still have the non-uefi, non-cryptlvm upgrade scenarios still as well as the uefi,cryptlvm scenario doing upgrade from 42.1 to current:

  • DVD-x86_64-update_Leap_42.1_cryptlvm@uefi
  • DVD-x86_64-update_Leap_42.2_cryptlvm@uefi
  • NET-x86_64-update_Leap_42.1_cryptlvm@uefi
  • NET-x86_64-update_Leap_42.2_cryptlvm@uefi

This should remove the urgency.

Next steps:

  • Ensure TW is a superset of Leap, i.e. add missing upgrade tests only on Leap to TW
  • If necessary according to lnussel add 42.1 upgrade tests by creating the assets somehow, manually or automatically
Actions #6

Updated by SLindoMansilla almost 6 years ago

  • Description updated (diff)
Actions #7

Updated by SLindoMansilla almost 6 years ago

  • Subject changed from [functional][u] uefi upgrade tests on TW+Leap (was: missing assets) to [functional][opensuse][u] uefi upgrade tests on TW+Leap (was: missing assets)
  • Description updated (diff)
Actions #8

Updated by SLindoMansilla almost 6 years ago

  • Subject changed from [functional][opensuse][u] uefi upgrade tests on TW+Leap (was: missing assets) to [functional][opensuse][u][medium] uefi upgrade tests on TW+Leap (was: missing assets)
  • Estimated time set to 5.00 h
  • Difficulty set to medium
Actions #9

Updated by okurz almost 6 years ago

  • Copied to action #40052: [functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2 added
Actions #10

Updated by SLindoMansilla almost 6 years ago

  • Related to action #39713: [functional][y] add openqa upgrade tests from 15.0 added
Actions #11

Updated by SLindoMansilla almost 6 years ago

  • Assignee set to SLindoMansilla
Actions #12

Updated by SLindoMansilla almost 6 years ago

  • Status changed from Workable to In Progress
Actions #14

Updated by mgriessmeier almost 6 years ago

  • Due date changed from 2018-09-11 to 2018-09-25

try to reproduce on o3 directly by e.g. a jobs post or fallback to manual verification

Actions #15

Updated by SLindoMansilla almost 6 years ago

Running clone of 42.1 to generate qcow2 for cryptlvm on O3: https://openqa.opensuse.org/tests/752499

Actions #16

Updated by okurz almost 6 years ago

  • Due date deleted (2018-09-25)
  • Priority changed from Normal to Low
  • Target version changed from Milestone 19 to Milestone 22

failed because the ISO is not there anymore: https://openqa.opensuse.org/tests/752499/file/autoinst-log.txt shows "Download of /var/lib/openqa/cache/openSUSE-Leap-42.1-DVD-x86_64-Build0265-Media.iso failed with: 404 - Not Found" . You could try to reference the ISO from download.o.o . As this becomes more effort now and with limited value I suggest to drop the work on it and revisit the situation later. We have a good coverage spanning different versions, desktops, flavors, etc., still and no currently incomplete or failing tests linked with this ticket here.

@sergio unassign ok?

Actions #17

Updated by okurz almost 6 years ago

  • Status changed from In Progress to Workable
  • Assignee deleted (SLindoMansilla)

@SLindoMansilla is fine with unassigning

Actions #18

Updated by okurz over 5 years ago

  • Related to action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assets added
Actions #19

Updated by SLindoMansilla over 5 years ago

  • Subject changed from [functional][opensuse][u][medium] uefi upgrade tests on TW+Leap (was: missing assets) to [functional][opensuse][epic][u][medium] uefi upgrade tests on TW+Leap (was: missing assets)

Since it was copied to the ticket I am working on, and my ticket is called "part 2", I consider this an epic ticket

Actions #20

Updated by okurz over 5 years ago

  • Related to action #46868: [aarch64][functional][y][migration] enable migration tests on aarch64 added
Actions #21

Updated by okurz over 5 years ago

  • Target version changed from Milestone 22 to Milestone 24
Actions #22

Updated by okurz over 5 years ago

  • Status changed from Workable to Blocked
  • Assignee set to SLindoMansilla

@SLindoMansilla please update after #40052 is resolved

Actions #23

Updated by SLindoMansilla over 5 years ago

  • Status changed from Blocked to Feedback

Blocker resolved.

I asked lnussel and DimStar in #opensuse-factory @freenode if the test coverage for each upgrade test should have uefi and legacy scenarios. Waiting for their answer. If the answer is yes, I would resolve this epic. If not, I would transform it into a normal task to complete the missing scenarios.

Actions #24

Updated by okurz over 5 years ago

  • Related to action #48725: [functional][u] Add upgrade tests from Leap 15.1 to openSUSE Tumbleweed added
Actions #25

Updated by okurz over 5 years ago

"each upgrade test" should not have UEFI and legacy but we want a "sparse test matrix", i.e. cover all variants in a least one combination

Actions #26

Updated by SLindoMansilla over 5 years ago

  • Status changed from Feedback to Blocked

Blocked by #48725

Actions #27

Updated by mgriessmeier about 5 years ago

  • Target version changed from Milestone 24 to Milestone 25
Actions #28

Updated by okurz about 5 years ago

  • Status changed from Blocked to Workable

@SLindoMansilla it seems like something went wrong here. You set the ticket to "Blocked" by #48725 which in turn is blockey by this one :D
Considering that Leap 15.1 is about to be released like … today that is a bit unfortunate in case we we missed some serious issue, let's just cross fingers that we not ;)

Actions #29

Updated by okurz about 5 years ago

  • Related to action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G) added
Actions #30

Updated by mgriessmeier about 5 years ago

  • Target version changed from Milestone 25 to Milestone 26
Actions #31

Updated by mgriessmeier almost 5 years ago

  • Target version changed from Milestone 26 to Milestone 27

we should revisit for leap 15.2
@Sergio wdyt?

Actions #32

Updated by SLindoMansilla almost 5 years ago

  • Assignee deleted (SLindoMansilla)

mgriessmeier wrote:

we should revisit for leap 15.2
@Sergio wdyt?

Sure. This needs more love.
Let's take this to be properly refined today.

Actions #33

Updated by SLindoMansilla almost 5 years ago

  • Assignee set to mgriessmeier

To be tracked

Actions #34

Updated by SLindoMansilla almost 5 years ago

  • Due date set to 2019-08-24

due to changes in a related task

Actions #35

Updated by SLindoMansilla almost 5 years ago

  • Related to deleted (action #48725: [functional][u] Add upgrade tests from Leap 15.1 to openSUSE Tumbleweed)
Actions #36

Updated by SLindoMansilla almost 5 years ago

  • Due date set to 2019-08-24

due to changes in a related task

Actions #37

Updated by SLindoMansilla almost 5 years ago

  • Related to deleted (action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G))
Actions #38

Updated by SLindoMansilla almost 5 years ago

  • Copied to deleted (action #40052: [functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2)
Actions #39

Updated by SLindoMansilla almost 5 years ago

  • Related to action #40349: [sle][functional][u][migration][sle15sp1] Jobs incomplete due to missing assets added
Actions #40

Updated by mgriessmeier almost 5 years ago

  • Target version changed from Milestone 27 to Milestone 28
Actions #41

Updated by mgriessmeier over 4 years ago

  • Target version changed from Milestone 28 to Milestone 30

needs to be discussed offline

Actions #42

Updated by SLindoMansilla about 4 years ago

  • Assignee changed from mgriessmeier to SLindoMansilla
Actions #43

Updated by szarate almost 4 years ago

  • Tracker changed from action to coordination
  • Status changed from Workable to New
  • Difficulty deleted (medium)
Actions #45

Updated by tjyrinki_suse over 3 years ago

  • Subject changed from [functional][opensuse][epic][u][medium] uefi upgrade tests on TW+Leap (was: missing assets) to [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets)
Actions #46

Updated by SLindoMansilla over 3 years ago

  • Assignee deleted (SLindoMansilla)

No time to work on this :(

Actions #47

Updated by slo-gin almost 2 years ago

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Actions

Also available in: Atom PDF