Project

General

Profile

Actions
Copy link

action #40052

closed

coordination #39302: [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets)

[functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2

Added by okurz over 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
SLindoMansilla
Category:
Bugs in existing tests
Target version:
SUSE QA - Milestone 23
Start date:
2018-08-08
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

Same as in #39302 we have some upgrade test for openSUSE Leap which do not have a corresponding scenario in openSUSE TW causing openQA GRU to cleanup assets.

Suggestions

  • DONE: Remove currently failing incompletes from the test schedule
  • DONE: Work on #39302 first
  • Discuss with lnussel and maxlin and DimStar if the "64bit" machine definition from TW with 1.5GB RAM should be used as default or "64bit-2G" with 2GB
  • Make sure TW is a superset of Leap upgrade cases
  • Bring back removed scenarios unless seen as not necessary

Further details

Original observation

Latest failures:

Actions
Copy link
 #1

Updated by okurz over 5 years ago

  • Copied from coordination #39302: [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets) added
Actions
Copy link
 #2

Updated by okurz over 5 years ago

  • Description updated (diff)

I removed the four mentioned scenarios.

Actions
Copy link
 #3

Updated by szarate over 5 years ago

  • Related to action #40349: [sle][functional][u][migration][sle15sp1] Jobs incomplete due to missing assets added
Actions
Copy link
 #4

Updated by okurz over 5 years ago

  • Due date changed from 2018-09-25 to 2018-10-23

we need #39302 first and everything takes longer than wished for so we need to delay the plan.

Actions
Copy link
 #5

Updated by okurz over 5 years ago

  • Description updated (diff)

updated #39302 and mentioned that we have a good enough test coverage for now and should now focus on the other points, e.g. straighten out the used RAM and such.

Actions
Copy link
 #6

Updated by okurz over 5 years ago

  • Target version changed from Milestone 19 to Milestone 20
Actions
Copy link
 #7

Updated by okurz over 5 years ago

  • Due date deleted (2018-10-23)
  • Target version changed from Milestone 20 to Milestone 21
Actions
Copy link
 #8

Updated by okurz over 5 years ago

  • Related to action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assets added
Actions
Copy link
 #9

Updated by SLindoMansilla over 5 years ago

  • Status changed from Workable to In Progress
  • Assignee set to SLindoMansilla
Actions
Copy link
 #10

Updated by SLindoMansilla over 5 years ago

Currently covered (in progress done)

openSUSE Leap 42.1 to openSUSE Tumbleweed

openSUSE Leap 42.2 to openSUSE Tumbleweed

openSUSE Leap 42.3 to openSUSE Tumbleweed

openSUSE Leap 15.0 to openSUSE Tumbleweed

Actions
Copy link
 #11

Updated by SLindoMansilla over 5 years ago

Scheduled test suites for tumbleweed on development job group: https://openqa.opensuse.org/admin/job_templates/38

opensuse-Tumbleweed-DVD

opensuse-Tumbleweed-NET

Uploaded openSUSE-Leap-42.3-DVD-x86_64-Build0331-Media.iso into O3

Actions
Copy link
 #12

Updated by SLindoMansilla over 5 years ago

  • Status changed from In Progress to Feedback
Actions
Copy link
 #13

Updated by SLindoMansilla over 5 years ago

  • Status changed from Feedback to In Progress
Actions
Copy link
 #14

Updated by okurz over 5 years ago

  • Target version changed from Milestone 21 to Milestone 22
Actions
Copy link
 #15

Updated by SLindoMansilla over 5 years ago

  • Status changed from In Progress to Workable

At the moment busy with:

I will continue with it as soon as I finish one of them.

Actions
Copy link
 #16

Updated by okurz about 5 years ago

  • Target version changed from Milestone 22 to Milestone 23
Actions
Copy link
 #17

Updated by SLindoMansilla about 5 years ago

  • Status changed from Workable to In Progress
Actions
Copy link
 #18

Updated by SLindoMansilla about 5 years ago

Needle screenlock-gnome-leap42.3-20190220 created

Actions
Copy link
 #19

Updated by okurz about 5 years ago

  • Blocks action #39305: [functional][u] 42.3 upgrade tests use GM image instead of updated one added
Actions
Copy link
 #20

Updated by SLindoMansilla about 5 years ago

3 new test suite moved to main job group openSUSE_Tumbleweed

  • zdup-Leap-42.1-gnome
  • zdup-Leap-42.2-gnome
  • zdup-Leap-42.3-gnome

Only 3 more test suites left to complete this ticket:

Actions
Copy link
 #21

Updated by SLindoMansilla about 5 years ago

  • Status changed from In Progress to Resolved

Please, find verification runs on https://progress.opensuse.org/issues/40052#note-10

Actions
Copy link
 #22

Updated by okurz about 5 years ago

  • Status changed from Resolved to In Progress

I am not sure if we understood the same with the two suggestions

  • Make sure TW is a superset of Leap upgrade cases
  • Bring back removed scenarios unless seen as not necessary

We should have these upgrade tests on openSUSE Leap still, i.e. Leap-$old_to_Leap-$current for each corresponding case. Do we have them?

Actions
Copy link
 #23

Updated by SLindoMansilla about 5 years ago

okurz wrote:

I am not sure if we understood the same with the two suggestions

  • Make sure TW is a superset of Leap upgrade cases

As discussed with you, I should not create/schedule any test that weren't needed for already existing upgrade scenarios. If this is still true, TW is now a superset of all existing upgrade scenarios from Leap 42.3 to Leap 15.1 (unless I miss any, then please send me the link, the scenarios I am aware of are these: https://progress.opensuse.org/issues/40052#note-10

  • Bring back removed scenarios unless seen as not necessary

I have brought back the needed scenarios so the existing TW upgrade scenarios don't miss assets and also created the needed TW upgrade scenarios to keep generated qcow2 from being cleaned up, so that upgrade scenarios from Leap 42.3 to Leap 15.1 don't miss assets.

We should have these upgrade tests on openSUSE Leap still, i.e. Leap-$old_to_Leap-$current for each corresponding case. Do we have them?

I don't understand this question. Which tests are you missing? The list I crosschecked is this: https://progress.opensuse.org/issues/40052#note-10

Actions
Copy link
 #24

Updated by okurz about 5 years ago

So take a look on https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1&build=423.2&groupid=50 . There is for example update_Leap_42.2_gnome@64bit-2G which does not have a corresponding scenario in the latest Tumbleweed test results. There is only update_Leap_42.2_gnome@64bit which differs in the machine definition, hence the suggestion "Discuss with lnussel and maxlin and DimStar if the "64bit" machine definition from TW with 1.5GB RAM should be used as default or "64bit-2G" with 2GB".

Second example: Tumbleweed has update_Leap_42.2_kde which Leap does not have. That is a scenario I consider missing from Leap and the ticket description mentions it as "opensuse-15.1-DVD-x86_64-update_Leap_42.2_kde@64bit-2G"

Actions
Copy link
 #25

Updated by SLindoMansilla about 5 years ago

True, I was missing the point about the 64bit-2G machine. Working on it.

Actions
Copy link
 #26

Updated by SLindoMansilla about 5 years ago

  • Status changed from In Progress to Resolved

As spoken with DimStar and lnussel,

  • the machine 64bit should be used as default
  • the machine 64bit-2G is not needed for coverage and is only a helper in case we need to provide more memory for an upgrade

Considering this ticket resolved.

Actions
Copy link
 #27

Updated by okurz about 5 years ago

  • Status changed from Resolved to In Progress

Sorry I don't understand what you mean with "the machine 64bit-2G is not needed for coverage". Of course I understand that 64bit-2G is equivalent to 64bit but just having 2GB of RAM. That is not the point: openQA gru does not know that the machines are "equivalent" so according assets can only be retained if the machine type matches.

Why do you consider the ticket to be "Resolved"? The question is still open: Should one or the other machine be used, just be consistent among Tumbleweed and Leap. And please only set the ticket to "Resolved" when all points have been adressed, e.g. you have not looked into the second part of #40052#note-24 . If it is still unclear you can talk to me in person as well.

Actions
Copy link
 #28

Updated by SLindoMansilla about 5 years ago

as spoken, I will adapt the current upgrade scenarios on Leap 15.1 to use 64bit machine instead of 64bit-2G, so that the proper qcow2 image is used by Leap and Tumbleweed scenarios.

To add the missing upgrade scenarios, this ticket was created: #48725

Actions
Copy link
 #29

Updated by SLindoMansilla about 5 years ago

  • Status changed from In Progress to Feedback

I need that someone with write access to O3 copies the following files:

cp opensuse-42.1-x86_64-Updates-20170213-1-gnome@64bit-2G.qcow2 opensuse-42.1-x86_64-Updates-20170213-1-gnome@64bit.qcow2
cp opensuse-42.1-x86_64-Updates-20170213-1-kde@64bit-2G.qcow2 opensuse-42.1-x86_64-Updates-20170213-1-kde@64bit.qcow2
cp opensuse-42.2-x86_64-Updates-20180313-2-gnome@64bit-2G.qcow2 opensuse-42.2-x86_64-Updates-20180313-2-gnome@64bit-2G.qcow2
cp opensuse-42.3-x86_64-GM-gnome@64bit-2G.qcow2 opensuse-42.3-x86_64-GM-gnome@64bit.qcow2
cp opensuse-42.3-x86_64-GM-kde@64bit-2G.qcow2 opensuse-42.3-x86_64-GM-kde@64bit.qcow2
Actions
Copy link
 #30

Updated by okurz about 5 years ago

A simple "cp" would waste useless disk space. I used hard links instead. However, all files existed already. Maybe someone else has done it however the dates are not from yesterday or today.

I was thinking more about the upgrade cases consolidation. In general I would be more conservative with changes for Leap 15.1 as it should become more stable now. Maybe time to try again for 15.2 to not disrupt the Leap 15.1 stabilization process.

Can you write down your specific plans how you would replace the 64bit-2G jobs with the 64bit variants?

Actions
Copy link
 #31

Updated by okurz about 5 years ago

openSUSE Leap 15 contains less scenarios on "64bit" than "64bit-2G" so after ensuring the according scenarios are stable on "64bit" as well we should change them in the build validation job group to 64bit

Actions
Copy link
 #32

Updated by SLindoMansilla about 5 years ago

  • Status changed from Feedback to In Progress

okurz wrote:

A simple "cp" would waste useless disk space. I used hard links instead. However, all files existed already. Maybe someone else has done it however the dates are not from yesterday or today.

sure, I though on a symlink, but a hard link is better.

I was thinking more about the upgrade cases consolidation. In general I would be more conservative with changes for Leap 15.1 as it should become more stable now. Maybe time to try again for 15.2 to not disrupt the Leap 15.1 stabilization process.

I am ok with that.

Can you write down your specific plans how you would replace the 64bit-2G jobs with the 64bit variants?

  1. I have already scheduled the upgrade scenarios using 64bit-2G with machine 64bit in the development job group.
  2. I planned to cloned the jobs yesterday overriding QEMURAM, MACHINE and HDD_1.
  3. Once I have then green/orange (so we have a last good and expected result), I would schedule them in the main job group and unschedule the upgrade jobs using 64bit-2G.
  4. The cleanup should happen automatically now. Since they are hardlinks, there is no rush.
Actions
Copy link
 #33

Updated by SLindoMansilla about 5 years ago

Already verified 64bit scenarios on O3, moved to main job group and equivalent 64bit-2G unscheduled from main job group:

  • update_Leap_42.1_gnome
  • update_Leap_42.1_kde
  • update_Leap_42.2_gnome
  • update_Leap_42.3_gnome
  • update_Leap_42.3_kde
  • zdup-Leap-42.1-gnome
  • zdup-Leap-42.3-kde
  • zdup-Leap-42.2-gnome
  • zdup-Leap-42.3-gnome
  • update_Leap_42.1_gnome
  • update_Leap_42.1_kde
  • update_Leap_42.2_gnome
  • update_Leap_42.3_gnome
  • update_Leap_42.3_kde
  • update_Leap_42.3_kde+system_performance
Actions
Copy link
 #34

Updated by SLindoMansilla about 5 years ago

  • Status changed from In Progress to Feedback

I cannot find any 64bit-2G scheduled job for upgrade scenarios in https://openqa.opensuse.org/admin/job_templates/50.

Waiting for next build to verifiy proper schedule.

okurz, if you think it is enough, you can resolve the ticket.

Actions
Copy link
 #35

Updated by SLindoMansilla about 5 years ago

  • Status changed from Feedback to Resolved

Verified schedule on O3: https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1&build=426.2&groupid=50

The failing ones have an expected previous result.
I have notice that the workqueue lockup happen more often. Maybe this time is related to a machine with less memory. The investigation will follow here: https://bugzilla.suse.com/show_bug.cgi?id=1126782

Actions
Copy link
 #36

Updated by okurz almost 5 years ago

  • Related to action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G) added
Actions
Copy link
 #37

Updated by SLindoMansilla over 4 years ago

  • Copied from deleted (coordination #39302: [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets))
Actions
Copy link
 #38

Updated by SLindoMansilla over 4 years ago

  • Related to deleted (action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G))
Actions
Copy link
 #39

Updated by SLindoMansilla over 4 years ago

  • Related to deleted (action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assets)
Actions
Copy link
 #40

Updated by SLindoMansilla over 4 years ago

  • Parent task set to #39302
Actions
Copy link
 #41

Updated by SLindoMansilla over 4 years ago

  • Blocks deleted (action #39305: [functional][u] 42.3 upgrade tests use GM image instead of updated one)
Actions
Copy link
 #42

Updated by SLindoMansilla over 4 years ago

  • Related to deleted (action #40349: [sle][functional][u][migration][sle15sp1] Jobs incomplete due to missing assets)
Actions
Copy link

Also available in: Atom PDF