action #40052
closedcoordination #39302: [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets)
[functional][u] inconsistent test cases / missing assets in openSUSE Leap upgrade cases vs. Tumbleweed, part 2
Added by okurz over 5 years ago. Updated over 4 years ago.
0%
Description
Observation¶
Same as in #39302 we have some upgrade test for openSUSE Leap which do not have a corresponding scenario in openSUSE TW causing openQA GRU to cleanup assets.
Suggestions¶
- DONE: Remove currently failing incompletes from the test schedule
- DONE: Work on #39302 first
- Discuss with lnussel and maxlin and DimStar if the "64bit" machine definition from TW with 1.5GB RAM should be used as default or "64bit-2G" with 2GB
- Make sure TW is a superset of Leap upgrade cases
- Bring back removed scenarios unless seen as not necessary
Further details¶
Original observation¶
Latest failures:
Updated by okurz over 5 years ago
- Copied from coordination #39302: [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets) added
Updated by okurz over 5 years ago
- Description updated (diff)
I removed the four mentioned scenarios.
Updated by szarate over 5 years ago
- Related to action #40349: [sle][functional][u][migration][sle15sp1] Jobs incomplete due to missing assets added
Updated by okurz over 5 years ago
- Due date changed from 2018-09-25 to 2018-10-23
we need #39302 first and everything takes longer than wished for so we need to delay the plan.
Updated by okurz over 5 years ago
- Target version changed from Milestone 19 to Milestone 20
Updated by okurz over 5 years ago
- Due date deleted (
2018-10-23) - Target version changed from Milestone 20 to Milestone 21
Updated by okurz over 5 years ago
- Related to action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assets added
Updated by SLindoMansilla over 5 years ago
- Status changed from Workable to In Progress
- Assignee set to SLindoMansilla
Updated by SLindoMansilla over 5 years ago
Currently covered (in progress done)¶
openSUSE Leap 42.1 to openSUSE Tumbleweed¶
- 42.1-GNOME-upgrade-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20181218-update_Leap_42.1_gnome@64bit
- 42.1-KDE-update-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20181218-update_Leap_42.1_kde@64bit
- 42.1-GNOME-zdup-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20190225-zdup-Leap-42.1-gnome@64bit
- 42.1-GNOME-upgrade-NET: opensuse-Tumbleweed-NET-x86_64-Build20181218-update_Leap_42.1_gnome@64bit
- 42.1-KDE-upgrade-NET: opensuse-Tumbleweed-NET-x86_64-Build20181218-update_Leap_42.1_kde@64bit
- 42.1-GNOME-zdup-NET: opensuse-Tumbleweed-NET-x86_64-Build20181218-zdup-Leap-42.1-gnome@64bit (not needed, but useful)
openSUSE Leap 42.2 to openSUSE Tumbleweed¶
- 42.2-GNOME-upgrade-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20181218-update_Leap_42.2_gnome@64bit
- 42.2-GNOME-zdup-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20190225-zdup-Leap-42.2-gnome@64bit
- 42.2-GNOME-update-NET: opensuse-Tumbleweed-NET-x86_64-Build20181218-update_Leap_42.2_gnome@64bit
- 42.2-GNOME-zdup-NET: opensuse-Tumbleweed-NET-x86_64-Build20181218-zdup-Leap-42.2-gnome@64bit (not needed, but useful)
openSUSE Leap 42.3 to openSUSE Tumbleweed¶
- 42.3-CRYPTLVM-upgrade-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20190301-update_Leap_42.3_cryptlvm@uefi
- 42.3-GNOME-upgrade-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20181218-update_Leap_42.3_gnome@64bit
- 42.3-KDE-upgrade-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20181218-update_Leap_42.3_kde@64bit
- 42.3-GNOME-zdup-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20190225-zdup-Leap-42.3-gnome@64bit
- 42.3-KDE-zdup-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20190226-zdup-Leap-42.3-kde@64bit
- 42.3-CRYPTLVM-upgrade-NET: opensuse-Tumbleweed-NET-x86_64-Build20190226-update_Leap_42.3_cryptlvm@uefi
- 42.3-GNOME-upgrade-NET: opensuse-Tumbleweed-NET-x86_64-Build20181218-update_Leap_42.3_gnome@64bit
- 42.3-KDE-upgrade-NET: opensuse-Tumbleweed-NET-x86_64-Build20181218-update_Leap_42.3_kde+system_performance@64bit
- 42.3-KDE+system_performance-update-NET: opensuse-Tumbleweed-NET-x86_64-Build20181218-update_Leap_42.3_kde+system_performance@64bit
openSUSE Leap 15.0 to openSUSE Tumbleweed¶
- 15.0-CRYPTLVM-upgrade-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20181218-update_Leap_15.0_cryptlvm@uefi
- 15.0-GNOME-upgrade-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20181218-update_Leap_15.0_gnome@64bit
- 15.0-KDE-upgrade-DVD: opensuse-Tumbleweed-DVD-x86_64-Build20181218-update_Leap_15.0_kde@64bit
Updated by SLindoMansilla over 5 years ago
Scheduled test suites for tumbleweed on development job group: https://openqa.opensuse.org/admin/job_templates/38
opensuse-Tumbleweed-DVD¶
- zdup-Leap-42.1-gnome@64bit (soft-failed, moved to main job group)
- zdup-Leap-42.2-gnome@64bit (soft-failed, moved to main job group)
- zdup-Leap-42.3-gnome@64bit (soft-failed, moved to main job group)
- zdup-Leap-42.3-kde@64bit (soft-failed, moved to main job group)
- zdup-Leap-15.0-gnome@64bit (soft-failed, moved to main job group)
opensuse-Tumbleweed-NET¶
Uploaded openSUSE-Leap-42.3-DVD-x86_64-Build0331-Media.iso into O3
- update_Leap_42.3_cryptlvm@64bit-2G
- Creating qcow2 legacy BIOS: https://openqa.opensuse.org/tests/859787
- update_Leap_42.3_cryptlvm@uefi
- Creating qcow2 uefi: https://openqa.opensuse.org/tests/859574
Updated by SLindoMansilla over 5 years ago
- Status changed from In Progress to Feedback
Updated by SLindoMansilla over 5 years ago
- Status changed from Feedback to In Progress
Updated by okurz over 5 years ago
- Target version changed from Milestone 21 to Milestone 22
Updated by SLindoMansilla over 5 years ago
- Status changed from In Progress to Workable
Updated by okurz about 5 years ago
- Target version changed from Milestone 22 to Milestone 23
Updated by SLindoMansilla about 5 years ago
- Status changed from Workable to In Progress
Updated by SLindoMansilla about 5 years ago
Needle screenlock-gnome-leap42.3-20190220 created
Updated by okurz about 5 years ago
- Blocks action #39305: [functional][u] 42.3 upgrade tests use GM image instead of updated one added
Updated by SLindoMansilla about 5 years ago
3 new test suite moved to main job group openSUSE_Tumbleweed
- zdup-Leap-42.1-gnome
- zdup-Leap-42.2-gnome
- zdup-Leap-42.3-gnome
Only 3 more test suites left to complete this ticket:
- 42.3-CRYPTLVM-upgrade-DVD-UEFI (soft-failed, moved to main job group)
- 42.3-KDE-zdup-DVD done
- 42.3-CRYPTLVM-upgrade-NET-UEFI (soft-failed, moved to main job group)
Updated by SLindoMansilla about 5 years ago
- Status changed from In Progress to Resolved
Please, find verification runs on https://progress.opensuse.org/issues/40052#note-10
Updated by okurz about 5 years ago
- Status changed from Resolved to In Progress
I am not sure if we understood the same with the two suggestions
- Make sure TW is a superset of Leap upgrade cases
- Bring back removed scenarios unless seen as not necessary
We should have these upgrade tests on openSUSE Leap still, i.e. Leap-$old_to_Leap-$current for each corresponding case. Do we have them?
Updated by SLindoMansilla about 5 years ago
okurz wrote:
I am not sure if we understood the same with the two suggestions
- Make sure TW is a superset of Leap upgrade cases
As discussed with you, I should not create/schedule any test that weren't needed for already existing upgrade scenarios. If this is still true, TW is now a superset of all existing upgrade scenarios from Leap 42.3 to Leap 15.1 (unless I miss any, then please send me the link, the scenarios I am aware of are these: https://progress.opensuse.org/issues/40052#note-10
- Bring back removed scenarios unless seen as not necessary
I have brought back the needed scenarios so the existing TW upgrade scenarios don't miss assets and also created the needed TW upgrade scenarios to keep generated qcow2 from being cleaned up, so that upgrade scenarios from Leap 42.3 to Leap 15.1 don't miss assets.
We should have these upgrade tests on openSUSE Leap still, i.e. Leap-$old_to_Leap-$current for each corresponding case. Do we have them?
I don't understand this question. Which tests are you missing? The list I crosschecked is this: https://progress.opensuse.org/issues/40052#note-10
Updated by okurz about 5 years ago
So take a look on https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1&build=423.2&groupid=50 . There is for example update_Leap_42.2_gnome@64bit-2G which does not have a corresponding scenario in the latest Tumbleweed test results. There is only update_Leap_42.2_gnome@64bit which differs in the machine definition, hence the suggestion "Discuss with lnussel and maxlin and DimStar if the "64bit" machine definition from TW with 1.5GB RAM should be used as default or "64bit-2G" with 2GB".
Second example: Tumbleweed has update_Leap_42.2_kde which Leap does not have. That is a scenario I consider missing from Leap and the ticket description mentions it as "opensuse-15.1-DVD-x86_64-update_Leap_42.2_kde@64bit-2G"
Updated by SLindoMansilla about 5 years ago
True, I was missing the point about the 64bit-2G machine. Working on it.
Updated by SLindoMansilla about 5 years ago
- Status changed from In Progress to Resolved
As spoken with DimStar and lnussel,
- the machine 64bit should be used as default
- the machine 64bit-2G is not needed for coverage and is only a helper in case we need to provide more memory for an upgrade
Considering this ticket resolved.
Updated by okurz about 5 years ago
- Status changed from Resolved to In Progress
Sorry I don't understand what you mean with "the machine 64bit-2G is not needed for coverage". Of course I understand that 64bit-2G is equivalent to 64bit but just having 2GB of RAM. That is not the point: openQA gru does not know that the machines are "equivalent" so according assets can only be retained if the machine type matches.
Why do you consider the ticket to be "Resolved"? The question is still open: Should one or the other machine be used, just be consistent among Tumbleweed and Leap. And please only set the ticket to "Resolved" when all points have been adressed, e.g. you have not looked into the second part of #40052#note-24 . If it is still unclear you can talk to me in person as well.
Updated by SLindoMansilla about 5 years ago
as spoken, I will adapt the current upgrade scenarios on Leap 15.1 to use 64bit machine instead of 64bit-2G, so that the proper qcow2 image is used by Leap and Tumbleweed scenarios.
To add the missing upgrade scenarios, this ticket was created: #48725
Updated by SLindoMansilla about 5 years ago
- Status changed from In Progress to Feedback
I need that someone with write access to O3 copies the following files:
cp opensuse-42.1-x86_64-Updates-20170213-1-gnome@64bit-2G.qcow2 opensuse-42.1-x86_64-Updates-20170213-1-gnome@64bit.qcow2
cp opensuse-42.1-x86_64-Updates-20170213-1-kde@64bit-2G.qcow2 opensuse-42.1-x86_64-Updates-20170213-1-kde@64bit.qcow2
cp opensuse-42.2-x86_64-Updates-20180313-2-gnome@64bit-2G.qcow2 opensuse-42.2-x86_64-Updates-20180313-2-gnome@64bit-2G.qcow2
cp opensuse-42.3-x86_64-GM-gnome@64bit-2G.qcow2 opensuse-42.3-x86_64-GM-gnome@64bit.qcow2
cp opensuse-42.3-x86_64-GM-kde@64bit-2G.qcow2 opensuse-42.3-x86_64-GM-kde@64bit.qcow2
Updated by okurz about 5 years ago
A simple "cp" would waste useless disk space. I used hard links instead. However, all files existed already. Maybe someone else has done it however the dates are not from yesterday or today.
I was thinking more about the upgrade cases consolidation. In general I would be more conservative with changes for Leap 15.1 as it should become more stable now. Maybe time to try again for 15.2 to not disrupt the Leap 15.1 stabilization process.
Can you write down your specific plans how you would replace the 64bit-2G jobs with the 64bit variants?
Updated by okurz about 5 years ago
openSUSE Leap 15 contains less scenarios on "64bit" than "64bit-2G" so after ensuring the according scenarios are stable on "64bit" as well we should change them in the build validation job group to 64bit
Updated by SLindoMansilla about 5 years ago
- Status changed from Feedback to In Progress
okurz wrote:
A simple "cp" would waste useless disk space. I used hard links instead. However, all files existed already. Maybe someone else has done it however the dates are not from yesterday or today.
sure, I though on a symlink, but a hard link is better.
I was thinking more about the upgrade cases consolidation. In general I would be more conservative with changes for Leap 15.1 as it should become more stable now. Maybe time to try again for 15.2 to not disrupt the Leap 15.1 stabilization process.
I am ok with that.
Can you write down your specific plans how you would replace the 64bit-2G jobs with the 64bit variants?
- I have already scheduled the upgrade scenarios using 64bit-2G with machine 64bit in the development job group.
- I planned to cloned the jobs yesterday overriding QEMURAM, MACHINE and HDD_1.
- Once I have then green/orange (so we have a last good and expected result), I would schedule them in the main job group and unschedule the upgrade jobs using 64bit-2G.
- The cleanup should happen automatically now. Since they are hardlinks, there is no rush.
Updated by SLindoMansilla about 5 years ago
Already verified 64bit scenarios on O3, moved to main job group and equivalent 64bit-2G unscheduled from main job group:
- update_Leap_42.1_gnome
- update_Leap_42.1_kde
- update_Leap_42.2_gnome
- update_Leap_42.3_gnome
- update_Leap_42.3_kde
- zdup-Leap-42.1-gnome
- zdup-Leap-42.3-kde
- zdup-Leap-42.2-gnome
- zdup-Leap-42.3-gnome
- update_Leap_42.1_gnome
- update_Leap_42.1_kde
- update_Leap_42.2_gnome
- update_Leap_42.3_gnome
- update_Leap_42.3_kde
- update_Leap_42.3_kde+system_performance
Updated by SLindoMansilla about 5 years ago
- Status changed from In Progress to Feedback
I cannot find any 64bit-2G scheduled job for upgrade scenarios in https://openqa.opensuse.org/admin/job_templates/50.
Waiting for next build to verifiy proper schedule.
okurz, if you think it is enough, you can resolve the ticket.
Updated by SLindoMansilla about 5 years ago
- Status changed from Feedback to Resolved
Verified schedule on O3: https://openqa.opensuse.org/tests/overview?distri=opensuse&version=15.1&build=426.2&groupid=50
The failing ones have an expected previous result.
I have notice that the workqueue lockup happen more often. Maybe this time is related to a machine with less memory. The investigation will follow here: https://bugzilla.suse.com/show_bug.cgi?id=1126782
Updated by okurz almost 5 years ago
- Related to action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G) added
Updated by SLindoMansilla over 4 years ago
- Copied from deleted (coordination #39302: [qe-core][functional][opensuse][epic][medium] uefi upgrade tests on TW+Leap (was: missing assets))
Updated by SLindoMansilla over 4 years ago
- Related to deleted (action #51824: [functional][u] test incomplete - missing qcow2 for 64bit machine (non-2G))
Updated by SLindoMansilla over 4 years ago
- Related to deleted (action #44546: [opensuse][functional][u] TW kde 42.2 upgrade tests missing assets)
Updated by SLindoMansilla over 4 years ago
- Blocks deleted (action #39305: [functional][u] 42.3 upgrade tests use GM image instead of updated one)
Updated by SLindoMansilla over 4 years ago
- Related to deleted (action #40349: [sle][functional][u][migration][sle15sp1] Jobs incomplete due to missing assets)