Project

General

Profile

action #107620

[qe-core] Please add upgrade test from Leap 15.3 to Leap 15.4

Added by lkocman 4 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
New test
Target version:
Start date:
2022-03-08
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Leap test suite does not have upgrade test from 15.3 to 15.4 could you please add it?

Thank you
(Tracked as part of) https://progress.opensuse.org/issues/99894

Notes

  • We need to add a job that does the upgrade from 15.3 to 15.4
  • Test coverage should be kept
  • Plan for migration towards YAML Schedule in a separate ticket (Unless changes in the schedule are required.)

Acceptance criteria


Subtasks

action #107965: [qe-core] add gm qcows to fixed hdd directory for leap in o3Resolvedszarate


Related issues

Related to openQA Tests - action #107812: [qe-core] Leap 15.2 GM deleted by mistakeResolved2022-03-03

Related to openSUSE Leap 15.4 - action #99894: add openqa upgrade tests to/from 15.3Blocked2021-10-06

Related to openQA Tests - action #91554: [qem][leap][opensuse][qe-core] Update GM images before running upgrade testsFeedback

Related to openQA Tests - action #108692: [qe-core] Leap offline migration - do not skip releases (15.0 -> 15.1 -> 15.2 -> 15.3)Resolved

Is duplicate of openQA Tests - action #108695: [desktop] test fails in opensuse_welcome - blocks migration from 15.3 - missing needle?Rejected2022-03-212022-04-04

History

#1 Updated by lkocman 4 months ago

Also please be aware that latest supported Leap is currently 15.2

Technically all migration tests from previous releases (42.X, 15.0, 15.1) could be dropped. Unless QA is willing to help maintaing them.

Thank you

#2 Updated by favogt 4 months ago

lkocman wrote:

Also please be aware that latest supported Leap is currently 15.2

Technically all migration tests from previous releases (42.X, 15.0, 15.1) could be dropped. Unless QA is willing to help maintaing them.

Technically, yes. As long as they work they should be kept though, they can still show product bugs of actual relevance.

#3 Updated by szarate 4 months ago

  • Subject changed from Please add upgrade test from Leap 15.3 to Leap 15.4 to [qe-core] Please add upgrade test from Leap 15.3 to Leap 15.4

#4 Updated by tjyrinki_suse 4 months ago

  • Status changed from New to Workable
  • Target version set to QE-Core: Ready
  • Start date deleted (2022-02-25)

#5 Updated by szarate 4 months ago

  • Description updated (diff)

#6 Updated by punkioudi 4 months ago

  • Assignee set to punkioudi

#7 Updated by szarate 4 months ago

  • Related to action #107812: [qe-core] Leap 15.2 GM deleted by mistake added

#8 Updated by punkioudi 4 months ago

  • Status changed from Workable to Blocked

Thanks Santiago for the update. For now, I set this ticket as blocked, until we have the Leap15.3 GM qcow2 image in o3 assets.

#9 Updated by tjyrinki_suse 4 months ago

  • Status changed from Blocked to Workable

Image was restored.

#10 Updated by punkioudi 4 months ago

tjyrinki_suse is leap15.3 GM qcow2 in o3 assets?

#11 Updated by punkioudi 4 months ago

  • Status changed from Workable to Blocked

I set this ticket back to blocked, as Leap15.3 GM qcow2 image is not there yet. We are waiting for the next build here : https://openqa.opensuse.org/group_overview/88.

#12 Updated by szarate 4 months ago

  • Status changed from Blocked to Workable

build is available

#13 Updated by szarate 4 months ago

  • Related to action #99894: add openqa upgrade tests to/from 15.3 added

#14 Updated by szarate 4 months ago

  • Related to action #91554: [qem][leap][opensuse][qe-core] Update GM images before running upgrade tests added

#15 Updated by punkioudi 4 months ago

  • Status changed from Workable to In Progress

The following testsuites have been added in Leap 15 runs:

  • upgrade_Leap_15.3_gnome
  • upgrade_Leap_15.3_kde They will run when the new Build is triggered.

I did some check runs :
https://openqa.opensuse.org/tests/2237332
https://openqa.opensuse.org/tests/2237356

#17 Updated by punkioudi 4 months ago

upgrade_Leap_15.3_kde : https://openqa.opensuse.org/tests/2243864
upgrade_Leap_15.3_gnome : https://openqa.opensuse.org/tests/2243863

In both cases, it seems that the welcome opensuse doesn't work in the same was as in create_hdd_gnome + create_hdd_kde, should I file a bug?
For the upgrade_Leap_15.3_cryptlvm@uefi, I think there was no qcow2 image created.

#18 Updated by punkioudi 4 months ago

I will schedule also the tests mentioned in the bugzilla link :

  • zdup-Leap-15.2-gnome
  • zdup-Leap-15.2-kde

Is there anything else I'm missing?

#19 Updated by punkioudi 4 months ago

  • Status changed from In Progress to Feedback

#21 Updated by szarate 4 months ago

  • Category set to New test

punkioudi wrote:

upgrade_Leap_15.3_kde : https://openqa.opensuse.org/tests/2243864
upgrade_Leap_15.3_gnome : https://openqa.opensuse.org/tests/2243863

In both cases, it seems that the welcome opensuse doesn't work in the same was as in create_hdd_gnome + create_hdd_kde, should I file a bug?
For the upgrade_Leap_15.3_cryptlvm@uefi, I think there was no qcow2 image created.

Cryptlvm has to be also added (atm the create job failed for aarch64, and is not scheduled for x86_64).

For the zdup, we'll wait for lkocman to create the ticket to start working on it, and make the tests scenarios to behave as a linked list (15.0* to 15.4 step by step) (asterisk means last EOL version)

#22 Updated by punkioudi 4 months ago

I re-triggered the cryptlvm test and the qcow2 image is now available:
https://openqa.opensuse.org/tests/2246951#downloads
szarate, could you copy it to the folder with the other qcow2 images? (:

#23 Updated by favogt 4 months ago

  • Status changed from Feedback to Workable

Let's reopen this for the opensuse_welcome failures.

It's not that trivial to fix, as in the live_upgrade case it'll have to deal with opensuse_welcome on the live CD, but not after the upgrade. So it might have to do a version switch like SLE.

#24 Updated by punkioudi 4 months ago

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.
For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
tjyrinki_suse, szarate wdyt?

#25 Updated by favogt 4 months ago

punkioudi wrote:

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.

No, the behaviour is identical - opensuse-welcome does not autostart in the upgraded system.

For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
tjyrinki_suse, szarate wdyt?

#26 Updated by punkioudi 4 months ago

favogt wrote:

punkioudi wrote:

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.

No, the behaviour is identical - opensuse-welcome does not autostart in the upgraded system.

For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
tjyrinki_suse, szarate wdyt?

How is it identical?
Leap 15.2 -> Leap 15.4

Leap 15.3 -> Leap 15.4

Do I see something wrong?

#27 Updated by lkocman 4 months ago

Please be aware of https://progress.opensuse.org/issues/108692 which is supposed to ensure that migrations from prior 15.2 are possible.

#28 Updated by szarate 3 months ago

  • Related to action #108692: [qe-core] Leap offline migration - do not skip releases (15.0 -> 15.1 -> 15.2 -> 15.3) added

#29 Updated by punkioudi 3 months ago

  • Related to action #108689: [qe-core][opensuse] KDE / Leap test fails in consoletest_finish - screnlocker missing needle ? added

#30 Updated by favogt 3 months ago

punkioudi wrote:

favogt wrote:

punkioudi wrote:

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.

No, the behaviour is identical - opensuse-welcome does not autostart in the upgraded system.

For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
tjyrinki_suse, szarate wdyt?

How is it identical?

15.3 -> 15.4 has identical behaviour on GNOME and KDE. The difference is that on GNOME, the test module itself start opensuse-welcome as a (meanwhile obsolete?) workaround for a product bug.

#31 Updated by punkioudi 3 months ago

favogt wrote:

punkioudi wrote:

favogt wrote:

punkioudi wrote:

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.

No, the behaviour is identical - opensuse-welcome does not autostart in the upgraded system.

For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
tjyrinki_suse, szarate wdyt?

How is it identical?

15.3 -> 15.4 has identical behaviour on GNOME and KDE. The difference is that on GNOME, the test module itself start opensuse-welcome as a (meanwhile obsolete?) workaround for a product bug.

Yes, indeed but I was speaking about its difference with the upgrade Leap15.2 -> Leap 15.4 test results.
In any case, I am preparing a PR to solve the test issue (:

#32 Updated by favogt 3 months ago

Yes, indeed but I was speaking about its difference with the upgrade Leap15.2 -> Leap 15.4 test results.

Yep. From 15.3 on, opensuse-welcome is installed by default and thus also already disabled in those images. So for future 15.4 -> 15.5 upgrades opensuse-welcome will also not appear after upgrades.

For some reason I'm not receiving mail notifications for changes to this ticket...

#33 Updated by favogt 3 months ago

  • Related to deleted (action #108689: [qe-core][opensuse] KDE / Leap test fails in consoletest_finish - screnlocker missing needle ?)

#34 Updated by punkioudi 3 months ago

Here is the proposed solution for opensuse_welcome issue : https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/14575, I am running some VRs and open it for review

#35 Updated by openqa_review 3 months ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde_live_upgrade_leap_15.3@64bit
https://openqa.opensuse.org/tests/2285635#step/opensuse_welcome#1/1

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

#36 Updated by szarate 3 months ago

Is the status of this ticket current? any updates? can we resolve the ticket?

#37 Updated by punkioudi 3 months ago

szarate, I am wating for you to upload the cryptlvm qcow2 image and then schedule the upgrade_cryptlvm testsuite. After that, the ticket can be closed.

#38 Updated by szarate 2 months ago

  • Sprint set to QE-Core: April Sprint (Apr 13 - May 11)
  • Tags set to outofsprint
  • Status changed from Workable to Feedback

punkioudi wrote:

szarate, I am wating for you to upload the cryptlvm qcow2 image and then schedule the upgrade_cryptlvm testsuite. After that, the ticket can be closed.

I have just added them to the pool

`fixed/opensuse-15.3-x86_64-38.1-cryptlvm@uefi.qcow2' -> `fixed/opensuse-15.3-x86_64-GM-QU-cryptlvm@uefi.qcow2'
`fixed/opensuse-15.3-x86_64-38.1-cryptlvm@uefi-uefi-vars.qcow2' -> `fixed/opensuse-15.3-x86_64-GM-QU-cryptlvm@uefi-uefi-vars.qcow2'

#39 Updated by szarate 2 months ago

  • Sprint deleted (QE-Core: April Sprint (Apr 13 - May 11))

#40 Updated by punkioudi 2 months ago

Thanks szarate, I just added the new testsuite and scheduled it in Leap runs. It will be visible in the next Build :)

#41 Updated by szarate 2 months ago

  • Blocks action #108695: [desktop] test fails in opensuse_welcome - blocks migration from 15.3 - missing needle? added

#42 Updated by szarate 2 months ago

  • Blocks deleted (action #108695: [desktop] test fails in opensuse_welcome - blocks migration from 15.3 - missing needle?)

#43 Updated by szarate 2 months ago

  • Is duplicate of action #108695: [desktop] test fails in opensuse_welcome - blocks migration from 15.3 - missing needle? added

#44 Updated by szarate 2 months ago

  • Status changed from Feedback to Rejected

See related issues

#45 Updated by szarate 2 months ago

  • Status changed from Rejected to Feedback

#46 Updated by punkioudi 2 months ago

Last upgrade scenario runs fine : https://openqa.opensuse.org/tests/2317436, so I think this ticket can be set as resolved now, wdyt szarate?

#47 Updated by szarate 2 months ago

  • Sprint set to QE-Core: April Sprint (Apr 13 - May 11)

#48 Updated by szarate 2 months ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF