Project

General

Profile

Actions

action #107620

closed

[qe-core] Please add upgrade test from Leap 15.3 to Leap 15.4

Added by lkocman about 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
New test
Target version:
Start date:
2022-03-08
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:
Sprint:
QE-Core: April Sprint (Apr 13 - May 11)

Description

Leap test suite does not have upgrade test from 15.3 to 15.4 could you please add it?

Thank you
(Tracked as part of) https://progress.opensuse.org/issues/99894

Notes

  • We need to add a job that does the upgrade from 15.3 to 15.4
  • Test coverage should be kept
  • Plan for migration towards YAML Schedule in a separate ticket (Unless changes in the schedule are required.)

Acceptance criteria


Subtasks 1 (0 open1 closed)

action #107965: [qe-core] add gm qcows to fixed hdd directory for leap in o3Resolvedszarate2022-03-08

Actions

Related issues 5 (1 open4 closed)

Related to openQA Tests - action #107812: [qe-core] Leap 15.2 GM deleted by mistakeResolvedszarate2022-03-03

Actions
Related to openSUSE Leap 15.4 - action #99894: add openqa upgrade tests to/from 15.3Resolvedlkocman2021-10-06

Actions
Related to openQA Tests - action #91554: [qem][leap][opensuse][qe-core] Update GM images before running upgrade testsFeedbackapappas

Actions
Related to openQA Tests - action #108692: [qe-core] Leap offline migration - do not skip releases (15.0 -> 15.1 -> 15.2 -> 15.3)Resolvedmloviska

Actions
Is duplicate of openQA Tests - action #108695: [desktop] test fails in opensuse_welcome - blocks migration from 15.3 - missing needle?Rejectedpunkioudi2022-03-212022-04-04

Actions
Actions #1

Updated by lkocman about 2 years ago

Also please be aware that latest supported Leap is currently 15.2

Technically all migration tests from previous releases (42.X, 15.0, 15.1) could be dropped. Unless QA is willing to help maintaing them.

Thank you

Actions #2

Updated by favogt about 2 years ago

lkocman wrote:

Also please be aware that latest supported Leap is currently 15.2

Technically all migration tests from previous releases (42.X, 15.0, 15.1) could be dropped. Unless QA is willing to help maintaing them.

Technically, yes. As long as they work they should be kept though, they can still show product bugs of actual relevance.

Actions #3

Updated by szarate about 2 years ago

  • Subject changed from Please add upgrade test from Leap 15.3 to Leap 15.4 to [qe-core] Please add upgrade test from Leap 15.3 to Leap 15.4
Actions #4

Updated by tjyrinki_suse about 2 years ago

  • Status changed from New to Workable
  • Target version set to QE-Core: Ready
  • Start date deleted (2022-02-25)
Actions #5

Updated by szarate about 2 years ago

  • Description updated (diff)
Actions #6

Updated by punkioudi about 2 years ago

  • Assignee set to punkioudi
Actions #7

Updated by szarate about 2 years ago

  • Related to action #107812: [qe-core] Leap 15.2 GM deleted by mistake added
Actions #8

Updated by punkioudi about 2 years ago

  • Status changed from Workable to Blocked

Thanks Santiago for the update. For now, I set this ticket as blocked, until we have the Leap15.3 GM qcow2 image in o3 assets.

Actions #9

Updated by tjyrinki_suse about 2 years ago

  • Status changed from Blocked to Workable

Image was restored.

Actions #10

Updated by punkioudi about 2 years ago

@tjyrinki_suse is leap15.3 GM qcow2 in o3 assets?

Actions #11

Updated by punkioudi about 2 years ago

  • Status changed from Workable to Blocked

I set this ticket back to blocked, as Leap15.3 GM qcow2 image is not there yet. We are waiting for the next build here : https://openqa.opensuse.org/group_overview/88.

Actions #12

Updated by szarate about 2 years ago

  • Status changed from Blocked to Workable

build is available

Actions #13

Updated by szarate about 2 years ago

  • Related to action #99894: add openqa upgrade tests to/from 15.3 added
Actions #14

Updated by szarate about 2 years ago

  • Related to action #91554: [qem][leap][opensuse][qe-core] Update GM images before running upgrade tests added
Actions #15

Updated by punkioudi about 2 years ago

  • Status changed from Workable to In Progress

The following testsuites have been added in Leap 15 runs:

  • upgrade_Leap_15.3_gnome
  • upgrade_Leap_15.3_kde They will run when the new Build is triggered.

I did some check runs :
https://openqa.opensuse.org/tests/2237332
https://openqa.opensuse.org/tests/2237356

Actions #17

Updated by punkioudi about 2 years ago

upgrade_Leap_15.3_kde : https://openqa.opensuse.org/tests/2243864
upgrade_Leap_15.3_gnome : https://openqa.opensuse.org/tests/2243863

In both cases, it seems that the welcome opensuse doesn't work in the same was as in create_hdd_gnome + create_hdd_kde, should I file a bug?
For the upgrade_Leap_15.3_cryptlvm@uefi, I think there was no qcow2 image created.

Actions #18

Updated by punkioudi about 2 years ago

I will schedule also the tests mentioned in the bugzilla link :

  • zdup-Leap-15.2-gnome
  • zdup-Leap-15.2-kde

Is there anything else I'm missing?

Actions #19

Updated by punkioudi about 2 years ago

  • Status changed from In Progress to Feedback
Actions #21

Updated by szarate about 2 years ago

  • Category set to New test

punkioudi wrote:

upgrade_Leap_15.3_kde : https://openqa.opensuse.org/tests/2243864
upgrade_Leap_15.3_gnome : https://openqa.opensuse.org/tests/2243863

In both cases, it seems that the welcome opensuse doesn't work in the same was as in create_hdd_gnome + create_hdd_kde, should I file a bug?
For the upgrade_Leap_15.3_cryptlvm@uefi, I think there was no qcow2 image created.

Cryptlvm has to be also added (atm the create job failed for aarch64, and is not scheduled for x86_64).

For the zdup, we'll wait for @lkocman to create the ticket to start working on it, and make the tests scenarios to behave as a linked list (15.0* to 15.4 step by step) (asterisk means last EOL version)

Actions #22

Updated by punkioudi about 2 years ago

I re-triggered the cryptlvm test and the qcow2 image is now available:
https://openqa.opensuse.org/tests/2246951#downloads
@szarate, could you copy it to the folder with the other qcow2 images? (:

Actions #23

Updated by favogt about 2 years ago

  • Status changed from Feedback to Workable

Let's reopen this for the opensuse_welcome failures.

It's not that trivial to fix, as in the live_upgrade case it'll have to deal with opensuse_welcome on the live CD, but not after the upgrade. So it might have to do a version switch like SLE.

Actions #24

Updated by punkioudi almost 2 years ago

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.
For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
@tjyrinki_suse, @szarate wdyt?

Actions #25

Updated by favogt almost 2 years ago

punkioudi wrote:

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.

No, the behaviour is identical - opensuse-welcome does not autostart in the upgraded system.

For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
@tjyrinki_suse, @szarate wdyt?

Actions #26

Updated by punkioudi almost 2 years ago

favogt wrote:

punkioudi wrote:

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.

No, the behaviour is identical - opensuse-welcome does not autostart in the upgraded system.

For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
@tjyrinki_suse, @szarate wdyt?

How is it identical?
Leap 15.2 -> Leap 15.4

Leap 15.3 -> Leap 15.4

Do I see something wrong?

Actions #27

Updated by lkocman almost 2 years ago

Please be aware of https://progress.opensuse.org/issues/108692 which is supposed to ensure that migrations from prior 15.2 are possible.

Actions #28

Updated by szarate almost 2 years ago

  • Related to action #108692: [qe-core] Leap offline migration - do not skip releases (15.0 -> 15.1 -> 15.2 -> 15.3) added
Actions #29

Updated by punkioudi almost 2 years ago

  • Related to action #108689: [qe-core][opensuse] KDE / Leap test fails in consoletest_finish - screnlocker missing needle ? added
Actions #30

Updated by favogt almost 2 years ago

punkioudi wrote:

favogt wrote:

punkioudi wrote:

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.

No, the behaviour is identical - opensuse-welcome does not autostart in the upgraded system.

For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
@tjyrinki_suse, @szarate wdyt?

How is it identical?

15.3 -> 15.4 has identical behaviour on GNOME and KDE. The difference is that on GNOME, the test module itself start opensuse-welcome as a (meanwhile obsolete?) workaround for a product bug.

Actions #31

Updated by punkioudi almost 2 years ago

favogt wrote:

punkioudi wrote:

favogt wrote:

punkioudi wrote:

I think that it would be better to fix the way that HDD is being created, instead of adding a workaround in the test, as we see different behaviors in gnome + kde.

No, the behaviour is identical - opensuse-welcome does not autostart in the upgraded system.

For example the similar scenario Leap 15.2 -> Leap 15.4 : https://openqa.opensuse.org/tests/2255751#step/opensuse_welcome/3 works fine, so we could probably try to achieve the same result.
@tjyrinki_suse, @szarate wdyt?

How is it identical?

15.3 -> 15.4 has identical behaviour on GNOME and KDE. The difference is that on GNOME, the test module itself start opensuse-welcome as a (meanwhile obsolete?) workaround for a product bug.

Yes, indeed but I was speaking about its difference with the upgrade Leap15.2 -> Leap 15.4 test results.
In any case, I am preparing a PR to solve the test issue (:

Actions #32

Updated by favogt almost 2 years ago

Yes, indeed but I was speaking about its difference with the upgrade Leap15.2 -> Leap 15.4 test results.

Yep. From 15.3 on, opensuse-welcome is installed by default and thus also already disabled in those images. So for future 15.4 -> 15.5 upgrades opensuse-welcome will also not appear after upgrades.

For some reason I'm not receiving mail notifications for changes to this ticket...

Actions #33

Updated by favogt almost 2 years ago

  • Related to deleted (action #108689: [qe-core][opensuse] KDE / Leap test fails in consoletest_finish - screnlocker missing needle ?)
Actions #34

Updated by punkioudi almost 2 years ago

Here is the proposed solution for opensuse_welcome issue : https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/14575, I am running some VRs and open it for review

Actions #35

Updated by openqa_review almost 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde_live_upgrade_leap_15.3@64bit
https://openqa.opensuse.org/tests/2285635#step/opensuse_welcome#1/1

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

Actions #36

Updated by szarate almost 2 years ago

Is the status of this ticket current? any updates? can we resolve the ticket?

Actions #37

Updated by punkioudi almost 2 years ago

@szarate, I am wating for you to upload the cryptlvm qcow2 image and then schedule the upgrade_cryptlvm testsuite. After that, the ticket can be closed.

Actions #38

Updated by szarate almost 2 years ago

  • Sprint set to QE-Core: April Sprint (Apr 13 - May 11)
  • Tags set to outofsprint
  • Status changed from Workable to Feedback

punkioudi wrote:

@szarate, I am wating for you to upload the cryptlvm qcow2 image and then schedule the upgrade_cryptlvm testsuite. After that, the ticket can be closed.

I have just added them to the pool

`fixed/opensuse-15.3-x86_64-38.1-cryptlvm@uefi.qcow2' -> `fixed/opensuse-15.3-x86_64-GM-QU-cryptlvm@uefi.qcow2'
`fixed/opensuse-15.3-x86_64-38.1-cryptlvm@uefi-uefi-vars.qcow2' -> `fixed/opensuse-15.3-x86_64-GM-QU-cryptlvm@uefi-uefi-vars.qcow2'
Actions #39

Updated by szarate almost 2 years ago

  • Sprint deleted (QE-Core: April Sprint (Apr 13 - May 11))
Actions #40

Updated by punkioudi almost 2 years ago

Thanks @szarate, I just added the new testsuite and scheduled it in Leap runs. It will be visible in the next Build :)

Actions #41

Updated by szarate almost 2 years ago

  • Blocks action #108695: [desktop] test fails in opensuse_welcome - blocks migration from 15.3 - missing needle? added
Actions #42

Updated by szarate almost 2 years ago

  • Blocks deleted (action #108695: [desktop] test fails in opensuse_welcome - blocks migration from 15.3 - missing needle?)
Actions #43

Updated by szarate almost 2 years ago

  • Is duplicate of action #108695: [desktop] test fails in opensuse_welcome - blocks migration from 15.3 - missing needle? added
Actions #44

Updated by szarate almost 2 years ago

  • Status changed from Feedback to Rejected

See related issues

Actions #45

Updated by szarate almost 2 years ago

  • Status changed from Rejected to Feedback
Actions #46

Updated by punkioudi almost 2 years ago

Last upgrade scenario runs fine : https://openqa.opensuse.org/tests/2317436, so I think this ticket can be set as resolved now, wdyt @szarate?

Actions #47

Updated by szarate almost 2 years ago

  • Sprint set to QE-Core: April Sprint (Apr 13 - May 11)
Actions #48

Updated by szarate almost 2 years ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF