QA » openQA Project » openQA Tests

I would only test version => version+2 at maximum, to reduce test matrix. Fedora is doing the same

An idea from Dominique. Utilize net-install image which has already the latest zypper to the date. The problem with our migration is that tools that do migration do not have required fixes in. I think he mentioned that TW uses it in this way for migration from old releases.

lkocman wrote:

An idea from Dominique. Utilize net-install image which has already the latest zypper to the date. The problem with our migration is that tools that do migration do not have required fixes in. I think he mentioned that TW uses it in this way for migration from old releases.

Right, on TW, from 'old' releases we only do 'offline upgrades' (i.e. not from a running, old system) using net-installer and dvd-installer, but not zdup.

This allows us to test extra old to current tumbleweed (we go back to 42.1 at the moment) (we tested zdup as long as we could, but TW changed compression payload of the RPM packages and those old distros could not unpack the new RPM format)

of course for the supported scenarios thee zdup cases should stay valid and functioning; the offline method as only method should only be considered on old platforms (and only if zdup cannot be used reasonably easy)

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: zdup-Leap-15.0-gnome
https://openqa.opensuse.org/tests/2316424#step/prepare_system_for_update_tests/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

Figure out the availability of the repositories for eol'd distros

szarate wrote:

Figure out the availability of the repositories for eol'd distros

Yup, they are present in http://download.opensuse.org/update/leap/, starting with 42.3

If we reduce scope we can also close this particular issue https://bugzilla.opensuse.org/show_bug.cgi?id=1197359

leap15.3 to leap15.4¶

• opensuse-15.4-NET-x86_64-Build222.1-zdup-Leap-kde-15.3to15.4
• opensuse-15.4-NET-x86_64-Build222.1-zdup-Leap-kde-15.3to15.4

This works without any issues, amarok and firefox_audio might not be directly related failures of upgrade procedure.

leap15.2 to leap15.4¶

• opensuse-15.4-NET-x86_64-Build230.2-zdup-Leap-15.2-kde@64bit

As per https://bugzilla.opensuse.org/show_bug.cgi?id=1196549#c6 installing updates before running upgrade ( I have updated the image before this run ) works fine

chained migration from leap15.1 -> leap15.2 -> leap15.4¶

• opensuse-15.4-NET-x86_64-Build222.1-zdup-Leap-gnome-15.2to15.4_chained_migration_15.1@64bit
• opensuse-15.4-NET-x86_64-Build222.1-zdup-Leap-kde-15.2to15.4_chained_migration_15.1@64bit

Works fine for both kde and gnome flavors

chained migration from leap15.0 -> leap15.2 -> leap15.4¶

Revealed two problems, the gnome migration chained job failed in my openQA instance, seems like a performance issue and maybe related to below messages from PackageKit (not sure). Manual test went fine though.

• opensuse-15.4-NET-x86_64-Build222.1-zdup-Leap-gnome-15.2to15.4_chained_migration_15.0@64bit

susetest login: [   36.064145] PackageKit[2294]: search-file transaction /11_eecdeacb from uid 1000 finished with success after 1305ms
[   37.484989] PackageKit[2294]: search-file transaction /12_babeeead from uid 1000 finished with success after 1415ms
[   38.764573] PackageKit[2294]: search-file transaction /13_cbdadcec from uid 1000 finished with success after 1274ms
[   39.275347] dbus-daemon[823]: [system] Failed to activate service 'org.bluez': timed out (service_start_timeout=25000ms)
[   39.276338] pulseaudio[2116]: E: [pulseaudio] bluez5-util.c: GetManagedObjects() failed: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
[   39.988575] PackageKit[2294]: search-file transaction /14_cbadbaab from uid 1000 finished with success after 1219ms
[   41.203013] PackageKit[2294]: search-file transaction /15_eabacbdb from uid 1000 finished with success after 1209ms
[   42.397529] PackageKit[2294]: search-file transaction /16_deebaade from uid 1000 finished with success after 1189ms
[   43.574572] PackageKit[2294]: search-file transaction /17_cbaadcdd from uid 1000 finished with success after 1172ms

In case of kde, there is a problem when updating to leap15.2 from leap15.0 ZYPPER_EXIT_INF_RPM_SCRIPT_FAILED.

• opensuse-15.4-NET-x86_64-Build222.1-zdup-Leap-kde-15.0to15.2@64bit

[  989.530582] [RPM][29992]: Transaction ID 62840732 started
[  989.599247] [RPM][29992]: erase sddm-0.17.0-lp150.8.1.x86_64: success
[  989.613054] useradd[29995]: failed adding user 'sddm', exit code: 9
[  989.763129] [RPM][29992]: install sddm-0.17.0-lp150.9.6.1.x86_64: success
[  989.769151] [RPM][29992]: erase sddm-0.17.0-lp150.8.1.x86_64: success
[  989.770640] [RPM][29992]: install sddm-0.17.0-lp150.9.6.1.x86_64: success
[  989.771318] [RPM][29992]: Transaction ID 62840732 finished: 0
.
.
.
[ 1131.470476] [RPM][878]: install libqt5-qtwebengine-5.10.1-lp150.3.4.2.x86_64: success
[ 1131.473261] [RPM][878]: scriptlet %postun(libqt5-qtwebengine-5.10.1-lp150.2.1.x86_64) failure: 2
[ 1131.487814] [RPM][878]: erase libqt5-qtwebengine-5.10.1-lp150.2.1.x86_64: success
[ 1131.491270] [RPM][878]: install libqt5-qtwebengine-5.10.1-lp150.3.4.2.x86_64: success
[ 1131.493408] [RPM][878]: 0 elements failed, 1 scripts failed
[ 1131.494968] [RPM][878]: Transaction ID 628407bc finished: 0

Running full chain migration scenario has another downside and that it takes a lot of time to execute even though I have stripped the test modules for bare minimum in the intermediate jobs and run the full scope test suite only at the last job aka leap15.4.

With respect to above, I would vote for Frederic's suggestion, maybe additionally have full chain test for minimal (desktop?) installations.

I have updated the Leap15.2 assets and the tests are passing the previous repo failure, however KDE fails with authentication problem, any ideas ?

• opensuse-15.4-NET-x86_64-Build236.1-zdup-Leap-15.2-gnome@64bit
• opensuse-15.4-NET-x86_64-Build236.1-zdup-Leap-15.2-kde@64bit

I would also like to know what to do with then older zdup test suites than 15.2 as we cannot fix them. Would you like to chain? If so should we keep gnome|kde or use minimal desktop story ?

mloviska wrote:

I have updated the Leap15.2 assets and the tests are passing the previous repo failure, however KDE fails with authentication problem, any ideas ?

• opensuse-15.4-NET-x86_64-Build236.1-zdup-Leap-15.2-gnome@64bit
• opensuse-15.4-NET-x86_64-Build236.1-zdup-Leap-15.2-kde@64bit

May 23 08:58:41.443455 susetest dbus-daemon[613]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.56' (uid=0 pid=1738 comm="pkcon refresh ")
May 23 08:58:41.452548 susetest systemd[1]: Starting PackageKit Daemon...
May 23 08:58:41.475770 susetest PackageKit[1743]: daemon start
May 23 08:58:41.699846 susetest dbus-daemon[613]: [system] Successfully activated service 'org.freedesktop.PackageKit'
May 23 08:58:41.701853 susetest systemd[1]: Started PackageKit Daemon.
May 23 08:58:41.710746 susetest plasmashell[1302]: plasma-pk-updates: Is net mobile: false
May 23 08:58:41.710780 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.711036 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.714625 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.715290 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.715345 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.715544 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.715661 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.745329 susetest PackageKit[1743]: uid 1000 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth (only_trusted:0)
May 23 08:58:41.715681 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.764936 susetest PackageKit[1743]: uid 1000 failed to obtain auth
May 23 08:58:41.715721 susetest plasmashell[1302]: plasma-pk-updates: Daemon changed
May 23 08:58:41.775703 susetest polkitd[622]: Registered Authentication Agent for unix-process:1738:14960 (system bus name :1.58 [/usr/bin/pkttyagent --notify-fd 10 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale POSIX)
May 23 08:58:41.720255 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.813052 susetest PackageKit[1743]: uid 0 is trying to obtain org.freedesktop.packagekit.system-sources-refresh auth (only_trusted:0)
May 23 08:58:41.720274 susetest plasmashell[1302]: plasma-pk-updates: Checking for updates
May 23 08:58:41.825327 susetest PackageKit[1743]: uid 0 obtained auth for org.freedesktop.packagekit.system-sources-refresh
May 23 08:58:41.731409 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.731677 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.871268 susetest plasmashell[1302]: plasma-pk-updates: Transaction status changed: "waiting-for-auth" "(101%)"
May 23 08:58:41.874194 susetest plasmashell[1302]: plasma-pk-updates: Transaction status changed: "waiting-for-auth" "(101%)"
May 23 08:58:41.884515 susetest plasmashell[1302]: PK error: "Failed to obtain authentication." type: "not-authorized"
May 23 08:58:41.884768 susetest plasmashell[1302]: plasma-pk-updates: Transaction "/1_aeecedcc" finished with status "failed" in 0 seconds
May 23 08:58:41.884810 susetest plasmashell[1302]: plasma-pk-updates: Cache transaction didn't finish successfully
May 23 08:58:41.884853 susetest plasmashell[1302]: plasma-pk-updates: Last check successful: false
May 23 08:58:41.885475 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.885825 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.886242 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.886300 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.886314 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 08:58:41.886434 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
.
.
.
May 23 09:00:29.906617 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 09:00:29.913957 susetest systemd[1]: Starting PackageKit Daemon...
May 23 09:00:30.067042 susetest systemd[1]: Started PackageKit Daemon.
May 23 09:00:30.089482 susetest PackageKit[1978]: new update-packages transaction /1_cebdccce scheduled from uid 1000
May 23 09:00:30.082987 susetest plasmashell[1302]: plasma-pk-updates: Is net mobile: false
May 23 09:00:30.083008 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 09:00:30.083088 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 09:00:30.083125 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 09:00:30.083146 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 09:00:30.083164 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 09:00:30.083190 susetest plasmashell[1302]: plasma-pk-updates: Is net online: true
May 23 09:00:30.083212 susetest plasmashell[1302]: plasma-pk-updates: Daemon changed
May 23 09:00:30.086917 susetest plasmashell[1302]: plasma-pk-updates: Transaction status changed: "wait" "(101%)"
May 23 09:00:30.115781 susetest plasmashell[1302]: plasma-pk-updates: Transaction status changed: "setup" "(101%)"
May 23 09:00:34.256060 susetest plasmashell[1302]: plasma-pk-updates: Transaction status changed: "dep-resolve" "(101%)"
May 23 09:00:34.595394 susetest plasmashell[1302]: plasma-pk-updates: Package updating: "MozillaFirefox;91.9.0-150200.152.37.3;x86_64;repo-sle-update" , info: "installing"
May 23 09:00:34.598511 susetest plasmashell[1302]: plasma-pk-updates: Package updating: "MozillaFirefox-translations-common;91.9.0-150200.152.37.3;x86_64;repo-sle-update" , info: "installing"
May 23 09:00:34.601482 susetest plasmashell[1302]: plasma-pk-updates: Package updating: "cups;2.2.7-150000.3.29.1;x86_64;repo-sle-update" , info: "installing"

Seems like after reload it started to work...

I would also like to know what to do with then older zdup test suites than 15.2 as we cannot fix them. Would you like to chain? If so should we keep gnome|kde or use minimal desktop story ?

https://github.com/os-autoinst/opensuse-jobgroups/pull/139

The agreement was to drop speficically zdup migration scenarios for unsupported releases.
We would keep it only 15.2+

Offline media migration and others would still be present as these do not suffer from missing zypper gpg key import feature.

https://github.com/os-autoinst/opensuse-jobgroups/pull/144

Workwise finished, unfortunately, the last PR got merged day after last build of Leap 15.4. In order to make o3 VRs, I can add zdup jobs to Updates test group.

some failures on zdup see #112169

https://github.com/os-autoinst/opensuse-jobgroups/pull/154

only GNOME scenarios: https://bugzilla.opensuse.org/show_bug.cgi?id=1200526