Project

General

Profile

action #12964

[qe-core][functional][migration] Boot to snapshot after upgrade and then rollback

Added by okurz over 4 years ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
New test
Target version:
SUSE QA tests - Milestone 31
Start date:
Due date:
% Done:

100%

Estimated time:
42.00 h
Difficulty:

Description

Observation

We have #9580 for SLE, now enable for openSUSE

Annoyingly, the test module boot_into_snapshot doesn't boot into a snapshot. It is only scheduled in test suite boot_to_snapshot and directly after grub_test, which boots into a RO snapshot if setting BOOT_TO_SNAPSHOT is present (See https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/installation/grub_test.pm#L80)
Then, boot_into_snapshot expects the SUT to be already booted into a RO snapshot and performs checks and the rollback (in my opinion, the test module boot_into_snapshot should be renamed to rollback_from_ro_snapshot to properly describe what it does).

Because boot_into_snapshot expects the system booted into a RO snapshot, the check assert_script_run('touch /etc/NOWRITE;test ! -f /etc/NOWRITE'); is mandatory.
If the check fails, it means that the SUT is on a writable snapshot, and that could mean that there is a product bug or a test bug.

IMPORTANT
If the test module boot_into_snapshot is scheduled after grub_test and the job doesn't have the setting BOOT_TO_SNAPSHOT=1, grub_test will boot into the default writable snapshot and touch...NOWRITE will of course fail, the failure is a test bug, as can be seen in the verification run http://10.162.23.47/tests/8183

  • grub_test (which is booting into default writable snapshot)
  • first_boot
  • opensuse_welcome

Acceptance criteria

  • AC1: boot_into_snapshot is scheduled at the end of upgrade_Leap_15.1_gnome scenario (Leap->Tumbleweed) for the last Leap version.
  • AC2: boot_into_snapshot is able to rollback from an upgraded tumbleweed to Leap.

Tasks

  1. Rename boot_into_snapshot to rollback_from_ro_snapshot.
  2. Schedule rollback_from_ro_snapshot as the last module from test suite upgrade_Leap_15.1_gnome.
  3. Create a test module called boot_into_snapshot which should be executed (instead of grub_test) just before rollback_from_ro_snapshot, which should take care of waiting for grub and booting into the RO snapshot.
  4. Test module schedule for test suite boot_to_snapshot should be
    • boot_into_snapshot (the new created one)
    • rollback_from_ro_snapshot (the previously wrongly called boot_into_snapshot)
  5. Test module schedule for upgrade_Leap_15.1_gnome should be
    • isosize
    • bootloader
    • welcome
    • upgrade_select
    • online_repos
    • resolve_dependency_issues
    • installation_overview
    • disable_grub_timeout
    • start_install
    • await_install
    • logs_from_installation_system
    • reboot_after_installation
    • grub_test
    • first_boot
    • opensuse_welcome
    • system_prepare
    • zypper_clear_repos
    • zypper_ar
    • zypper_ref
    • check_network
    • system_state
    • prepare_test_data
    • consoletest_setup
    • keymap_or_locale
    • force_scheduled_tasks
    • textinfo
    • hostname
    • x_vt
    • zypper_lr
    • ncurses
    • yast2_lan
    • curl_https
    • salt
    • glibc_sanity
    • zypper_in
    • zypper_log
    • yast2_i
    • yast2_bootloader
    • vim
    • firewall_enabled
    • sshd
    • ssh_cleanup
    • sshfs
    • mtab
    • orphaned_packages_check
    • consoletest_finish
    • user_gui_login
    • desktop_runner
    • xterm
    • keymap_or_locale_x11
    • sshxterm
    • gnome_control_center
    • gnome_tweak_tool
    • gnome_terminal
    • gedit
    • firefox
    • firefox_audio
    • chromium
    • graphicsMagick
    • ooffice
    • oomath
    • oocalc
    • glxgears
    • nautilus
    • gnome_music
    • evolution
    • desktop_mainmenu
    • inkscape
    • gimp
    • hexchat
    • vlc
    • reboot_gnome
    • boot_into_snapshot (the new created one)
    • rollback_from_ro_snapshot (the previously wrongly called boot_into_snapshot)
    • shutdown
  6. Verify changes on following scenarios

Further information

Verify that changes do not break:

Scenario to add boot_into_snapshot at the end:


Checklist

  • TW
  • Leap

Related issues

Related to openQA Tests - action #18014: [migration]test fails in snapper_rollback by select root console failed.Resolved2017-03-27

Related to openQA Tests - action #34069: [sle][migration][sle15] Update snapper_rollback test module to check registration statusResolved2018-03-30

Related to openQA Tests - action #75295: [qe-core][opensuse][functional]test fails in shutdownNew

Blocks openQA Tests - action #55877: [qe-core][functional] Describe the process on how we get a Leap scenario to a TumbleweedWorkable2019-08-24

Copied from openQA Tests - action #9580: Boot to snapshot after upgrade and then rollbackResolved2015-11-17

History

#1 Updated by okurz over 4 years ago

  • Copied from action #9580: Boot to snapshot after upgrade and then rollback added

#2 Updated by okurz almost 4 years ago

  • Subject changed from openSUSE: Boot to snapshot after upgrade and then rollback to [opensuse] Boot to snapshot after upgrade and then rollback

#4 Updated by okurz almost 3 years ago

  • Subject changed from [opensuse] Boot to snapshot after upgrade and then rollback to [opensuse][functional][u] Boot to snapshot after upgrade and then rollback
  • Target version set to Milestone 18

#5 Updated by okurz almost 3 years ago

  • Related to action #18014: [migration]test fails in snapper_rollback by select root console failed. added

#6 Updated by okurz almost 3 years ago

  • Related to action #34069: [sle][migration][sle15] Update snapper_rollback test module to check registration status added

#7 Updated by okurz over 2 years ago

  • Target version changed from Milestone 18 to Milestone 18

#8 Updated by okurz over 2 years ago

  • Target version changed from Milestone 18 to future

#9 Updated by mgriessmeier 11 months ago

  • Status changed from New to Rejected

rejected, our guess is that it's already done, please reopen with updated information if not

#10 Updated by okurz 10 months ago

  • Status changed from Rejected to New

No, nothing has been done as far as I can see. While there might be progress for SLE this ticket is about the corresponding part for openSUSE. There is only boot_to_snapshot which is boot into a snapshot after a clean installation but no "boot to snapshot after upgrade". I think it's even worse because there are by now also multiple modules doing the same or similar but different. The module "boot_into_snapshot" (and rollback) is not mentioned for migration cases anymore. I checked with

$ for i in suse.de opensuse.org; do openqa-db_query_last_use_of_module -vvvv  --openqa-host openqa.$i --module boot_into_snapshot; done
DEBUG:/home/okurz/bin/openqa-db_query_last_use_of_module:call: ['ssh', 'openqa.suse.de', 'sudo -u geekotest psql -t --command="select distinct distri,version,flavor,test,arch,machine from jobs where id in (select job_id from job_modules where (name=\'boot_into_snapshot\')and t_updated > now() - interval \'90 days\') limit 30;" openqa']
 sle    | 15-SP2  | Online | boot_to_snapshot                                     | aarch64 | aarch64
 sle    | 15-SP2  | Online | boot_to_snapshot                                     | ppc64le | ppc64le
 sle    | 15-SP2  | Online | boot_to_snapshot                                     | x86_64  | 64bit
 sle    | 15-SP2  | Online | poo41882_okurz_boot_to_snapshot_new_kernel_on_worker | aarch64 | aarch64

DEBUG:/home/okurz/bin/openqa-db_query_last_use_of_module:call: ['ssh', 'openqa.opensuse.org', 'sudo -u geekotest psql -t --command="select distinct distri,version,flavor,test,arch,machine from jobs where id in (select job_id from job_modules where (name=\'boot_into_snapshot\')and t_updated > now() - interval \'90 days\') limit 30;" openqa']
 opensuse | 15.2       | DVD    | boot_to_snapshot                                                                      | aarch64 | aarch64
 opensuse | 15.2       | DVD    | boot_to_snapshot                                                                      | ppc64le | ppc64le
 opensuse | 15.2       | DVD    | boot_to_snapshot                                                                      | x86_64  | 64bit
 opensuse | Tumbleweed | DVD    | boot_to_snapshot                                                                      | aarch64 | aarch64
 opensuse | Tumbleweed | DVD    | boot_to_snapshot                                                                      | ppc64   | ppc64
 opensuse | Tumbleweed | DVD    | boot_to_snapshot                                                                      | ppc64le | ppc64le
 opensuse | Tumbleweed | DVD    | boot_to_snapshot                                                                      | x86_64  | 64bit
 opensuse | Tumbleweed | DVD    | boot_to_snapshot@StefanBruens/os-autoinst-distri-opensuse#accept-snapshot-description | x86_64  | 64bit

for the other test modules:

$ git grep 'snapper.*rollback'
data/console/check_registration_status.py:    # after snapper rollback to the snapshot before migration.  In such case,
lib/main_common.pm:        loadtest "boot/snapper_rollback";
lib/main_common.pm:        loadtest "migration/sle12_online_migration/snapper_rollback";
lib/migration.pm:# System check after snapper rollback
lib/utils.pm:    record_soft_failure 'bsc#1122591 - Create subvolume for aarch64 to make snapper rollback works';
tests/boot/snapper_rollback.pm:    script_run("snapper rollback -d rollback-before-migration");
tests/boot/snapper_rollback.pm:    assert_script_run("snapper list | tail -n 2 | grep rollback", 180);
tests/console/snapper_jeos_cli.pm:    assert_script_run("snapper rollback -d $openqarollback $init_snapshot");
tests/console/snapper_jeos_cli.pm:    assert_script_run("snapper rollback $latest_snapshot");
tests/console/snapper_jeos_cli.pm:    assert_script_run("snapper rollback $openqainit_snapshot");
tests/installation/boot_into_snapshot.pm:#  The test is also used for testing the snapper rollback functionality.
tests/installation/boot_into_snapshot.pm:    assert_script_run('snapper rollback');
tests/installation/boot_into_snapshot.pm:    assert_script_run('snapper --iso list | grep \'number.*rollback\'.*important=yes',
tests/installation/setup_zdup.pm:        # Create a snapshot with specified description to do snapper rollback
tests/migration/sle12_online_migration/pre_migration.pm:        # also update snapper to ensure rollback service work properly after migration
tests/migration/sle12_online_migration/snapper_rollback.pm:    script_run "snapper rollback";
okurz@linux-28d6:~/local/os-autoinst/opensuse 0 (fix/postgres_unzip) $ for i in suse.de opensuse.org; do openqa-db_query_last_use_of_module -vvvv  --openqa-host openqa.$i --module snapper_rollback; done
DEBUG:/home/okurz/bin/openqa-db_query_last_use_of_module:call: ['ssh', 'openqa.suse.de', 'sudo -u geekotest psql -t --command="select distinct distri,version,flavor,test,arch,machine from jobs where id in (select job_id from job_modules where (name=\'snapper_rollback\')and t_updated > now() - interval \'90 days\') limit 30;" openqa']
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_pcm-contm_full                                                                                   | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_sdk-lp-tcm_full                                                                                  | ppc64le | ppc64le
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_sdk-tcm_full                                                                                     | aarch64 | aarch64
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_sdk-tcm_full@lemon-suse/os-autoinst-distri-opensuse#Fix_timeout_higher_TIMEOUT_MAX_JOB_TIME      | aarch64 | aarch64
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_sdk-tcm_full@lemon-suse/os-autoinst-distri-opensuse#Record_timeoutvalue_higher_than_Max_JOB_TIME | aarch64 | aarch64
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_sdk-we-live-tcm_full                                                                             | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_sdk-we_full                                                                                      | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_sdk-we_full@ldevulder/os-autoinst-distri-opensuse#fix-addon_products_sle-with-full-media         | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_tcm-wsm_full                                                                                     | aarch64 | aarch64
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_tcm-wsm_full                                                                                     | ppc64le | ppc64le
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_media_tcm-wsm_full                                                                                     | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_asmm-lgm_all_full                                                                                 | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_asmm-lgm_all_full@tinawang123/os-autoinst-distri-opensuse#python                                  | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_base_all_minimal                                                                                  | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_base_all_minimal@Zaoliang/os-autoinst-distri-opensuse#reboot_after_installation-2020              | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_hpc_def_full                                                                                      | aarch64 | aarch64
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_hpc_def_full                                                                                      | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_hpc_def_full@lemon-suse/os-autoinst-distri-opensuse#Remove_workaround_bug_1040758                 | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_idu-ids_all_full                                                                                  | ppc64le | ppc64le
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_idu-ids_all_full@tinawang123/os-autoinst-distri-opensuse#ibmmodule                                | ppc64le | ppc64le
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_lp_all_full                                                                                       | ppc64le | ppc64le
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_lp_all_full                                                                                       | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_lp_all_full@lemon-suse/os-autoinst-distri-opensuse#Remove_workaound_bug1129504                    | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_phub_def_full                                                                                     | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_sdk_full                                                                                          | aarch64 | aarch64
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_sdk_full                                                                                          | ppc64le | ppc64le
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_sdk_full                                                                                          | x86_64  | 64bit_cirrus
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2           | offline_sles12sp5_pscc_sdk_full@tinawang123/os-autoinst-distri-opensuse#ibmmodule                                        | ppc64le | ppc64le
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2-Milestone | offline_sles12sp5_media_base_minimal                                                                                     | ppc64le | ppc64le
 sle    | 15-SP2  | Migration-from-SLE12-SP5-to-SLE15-SP2-Milestone | offline_sles12sp5_media_tcm-wsm_full                                                                                     | aarch64 | aarch64

DEBUG:/home/okurz/bin/openqa-db_query_last_use_of_module:call: ['ssh', 'openqa.opensuse.org', 'sudo -u geekotest psql -t --command="select distinct distri,version,flavor,test,arch,machine from jobs where id in (select job_id from job_modules where (name=\'snapper_rollback\')and t_updated > now() - interval \'90 days\') limit 30;" openqa']

so what I see as tasks: Have scenarios for openSUSE and SLE that ensure to boot an upgraded HDD image, boot_into_snapshot, rollback, reboot, check system can be booted with no code duplication in the different test modules. For this I think it makes sense to collaborate with "QA SLE Migration".

#11 Updated by szarate 9 months ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from Low to Normal
  • Target version changed from future to Milestone 31
  • Start date deleted (2015-11-17)
  • Estimated time set to 42.00 h

#12 Updated by zluo 5 months ago

  • Status changed from Workable to In Progress
  • Assignee set to zluo

let me check this at first.

#13 Updated by zluo 5 months ago

first of all: need to define which HDD should be used to upgrade or just run upgrade, then run boot_to_snapshot

1: pass UPGRADE=1 to bootloader to run upgrade installation, reboot into snapshot
2: run upgrade after boot_to_desktop and reboot into snapshot

#14 Updated by zluo 5 months ago

trying to use test zdup-Leap-15.1-gnome and add grub_test and boot_into_snapshot.

At moment I have problem with grub_test or reboot_gnome:

http://10.162.23.47/tests/7948#step/grub_test/3
http://10.162.23.47/tests/7950#step/reboot_gnome/11

#15 Updated by SLindoMansilla 5 months ago

  • Description updated (diff)

#16 Updated by SLindoMansilla 5 months ago

  • Subject changed from [opensuse][functional][u] Boot to snapshot after upgrade and then rollback to [functional][u][migration] Boot to snapshot after upgrade and then rollback

It is also for SLE. I haven't found any scenario that uses boot_into_snapshot on an upgraded SUT.

#17 Updated by SLindoMansilla 5 months ago

  • Description updated (diff)

#18 Updated by zluo 5 months ago

simply run boot_into_snapshot after upgrade doesn't work.

assert_screen 'linux-login', 200; breaks the test and assert_script_run('touch /etc/NOWRITE;test ! -f /etc/NOWRITE'); doesn't work for write only file system.

http://10.162.23.47/tests/8024 shows possible scenarios: add boot_desktop, opensuse_welcome

#19 Updated by zluo 5 months ago

https://progress.opensuse.org/issues/70588 makes problem because if zdup will be removed.

#20 Updated by zluo 5 months ago

so trying to get DVD upgrade scenarios like:

https://openqa.opensuse.org/tests/1373662

#21 Updated by zluo 5 months ago

created:
schedule/functional/boot_into_snapshot_after_upgrade.yaml

http://10.162.23.47/tests/8048#step/boot_into_snapshot looks good for now.

#22 Updated by zluo 5 months ago

PR created:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/10919

After merge I will add test suite on O3 and run verification.

#23 Updated by zluo 5 months ago

now think about how to create the same test for SLES

#24 Updated by zluo 5 months ago

http://10.162.23.47/tests/8084 for record, still need to reduce some modules f possible.

#25 Updated by zluo 5 months ago

http://10.162.23.47/tests/8095 looks good for now. YAML_SCHEDULE is used.

#27 Updated by zluo 5 months ago

the idea to get an upgrade by zdup from sle 12 to 15 seems to be not working atm:

http://10.162.23.47/tests/8099#step/zdup/17

check zdup upgrade from 15 sp1 to sp3 later:

https://openqa.suse.de/tests/4639727
http://10.162.23.47/tests/8100 (x86_64)

#28 Updated by zluo 5 months ago

zdup shows issue repositories:

http://10.162.23.47/tests/8100#step/zdup/4 shows old repos got disabled, but it still doesn't work for upgrade:
http://10.162.23.47/tests/8100#step/zdup/8 'Read error on old image'
Later I see:
Warning: You are about to do a distribution upgrade with all enabled repositories. Make sure these repositories are compatible before you continue. See 'man zypper' for more information about this command.

See http://10.162.23.47/tests/8099#step/zdup/8 (old image is sles 12 sp5). I think this has not to do with issue reported in poo#70588.
So we should use yaml schedule in my PR for sles: https://gitlab.suse.de/qsf-u/qa-sle-functional-userspace/-/merge_requests/103

#29 Updated by SLindoMansilla 4 months ago

  • Description updated (diff)

There is no more zdup scenario from Leap to Tumbleweed

#30 Updated by zluo 4 months ago

use BOOT_INTO_SNAPSHOT of grub_test, adapt lib/bootloader_setup.pm at function BOOT_INTO_SNAPSHOT:
http://10.162.23.47/tests/8135#step/grub_test/7

BOOT_INTO_SNAPSHOT_AFTER_UPGRADE, BOOT_TO_SNAPSHOT need to be used in setting, add required needles, check that boot_into_snapshot_after_upgrade works fully by assert_screen 'opensuse-welcome'

#31 Updated by zluo 4 months ago

http://10.162.23.47/tests/8142 (Boot_into_snapshot_after_upgrade)
http://10.162.23.47/tests/8143 (Roll_back_after_upgrade)

need to check changes in lib/bootloader_setup.pm doesn't break other tests which uses BOOT_INTO_SNAPSHOT in grub_test.

#35 Updated by zluo 4 months ago

PR re-worked and updated:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/10919/

After review and merge I'll add settings for 2 new tests

#37 Updated by zluo 4 months ago

http://10.162.23.47/tests/8199#step/boot_into_snapshot shows that boot_into_snapshot failed by using HDD_1 qcow2 image which got created from:
http://10.162.23.47/tests/8197

becaues UPGRADE=1 is used, the setting is used by bootloader to boot into an upgrade installation, but if we use BOOT_INTO_SNAPSHOT=1 in same test, this won't work (see http://10.162.23.47/tests/8188)

So it requires grub_test to boot into snapshot which actually works in first test example.
but boot_into_snapshot itself still fails.

#38 Updated by SLindoMansilla 4 months ago

zluo wrote:

http://10.162.23.47/tests/8199#step/boot_into_snapshot shows that boot_into_snapshot failed by using HDD_1 qcow2 image which got created from:
http://10.162.23.47/tests/8197

So it requires grub_test to boot into snapshot which actually works in first test example.
but boot_into_snapshot itself still fails.

It didn't. If grub_test worked, it wouldn't have shown the grub menu on boot_into_snapshot. Please, see a proper example of how grub_test was expected to behave on https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=opensuse&flavor=DVD&machine=64bit&test=boot_to_snapshot&version=Tumbleweed#step/boot_into_snapshot/1
In your example grub_test didn't managed to boot from the RO snapshot. So, boot_into_snapshot is waiting forever for the login prompt.

#39 Updated by zluo 4 months ago

qcow2 comes from upgrade installation with gnome desktop.
but https://openqa.opensuse.org/tests/1410329 uses qcow2 from textmode installation.
The workflow might be different in GRUB2 to get to boot up snapshot.

And on O3 we don't have upgrade installation for textmode. I can change function boot_into_snapshot in bootloader_setup.pm to make it boot into snapshot, but after desktop is reached, the current boot_into_snapshot is still expecting linux-login. And this needs to be adapted to desktop.

#40 Updated by SLindoMansilla 4 months ago

zluo wrote:

qcow2 comes from upgrade installation with gnome desktop.
but https://openqa.opensuse.org/tests/1410329 uses qcow2 from textmode installation.
The workflow might be different in GRUB2 to get to boot up snapshot.

And on O3 we don't have upgrade installation for textmode. I can change function boot_into_snapshot in bootloader_setup.pm to make it boot into snapshot, but after desktop is reached, the current boot_into_snapshot is still expecting linux-login. And this needs to be adapted to desktop.

Upgrade in textmode was done by zdup scenarios, which are not scheduled for O3 since some time now (I don't know why).
That doesn't matter, because boot_into_snapshot selects root-console, but it can only select the root-console if the system has booted, and the system can only be booted if current grub_test behaves as programmed.

Since we are now discussing about this for more than 2 days, it is more than clear to me that my proposed changes are necessary, because anyone in the future will also be confused and this will be a pain to maintain.

#41 Updated by SLindoMansilla 4 months ago

zluo wrote:

becaues UPGRADE=1 is used, the setting is used by bootloader to boot into an upgrade installation, but if we use BOOT_INTO_SNAPSHOT=1 in same test, this won't work (see http://10.162.23.47/tests/8188)

This is because in test suite boot_to_snapshot, the module grub_test is misused with a test setting to boot from a RO snapshot.
But, the problem is that we need the module grub_test twice (once after upgrade, and then once before rollback) and the behavior can only be one based on the setting.

#42 Updated by SLindoMansilla 4 months ago

  • Description updated (diff)

I have moved my comments from PR to the ticket description and put my suggestions as tasks to reduce further miscommunication.

#43 Updated by okurz 4 months ago

Upgrade in textmode was done by zdup scenarios, which are not scheduled for O3 since some time now (I don't know why).

Some zdup scenarios were recently removed for Tumbleweed as the rpm package format was changed so direct zdup migrations from older openSUSE or Leap do not work anymore. However zdup in general still works and should be tested especially for N-1 to N migrations or from currently supported Leap versions to Tumbleweed. If any are missing better cross-check with TW and Leap RMs to be sure

#44 Updated by zluo 4 months ago

http://10.162.23.47/tests/8227 works with boot_into_snapshot (new), but it is not working yet in:

http://10.162.23.47/tests/8230#step/boot_into_snapshot

#45 Updated by zluo 4 months ago

http://10.162.23.47/tests/8243 works so far.
http://10.162.23.47/tests/8238 (sles boot_into_snapshot)

#47 Updated by szarate 3 months ago

  • Blocks action #55877: [qe-core][functional] Describe the process on how we get a Leap scenario to a Tumbleweed added

#48 Updated by zluo 3 months ago

http://10.162.23.47/tests/8334#step/boot_into_snapshot/46 works for boot_into_snapshot (leap)
http://10.162.23.47/tests/8338 boot_into_snapshot after grub_test works as well.

But http://10.162.23.47/tests/8334#step/rollback_from_ro_snapshot/4 shows issue with rollback_from_ro_snapshot atm.

#50 Updated by zluo 3 months ago

http://10.162.23.47/tests/8344#step/rollback_from_ro_snapshot/3 shows issue with select_console.
checking why this is not working.

#51 Updated by zluo 3 months ago

rollback_from_ro_snapshot works now, but shutdown failed:
http://10.162.23.47/tests/8349#step/shutdown/2

#52 Updated by SLindoMansilla 3 months ago

zluo wrote:

rollback_from_ro_snapshot works now, but shutdown failed:
http://10.162.23.47/tests/8349#step/shutdown/2

Interesting, it was supposed to send_key('alt-f2') to open the desktop runner, but the system changed to tty2.
Normally ctrl-alt-f2 would be needed. But, maybe we have found a product bug for gnome shorcuts when rolling back.
I know that in old times, alt-fN was enough to switch to ttyN.

We need to investigate that manually on the SUT.

#53 Updated by zluo 3 months ago

debug shutdown issue:
http://10.162.23.47/tests/8378#live

vncviewer couperin.arch.suse.de:5992 -Shared

send_key 'alt-f2' is working and it can open desktop runner.

#54 Updated by zluo 3 months ago

PR updated again, grub_test got replaced by boot_into_desktop:

http://10.162.23.47/tests/8383

#55 Updated by zluo 3 months ago

  • Related to action #75295: [qe-core][opensuse][functional]test fails in shutdown added

#56 Updated by tjyrinki_suse 2 months ago

  • Subject changed from [functional][u][migration] Boot to snapshot after upgrade and then rollback to [qe-core][functional][migration] Boot to snapshot after upgrade and then rollback

#57 Updated by zluo 2 months ago

@SLindoMansilla
szarate

At moment we stuck with shutdown issue. Please advise further how to handle this issue, so I might be able to change or try your code suggestions, thanks!

#58 Updated by szarate 2 months ago

Zaoliang, please try doing the whole upgrade and then rollback without openQA (Manually), there has to be a difference between what openQA is doing and the SUT, which could mean a possible product bug, but we need to figure out whee is it.

#59 Updated by zluo 2 months ago

I did this already and I cannot find any issue for shutdown. I will check my PR code again, thanks!

#61 Updated by zluo 2 months ago

shutdown issue is not related to rollback because it is working during my debug session and this is not a product issue. The problem is that after reboot of leap we don't have gnome desktop. gdm is not crashed. There is only an issue with tty. For this I created #75295.

I added shutdown directly in rollback_from_ro_snapshot after reboot. This is working. So I hope we can go further for the ticket at least not to wait too long, thanks.

Please see http://10.162.23.47/tests/8495#step/rollback_from_ro_snapshot

PR is already updated.

#62 Updated by zluo 2 months ago

  • Status changed from In Progress to Feedback

#63 Updated by zluo about 2 months ago

test suite upgrade_boot-into_snapshot_rollback_Leap_15.1_gnome added on O3.
upgrade_boot-into_snapshot_rollback_Leap_15.1_gnome has been added now to job group.

check test result for next release of tw.

#64 Updated by zluo about 2 months ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF