action #34471
open[qe-core][functional][opensuse][medium] too early matching in too generic needle text-login-20160812
80%
Description
Observation¶
openQA test in scenario opensuse-42.3-Updates-x86_64-textmode@64bit fails in
zypper_up
[7 Apr 2018 21:01:23] <tacit> okurz: Somewhat more frequent unstable failure: https://openqa.opensuse.org/tests/652064#step/zypper_up/4 suspecting timing here as the root username is entered before the login promot
[7 Apr 2018 21:57:16] <okurz> tacit: well, problem is the not so good needle in https://openqa.opensuse.org/tests/652064#step/zypper_up/1 . as you can see on the '(i)' icon the test looks for tty6-selected and this is what we should use. What happens now that the test expects tt6, checks for just the generic needle, is happy with tty1, then the VM actually switches to tty6 but in the process the root name is already typed, only after that the getty is
[7 Apr 2018 21:57:16] <okurz> activated. But you know what, I think I have seen this slower behavior even manually. Was there some update to getty or systemd that could have caused a slow down here?
[8 Apr 2018 08:10:09] <okurz> tacit: So actually I would consider a product bug but try to remove our needles nevertheless. We can remove the needle but it might break some tests.
Reproducible¶
Fails since (at least) Build 20180407-3 (current job)
Expected result¶
- AC: There should be no needle with tag "text-login" anymore but only tty$nr-specific ones as we did already for SLE to prevent premature matching
Tasks¶
- Remove all needles with tag "text-login"
- Fix the tests that use that needle tag.
Suggestions¶
- Find current uses of the generic needles, e.g. "text-login-20160812" using https://openqa.opensuse.org/admin/needles
- Make sure there exist valid tty$nr-selected needles for all cases where text-login is matched on
- Delete the too generic needles
- Crosscheck carefully that all tests on o3 do not fail because of now deleted needles where there is no tty$nr-replacement.
Further details¶
Always latest result in this scenario: latest
See #33097 for the work that has been done for the corresponding SLE needles.
Updated by okurz about 6 years ago
- Due date changed from 2018-05-22 to 2018-05-08
- Priority changed from Normal to High
https://openqa.opensuse.org/tests/653247#step/firefox_audio/6 shows what I think is the same problem, in the post_fail_hook the needle text-login-20170427 is matching but we are not yet on the right tty, expecting tty5, seeing tty4. This prevents a proper analysis if the firefox_audio module itself is at fault or the system is bogged down due to something else.
Updated by okurz about 6 years ago
- Blocks action #33019: [opensuse][functional][sporadic][u][medium] Frequent fails in firefox_audio, firefox not started added
Updated by mgriessmeier about 6 years ago
- Subject changed from [functional][opensuse][u]too early matching in too generic needle text-login-20160812 to [functional][opensuse][u][medium] too early matching in too generic needle text-login-20160812
- Status changed from New to Workable
Updated by SLindoMansilla almost 6 years ago
- Status changed from Workable to In Progress
- Assignee set to SLindoMansilla
Updated by SLindoMansilla almost 6 years ago
Updated by SLindoMansilla almost 6 years ago
Scenarios that would fail without a needle "text-login".
Found on https://openqa.suse.de/admin/needles (search: text-login)¶
OSD
- virt-pvusb-sles12sp2-fv-on-developing-xen@64bit-ipmi
- gi-guest_sles12sp2-on-host-developing-kvm@64bit-ipmi
- gi-guest_developing-on-host_sles12sp1-kvm@64bit-ipmi
- prj2_host_upgrade_sles12sp3_to_developing_xen@64bit-ipmi
Found on https://openqa.opensuse.org/admin/needles (search: text-login)¶
O3
- zdup-13.1-gnome@64bit setup_zdup/3 Needle created, but need to track (tty-6-selected-20180425) (verified)
- Same job, but in other module consoletest_setup/33 (verified)
- textmode@64bit Needle created, but need to track (tty4-selected-20180425) (verified)
Updated by SLindoMansilla almost 6 years ago
Adapt code to not use generic needle text-login: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4942
Updated by okurz almost 6 years ago
- Related to action #34339: [qe-core][sle][functional][medium] test fails in consoletest_setup - generic text login matching too early, causing mistyping -- was " login prompt not ready to type - missing keys while typing login" added
Updated by SLindoMansilla almost 6 years ago
- Related to action #32926: [sle][functional][y][hyperv][medium] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful) added
Updated by mgriessmeier almost 6 years ago
- Due date changed from 2018-05-08 to 2018-05-22
Updated by okurz almost 6 years ago
Let's work on #32926 first, then continue here.
Updated by SLindoMansilla almost 6 years ago
- Status changed from In Progress to Workable
Waiting for https://progress.opensuse.org/issues/32926
and
Problems with IPMI machines:
- openqaw4-sp.qa.suse.de
- sonic-sp.qa.suse.de
szarate and nsinger are investigating the issue.
Updated by mgriessmeier almost 6 years ago
- Due date changed from 2018-05-22 to 2018-06-05
dependency resolved, this one is workable now
Updated by mgriessmeier almost 6 years ago
- Blocks deleted (action #33019: [opensuse][functional][sporadic][u][medium] Frequent fails in firefox_audio, firefox not started)
Updated by mgriessmeier almost 6 years ago
- Related to action #33019: [opensuse][functional][sporadic][u][medium] Frequent fails in firefox_audio, firefox not started added
Updated by jorauch almost 6 years ago
- Related to deleted (action #34339: [qe-core][sle][functional][medium] test fails in consoletest_setup - generic text login matching too early, causing mistyping -- was " login prompt not ready to type - missing keys while typing login")
Updated by jorauch almost 6 years ago
- Blocks action #34339: [qe-core][sle][functional][medium] test fails in consoletest_setup - generic text login matching too early, causing mistyping -- was " login prompt not ready to type - missing keys while typing login" added
Updated by mgriessmeier almost 6 years ago
- Due date changed from 2018-06-05 to 2018-06-19
- Assignee deleted (
SLindoMansilla) - Target version changed from Milestone 16 to Milestone 17
Updated by zluo almost 6 years ago
- Status changed from Workable to In Progress
check and replace assert_screen 'text_login' by wait_boot and select_console 'root-console':
sle-15-Installer-DVD-x86_64-Build665.2-external_iso@64bit
copy_iso_to_external_drive
post_run_hook contains assert_screen 'text_login' which needs to be changed
http://e13.suse.de/tests/2731#step/copy_iso_to_external_drive/29
looks good.
Updated by zluo almost 6 years ago
PR:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5206
and continue on other files...
Updated by zluo almost 6 years ago
PR for tests/virt_autotest/reboot_and_wait_up.pm:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5208
Updated by zluo almost 6 years ago
This is not working, so no change then:
http://e13.suse.de/tests/2742#step/reboot_and_wait_up_normal/7
Updated by zluo almost 6 years ago
checked tests/console/sle15_workarounds.pm as well, keep text-login because workaround is still needed.
Updated by zluo almost 6 years ago
lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60); no need to change, just keep it
Updated by zluo almost 6 years ago
lib/susedistribution.pm: assert_screen "text-login", 10;
is used in the live-cd accessing the "install-shell". That is used for example here:
https://openqa.opensuse.org/tests/688993#step/logs_from_installation_system/1¶
We keep if (LIVECD) part as well.
Updated by zluo almost 6 years ago
lib/susedistribution.pm: # Accept 'text-login' by default
remove the old comment.
Updated by zluo almost 6 years ago
Updated by okurz almost 6 years ago
zluo wrote:
checked tests/console/sle15_workarounds.pm as well, keep text-login because workaround is still needed.
zluo wrote:
lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60);no need to change, just keep it
Please adress my comment in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5206#issuecomment-396524370
Updated by zluo almost 6 years ago
tried to change tests/virt_autotest/reboot_and_wait_up.pm: assert_screen("text-login", 600); by using:
$self->wait_boot();
select_console 'root-console';
And change wait_boot function in lib/opensusebasetest.pm to avoid BACKEND impi run into GRUB:
elsif ((!(check_var('VIRSH_VMM_FAMILY', 'xen') && check_var('VIRSH_VMM_TYPE', 'linux') && check_var('BACKEND', 'svirt'))) || !(check_var('BACKEND', 'ipmi')))
This doesn't help:
http://e13.suse.de/tests/2749#step/reboot_and_wait_up_normal/6
Updated by zluo almost 6 years ago
at moment the ipmi worker is not stable and it skips many times my test runs :(
Updated by okurz almost 6 years ago
- Target version changed from Milestone 17 to Milestone 17
Updated by zluo almost 6 years ago
for ipmi use root-ssh instead of root-console and use PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5236 for another try:
Updated by zluo almost 6 years ago
it seems to have same issue for:
at /usr/lib/os-autoinst/backend/baseclass.pm line 80.
backend::baseclass::die_handler('The console isn\'t responding correctly. Maybe half-open sock...') called at /usr/lib/os-autoinst/backend/baseclass.pm line 241
eval {...} called at /usr/lib/os-autoinst/backend/baseclass.pm line 156
backend::baseclass::run_capture_loop('backend::ipmi=HASH(0x699cc40)') called at /usr/lib/os-autoinst/backend/baseclass.pm line 129
backend::baseclass::run('backend::ipmi=HASH(0x699cc40)', 5, 8) called at /usr/lib/os-autoinst/backend/driver.pm line 85
backend::driver::start('backend::driver=HASH(0x6493890)') called at /usr/lib/os-autoinst/backend/driver.pm line 48
backend::driver::new('backend::driver', 'ipmi') called at /usr/bin/isotovideo line 236
main::init_backend() called at /usr/bin/isotovideo line 305
Updated by zluo almost 6 years ago
the issue reported in (https://progress.opensuse.org/issues/32746) needs to revolved at first if we want to make changes in reboot_and_wait_up.pm
Updated by mgriessmeier almost 6 years ago
- Due date changed from 2018-06-19 to 2018-07-03
- Status changed from In Progress to Blocked
so it's basically blocked by https://progress.opensuse.org/issues/32746
Updated by mgriessmeier almost 6 years ago
- Blocked by action #32746: [sle][tools][remote-backends][hard] Incomplete job because console isn't responding correctly. Half-open socket on IPMI added
Updated by zluo almost 6 years ago
- % Done changed from 0 to 80
I think we are fine with the status of this ticket for now. Only reboot_and_wait_up.pm cannot be handled as we wish. So this can be worked on later when #32746 got resolved.
Updated by mgriessmeier almost 6 years ago
- Due date changed from 2018-07-03 to 2018-07-17
Updated by mgriessmeier almost 6 years ago
- Due date changed from 2018-07-17 to 2018-07-31
- Status changed from Blocked to Feedback
Updated by okurz almost 6 years ago
- Due date deleted (
2018-07-31) - Status changed from Feedback to Workable
- Assignee deleted (
zluo) - Target version changed from Milestone 17 to Milestone 21+
ok, so we have covered a lot already. As we understood in SR zluo is not currently working on this at the moment. We should revisit the situation in some months again.
Updated by okurz over 5 years ago
- Priority changed from High to Normal
still valid but not "High" prio.
Updated by okurz over 5 years ago
- Target version changed from Milestone 21+ to future
Updated by okurz about 5 years ago
I found that there is "linux-login" as well as "text-login". I assume both are equivalent and should be merged.
$ git grep '\(text\|linux\)-login\>'
lib/migration.pm: if (!check_screen('linux-login', 200)) {
lib/opensusebasetest.pm: push @tags, 'linux-login' if get_var('KEEP_GRUB_TIMEOUT'); # Also wait for linux-login if grub timeout was not disabled
lib/opensusebasetest.pm: elsif (!match_has_tag("grub2") and !match_has_tag('linux-login')) {
lib/opensusebasetest.pm: my $failneedle = get_var('KEEP_GRUB_TIMEOUT') ? 'linux-login' : 'grub2';
lib/opensusebasetest.pm: my $textmode_needles = [qw(linux-login emergency-shell emergency-mode)];
lib/susedistribution.pm: assert_screen "text-login", 10;
lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60);
tests/boot/boot_from_pxe.pm: assert_screen 'linux-login', 100;
tests/console/copy_iso_to_external_drive.pm: assert_screen "text-login";
tests/console/sle15_workarounds.pm: assert_screen(["tty2-selected", 'text-login', 'text-logged-in-root', 'generic-desktop']);
tests/installation/boot_into_snapshot.pm: assert_screen 'linux-login', 200;
tests/installation/first_boot.pm: assert_screen('linux-login', $boot_timeout) unless check_var('ARCH', 's390x');
tests/installation/grub_test.pm: # If grub timeout was not disabled, we wait for linux-login instead
tests/installation/grub_test.pm: my $tag = get_var('KEEP_GRUB_TIMEOUT') ? 'linux-login' : 'grub2';
tests/installation/reboot_eject_cd.pm: assert_screen('text-login');
tests/jeos/firstrun.pm: assert_screen_with_soft_timeout('linux-login', timeout => 1000, soft_timeout => 300, bugref => 'bsc#1077007');
tests/locale/keymap_or_locale.pm: assert_screen([qw(linux-login cleared-console)]);
tests/locale/keymap_or_locale.pm: assert_screen([qw(linux-login cleared-console)]);
tests/qa_automation/kernel_kexec.pm: assert_screen('linux-login', 300);
tests/qam-minimal/install_update.pm: assert_screen 'linux-login';
tests/sles4sap/netweaver_ascs_install.pm: text-login linux-login started-x-displaymanager-info)
tests/virt_autotest/login_console.pm: send_key_until_needlematch(['linux-login', 'virttest-displaymanager'], 'ret', $timeout / 5, 5);
tests/virt_autotest/reboot_and_wait_up.pm: assert_screen("text-login", 600);
tests/virtualization/prepare.pm: assert_screen 'linux-login', 600;
tests/virtualization/xen/patch_and_reboot.pm: send_key_until_needlematch('text-login', 'ret', 360, 5);
So let's take a look for places where we have "text-login" and decide which are ok to keep or not
$ git grep '\<text-login\>'
lib/susedistribution.pm: assert_screen "text-login", 10;
lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60);
tests/console/copy_iso_to_external_drive.pm: assert_screen "text-login";
tests/console/sle15_workarounds.pm: assert_screen(["tty2-selected", 'text-login', 'text-logged-in-root', 'generic-desktop']);
tests/installation/reboot_eject_cd.pm: assert_screen('text-login');
tests/sles4sap/netweaver_ascs_install.pm: text-login linux-login started-x-displaymanager-info)
tests/virt_autotest/reboot_and_wait_up.pm: assert_screen("text-login", 600);
- 1. [ ]
lib/susedistribution.pm: assert_screen "text-login", 10;
is used in the live-cd accessing the "install-shell". That is used for example here:
https://openqa.opensuse.org/tests/688993#step/logs_from_installation_system/1
Could you please try to just remove that if (LIVECD) part?
- 2. [x]
lib/susedistribution.pm: # Accept 'text-login' by default
should just be removed -> done with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5219
- 3. [ ]
lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60);
is within $type eq 'ssh' and I do not know if we still need that. Please take a look in used needles in scenarios using ssh and check where this part of the code is covered.
- 4. [ ]
tests/console/sle15_workarounds.pm: assert_screen(["tty2-selected", 'text-login', 'text-logged-in-root', 'generic-desktop']);
Please discuss with test module maintainer @rwx788 how to proceed with this test module. The bug for which this code is used as workaround is still valid and open. One solution I could think if is renaming the test module. Maybe we do not need the text-login needle here but can just work with the tty2-selected needle although explicitly mentioning the tty number is not so good. Please look into extracting the current console number from the root console object.
- 5. [ ]
tests/installation/reboot_eject_cd.pm: assert_screen('text-login');
is used for example in https://openqa.opensuse.org/tests/677992#step/reboot_eject_cd/1 . I guess you can simply replace text-login by tty3-selected because one line above there is a send_key 'ctrl-alt-f3'
- 6. [ ]
tests/sles4sap/netweaver_ascs_install.pm: text-login linux-login started-x-displaymanager-info)
should be replaced by wait_boot
- 7. [ ]
tests/virt_autotest/reboot_and_wait_up.pm: assert_screen("text-login", 600);
same, as already discussed yesterday, to be replaced by wait_boot
Updated by zluo over 4 years ago
- Status changed from Workable to In Progress
- Assignee set to zluo
- Target version changed from future to Milestone 26
take over and checking current status.
Updated by zluo over 4 years ago
to check this later:
http://f40.suse.de/tests/4821#next_previous
Updated by zluo over 4 years ago
- Status changed from In Progress to Rejected
this is not an issue anymore:
http://f40.suse.de/tests/4869#next_previous
Updated by okurz over 4 years ago
- Status changed from Rejected to In Progress
Why do you think this is not an issue anymore when we still haven't fulfilled the requirements we have set on ourselves? Please read the ticket description and comments again.
Updated by zluo over 4 years ago
- Assignee deleted (
zluo)
since the original issue doesn't happen anymore and this is not a priority at moment, I won't work on it.
Updated by mgriessmeier over 4 years ago
- Status changed from Workable to New
- Priority changed from Normal to High
- Target version changed from Milestone 26 to Milestone 27
discuss in grooming again ...
Updated by SLindoMansilla over 4 years ago
- Description updated (diff)
- Status changed from New to Workable
- Priority changed from High to Normal
Updated by mgriessmeier over 4 years ago
- Target version changed from Milestone 27 to Milestone 28
Updated by okurz over 4 years ago
- Related to action #36126: [functional][u] post_fail_hook matches on "text_login_root" before actual tty switch and therefore never logs in added
Updated by zluo over 4 years ago
- Status changed from Workable to In Progress
- Assignee set to zluo
take over and check if this is a similar issue because I'm working on https://progress.opensuse.org/issues/36126
Updated by zluo over 4 years ago
found still some text-login:
text-login-20130711.json: "text-login"
text-login-20160812.json: "text-login"
text-login-20170427.json: "text-login"
text-login-20170504.json: "text-login"
We still have following test modules:
lib/sles4sap.pm: text-login linux-login started-x-displaymanager-info)
lib/susedistribution.pm: assert_screen "text-login", 10;
lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60);
tests/console/copy_iso_to_external_drive.pm: assert_screen "text-login";
tests/console/sle15_workarounds.pm: assert_screen(["tty2-selected", 'text-login', 'text-logged-in-root', 'generic-desktop']);
tests/installation/reboot_eject_cd.pm: assert_screen('text-login');
tests/systemd_testsuite/test_09_issue_2691.pm: send_key_until_needlematch('text-login', 'ret', 360, 5);
tests/virt_autotest/reboot_and_wait_up.pm: assert_screen("text-login", 600);
above test modules are tested for sle but not for opensuse.
To compare with #46 and consider to how to process with this ticket again.
Updated by zluo over 4 years ago
found:
lib/caasp.pm: assert_screen 'linux-login-casp', 150;
lib/caasp.pm: # Don't match linux-login-casp twice
lib/caasp.pm: assert_screen [qw(grub2 linux-login-casp)], 150;
lib/locale.pm: assert_screen([qw(linux-login cleared-console)]);
lib/locale.pm: assert_screen(has_ttys() ? 'linux-login' : 'cleared_console');
lib/migration.pm:# Assert screen 'linux-login' with 200s
lib/migration.pm: if (!check_screen('linux-login', 200)) {
lib/opensusebasetest.pm: my $textmode_needles = [qw(linux-login emergency-shell emergency-mode)];
lib/opensusebasetest.pm: # 2nd stage of autoyast can be considered as linux-login
lib/sles4sap.pm: text-login linux-login started-x-displaymanager-info)
lib/systemd_testsuite_test.pm: assert_screen('linux-login', 300);
lib/transactional.pm: assert_screen 'linux-login', 200;
tests/autoyast/installation.pm: push @needles, 'linux-login-casp' if is_caasp;
tests/autoyast/installation.pm: || match_has_tag('linux-login-casp')
tests/boot/boot_from_pxe.pm: assert_screen 'linux-login', 100;
tests/boot/boot_linuxrc.pm: assert_screen([qw(linux-login displaymanager generic-desktop)], 180);
tests/boot/grub2_test.pm: assert_screen 'linux-login', 200;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 300;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 200;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 100;
tests/installation/boot_into_snapshot.pm: assert_screen 'linux-login', 200;
tests/installation/first_boot.pm: assert_screen('linux-login', $boot_timeout) unless check_var('ARCH', 's390x');
tests/installation/ipxe_install.pm: assert_screen('linux-login', 1800);
tests/jeos/firstrun.pm: assert_screen [qw(linux-login reached-power-off)], 1000;
tests/jeos/prepare_firstboot.pm: assert_screen('linux-login', 300);
tests/kernel/boot_ltp.pm: assert_screen('linux-login', 1800);
tests/kernel/build_git_kernel.pm: assert_screen('linux-login', 1800);
tests/kernel/kernel_kexec.pm: # clear console to prevent linux-login to match before reboot
tests/kernel/kernel_kexec.pm: assert_screen('linux-login', 300);
tests/kernel/mellanox_config.pm: assert_screen('linux-login', 1800);
tests/kernel_performance/run_perf_case.pm: assert_screen('linux-login', $timeout);
tests/kiwi_images_test/kiwi_boot.pm: assert_screen('linux-login', 1200);
tests/kiwi_images_test/kiwi_boot.pm: assert_screen('linux-login', 1000);
tests/kiwi_images_test/login_reboot.pm: assert_screen('linux-login', 120);
tests/kiwi_images_test/validate_build.pm: assert_screen('linux-login', 120);
tests/locale/keymap_or_locale.pm: assert_screen([qw(linux-login cleared-console)]);
tests/qam-minimal/install_update.pm: assert_screen 'linux-login', 90;
tests/virt_autotest/login_console.pm: send_key_until_needlematch(['linux-login', 'virttest-displaymanager'], 'ret', $counter, $timeout);
tests/virt_autotest/login_console.pm: assert_screen('will-linux-login', $timeout);
tests/virt_autotest/login_console.pm: last if (check_screen([qw(linux-login virttest-displaymanager)], 60));
tests/virtualization/prepare.pm: assert_screen 'linux-login', 600;
tag 'linux-login' found from needles:
linux-login-20140916.json: "linux-login"
linux-login-autoyast-20150724.json: "linux-login"
linux-login-autoyast-20160822.json: "linux-login"
linux-login-autoyast-20191126.json: "linux-login",
linux-login-kubic-20180201.json: "linux-login-casp"
linux-login-kubic-20190408.json: "linux-login-casp"
linux-login-kubic-20190415.json: "linux-login-casp"
linux-login-kubic-20191031.json: "linux-login-casp"
Updated by zluo over 4 years ago
http://f40.suse.de/tests/5546#step/zypper_up/1 (JeOS-for-kvm-and-xen-x86_64-Build17.7-jeos@64bit_virtio-2G)
http://f40.suse.de/tests/5547#step/zypper_up/1 (Tumbleweed-NET-x86_64-Build20191126-textmode@64bit)
http://f40.suse.de/tests/5548#step/zypper_up/1 (opensuse-15.1-DVD-Updates-x86_64-Build20191127-2-textmode@64bit)
http://f40.suse.de/tests/5549#step/zypper_up/1 (opensuse-15.0-DVD-Updates-x86_64-Build20191127-2-textmode@64bit)
show that root-console-20170321 is used. We don't have problem at moment for opensuse. And we can delete text-login from opensuse needle repository.
Updated by zluo over 4 years ago
needle PR for opensuse:
https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/617
https://openqa.opensuse.org/tests/1096840#step/reboot_eject_cd/1 still use text-login
will update this.
Updated by zluo over 4 years ago
http://f40.suse.de/tests/5551#step/zypper_up/41 shows network issue and triggers later post_fail_hook: http://f40.suse.de/tests/5551#step/zypper_up/41
tty5-selected is then correctly use now. So we are safe to remove text-login.
Updated by zluo over 4 years ago
Updated by zluo over 4 years ago
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9014 looks good on o3.
Updated by zluo over 4 years ago
Now checking linux-login tag which is used by opensuse:
lib/caasp.pm: assert_screen 'linux-login-casp', 150;
lib/caasp.pm: # Don't match linux-login-casp twice
lib/caasp.pm: assert_screen [qw(grub2 linux-login-casp)], 150;
lib/locale.pm: assert_screen([qw(linux-login cleared-console)]);
lib/locale.pm: assert_screen(has_ttys() ? 'linux-login' : 'cleared_console');
lib/migration.pm:# Assert screen 'linux-login' with 200s
lib/migration.pm: if (!check_screen('linux-login', 200)) {
lib/opensusebasetest.pm: my $textmode_needles = [qw(linux-login emergency-shell emergency-mode)];
lib/opensusebasetest.pm: # 2nd stage of autoyast can be considered as linux-login
lib/sles4sap.pm: text-login linux-login started-x-displaymanager-info)
lib/systemd_testsuite_test.pm: assert_screen('linux-login', 300);
lib/transactional.pm: assert_screen 'linux-login', 200;
tests/autoyast/installation.pm: push @needles, 'linux-login-casp' if is_caasp;
tests/autoyast/installation.pm: || match_has_tag('linux-login-casp')
tests/boot/boot_from_pxe.pm: assert_screen 'linux-login', 100;
tests/boot/boot_linuxrc.pm: assert_screen([qw(linux-login displaymanager generic-desktop)], 180);
tests/boot/grub2_test.pm: assert_screen 'linux-login', 200;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 300;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 200;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 100;
tests/installation/boot_into_snapshot.pm: assert_screen 'linux-login', 200;
tests/installation/first_boot.pm: assert_screen('linux-login', $boot_timeout) unless check_var('ARCH', 's390x');
tests/installation/ipxe_install.pm: assert_screen('linux-login', 1800);
tests/jeos/firstrun.pm: assert_screen [qw(linux-login reached-power-off)], 1000;
tests/jeos/prepare_firstboot.pm: assert_screen('linux-login', 300);
tests/kernel/boot_ltp.pm: assert_screen('linux-login', 1800);
tests/kernel/build_git_kernel.pm: assert_screen('linux-login', 1800);
tests/kernel/kernel_kexec.pm: # clear console to prevent linux-login to match before reboot
tests/kernel/kernel_kexec.pm: assert_screen('linux-login', 300);
tests/kernel/mellanox_config.pm: assert_screen('linux-login', 1800);
tests/kernel_performance/run_perf_case.pm: assert_screen('linux-login', $timeout);
tests/kiwi_images_test/kiwi_boot.pm: assert_screen('linux-login', 1200);
tests/kiwi_images_test/kiwi_boot.pm: assert_screen('linux-login', 1000);
tests/kiwi_images_test/login_reboot.pm: assert_screen('linux-login', 120);
tests/kiwi_images_test/validate_build.pm: assert_screen('linux-login', 120);
tests/locale/keymap_or_locale.pm: assert_screen([qw(linux-login cleared-console)]);
tests/qam-minimal/install_update.pm: assert_screen 'linux-login', 90;
tests/virt_autotest/login_console.pm: send_key_until_needlematch(['linux-login', 'virttest-displaymanager'], 'ret', $counter, $timeout);
tests/virt_autotest/login_console.pm: assert_screen('will-linux-login', $timeout);
tests/virt_autotest/login_console.pm: last if (check_screen([qw(linux-login virttest-displaymanager)], 60));
tests/virtualization/prepare.pm: assert_screen 'linux-login', 600;
--
found: first_boot.pm, firstrun.pm, keymap_or_locale.pm are valid for opensuse tw.
Need to check these test modules for sle and create required needle 'tty$Nr-selected' as well!
Updated by zluo over 4 years ago
http://f40.suse.de/tests/5567 done for first_boot and keymap_or_locale.
Updated by zluo over 4 years ago
http://f40.suse.de/tests/5574#step/firstrun/7 shows successful test run with a new needle 'tty6-selected'.
Updated by zluo over 4 years ago
Updated by zluo over 4 years ago
for sle-15-SP2-Online-x86_64-Build93.1-gnome@64bit:
http://f40.suse.de/tests/5576
first_boot and keymap_or_locale are not effected by code changes:
elsif ((get_var('DESKTOP', '') =~ /textmode|serverro/) || get_var('BOOT_TO_SNAPSHOT')) {
my $tty = get_root_console_tty;
assert_screen("tty$tty-selected", $boot_timeout) unless check_var('ARCH', 's390x');
return;
}
and
assert_screen([qw("tty$tty-selected" cleared-console)]);
Updated by dimstar over 4 years ago
- Priority changed from Normal to Urgent
zluo wrote:
needle PR: https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/619
This has been merged and the live installer in Tumbleweed are now broken
Updated by okurz over 4 years ago
I have merged a revert https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/620 for now. Please see my comment https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/617#issuecomment-559749693
Updated by zluo over 4 years ago
PR to handle linux-login:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034
Updated by zluo over 4 years ago
test_09_issue_2691:
http://f40.suse.de/tests/5585#step/test_09_issue_2691/1 shows that needle match and it has tags: text-login and tty6-selected
Updated by zluo over 4 years ago
installation_overview.pm still uses tag 'text-login', need to check where this is coming from.
it comes from install-shell - > text-login. Change it to tty2-selected:
Updated by zluo over 4 years ago
clone https://openqa.suse.de/tests/3641764 for change 'tty1-selected' in tests/virt_autotest/reboot_and_wait_up.pm
Updated by zluo over 4 years ago
Updated by zluo over 4 years ago
- Status changed from In Progress to Feedback
- Priority changed from Urgent to High
Updated by mgriessmeier over 4 years ago
- Target version changed from Milestone 28 to Milestone 30
needs to be discussed offline
Updated by okurz over 4 years ago
- Related to action #57281: [sle][Migration][SLE15SP2] test fails in orphaned_packages_check - switch to tty failed added
Updated by zluo about 4 years ago
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9048 merged now.
checking now old PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034
first_boot.pm got changed and it uses now function wait_boot_past_bootloader and verify this change by test runs:
diff --git a/lib/opensusebasetest.pm b/lib/opensusebasetest.pm
index 08648c884..6b5ee125b 100644
--- a/lib/opensusebasetest.pm
+++ b/lib/opensusebasetest.pm
@@ -806,7 +806,7 @@ sub wait_boot_textmode {
return if check_var('ARCH', 's390x');
my $ready_time = $args{ready_time};
- my $textmode_needles = [qw(linux-login emergency-shell emergency-mode)];
+ my $textmode_needles = [qw(tty1-selected emergency-shell emergency-mode)];
# 2nd stage of autoyast can be considered as linux-login
push @{$textmode_needles}, 'autoyast-init-second-stage' if get_var('AUTOYAST');
# Soft-fail for user_defined_snapshot in extra_tests_on_gnome and extra_tests_on_gnome_on_ppc
Updated by zluo about 4 years ago
Updated by xlai about 4 years ago
zluo wrote:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 updated now.
@zluo @okurz Would really appreciate if we can be involved in solution discussion and PR review earlier if next time such big influence changes are made. Our tests in latest build 134.1 fail a lot at matching tty1-selected in https://openqa.nue.suse.com/tests/overview?distri=sle&version=15-SP2&build=134.1&groupid=263.
Updated by okurz about 4 years ago
xlai wrote:
zluo wrote:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 updated now.
@zluo @okurz Would really appreciate if we can be involved in solution discussion and PR review earlier if next time such big influence changes are made.
I didn't do any changes within the last year. I only participated in review and merged a revert so far. The above mentioned PR was already reverted in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9473 so maybe a retrigger should work. In general you can participate in the solution discussion since two years (!) but I did not see any comment by you in this ticket. Also the PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 was merged after being open for more than 2 months (!) so you could have looked into the PR for a very long time already. Expecting anyone would decide when to personally ping you or not on indvidual pull requests is an approach which can be used in exceptions but it isn't going to scale well. IMHO the pull request being open for two months is more than enough. However I grant you that the PR has a lacking description and was ill-prepared and is missing a lot of verification runs. Early review can all help with this.
EDIT: sorry, meant to cross-check mainly IPMI, not (only) ssh
Updated by xlai about 4 years ago
okurz wrote:
xlai wrote:
zluo wrote:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 updated now.
@zluo @okurz Would really appreciate if we can be involved in solution discussion and PR review earlier if next time such big influence changes are made.
I didn't do any changes within the last year. I only participated in review and merged a revert so far. The above mentioned PR was already reverted in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9473 so maybe a retrigger should work.
I at you because you are the creator of this ticket and I thought you created the "expected results" and "tasks" , which showed the solution. Hope you do not mind :).
In general you can participate in the solution discussion since two years (!) but I did not see any comment by you in this ticket. Also the PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 was merged after being open for more than 2 months (!) so you could have looked into the PR for a very long time already. Expecting anyone would decide when to personally ping you or not on indvidual pull requests is an approach which can be used in exceptions but it isn't going to scale well. IMHO the pull request being open for two months is more than enough. However I grant you that the PR has a lacking description and was ill-prepared and is missing a lot of verification runs. Early review can all help with this.
Yes, you are right. But looking into all open PRs and reviewing them will take a lot of time. It is quite expensive to be aware of the big changes from this PR review way. It is a common challenge for most openqa test code maintainers.
@zluo @SLindoMansilla Introducing a check for "tty1" is IMHO a pretty bad idea and far from the original idea of this ticket. Our tests should stay generic and keep systems in mind that do not have tty's, e.g. remote systems where we login over ssh.
I am afraid so. There should be a better way to solve the original ticket. Now virtualization tests are blocked a lot. And the influence of replacing such general needle "linux-login" should be much bigger.
Updated by okurz about 4 years ago
I at you because you are the creator of this ticket and I thought you created the "expected results" and "tasks" , which showed the solution. Hope you do not mind :).
I think you did good. Just wanted to put in perspective what my current impact is :)
Yes, you are right. But looking into all open PRs and reviewing them will take a lot of time. It is quite expensive to be aware of the big changes from this PR review way. It is a common challenge for most openqa test code maintainers.
I understand that. Basically I see three different choices as of now:
- Review PRs, at best before merge
- Investigate which test changes can explain a test fail. This is also now provided in the "Investigation" tab of each test
Updated by SLindoMansilla about 4 years ago
Suggestion:
- create a new needle covering the ttyS1 terminal with tag 'ttyS1-selected'.
- Or revert this specific change and leave text-login as general needle.
Updated by zluo about 4 years ago
since PR for linux-login got reverted, and I created now a new PR to fix issue:
https://openqa.nue.suse.com/tests/3853054
This should be safe from now with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9487 (still need verification run).
Updated by zluo about 4 years ago
https://openqa.suse.de/tests/3858631#step/reboot_and_wait_up_normal/4 looks good. The new needle created contains 2 tags: ttys1-selected and tty1-selected
Updated by zluo about 4 years ago
delete ttys1-selected:
https://gitlab.suse.de/openqa/os-autoinst-needles-sles/merge_requests/1318
and created new PR to fix the issue for reboot_ans_wait_up:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9487
Updated by zluo about 4 years ago
since my PR got reverted again, we can do this now:
Or revert this specific change and leave text-login as general needle
However with a new needle it fixed the issue, but actually this not easy to cover all cases which text-login still used widely for other tests/team, maybe this is better, don't to try this for now till we have a better idea.
See example:
https://149.44.176.58/tests/3869862#step/reboot_and_wait_up_normal/4
Updated by zluo about 4 years ago
- Status changed from Feedback to Workable
- Assignee deleted (
zluo)
since my PR got reverted again, we can do this now:
Or revert this specific change and leave text-login as general needle
However with a new needle it fixed the issue, but actually this not easy to cover all cases where text-login is still used widely. Maybe this is better, don't to try this for now till we have a "final" solution.
Updated by xlai about 4 years ago
This change still fails 3 tests in virtualization-acceptance job group, https://openqa.nue.suse.com/tests/overview?distri=sle&version=15-SP2&build=136.2&groupid=263. It also fails several tests in virtualization-milestone group, although some are still scheduled, https://openqa.nue.suse.com/tests/overview?distri=sle&version=15-SP2&build=136.2&groupid=264.
So due to this change, we can not deliver the public beta candidate acceptance test report in time.
If a full support can not be done, I strongly suggest a full revert of all PRs, rather than partially revert and partially keep. @mgressimier @zluo, how do you think?
Updated by SLindoMansilla about 4 years ago
- Status changed from Workable to New
To be discussed again
Updated by SLindoMansilla about 4 years ago
- Priority changed from High to Normal
Updated by okurz about 4 years ago
- Category changed from Bugs in existing tests to Enhancement to existing tests
Actually the original issue is fixed since long but the enhancement as mentioned in the subject is still valid.
Updated by tjyrinki_suse over 3 years ago
- Subject changed from [functional][opensuse][u][medium] too early matching in too generic needle text-login-20160812 to [qe-core][functional][opensuse][medium] too early matching in too generic needle text-login-20160812
Updated by okurz over 2 years ago
I came to this ticket due to periodically reviewing tickets as described on https://progress.opensuse.org/projects/openqatests/wiki#How-we-work-on-tickets
This ticket was set to "Normal" priority but was not updated within the SLO period for "Normal" tickets (365 days) as described on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#SLOs-service-level-objectives
First reminder: Please consider picking up this ticket within the next 365 days or just set the ticket to the next lower priority of "Low" (no SLO related time period).
Updated by slo-gin over 1 year ago
This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.
Updated by slo-gin about 1 month ago
This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.