Project

General

Profile

Actions

action #34471

closed

[qe-core][functional][opensuse][medium] too early matching in too generic needle text-login-20160812

Added by okurz over 6 years ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Enhancement to existing tests
Target version:
SUSE QA - Milestone 30
Start date:
2018-04-08
Due date:
% Done:

80%

Estimated time:
42.00 h
Difficulty:
medium

Description

Observation

openQA test in scenario opensuse-42.3-Updates-x86_64-textmode@64bit fails in
zypper_up

[7 Apr 2018 21:01:23] <tacit> okurz: Somewhat more frequent unstable failure: https://openqa.opensuse.org/tests/652064#step/zypper_up/4    suspecting timing here as the root username is entered before the login promot
[7 Apr 2018 21:57:16] <okurz> tacit: well, problem is the not so good needle in https://openqa.opensuse.org/tests/652064#step/zypper_up/1 . as you can see on the '(i)' icon the test looks for tty6-selected and this is what we should use. What happens now that the test expects tt6, checks for just the generic needle, is happy with tty1, then the VM actually switches to tty6 but in the process the root name is already typed, only after that the getty is 
[7 Apr 2018 21:57:16] <okurz> activated. But you know what, I think I have seen this slower behavior even manually. Was there some update to getty or systemd that could have caused a slow down here?
[8 Apr 2018 08:10:09] <okurz> tacit: So actually I would consider a product bug but try to remove our needles nevertheless. We can remove the needle but it might break some tests.

Reproducible

Fails since (at least) Build 20180407-3 (current job)

Expected result

  • AC: There should be no needle with tag "text-login" anymore but only tty$nr-specific ones as we did already for SLE to prevent premature matching

Tasks

  1. Remove all needles with tag "text-login"
  2. Fix the tests that use that needle tag.

Suggestions

  • Find current uses of the generic needles, e.g. "text-login-20160812" using https://openqa.opensuse.org/admin/needles
  • Make sure there exist valid tty$nr-selected needles for all cases where text-login is matched on
  • Delete the too generic needles
  • Crosscheck carefully that all tests on o3 do not fail because of now deleted needles where there is no tty$nr-replacement.

Further details

Always latest result in this scenario: latest

See #33097 for the work that has been done for the corresponding SLE needles.


Related issues 6 (1 open5 closed)

Related to openQA Tests - action #32926: [sle][functional][y][hyperv][medium] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful)Resolvedokurz2018-03-082018-05-22

Actions
Related to openQA Tests - action #33019: [opensuse][functional][sporadic][u][medium] Frequent fails in firefox_audio, firefox not startedResolvedjorauch2018-03-092018-05-22

Actions
Related to openQA Tests - action #36126: [functional][u] post_fail_hook matches on "text_login_root" before actual tty switch and therefore never logs inResolvedzluo2018-05-14

Actions
Related to openQA Tests - action #57281: [sle][Migration][SLE15SP2] test fails in orphaned_packages_check - switch to tty failedResolvedhjluo2019-09-24

Actions
Blocks openQA Tests - action #34339: [qe-core][sle][functional][medium] test fails in consoletest_setup - generic text login matching too early, causing mistyping -- was " login prompt not ready to type - missing keys while typing login"Blockedzluo2018-04-05

Actions
Blocked by openQA Tests - action #32746: [sle][tools][remote-backends][hard] Incomplete job because console isn't responding correctly. Half-open socket on IPMIResolvedokurz2018-03-05

Actions
Actions #1

Updated by okurz over 6 years ago

  • Description updated (diff)
Actions #2

Updated by okurz over 6 years ago

  • Due date changed from 2018-05-22 to 2018-05-08
  • Priority changed from Normal to High

https://openqa.opensuse.org/tests/653247#step/firefox_audio/6 shows what I think is the same problem, in the post_fail_hook the needle text-login-20170427 is matching but we are not yet on the right tty, expecting tty5, seeing tty4. This prevents a proper analysis if the firefox_audio module itself is at fault or the system is bogged down due to something else.

Actions #3

Updated by okurz over 6 years ago

  • Blocks action #33019: [opensuse][functional][sporadic][u][medium] Frequent fails in firefox_audio, firefox not started added
Actions #4

Updated by mgriessmeier over 6 years ago

  • Subject changed from [functional][opensuse][u]too early matching in too generic needle text-login-20160812 to [functional][opensuse][u][medium] too early matching in too generic needle text-login-20160812
  • Status changed from New to Workable
Actions #5

Updated by cwh over 6 years ago

  • Difficulty set to medium
Actions #6

Updated by SLindoMansilla over 6 years ago

  • Status changed from Workable to In Progress
  • Assignee set to SLindoMansilla
Actions #8

Updated by SLindoMansilla over 6 years ago

Scenarios that would fail without a needle "text-login".

Found on https://openqa.suse.de/admin/needles (search: text-login)

OSD

Found on https://openqa.opensuse.org/admin/needles (search: text-login)

O3

Actions #9

Updated by SLindoMansilla over 6 years ago

Adapt code to not use generic needle text-login: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4942

Actions #10

Updated by okurz over 6 years ago

  • Related to action #34339: [qe-core][sle][functional][medium] test fails in consoletest_setup - generic text login matching too early, causing mistyping -- was " login prompt not ready to type - missing keys while typing login" added
Actions #11

Updated by SLindoMansilla over 6 years ago

  • Related to action #32926: [sle][functional][y][hyperv][medium] avoid typing username before switched tty (was: test fails in yast2_i - (mising needles?, rather too low timeout for hyperv) for Installation Report succesful) added
Actions #12

Updated by mgriessmeier over 6 years ago

  • Due date changed from 2018-05-08 to 2018-05-22
Actions #13

Updated by okurz over 6 years ago

Let's work on #32926 first, then continue here.

Actions #14

Updated by SLindoMansilla over 6 years ago

  • Status changed from In Progress to Workable

Waiting for https://progress.opensuse.org/issues/32926

and

Problems with IPMI machines:

  • openqaw4-sp.qa.suse.de
  • sonic-sp.qa.suse.de

szarate and nsinger are investigating the issue.

Actions #15

Updated by mgriessmeier over 6 years ago

  • Due date changed from 2018-05-22 to 2018-06-05

dependency resolved, this one is workable now

Actions #16

Updated by mgriessmeier over 6 years ago

  • Blocks deleted (action #33019: [opensuse][functional][sporadic][u][medium] Frequent fails in firefox_audio, firefox not started)
Actions #17

Updated by mgriessmeier over 6 years ago

  • Related to action #33019: [opensuse][functional][sporadic][u][medium] Frequent fails in firefox_audio, firefox not started added
Actions #18

Updated by jorauch over 6 years ago

  • Related to deleted (action #34339: [qe-core][sle][functional][medium] test fails in consoletest_setup - generic text login matching too early, causing mistyping -- was " login prompt not ready to type - missing keys while typing login")
Actions #19

Updated by jorauch over 6 years ago

  • Blocks action #34339: [qe-core][sle][functional][medium] test fails in consoletest_setup - generic text login matching too early, causing mistyping -- was " login prompt not ready to type - missing keys while typing login" added
Actions #20

Updated by mgriessmeier over 6 years ago

  • Due date changed from 2018-06-05 to 2018-06-19
  • Assignee deleted (SLindoMansilla)
  • Target version changed from Milestone 16 to Milestone 17
Actions #21

Updated by zluo over 6 years ago

  • Assignee set to zluo

take over...

Actions #22

Updated by zluo over 6 years ago

  • Status changed from Workable to In Progress

check and replace assert_screen 'text_login' by wait_boot and select_console 'root-console':

sle-15-Installer-DVD-x86_64-Build665.2-external_iso@64bit
copy_iso_to_external_drive
post_run_hook contains assert_screen 'text_login' which needs to be changed

http://e13.suse.de/tests/2731#step/copy_iso_to_external_drive/29
looks good.

Actions #24

Updated by zluo over 6 years ago

PR for tests/virt_autotest/reboot_and_wait_up.pm:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5208

Actions #25

Updated by zluo over 6 years ago

Actions #26

Updated by zluo over 6 years ago

checked tests/console/sle15_workarounds.pm as well, keep text-login because workaround is still needed.

Actions #27

Updated by zluo over 6 years ago

lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60); no need to change, just keep it

Actions #28

Updated by zluo over 6 years ago

lib/susedistribution.pm: assert_screen "text-login", 10;

is used in the live-cd accessing the "install-shell". That is used for example here:

https://openqa.opensuse.org/tests/688993#step/logs_from_installation_system/1

We keep if (LIVECD) part as well.

Actions #29

Updated by zluo over 6 years ago

lib/susedistribution.pm: # Accept 'text-login' by default

remove the old comment.

Actions #31

Updated by okurz over 6 years ago

zluo wrote:

checked tests/console/sle15_workarounds.pm as well, keep text-login because workaround is still needed.

zluo wrote:

lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60); no need to change, just keep it

Please adress my comment in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5206#issuecomment-396524370

Actions #32

Updated by zluo over 6 years ago

tried to change tests/virt_autotest/reboot_and_wait_up.pm: assert_screen("text-login", 600); by using:

$self->wait_boot();
select_console 'root-console';

And change wait_boot function in lib/opensusebasetest.pm to avoid BACKEND impi run into GRUB:

elsif ((!(check_var('VIRSH_VMM_FAMILY', 'xen') && check_var('VIRSH_VMM_TYPE', 'linux') && check_var('BACKEND', 'svirt'))) || !(check_var('BACKEND', 'ipmi')))

This doesn't help:

http://e13.suse.de/tests/2749#step/reboot_and_wait_up_normal/6

Actions #33

Updated by zluo over 6 years ago

at moment the ipmi worker is not stable and it skips many times my test runs :(

http://e13.suse.de/tests/2833

Actions #34

Updated by okurz over 6 years ago

  • Target version changed from Milestone 17 to Milestone 17
Actions #35

Updated by zluo over 6 years ago

for ipmi use root-ssh instead of root-console and use PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5236 for another try:

http://e13.suse.de/tests/2840

Actions #36

Updated by zluo over 6 years ago

http://e13.suse.de/tests/2842

it seems to have same issue for:

at /usr/lib/os-autoinst/backend/baseclass.pm line 80.
backend::baseclass::die_handler('The console isn\'t responding correctly. Maybe half-open sock...') called at /usr/lib/os-autoinst/backend/baseclass.pm line 241
eval {...} called at /usr/lib/os-autoinst/backend/baseclass.pm line 156
backend::baseclass::run_capture_loop('backend::ipmi=HASH(0x699cc40)') called at /usr/lib/os-autoinst/backend/baseclass.pm line 129
backend::baseclass::run('backend::ipmi=HASH(0x699cc40)', 5, 8) called at /usr/lib/os-autoinst/backend/driver.pm line 85
backend::driver::start('backend::driver=HASH(0x6493890)') called at /usr/lib/os-autoinst/backend/driver.pm line 48
backend::driver::new('backend::driver', 'ipmi') called at /usr/bin/isotovideo line 236
main::init_backend() called at /usr/bin/isotovideo line 305
Actions #37

Updated by zluo over 6 years ago

the issue reported in (https://progress.opensuse.org/issues/32746) needs to revolved at first if we want to make changes in reboot_and_wait_up.pm

Actions #38

Updated by mgriessmeier over 6 years ago

  • Due date changed from 2018-06-19 to 2018-07-03
  • Status changed from In Progress to Blocked

so it's basically blocked by https://progress.opensuse.org/issues/32746

Actions #39

Updated by mgriessmeier over 6 years ago

  • Blocked by action #32746: [sle][tools][remote-backends][hard] Incomplete job because console isn't responding correctly. Half-open socket on IPMI added
Actions #40

Updated by zluo over 6 years ago

  • % Done changed from 0 to 80

I think we are fine with the status of this ticket for now. Only reboot_and_wait_up.pm cannot be handled as we wish. So this can be worked on later when #32746 got resolved.

Actions #41

Updated by mgriessmeier over 6 years ago

  • Due date changed from 2018-07-03 to 2018-07-17
Actions #42

Updated by mgriessmeier about 6 years ago

  • Due date changed from 2018-07-17 to 2018-07-31
  • Status changed from Blocked to Feedback
Actions #43

Updated by okurz about 6 years ago

  • Due date deleted (2018-07-31)
  • Status changed from Feedback to Workable
  • Assignee deleted (zluo)
  • Target version changed from Milestone 17 to Milestone 21+

ok, so we have covered a lot already. As we understood in SR zluo is not currently working on this at the moment. We should revisit the situation in some months again.

Actions #44

Updated by okurz almost 6 years ago

  • Priority changed from High to Normal

still valid but not "High" prio.

Actions #45

Updated by okurz almost 6 years ago

  • Target version changed from Milestone 21+ to future
Actions #46

Updated by okurz over 5 years ago

I found that there is "linux-login" as well as "text-login". I assume both are equivalent and should be merged.

$ git grep '\(text\|linux\)-login\>'
lib/migration.pm:    if (!check_screen('linux-login', 200)) {
lib/opensusebasetest.pm:        push @tags, 'linux-login'                     if get_var('KEEP_GRUB_TIMEOUT');    # Also wait for linux-login if grub timeout was not disabled
lib/opensusebasetest.pm:        elsif (!match_has_tag("grub2") and !match_has_tag('linux-login')) {
lib/opensusebasetest.pm:            my $failneedle = get_var('KEEP_GRUB_TIMEOUT') ? 'linux-login' : 'grub2';
lib/opensusebasetest.pm:        my $textmode_needles = [qw(linux-login emergency-shell emergency-mode)];
lib/susedistribution.pm:            assert_screen "text-login", 10;
lib/susedistribution.pm:        assert_screen(["text-logged-in-$user", "text-login"], 60);
tests/boot/boot_from_pxe.pm:        assert_screen 'linux-login', 100;
tests/console/copy_iso_to_external_drive.pm:    assert_screen "text-login";
tests/console/sle15_workarounds.pm:        assert_screen(["tty2-selected", 'text-login', 'text-logged-in-root', 'generic-desktop']);
tests/installation/boot_into_snapshot.pm:    assert_screen 'linux-login', 200;
tests/installation/first_boot.pm:        assert_screen('linux-login', $boot_timeout) unless check_var('ARCH', 's390x');
tests/installation/grub_test.pm:    # If grub timeout was not disabled, we wait for linux-login instead
tests/installation/grub_test.pm:    my $tag = get_var('KEEP_GRUB_TIMEOUT') ? 'linux-login' : 'grub2';
tests/installation/reboot_eject_cd.pm:    assert_screen('text-login');
tests/jeos/firstrun.pm:    assert_screen_with_soft_timeout('linux-login', timeout => 1000, soft_timeout => 300, bugref => 'bsc#1077007');
tests/locale/keymap_or_locale.pm:        assert_screen([qw(linux-login cleared-console)]);
tests/locale/keymap_or_locale.pm:        assert_screen([qw(linux-login cleared-console)]);
tests/qa_automation/kernel_kexec.pm:    assert_screen('linux-login', 300);
tests/qam-minimal/install_update.pm:            assert_screen 'linux-login';
tests/sles4sap/netweaver_ascs_install.pm:              text-login linux-login started-x-displaymanager-info)
tests/virt_autotest/login_console.pm:    send_key_until_needlematch(['linux-login', 'virttest-displaymanager'], 'ret', $timeout / 5, 5);
tests/virt_autotest/reboot_and_wait_up.pm:        assert_screen("text-login", 600);
tests/virtualization/prepare.pm:        assert_screen 'linux-login', 600;
tests/virtualization/xen/patch_and_reboot.pm:    send_key_until_needlematch('text-login', 'ret', 360, 5);

So let's take a look for places where we have "text-login" and decide which are ok to keep or not

$ git grep '\<text-login\>'
lib/susedistribution.pm:            assert_screen "text-login", 10;

lib/susedistribution.pm:        assert_screen(["text-logged-in-$user", "text-login"], 60);
tests/console/copy_iso_to_external_drive.pm:    assert_screen "text-login";
tests/console/sle15_workarounds.pm:        assert_screen(["tty2-selected", 'text-login', 'text-logged-in-root', 'generic-desktop']);
tests/installation/reboot_eject_cd.pm:    assert_screen('text-login');
tests/sles4sap/netweaver_ascs_install.pm:              text-login linux-login started-x-displaymanager-info)
tests/virt_autotest/reboot_and_wait_up.pm:        assert_screen("text-login", 600);
  • 1. [ ] lib/susedistribution.pm: assert_screen "text-login", 10;

is used in the live-cd accessing the "install-shell". That is used for example here:
https://openqa.opensuse.org/tests/688993#step/logs_from_installation_system/1

Could you please try to just remove that if (LIVECD) part?

  • 2. [x] lib/susedistribution.pm: # Accept 'text-login' by default

should just be removed -> done with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5219

  • 3. [ ] lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60);

is within $type eq 'ssh' and I do not know if we still need that. Please take a look in used needles in scenarios using ssh and check where this part of the code is covered.

  • 4. [ ] tests/console/sle15_workarounds.pm: assert_screen(["tty2-selected", 'text-login', 'text-logged-in-root', 'generic-desktop']);

Please discuss with test module maintainer @rwx788 how to proceed with this test module. The bug for which this code is used as workaround is still valid and open. One solution I could think if is renaming the test module. Maybe we do not need the text-login needle here but can just work with the tty2-selected needle although explicitly mentioning the tty number is not so good. Please look into extracting the current console number from the root console object.

  • 5. [ ] tests/installation/reboot_eject_cd.pm: assert_screen('text-login');

is used for example in https://openqa.opensuse.org/tests/677992#step/reboot_eject_cd/1 . I guess you can simply replace text-login by tty3-selected because one line above there is a send_key 'ctrl-alt-f3'

  • 6. [ ] tests/sles4sap/netweaver_ascs_install.pm: text-login linux-login started-x-displaymanager-info)

should be replaced by wait_boot

  • 7. [ ] tests/virt_autotest/reboot_and_wait_up.pm: assert_screen("text-login", 600);

same, as already discussed yesterday, to be replaced by wait_boot

Actions #47

Updated by zluo about 5 years ago

  • Status changed from Workable to In Progress
  • Assignee set to zluo
  • Target version changed from future to Milestone 26

take over and checking current status.

Actions #48

Updated by zluo about 5 years ago

Actions #49

Updated by zluo about 5 years ago

  • Status changed from In Progress to Rejected

this is not an issue anymore:
http://f40.suse.de/tests/4869#next_previous

Actions #50

Updated by okurz about 5 years ago

  • Status changed from Rejected to In Progress

Why do you think this is not an issue anymore when we still haven't fulfilled the requirements we have set on ourselves? Please read the ticket description and comments again.

Actions #51

Updated by zluo about 5 years ago

  • Assignee deleted (zluo)

since the original issue doesn't happen anymore and this is not a priority at moment, I won't work on it.

Actions #52

Updated by zluo about 5 years ago

  • Status changed from In Progress to Workable
Actions #53

Updated by mgriessmeier about 5 years ago

  • Status changed from Workable to New
  • Priority changed from Normal to High
  • Target version changed from Milestone 26 to Milestone 27

discuss in grooming again ...

Actions #54

Updated by SLindoMansilla about 5 years ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from High to Normal
Actions #55

Updated by SLindoMansilla about 5 years ago

  • Estimated time set to 42.00 h
Actions #56

Updated by mgriessmeier about 5 years ago

  • Target version changed from Milestone 27 to Milestone 28
Actions #57

Updated by okurz almost 5 years ago

  • Related to action #36126: [functional][u] post_fail_hook matches on "text_login_root" before actual tty switch and therefore never logs in added
Actions #58

Updated by zluo almost 5 years ago

  • Status changed from Workable to In Progress
  • Assignee set to zluo

take over and check if this is a similar issue because I'm working on https://progress.opensuse.org/issues/36126

Actions #59

Updated by zluo almost 5 years ago

found still some text-login:
text-login-20130711.json: "text-login"
text-login-20160812.json: "text-login"
text-login-20170427.json: "text-login"
text-login-20170504.json: "text-login"

We still have following test modules:

lib/sles4sap.pm: text-login linux-login started-x-displaymanager-info)
lib/susedistribution.pm: assert_screen "text-login", 10;
lib/susedistribution.pm: assert_screen(["text-logged-in-$user", "text-login"], 60);
tests/console/copy_iso_to_external_drive.pm: assert_screen "text-login";
tests/console/sle15_workarounds.pm: assert_screen(["tty2-selected", 'text-login', 'text-logged-in-root', 'generic-desktop']);
tests/installation/reboot_eject_cd.pm: assert_screen('text-login');
tests/systemd_testsuite/test_09_issue_2691.pm: send_key_until_needlematch('text-login', 'ret', 360, 5);
tests/virt_autotest/reboot_and_wait_up.pm: assert_screen("text-login", 600);

above test modules are tested for sle but not for opensuse.

To compare with #46 and consider to how to process with this ticket again.

Actions #60

Updated by zluo almost 5 years ago

found:

lib/caasp.pm: assert_screen 'linux-login-casp', 150;
lib/caasp.pm: # Don't match linux-login-casp twice
lib/caasp.pm: assert_screen [qw(grub2 linux-login-casp)], 150;
lib/locale.pm: assert_screen([qw(linux-login cleared-console)]);
lib/locale.pm: assert_screen(has_ttys() ? 'linux-login' : 'cleared_console');
lib/migration.pm:# Assert screen 'linux-login' with 200s
lib/migration.pm: if (!check_screen('linux-login', 200)) {
lib/opensusebasetest.pm: my $textmode_needles = [qw(linux-login emergency-shell emergency-mode)];
lib/opensusebasetest.pm: # 2nd stage of autoyast can be considered as linux-login
lib/sles4sap.pm: text-login linux-login started-x-displaymanager-info)
lib/systemd_testsuite_test.pm: assert_screen('linux-login', 300);
lib/transactional.pm: assert_screen 'linux-login', 200;
tests/autoyast/installation.pm: push @needles, 'linux-login-casp' if is_caasp;
tests/autoyast/installation.pm: || match_has_tag('linux-login-casp')
tests/boot/boot_from_pxe.pm: assert_screen 'linux-login', 100;
tests/boot/boot_linuxrc.pm: assert_screen([qw(linux-login displaymanager generic-desktop)], 180);
tests/boot/grub2_test.pm: assert_screen 'linux-login', 200;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 300;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 200;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 100;
tests/installation/boot_into_snapshot.pm: assert_screen 'linux-login', 200;
tests/installation/first_boot.pm: assert_screen('linux-login', $boot_timeout) unless check_var('ARCH', 's390x');
tests/installation/ipxe_install.pm: assert_screen('linux-login', 1800);
tests/jeos/firstrun.pm: assert_screen [qw(linux-login reached-power-off)], 1000;
tests/jeos/prepare_firstboot.pm: assert_screen('linux-login', 300);
tests/kernel/boot_ltp.pm: assert_screen('linux-login', 1800);
tests/kernel/build_git_kernel.pm: assert_screen('linux-login', 1800);
tests/kernel/kernel_kexec.pm: # clear console to prevent linux-login to match before reboot
tests/kernel/kernel_kexec.pm: assert_screen('linux-login', 300);
tests/kernel/mellanox_config.pm: assert_screen('linux-login', 1800);
tests/kernel_performance/run_perf_case.pm: assert_screen('linux-login', $timeout);
tests/kiwi_images_test/kiwi_boot.pm: assert_screen('linux-login', 1200);
tests/kiwi_images_test/kiwi_boot.pm: assert_screen('linux-login', 1000);
tests/kiwi_images_test/login_reboot.pm: assert_screen('linux-login', 120);
tests/kiwi_images_test/validate_build.pm: assert_screen('linux-login', 120);
tests/locale/keymap_or_locale.pm: assert_screen([qw(linux-login cleared-console)]);
tests/qam-minimal/install_update.pm: assert_screen 'linux-login', 90;
tests/virt_autotest/login_console.pm: send_key_until_needlematch(['linux-login', 'virttest-displaymanager'], 'ret', $counter, $timeout);
tests/virt_autotest/login_console.pm: assert_screen('will-linux-login', $timeout);
tests/virt_autotest/login_console.pm: last if (check_screen([qw(linux-login virttest-displaymanager)], 60));
tests/virtualization/prepare.pm: assert_screen 'linux-login', 600;

tag 'linux-login' found from needles:

linux-login-20140916.json: "linux-login"
linux-login-autoyast-20150724.json: "linux-login"
linux-login-autoyast-20160822.json: "linux-login"
linux-login-autoyast-20191126.json: "linux-login",
linux-login-kubic-20180201.json: "linux-login-casp"
linux-login-kubic-20190408.json: "linux-login-casp"
linux-login-kubic-20190415.json: "linux-login-casp"
linux-login-kubic-20191031.json: "linux-login-casp"

Actions #61

Updated by zluo almost 5 years ago

http://f40.suse.de/tests/5546#step/zypper_up/1 (JeOS-for-kvm-and-xen-x86_64-Build17.7-jeos@64bit_virtio-2G)
http://f40.suse.de/tests/5547#step/zypper_up/1 (Tumbleweed-NET-x86_64-Build20191126-textmode@64bit)
http://f40.suse.de/tests/5548#step/zypper_up/1 (opensuse-15.1-DVD-Updates-x86_64-Build20191127-2-textmode@64bit)
http://f40.suse.de/tests/5549#step/zypper_up/1 (opensuse-15.0-DVD-Updates-x86_64-Build20191127-2-textmode@64bit)

show that root-console-20170321 is used. We don't have problem at moment for opensuse. And we can delete text-login from opensuse needle repository.

Actions #63

Updated by zluo almost 5 years ago

http://f40.suse.de/tests/5551#step/zypper_up/41 shows network issue and triggers later post_fail_hook: http://f40.suse.de/tests/5551#step/zypper_up/41

tty5-selected is then correctly use now. So we are safe to remove text-login.

Actions #66

Updated by zluo almost 5 years ago

Now checking linux-login tag which is used by opensuse:

lib/caasp.pm: assert_screen 'linux-login-casp', 150;
lib/caasp.pm: # Don't match linux-login-casp twice
lib/caasp.pm: assert_screen [qw(grub2 linux-login-casp)], 150;
lib/locale.pm: assert_screen([qw(linux-login cleared-console)]);
lib/locale.pm: assert_screen(has_ttys() ? 'linux-login' : 'cleared_console');
lib/migration.pm:# Assert screen 'linux-login' with 200s
lib/migration.pm: if (!check_screen('linux-login', 200)) {
lib/opensusebasetest.pm: my $textmode_needles = [qw(linux-login emergency-shell emergency-mode)];
lib/opensusebasetest.pm: # 2nd stage of autoyast can be considered as linux-login
lib/sles4sap.pm: text-login linux-login started-x-displaymanager-info)
lib/systemd_testsuite_test.pm: assert_screen('linux-login', 300);
lib/transactional.pm: assert_screen 'linux-login', 200;
tests/autoyast/installation.pm: push @needles, 'linux-login-casp' if is_caasp;
tests/autoyast/installation.pm: || match_has_tag('linux-login-casp')
tests/boot/boot_from_pxe.pm: assert_screen 'linux-login', 100;
tests/boot/boot_linuxrc.pm: assert_screen([qw(linux-login displaymanager generic-desktop)], 180);
tests/boot/grub2_test.pm: assert_screen 'linux-login', 200;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 300;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 200;
tests/boot/grub2_test.pm: assert_screen 'linux-login', 100;
tests/installation/boot_into_snapshot.pm: assert_screen 'linux-login', 200;
tests/installation/first_boot.pm: assert_screen('linux-login', $boot_timeout) unless check_var('ARCH', 's390x');
tests/installation/ipxe_install.pm: assert_screen('linux-login', 1800);
tests/jeos/firstrun.pm: assert_screen [qw(linux-login reached-power-off)], 1000;
tests/jeos/prepare_firstboot.pm: assert_screen('linux-login', 300);
tests/kernel/boot_ltp.pm: assert_screen('linux-login', 1800);
tests/kernel/build_git_kernel.pm: assert_screen('linux-login', 1800);
tests/kernel/kernel_kexec.pm: # clear console to prevent linux-login to match before reboot
tests/kernel/kernel_kexec.pm: assert_screen('linux-login', 300);
tests/kernel/mellanox_config.pm: assert_screen('linux-login', 1800);
tests/kernel_performance/run_perf_case.pm: assert_screen('linux-login', $timeout);
tests/kiwi_images_test/kiwi_boot.pm: assert_screen('linux-login', 1200);
tests/kiwi_images_test/kiwi_boot.pm: assert_screen('linux-login', 1000);
tests/kiwi_images_test/login_reboot.pm: assert_screen('linux-login', 120);
tests/kiwi_images_test/validate_build.pm: assert_screen('linux-login', 120);
tests/locale/keymap_or_locale.pm: assert_screen([qw(linux-login cleared-console)]);
tests/qam-minimal/install_update.pm: assert_screen 'linux-login', 90;
tests/virt_autotest/login_console.pm: send_key_until_needlematch(['linux-login', 'virttest-displaymanager'], 'ret', $counter, $timeout);
tests/virt_autotest/login_console.pm: assert_screen('will-linux-login', $timeout);
tests/virt_autotest/login_console.pm: last if (check_screen([qw(linux-login virttest-displaymanager)], 60));
tests/virtualization/prepare.pm: assert_screen 'linux-login', 600;

--
found: first_boot.pm, firstrun.pm, keymap_or_locale.pm are valid for opensuse tw.
Need to check these test modules for sle and create required needle 'tty$Nr-selected' as well!

Actions #67

Updated by zluo almost 5 years ago

http://f40.suse.de/tests/5567 done for first_boot and keymap_or_locale.

Actions #68

Updated by zluo almost 5 years ago

http://f40.suse.de/tests/5574#step/firstrun/7 shows successful test run with a new needle 'tty6-selected'.

Actions #70

Updated by zluo almost 5 years ago

for sle-15-SP2-Online-x86_64-Build93.1-gnome@64bit:
http://f40.suse.de/tests/5576
first_boot and keymap_or_locale are not effected by code changes:

elsif ((get_var('DESKTOP', '') =~ /textmode|serverro/) || get_var('BOOT_TO_SNAPSHOT')) {
my $tty = get_root_console_tty;
assert_screen("tty$tty-selected", $boot_timeout) unless check_var('ARCH', 's390x');
return;
}
and 
assert_screen([qw("tty$tty-selected" cleared-console)]);
Actions #71

Updated by dimstar almost 5 years ago

  • Priority changed from Normal to Urgent

zluo wrote:

needle PR: https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/619

This has been merged and the live installer in Tumbleweed are now broken

https://openqa.opensuse.org/tests/1099159#next_previous

Actions #73

Updated by zluo almost 5 years ago

okay, will check this then.

Actions #75

Updated by zluo almost 5 years ago

test_09_issue_2691:
http://f40.suse.de/tests/5585#step/test_09_issue_2691/1 shows that needle match and it has tags: text-login and tty6-selected

Actions #76

Updated by zluo almost 5 years ago

installation_overview.pm still uses tag 'text-login', need to check where this is coming from.
it comes from install-shell - > text-login. Change it to tty2-selected:

http://f40.suse.de/tests/5589#step/installation_overview/2

Actions #77

Updated by zluo almost 5 years ago

clone https://openqa.suse.de/tests/3641764 for change 'tty1-selected' in tests/virt_autotest/reboot_and_wait_up.pm

Actions #80

Updated by zluo almost 5 years ago

  • Status changed from In Progress to Feedback
  • Priority changed from Urgent to High
Actions #81

Updated by mgriessmeier almost 5 years ago

  • Target version changed from Milestone 28 to Milestone 30

needs to be discussed offline

Actions #82

Updated by okurz almost 5 years ago

  • Related to action #57281: [sle][Migration][SLE15SP2] test fails in orphaned_packages_check - switch to tty failed added
Actions #83

Updated by zluo over 4 years ago

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9048 merged now.

checking now old PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034

first_boot.pm got changed and it uses now function wait_boot_past_bootloader and verify this change by test runs:

diff --git a/lib/opensusebasetest.pm b/lib/opensusebasetest.pm
index 08648c884..6b5ee125b 100644
--- a/lib/opensusebasetest.pm
+++ b/lib/opensusebasetest.pm
@@ -806,7 +806,7 @@ sub wait_boot_textmode {
return if check_var('ARCH', 's390x');
my $ready_time       = $args{ready_time};
-    my $textmode_needles = [qw(linux-login emergency-shell emergency-mode)];
+    my $textmode_needles = [qw(tty1-selected emergency-shell emergency-mode)];
# 2nd stage of autoyast can be considered as linux-login
push @{$textmode_needles}, 'autoyast-init-second-stage' if get_var('AUTOYAST');
# Soft-fail for user_defined_snapshot in extra_tests_on_gnome and extra_tests_on_gnome_on_ppc
Actions #85

Updated by xlai over 4 years ago

zluo wrote:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 updated now.

@zluo @okurz Would really appreciate if we can be involved in solution discussion and PR review earlier if next time such big influence changes are made. Our tests in latest build 134.1 fail a lot at matching tty1-selected in https://openqa.nue.suse.com/tests/overview?distri=sle&version=15-SP2&build=134.1&groupid=263.

Actions #86

Updated by okurz over 4 years ago

xlai wrote:

zluo wrote:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 updated now.

@zluo @okurz Would really appreciate if we can be involved in solution discussion and PR review earlier if next time such big influence changes are made.

I didn't do any changes within the last year. I only participated in review and merged a revert so far. The above mentioned PR was already reverted in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9473 so maybe a retrigger should work. In general you can participate in the solution discussion since two years (!) but I did not see any comment by you in this ticket. Also the PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 was merged after being open for more than 2 months (!) so you could have looked into the PR for a very long time already. Expecting anyone would decide when to personally ping you or not on indvidual pull requests is an approach which can be used in exceptions but it isn't going to scale well. IMHO the pull request being open for two months is more than enough. However I grant you that the PR has a lacking description and was ill-prepared and is missing a lot of verification runs. Early review can all help with this.

EDIT: sorry, meant to cross-check mainly IPMI, not (only) ssh

Actions #87

Updated by xlai over 4 years ago

okurz wrote:

xlai wrote:

zluo wrote:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 updated now.

@zluo @okurz Would really appreciate if we can be involved in solution discussion and PR review earlier if next time such big influence changes are made.

I didn't do any changes within the last year. I only participated in review and merged a revert so far. The above mentioned PR was already reverted in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9473 so maybe a retrigger should work.

I at you because you are the creator of this ticket and I thought you created the "expected results" and "tasks" , which showed the solution. Hope you do not mind :).

In general you can participate in the solution discussion since two years (!) but I did not see any comment by you in this ticket. Also the PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9034 was merged after being open for more than 2 months (!) so you could have looked into the PR for a very long time already. Expecting anyone would decide when to personally ping you or not on indvidual pull requests is an approach which can be used in exceptions but it isn't going to scale well. IMHO the pull request being open for two months is more than enough. However I grant you that the PR has a lacking description and was ill-prepared and is missing a lot of verification runs. Early review can all help with this.

Yes, you are right. But looking into all open PRs and reviewing them will take a lot of time. It is quite expensive to be aware of the big changes from this PR review way. It is a common challenge for most openqa test code maintainers.

@zluo @SLindoMansilla Introducing a check for "tty1" is IMHO a pretty bad idea and far from the original idea of this ticket. Our tests should stay generic and keep systems in mind that do not have tty's, e.g. remote systems where we login over ssh.

I am afraid so. There should be a better way to solve the original ticket. Now virtualization tests are blocked a lot. And the influence of replacing such general needle "linux-login" should be much bigger.

Actions #88

Updated by okurz over 4 years ago

I at you because you are the creator of this ticket and I thought you created the "expected results" and "tasks" , which showed the solution. Hope you do not mind :).

I think you did good. Just wanted to put in perspective what my current impact is :)

Yes, you are right. But looking into all open PRs and reviewing them will take a lot of time. It is quite expensive to be aware of the big changes from this PR review way. It is a common challenge for most openqa test code maintainers.

I understand that. Basically I see three different choices as of now:

  1. Review PRs, at best before merge
  2. Investigate which test changes can explain a test fail. This is also now provided in the "Investigation" tab of each test
Actions #89

Updated by SLindoMansilla over 4 years ago

Suggestion:

  • create a new needle covering the ttyS1 terminal with tag 'ttyS1-selected'.
  • Or revert this specific change and leave text-login as general needle.
Actions #90

Updated by zluo over 4 years ago

since PR for linux-login got reverted, and I created now a new PR to fix issue:

https://openqa.nue.suse.com/tests/3853054

This should be safe from now with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9487 (still need verification run).

Actions #91

Updated by zluo over 4 years ago

https://openqa.suse.de/tests/3858631#step/reboot_and_wait_up_normal/4 looks good. The new needle created contains 2 tags: ttys1-selected and tty1-selected

Actions #93

Updated by zluo over 4 years ago

since my PR got reverted again, we can do this now:

Or revert this specific change and leave text-login as general needle

However with a new needle it fixed the issue, but actually this not easy to cover all cases which text-login still used widely for other tests/team, maybe this is better, don't to try this for now till we have a better idea.

See example:
https://149.44.176.58/tests/3869862#step/reboot_and_wait_up_normal/4

Actions #94

Updated by zluo over 4 years ago

  • Status changed from Feedback to Workable
  • Assignee deleted (zluo)

since my PR got reverted again, we can do this now:

Or revert this specific change and leave text-login as general needle

However with a new needle it fixed the issue, but actually this not easy to cover all cases where text-login is still used widely. Maybe this is better, don't to try this for now till we have a "final" solution.

Actions #95

Updated by xlai over 4 years ago

This change still fails 3 tests in virtualization-acceptance job group, https://openqa.nue.suse.com/tests/overview?distri=sle&version=15-SP2&build=136.2&groupid=263. It also fails several tests in virtualization-milestone group, although some are still scheduled, https://openqa.nue.suse.com/tests/overview?distri=sle&version=15-SP2&build=136.2&groupid=264.

So due to this change, we can not deliver the public beta candidate acceptance test report in time.

If a full support can not be done, I strongly suggest a full revert of all PRs, rather than partially revert and partially keep. @mgressimier @zluo, how do you think?

Actions #96

Updated by SLindoMansilla over 4 years ago

  • Status changed from Workable to New

To be discussed again

Actions #97

Updated by SLindoMansilla over 4 years ago

  • Priority changed from High to Normal
Actions #98

Updated by okurz over 4 years ago

  • Category changed from Bugs in existing tests to Enhancement to existing tests

Actually the original issue is fixed since long but the enhancement as mentioned in the subject is still valid.

Actions #99

Updated by tjyrinki_suse almost 4 years ago

  • Subject changed from [functional][opensuse][u][medium] too early matching in too generic needle text-login-20160812 to [qe-core][functional][opensuse][medium] too early matching in too generic needle text-login-20160812
Actions #100

Updated by okurz almost 3 years ago

I came to this ticket due to periodically reviewing tickets as described on https://progress.opensuse.org/projects/openqatests/wiki#How-we-work-on-tickets

This ticket was set to "Normal" priority but was not updated within the SLO period for "Normal" tickets (365 days) as described on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#SLOs-service-level-objectives

First reminder: Please consider picking up this ticket within the next 365 days or just set the ticket to the next lower priority of "Low" (no SLO related time period).

Actions #101

Updated by slo-gin almost 2 years ago

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Actions #102

Updated by slo-gin 7 months ago

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Actions #103

Updated by mgrifalconi 5 months ago

  • Tags set to qecore-cleanup
  • Status changed from New to Resolved

Please create a new follow-up for any enhancements needed.

Actions

Also available in: Atom PDF