QA » openQA Project » openQA Tests

ok, the validation within the PR was just triggered to show the actual problem: https://openqa.opensuse.org/tests/835514#step/keymap_or_locale/5 failing trying to login as extra but as discussed in [#opensuse-factory](irc://chat.freenode.net/opensuse-factory) :

[22/01/2019 20:04:32] <guillaume_g> okurz: btw, for the keymap test, I think I did not need new needle as it check for 'login' needle... I need to find out what happened. I will check tomorrow!
[22/01/2019 20:07:21] <okurz> guillaume_g: sure, I will handle the test failures for now, no rush now :)
[22/01/2019 20:13:30] <okurz> guillaume_g: see https://openqa.opensuse.org/tests/835514#step/keymap_or_locale/4, it's a test triggered on o3 from a not-yet-merged PR branch, how cool is that? :)
[22/01/2019 20:26:52] <guillaume_g> okurz: the test should not perform a login and just type the string...
[22/01/2019 20:29:25] <guillaume_g> okurz: maybe just an assert_screen on tty3 (with alt-f3) should be enough
[22/01/2019 20:30:11] <okurz> guillaume_g: remember the original problem that on the remote backends we do not have the notion of "tty3", we only have ssh terminals and such
[22/01/2019 20:31:02] <okurz> but I agree that the test should not login and maybe the same test is not useful at all to execute over a remote ssh. This is why I created a progress ticket to do it properly and think about it first. Please add your thoughts in the ticket as well. I am currently unable to grasp everything from just IRC chat log due to needing to handle the immediate failures :)
[22/01/2019 20:31:43] <guillaume_g> okurz: ok. Maybe just skip tty3 test on aarch64 for now. ;)
[22/01/2019 20:32:17] <okurz> no aarch64 should be fine because we have a local tty3 there
[22/01/2019 20:37:10] <guillaume_g> okurz: we have a tty3, but test fails because worker is a bit slow... ;)
[22/01/2019 20:38:40] <guillaume_g> https://openqa.opensuse.org/tests/831803#step/keymap_or_locale/3 shows we match a cleared console _before_ the switch to tty3 happen. So, we start to type too early.

For aarch64 the problem is visible in https://openqa.opensuse.org/tests/835515#step/keymap_or_locale/3 causing tests to fail. The test finds a "cleared_console", not the "login prompt" which it should so it starts to type but that is too early.

@mloviska your commit

commit 08d0e63fd
Author: Martin Loviska <mloviska@suse.com>
Date:   Mon Mar 26 16:49:04 2018 +0200

    Add cleared-console tag for verify keymap textmode subroutine s390x-specific

diff --git a/tests/locale/keymap_or_locale.pm b/tests/locale/keymap_or_locale.pm
index d4b7cdbe8..93b2778cb 100644
--- a/tests/locale/keymap_or_locale.pm
+++ b/tests/locale/keymap_or_locale.pm
@@ -23,7 +23,7 @@ sub verify_default_keymap_textmode {
     }
     else {
         send_key('alt-f3');
-        assert_screen("linux-login");
+        assert_screen([qw(linux-login cleared-console)]);

added the tag "cleared-console" to the screens to check but unconditionally so failing aarch64. Probably we should just add these tag on remote backends, WDYT?

Not aware of further failures, back to "Workable" for the next step of a dedicated "extra" console

Merged https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6884 to revert for now. We should keep https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6884#discussion_r259826505 in mind, e.g. check that rollback to snapshot also still works, e.g. for vmware, etc.

@mgriesmeier please update after #46532 and check what are the next steps to be done. Thx.

blocker resolved.

Next steps:
Reapply https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6220 and test properly what is missing, e.g. could be necessary or useful to prevent select_console from actually logging in into the "extra-console" and just use it as we currently use "tty3"