QA » openQA Project » openQA Tests

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde_dual_windows10@uefi_win
https://openqa.opensuse.org/tests/739332

it also makes sense to update that windows version

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde_dual_windows10@uefi_win
https://openqa.opensuse.org/tests/751296

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde_dual_windows10@uefi_win
https://openqa.opensuse.org/tests/768516

okurz wrote:

@GraceWang may I ask what is your expectation on this? Any ETA that we could work with to fix the tests that currently fail?

Terribly sorry for the late reply.
I was busy with SLED12SP4 stuff in the past weeks.
Now, I can spare some time to work on this poo and poo#39125.

Since the windows qcow hasn't been changed for years, so there might be changes in TW or openQA caused this failure.

I did the below tests to find out the cause of this issue

The openQA version on my local server is Version 4.6.1539580083.99583ca7

The kde_dual_windows10@uefi_win failed since Build 20180802, so I find a TW ISO which Build is 20180722 to do the contrast testing (at that time the case kde_dual_windows10@uefi_win passed without any error)

Run the kde_dual_windows10@uefi_win on my local server with Build 20180722 and Build0926, both of the two are failed due to the missing of 'windows-bootsplash' window.

http://147.2.212.93/tests/600#step/boot_windows/7 (Build 20180722)
http://147.2.212.93/tests/593#step/boot_windows/6 (Build 20180926)

The earlier passed scenario (kde_dual_windows10@uefi_win with Build 20180722) no longer passed on my local openQA server, this means this issue is not caused by changes in TW but the update of openQA.

We have gnome-dual-windows10@64bit_win in SLED15SP1 which is not affected by the update of openQA (The 'windows-bootsplash' still exists).
See https://openqa.suse.de/tests/2170744#step/boot_windows/2
I also tried kde_dual_windows10@64bit_win with TW Build0926, the 'windows-bootsplash' exists. (See http://147.2.212.93/tests/604#)

Taking both dual_windows10@uefi_win and dual-windows10@64bit_win into account, the simple solution for this issue is to change
assert_screen 'windows-bootsplash', 600;
into
check_screen 'windows-bootsplash', 600;

@okurz, may I know your opinion about this?

GraceWang wrote:

Taking both dual_windows10@uefi_win and dual-windows10@64bit_win into account, the simple solution for this issue is to change
assert_screen 'windows-bootsplash', 600;
into
check_screen 'windows-bootsplash', 600;

Simple as always: Use a multi-tag assert_screen with match_has_tag instead :) Think of openQA tests as you would test as human. You would not sit in front of the screen for 600s even though you understand the current, not expected screen and can immediately act on it, right? So better do

assert_screen([qw(windows-bootsplash windows-screensaver)], 600);
if (match_has_tag('windows-screensaver')) {
    // do something about it
}

but what I can see from the video it just looks like the bootsplash does not appear anymore because Windows boots up faster.

I am not exactly sure why lib/windows_utils.pm looks for the bootsplash and then the login screen because I think it can be as simple as just waiting for windows-screensaver and then calling reset_consoles so just like this:

    // no assert_screen on bootsplash

    assert_screen 'windows-screensaver', 600;

    # Reset the consoles after the reboot: there is no user logged in anywhere
    reset_consoles;

could you try that?

okurz wrote:

Simple as always: Use a multi-tag assert_screen with match_has_tag instead :) Think of openQA tests as you would test as human. You would not sit in front of the screen for 600s even though you understand the current, not expected screen and can immediately act on it, right? So better do

assert_screen([qw(windows-bootsplash windows-screensaver)], 600);
if (match_has_tag('windows-screensaver')) {
    // do something about it
}

but what I can see from the video it just looks like the bootsplash does not appear anymore because Windows boots up faster.

Thanks for the nice solution. :-)
Yes, for the uefi_win test case, the bootsplash does not appear. But for the 64bit_win test case, the bootsplash still exists.

I am not exactly sure why lib/windows_utils.pm looks for the bootsplash and then the login screen because I think it can be as simple as just waiting for windows-screensaver and then calling reset_consoles so just like this:

I agree with you. Whether the bootsplash windows appears or not does not matter in this scenario.

    // no assert_screen on bootsplash

    assert_screen 'windows-screensaver', 600;

    # Reset the consoles after the reboot: there is no user logged in anywhere
    reset_consoles;

could you try that?

I tried this, and the result is softfailed due to an existing issue poo#19616.
See http://147.2.212.93/tests/609 and http://147.2.212.93/tests/610

So I made some changes to avoid the occurrence of poo#19616

    # Wait for the network is ready
    sleep 60;

    assert_screen 'windows-screensaver', 600;

    # Reset the consoles after the reboot: there is no user logged in anywhere
    reset_consoles;

Verification run:
http://147.2.212.93/tests/611
http://147.2.212.93/tests/612

If you are okay with the above solution, I will submit the changes.

sorry but I do not think that a sleep 60 is ever a good approach. We are just wasting time and people reading the code are not sure what we actually wait for. Again as in before: Think like a human tester: You would not blindly wait for 60 seconds but for a certain situation shown on the screen and then act upon it. In the above example the yellow triangle just shows up for a short time and then goes away again. I recommend you simply delete this needle and assign #19616 to yourself and track if it is still valid. If yes, create a better check for the error condition because the needle check could not reliably tell if the network never comes back, only that it is not yet ready.

As discussed I deleted the workaround needle win10_reboot-windows-screensaver-poo19616-20170606.json & png.

Updated the 'windows-screensaver' needle with the yellow triangle shown on the network icon.

Add assert_screen 'windows-network';
before assert_screen 'windows-login';

Under this situation, we can avoid wasting time to wait for a certain screen appear and at the same time we can check if the network is enabled correctly.

Needles:
https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/455

PR:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6002

Verification run:
http://147.2.212.93/tests/615 (uefi_win)
http://147.2.212.93/tests/616 (64bit_win)