QA » openQA Project » openQA Tests

@zluo please take this ticket

checked this issue and found that the needle doesn't match, only 16% coverage. It means there were changes. And I found also some other dismatched needles as well. So I need to check this test overall and provide new needles...

I found the reason for a lot of mismatched needles:

for example:
https://openqa.opensuse.org/tests/401500#step/yast2_control_center/24

The dialogue dosen't have a central position as it should be, therefore the needles cannot be matched... So this is an issue on openQA server itself.

found small changes for openldap: yast2_control-center_openldap-mirrormode-configuration_error is not needed anymore, so the test failed because following needle cannot be found.
Fix this at first.

fixed now:

http://e13.suse.de/tests/2972#step/yast2_control_center/10

for record:

openldap failed at:
http://e13.suse.de/tests/2970#step/yast2_control_center/199

tftp-server failed as well because of SLE has already installed tftp server, fix this now by changing following:

assert_screen [qw(yast2_control-center_tftp-server-install yast2_control-center_tftp-server-configuration_ready)];
if (match_has_tag 'yast2_control-center_tftp-server-install') {
send_key 'alt-i';
}

http://e13.suse.de/tests/2974#step/yast2_control_center/10

tftp-server has problem with needle macht, 82% only, will fixe this as well.

fixed: http://e13.suse.de/tests/2975#step/yast2_control_center/9

why should it be acceptable that the tftp server is already enabled? I assume you changed your test flow and are continuing from a snapshot where the tftp server was previously enabled. I suggest you run a clean job again to confirm.

@okurz

I use the latest build for the test (clone job from openqa.suse.de). And I also installed the latest build to verify tftp-server manually.

yast2_tftp was tested before yast2_control_center! So this is possible that has been already enabled.

For SLE we assume most server module like tftp-server got already installed, but it could the case that it is not enabled. For cancel installation it follows an error, which is not coming up now. Remove this assert_screen (error needle) make the test working again.

Having a server installed should not mean by default that a server is enabled and immediately running. The latest failure on openSUSE Leap fails in the printer dialog: https://openqa.opensuse.org/tests/403804#step/yast2_control_center/30
I suggest you crosscheck again on openSUSE Leap. If necessary add some "workaround" needles and use a higher TIMEOUT_SCALE to pass consoletest_finish.

AFter looking at the code I think the control center test is just way too big and therefore hard to maintain. It calls basically all yast modules. I'd expect that the yast control center test tests specifics of the control center rather than every single yast module. Please split it up. To test the control center launching, just launch some yast modules that are tested in gui anyways via control center instead of with alt-f2.

Agreed. Talked with zluo yesterday and we are fine with that conclusion.

I've removed parts of this test and I've tested against openSUSE Leap at first.
It looks good for now.

All server services are removed if they have been tested somewhere else as in yast console tests or yast2 gui tests. Next step is to implement single test module which is important and could not be tested easily on openQA.

Please take a look at my reference test and let me know if this is okay to you:

http://e13.suse.de/tests/2992#

PR on github:
Split yast2_control_center.pm #2913

Please check and merge, thanks!

PR:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2882

Test results for openSUSE Leap:

https://openqa.opensuse.org/tests/407571#step/yast2_control_center/

All created needles for Leap work fine as well.

the recent runs for SLES 12 show still a broken needle. I fixed this now and started a test run:
https://openqa.opensuse.org/tests/409459#step/yast2_control_center/

It looks good.

And latest runs for openSUSE TW and Leap are okay:
https://openqa.opensuse.org/tests/410062#step/yast2_control_center
htt://popenqa.opensuse.org/tests/latest?flavor=DVD&arch=x86_64&test=yast2_ui&machine=64bit&distri=opensuse&version=42.3#step/yast2_control_center

leap looks ok for now but latest failure on tw: https://openqa.opensuse.org/tests/411243#step/yast2_control_center/33

This could be probably performance issue on openQA server.

Check the latest run and it works fine:

https://openqa.opensuse.org/tests/latest?version=Tumbleweed&arch=x86_64&machine=64bit&flavor=DVD&test=yast2_ui&distri=opensuse#step/yast2_control_center

I consider a "server issue" unlikely considering that we don't see issues like these in other scenarios. Either the test is too demanding in general or has a race condition inside which needs to be covered by test code.

well, other test cases are much smaller in general. This is not the first time that we experience server issue. If you have any idea to improve the code, please let me know. Otherwise we need to observe the test results for a while...