QA » openQA Project » openQA Tests

Similar issue
https://openqa.opensuse.org/tests/664692#step/kontact/21

Amarok was not closed by previous test.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde-wayland@64bit_virtio-2G
https://openqa.opensuse.org/tests/676017

From the video it seems like desktop runner is ready to accept whole input. As the script types first characters "ak" runner gives suggestions pointing to amarok. Somehow the first characters get deleted therefore we see incomplete input.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde-wayland@64bit_virtio-2G
https://openqa.opensuse.org/tests/677523

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde-wayland@64bit_virtio
https://openqa.opensuse.org/tests/712629

filed https://progress.opensuse.org/issues/38864 for compilation issue.

I see a ticket blocker: https://progress.opensuse.org/issues/37501

http://e13.suse.de/tests/7120#step/desktop_runner :(

desktop runner doesn't start up.

Your customized test scheduled can not work this way because x11/kontact does not call select_console('x11') so you are stuck on the terminal that consoletest_setup selected. Either schedule another module in between, e.g. consoletest_finish, don't schedule consoletest_setup or put the call to select_console('x11') temporarily into the test module kontact

@okurz yes, thanks!

at moment I cannot reproduce this issue with akregator. It seems to not started at all in my test runs:

x11_start_program('akonadictl start', valid => 0);
# Workaround: sometimes the account assistant behind of mainwindow or tips window
# To disable it run at first time start
x11_start_program("echo \"[General]\" >> ~/.kde4/share/config/kmail2rc",         valid => 0);
x11_start_program("echo \"first-start=false\" >> ~/.kde4/share/config/kmail2rc", valid => 0);
x11_start_program("echo \"[General]\" >> ~/.config/kmail2rc",                    valid => 0);
x11_start_program("echo \"first-start=false\" >> ~/.config/kmail2rc",            valid => 0);

There is no checks for status, and video doesn't show akregator at all.

Thanks for zluo for investigating this.

I've tried to reproduce the issue locally, but the test failed on 'start_wayland_plasma5' module.

The appropriate bcs ticket is marked as RESOLVED FIXED on 13.08.2018, but I've found out that the issue is still happening on o3 on 15.08.2018 and also I've reproduced it locally with the latest build.

So, I've asked fvogt if the fix already applied in the last build, but not he didn't reply me yet.

A fix 'start_wayland_plasma5' module was applied in https://bugzilla.opensuse.org/show_bug.cgi?id=1105798.

So, after the fix will appear on o3, it may be possible to reproduce the issue with 'kontact' module.

Fix for 'start_wayland_plasma5' is on o3, so I'm in progress of investigating the 'kontakt' issue.

use assert_and_click, create a new needle for the fix:

http://e13.suse.de/tests/9420#step/kontact/15

PR:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5916

need a test run against latest build and create a needle for close dialog with fatal error:

https://openqa.opensuse.org/tests/735133#step/kontact/15

PR:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6226

http://e13.suse.de/tests/10533#step/kontact/16 shows the example where x11_start_program('killall kontact', valid => 0); doesn't work.

http://e13.suse.de/tests/10533#step/kontact/16 shows the example: after killall kontact, akregator is still there

Now trying with:

record_soft_failure('akregator cannot be closed, related to issue of bsc#1105207') && return if (check_screen 'akregator-not-closed');

needle PR:
https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/481

http://e13.suse.de/tests/10638#step/kontact/11 shows performance issue, this is really weird with tying character, it starts Amarok.

QEMURAM 1536 is not so much to run the whole tests...

https://openqa.opensuse.org/tests/805158#step/kontact/15 shows fatal error, so this problem is not sporadic.

PR updated again.

PR updated again.

needle PR got updated.

PR merged. Now we can focus again on the original issue of akregator showing up instead of kontact.

zluo@f40:/var/lib/openqa/tests/opensuse/tests> cnf akregator
The program 'akregator' can be found in the following package:
* akregator [ path: /usr/bin/akregator, repository: zypp (download.opensuse.org-oss) ]
Try installing with:
sudo zypper install akregator
zluo@f40:/var/lib/openqa/tests/opensuse/tests> cnf kontact
The program 'kontact' can be found in the following package:
* kontact [ path: /usr/bin/kontact, repository: zypp (download.opensuse.org-oss) ]
Try installing with:
sudo zypper install kontact

Not sure what you want to tell with the latest two comments however the original problem was, as described in #35589#note-20 that akregator is started by a partially typed or mistyped command in the desktop runner. Immediately after the start of the test module "kontact" the test starts to type "a" for "akonadictl" in https://openqa.opensuse.org/tests/832330/file/video.ogv#t=306.21,306.25 for which krunner suggests amarok which then somehow consumes the next character "k" and the typing continues with "o", see https://openqa.opensuse.org/tests/832330/file/video.ogv#t=306.41,306.45 so it can only turn out wrong.

well, this kind of typing issue happens sometimes and we don't have a good idea to fix this which is more related to setup/performance. My tests runs don't show this issue. So it is very hard to reproduce.

of course we should try to make test module more robust and provide workaround if needed. In this case with Akonadi service, if akondadi service is not running, starting kontact should work anyway. If akregator got started wrongly by typing issue, then we cannot fix this, but try to check this error and provide softfail for that.
This is already provided by my PR.

zluo wrote:

If akregator got started wrongly by typing issue, then we cannot fix this

I am sure we can fix it. If this is only shown by our automated tests then we can either ensure that this is handled as a valid bug and fixed accordingly (I am not aware of any bug report) or at least provide a workaround if we can not find a fix.

If you can not reproduce it locally you can try out https://progress.opensuse.org/issues/44327 and trigger tests on production based on your local branch.

zluo wrote:

This is already provided by my PR.

Why PR? https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6226 ? It only aborts the test early on boo#1105207

If you do not have further plans for this ticket yourself it's ok to unassign.

https://openqa.opensuse.org/tests/835854#step/kontact/8 is one of the latest failures. The screenshot here directly shows the incorrectly typed string in krunner.

Hm, maybe I did something wrong on cloning because of not configuring the VM for virtio.

First a verification for the helper commits in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6709 with

openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 845661 TEST=poo35589_okurz_kde_wayland_mitigation_off_001 BUILD=poo35589_okurz_kde_wayland_mitigation_off _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop EXTRABOOTPARAMS_BOOT_LOCAL="nopti nospec nospectre_v2 nospec_store_bypass_disable spectre_v2_user=off" _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1

Created job #846236: opensuse-Tumbleweed-DVD-x86_64-Build20190202-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t846236

Looks good. Now, onto loading the right modules again:

openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 845661 TEST=poo35589_okurz_kde_wayland_mitigation_off_001 BUILD=poo35589_okurz_kde_wayland_mitigation_off _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/amarok,tests/x11/kontact EXTRABOOTPARAMS_BOOT_LOCAL="nopti nospec nospectre_v2 nospec_store_bypass_disable spectre_v2_user=off" _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1

Created job #846305: opensuse-Tumbleweed-DVD-x86_64-Build20190202-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t846305

failed in amarok. Maybe it's better to start tests first with kontact only?

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 845661 TEST=poo35589_okurz_kde_wayland_mitigation_off_$i BUILD=poo35589_okurz_kde_wayland_mitigation_off _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact EXTRABOOTPARAMS_BOOT_LOCAL="nopti nospec nospectre_v2 nospec_store_bypass_disable spectre_v2_user=off" _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1 ; done

-> https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=poo35589_okurz_kde_wayland_mitigation_off

and for crosschecking with mitigation on:

$ for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 845661 TEST=poo35589_okurz_kde_wayland_kontact_only_$i BUILD=poo35589_okurz_kde_wayland_kontact_only _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1 ; done

-> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&distri=opensuse&build=poo35589_okurz_kde_wayland_kontact_only

Comparing failure rate in both and see if 1) mitigation off makes a difference 2) kontact is stable or fails even if it's the only module (might need krunner module for settle-down anyway).

Result¶

https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=poo35589_okurz_kde_wayland_mitigation_off failed in 17/100 jobs vs. https://openqa.opensuse.org/tests/overview?version=Tumbleweed&distri=opensuse&build=poo35589_okurz_kde_wayland_kontact_only failed in 25/100 jobs. Let's schedule 2x100 more for better statistics. I have the assumption though that the mitigation off helps however not all.

EDIT: That's now 39/200 -> 19.5% failure rate for mitigation off, 44/200 -> 22% failure rate for mitigation on -> no significant difference

Next steps¶

sysrich mentioned a potential relation to boo#1112824 because the Tumbleweed kernel as preemption enforced whereas on older versions and SLE and Leap we use voluntary preemption (or off? Something along those lines). Would be worth a try to run "kernel-vanilla" but I am not sure if we have a "test module" ensuring the installation of that package. Next step after that would be to rebuild the kernel with the changed setting and trying that one.

EDIT: We do have a test module to change the kernel, tests/kernel/change_kernel.pm

for i in {001..001}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 845661 TEST=poo35589_okurz_kde_wayland_mitigation_offkernel-vanilla_$i BUILD=poo35589_okurz_kde_wayland_mitigation_off_kernel-vanilla _GROUP=0 SCHEDULE=tests/kernel/change_kernel,tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact EXTRABOOTPARAMS_BOOT_LOCAL="nopti nospec nospectre_v2 nospec_store_bypass_disable spectre_v2_user=off" _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1 CHANGE_KERNEL_REPO=https://download.opensuse.org/repositories/Kernel:/stable/standard/ CHANGE_KERNEL_PKG=kernel-vanilla; done

Created job #846861: opensuse-Tumbleweed-DVD-x86_64-Build20190202-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t846861 -> passed so this approach works. 12 minutes vs. 8 minutes runtime. Let's schedule more of these:

for i in {002..200}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 845661 TEST=poo35589_okurz_kde_wayland_mitigation_offkernel-vanilla_$i BUILD=poo35589_okurz_kde_wayland_mitigation_off_kernel-vanilla _GROUP=0 SCHEDULE=tests/kernel/change_kernel,tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact EXTRABOOTPARAMS_BOOT_LOCAL="nopti nospec nospectre_v2 nospec_store_bypass_disable spectre_v2_user=off" _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1 CHANGE_KERNEL_REPO=https://download.opensuse.org/repositories/Kernel:/stable/standard/ CHANGE_KERNEL_PKG=kernel-vanilla; done

-> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&distri=opensuse&build=poo35589_okurz_kde_wayland_mitigation_off_kernel-vanilla

EDIT: 40/200 failed -> 20% failure rate -> no significant difference, "kernel-default" is not worse than "kernel-vanilla"

How about the same as above with default kernel and no changes to mitigation with Leap 15.0:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 844165 TEST=poo35589_okurz_kde_wayland_kontact_only_leap151_$i BUILD=poo35589_okurz_kde_wayland_kontact_only_leap151 _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1 ; done

-> https://openqa.opensuse.org/tests/overview?build=poo35589_okurz_kde_wayland_kontact_only_leap151&version=15.1&distri=opensuse

The last check on Leap15.1 did not work. All 100 jobs failed in https://openqa.opensuse.org/tests/847346#step/boot_to_desktop/5 to boot to a graphical desktop although the serial port shows that after 700s there are still some services responding with output. Currently no good idea what is wrong here.

Crosschecking as suggested in https://bugzilla.opensuse.org/show_bug.cgi?id=1112824#c143

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 847596 TEST=poo35589_okurz_kde_wayland_mitigation_off_kernel-twvolun_$i BUILD=poo35589_okurz_kde_wayland_mitigation_off_kernel-twvolun _GROUP=0 SCHEDULE=tests/kernel/change_kernel,tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact EXTRABOOTPARAMS_BOOT_LOCAL="nopti nospec nospectre_v2 nospec_store_bypass_disable spectre_v2_user=off" _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1 CHANGE_KERNEL_REPO=https://download.opensuse.org/repositories/home:/favogt:/twvolun/standard/; done

-> https://openqa.opensuse.org/tests/overview?distri=opensuse&build=poo35589_okurz_kde_wayland_mitigation_off_kernel-twvolun&version=Tumbleweed

EDIT: 17/99 failed (1 incomplete) -> 17% failure rate -> no significant difference

TODO: try to test successfully on Leap, any version. I have the feeling that we are using krunner too much in kontact. We could make that more stable with more waiting or using e.g. the user-console although that is most likely not faster because of switching and console enabling and such. Even worse is probably xterm where we set the prompt and disable the serial terminal and such multiple times.

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6942 created to improve robustness of start_wayland_plasma5 as a partial improvement of a side-failure.

So based on above results I come to the conclusion that we should either type even slower in krunner in the wayland scenario or use an xterm. However, for failures like https://openqa.opensuse.org/tests/848413#step/kontact/18 where we fail to type a single word correctly I doubt that xterm is really any better. So let's try to type slower in krunner@wayland.

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 864281 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #866260: opensuse-Tumbleweed-DVD-x86_64-Build20190226-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t866260 -> https://openqa.opensuse.org/tests/overview?distri=opensuse&build=poo35589_okurz_kde_wayland_very_slow_krunner&version=Tumbleweed

Still failed in some instances, e.g. https://openqa.opensuse.org/tests/866262#step/kontact/8 mentioning "ho" instead of "echo" so I added a wait_still_screen(3) for the case of WAYLAND in init_desktop_runner:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 864281 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

-> https://openqa.opensuse.org/tests/overview?distri=opensuse&build=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen&version=Tumbleweed

Still some instances of incomplete strings. https://openqa.opensuse.org/tests/866980/file/video.ogv#t=27.30,27.31 shows that of the word "kontact" only "nt…" is showing up in the krunner dialog and the first two characters ended up with looks like yet another, previous krunner dialog. Trying with substrings.

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 864281 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact _SKIP_POST_FAIL_HOOKS=1 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

-> https://openqa.opensuse.org/tests/overview?build=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split&distri=opensuse&version=Tumbleweed

6/99 failed, that's significantly less. However the failures in most cases look like still the same, e.g. some characters are typed in a krunner dialog, the next characters end up in a krunner dialog which pops up in a different location. I assume krunner is really crashing but nevertheless I guess we need a workaround.

To try one more time something different, typing one character, waiting, second character, waiting, typing rest plus collect logs from post_fail_hook:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 864281 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

-> Created job #868058: opensuse-Tumbleweed-DVD-x86_64-Build20190226-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t868058 -> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&build=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs&distri=opensuse

That's funny, when we fail, we reproducibly fail to collect and upload logs as in https://openqa.opensuse.org/tests/868131#step/kontact/38 showing only the help for "ar" when it should be "tar …".

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 864281 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #869814: opensuse-Tumbleweed-DVD-x86_64-Build20190226-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t869814 ->
https://openqa.opensuse.org/tests/overview?build=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs&distri=opensuse&version=Tumbleweed

Still missing the first character of "tar" in https://openqa.opensuse.org/tests/868131#step/kontact/38 . Writing an explicit single whitespace in export_kde_logs.

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 864281 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs_type_explicit_whitespace _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #869914: opensuse-Tumbleweed-DVD-x86_64-Build20190226-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t869914 -> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&build=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs_type_explicit_whitespace&distri=opensuse

https://openqa.opensuse.org/tests/870044 has more logs now but could not find any "coredumps" or similar. The krunnerrc consists of:

[General]
history=kontact,echo "first-start=false" >> ~/.config/kmail2rc,echo "first-start=false" >> ~/.kde4/share/config/kmail2rc,ho "[General]" >> ~/.kde4/share/config/kmail2rc,ec,akonadictl start,xterm

[PlasmaRunnerManager]
LaunchCounts=1 services_xterm.desktop

so no "plugins" explicitly enabled but also interesting that the characters "ec" of the last "echo" are showing in the "history" variable, separated from "ho …"

Further ideas: Call check_desktop_runner in start_wayland_plasma5 explicitly as well as an explicit test module for krunner which types the potentially problematic string "echo":

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 864281 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs_type_explicit_whitespace_plus_krunner_test _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/krunner,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #870056: opensuse-Tumbleweed-DVD-x86_64-Build20190226-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t870056 -> https://openqa.opensuse.org/tests/overview?build=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs_type_explicit_whitespace_plus_krunner_test&distri=opensuse&version=Tumbleweed

Could not find anything obvious on logs. Of course, quite some messages that look like error messages but I am not sure which one would point to the problem I observe and I guess there is no use in me reporting openSUSE bugs about any of the specific issues. Could be upstream bugs however that's not my primary concern now.

On a VM running openSUSE Leap 15.1 staring "plasma (wayland)" I could not reproduce the problems manually but I also wanted to try out what happens when I try to crash krunner. Starting krunner in konsole and calling pkill -SIGSEGV krunner causes the krash dialog with the frowning smiley in systray to show up and also console messages state "KCrash: Attemping to start /usr/bin/krunner from kdeinit" and "KCrash: Application 'krunner' crashing..." so an explicit message. Nothing I have seen in logs. What I have learned is that in the xsession-errors log, e.g. in https://openqa.opensuse.org/tests/870044/file/kontact-XSE.log the message "Using Wayland-EGL" should point to krunner starting initially. Whenever I open krunner in a VM manually I see another error message about an unexpected attribute on a virtual screen but none of that kind for the openQA VMs.

The web research results are very limited so far, best match https://opensuse.opensuse.narkive.com/OkKXNi2z/aargh-why-does-krunner-keep-disappearing suggesting to disable plugins. https://lists.opensuse.org/opensuse/2010-03/msg00039.html mentions how I could debug. If disabling plugins by config file does not work maybe I can delete some files

Trying to disable krunner plugins:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 864281 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs_type_explicit_whitespace_plus_krunner_test_krunner_plugins_disabled _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/krunner,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

-> Created job #870840: opensuse-Tumbleweed-DVD-x86_64-Build20190226-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t870840 -> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&build=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs_type_explicit_whitespace_plus_krunner_test_krunner_plugins_disabled&distri=opensuse

Seems a lot of jobs are incomplete now because of missing assets, let's try based on more recent base job 870613:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 870613 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs_type_explicit_whitespace_plus_krunner_test_krunner_plugins_disabled _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/krunner,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #871054: opensuse-Tumbleweed-DVD-x86_64-Build20190305-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t871054 -> https://openqa.opensuse.org/tests/overview?build=poo35589_okurz_kde_wayland_very_slow_krunner_wait_still_screen_split_twice_with_logs_fixed_export_kde_logs_type_explicit_whitespace_plus_krunner_test_krunner_plugins_disabled&distri=opensuse&version=Tumbleweed

all incomplete well, not sure why.

A retriggered job https://openqa.opensuse.org/tests/8713700 works ok still so something with the code? Why no autoinst-log.txt then?

I have the suspicion that the build string is getting too long :D

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 870613 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_slow_krunner_plugins_disabled _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/krunner,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

-> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&build=poo35589_okurz_kde_wayland_slow_krunner_plugins_disabled&distri=opensuse

same fail ratio, still failing like in
https://openqa.opensuse.org/tests/871386/file/video.ogv#t=32.08,32.14
Only shell commands should be enabled but it looks like some software center suggestions are still there. I guess I need to recreate krunnerrc from a more recent base system, Trying in TW and Krypton VM.

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 870613 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_slow_more_krunner_plugins_disabled _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/krunner,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #871511: opensuse-Tumbleweed-DVD-x86_64-Build20190305-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t871511 -> https://openqa.opensuse.org/tests/overview?build=poo35589_okurz_kde_wayland_slow_more_krunner_plugins_disabled&distri=opensuse&version=Tumbleweed

still failing to type "first-start" stuff not that stable

We should type less commands in kontact:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 870613 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_slow_krunner_plugins_disabled_less_kontact_commands _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/krunner,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner2 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

-> Created job #871671: opensuse-Tumbleweed-DVD-x86_64-Build20190305-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t871671 -> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&build=poo35589_okurz_kde_wayland_slow_krunner_plugins_disabled_less_kontact_commands&distri=opensuse

This reduced the fail ratio to 4/100 failed. In all four cases the problem is that the "echo" command is ineffective due to an incorrect string in krunner, e.g. as visible in https://openqa.opensuse.org/tests/871752/file/video.ogv#t=19.42,19.46 we see the incomplete string e "[Ge instead of expected echo "[Ge. Should we really type that command in the user console instead?

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 870613 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_kontact_commands_user_console_skip_krunner _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner4 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #871811: opensuse-Tumbleweed-DVD-x86_64-Build20190305-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t871811 -> https://openqa.opensuse.org/tests/overview?distri=opensuse&build=poo35589_okurz_kde_wayland_kontact_commands_user_console_skip_krunner&version=Tumbleweed

Hm, still failing to type "kontact" or "killall" correctly, same fail rate.

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 870613 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_krunner_plugins_disabled_user_console_match_typed _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/krunner,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner4 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #871951: opensuse-Tumbleweed-DVD-x86_64-Build20190305-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t871951 -> https://openqa.opensuse.org/tests/overview?distri=opensuse&build=poo35589_okurz_kde_wayland_krunner_plugins_disabled_user_console_match_typed&version=Tumbleweed

Added another 100 to the set. All 200 passed now \o/ That is 38fafc7c5 in fix/krunner4

Wanted to try out with the "match_typed" parameter but actually made the mistake to not actually commit it. So actually there should be no difference to before when it was last failing. Adding another 200 to the set.

I could try to enable more debug info for krunner, e.g. with kdebugsettings, but manually that does not bring so much more information.

As 400 (!) jobs pass now I want to crosscheck if I need all the changes, partially reverting, e.g. no string splitting, no krunner module, no wait_still_screen:

for i in {001..40}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 870613 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_user_console_rest_reverted _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner5 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #872359: opensuse-Tumbleweed-DVD-x86_64-Build20190305-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t872359 -> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&distri=opensuse&build=poo35589_okurz_kde_wayland_user_console_rest_reverted

1/40 failed. Keeping the string splitting + wait_still_screen:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 870613 TEST=poo35589_okurz_kde_wayland_$i BUILD=poo35589_okurz_kde_wayland_user_console_rest_reverted2 _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner5 MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #872830: opensuse-Tumbleweed-DVD-x86_64-Build20190305-extra_tests_on_kde@64bit -> https://openqa.opensuse.org/t872830 -> https://openqa.opensuse.org/tests/overview?version=Tumbleweed&distri=opensuse&build=poo35589_okurz_kde_wayland_user_console_rest_reverted2

Still 4/41 failed. I should stay on fix/krunner4 for now

How does the the current "kde-wayland" scenario fare based on fix/krunner4:

for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 872655 TEST=poo35589_okurz_kde_wayland_base_$i BUILD=poo35589_okurz_kde_wayland_base_fix_krunner4 _GROUP=0 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner4; done

-> https://openqa.opensuse.org/tests/overview?distri=opensuse&build=poo35589_okurz_kde_wayland_base_fix_krunner4&version=Tumbleweed

Some additional fails in firefox_audio but it's hard to see as the modules "ooffice, oocalc, oomath" always fail. I should exclude them. Also, should run these longer scenarios with reduced prio:

build=poo35589_okurz_kde_wayland_base_fix_krunner4; for i in {001..100}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 872655 TEST=poo35589_okurz_kde_wayland_base_$i BUILD=$build _GROUP=0 CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#fix/krunner4 _SKIP_POST_FAIL_HOOKS=1 EXCLUDE_MODULES=ooffice,oomath,oocalc; done ; for i in $(openqa_client_o3 --json-output jobs build=$build state=scheduled | jq '.jobs | .[] | .id') ; do openqa_client_o3 jobs/$i put --json-data '{"priority": 90}'; done

-> https://openqa.opensuse.org/tests/overview?distri=opensuse&build=poo35589_okurz_kde_wayland_base_fix_krunner4&version=Tumbleweed

PR reverted due to https://progress.opensuse.org/issues/49406

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: kde-wayland
https://openqa.opensuse.org/tests/907714

Followup to PR by others:

build=poo35589_okurz_kde_wayland_qt_log; for i in {001..001}; do openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org 883005 TEST=poo35589_okurz_kde_wayland_$i BUILD=$build _GROUP=0 SCHEDULE=tests/boot/boot_to_desktop,tests/x11/start_wayland_plasma5,tests/x11/krunner,tests/x11/kontact CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse#enhance/krunner MACHINE=64bit_virtio QEMUVGA=virtio WAYLAND=1; done

Created job #963260: opensuse-Tumbleweed-DVD-x86_64-Buildpoo35589_okurz_kde_wayland_qt_log-poo35589_okurz_kde_wayland_001@64bit_virtio -> https://openqa.opensuse.org/t963260

I checked on o3 for tw, atm we don't have any issue with kontact.

trigger 50 test run on f40.suse.de and see whether I can reproduce any issue with it.

check_bsc982138; of start_install.pm is not necessary since the production issue got fixed.

found issue with bootloader, this is quite strange:

http://f40.suse.de/tests/5844#step/bootloader/1 bootmenu-TW-xmas-20191209 matched but then the test failed :((

http://f40.suse.de/tests/5846#step/bootloader/1 successful