QA » openQA Project » openQA Tests

interesting to note that on ppc64le, there is also an unfocused tip window, but that one is not placed foreground, eg: https://openqa.opensuse.org/tests/921536#step/gnucash/23

https://openqa.opensuse.org/tests/947641#step/gnucash/24 shows no issue atm.

http://f40.suse.de/tests/3808 shows expected result.

need to check:

https://openqa.opensuse.org/tests/947720#next_previous

issue on 3O because of repo which got removed already. Need to clone current job for now.

if (match_has_tag('gnucash-tip-close')) {
send_key 'esc';
}

In past we had issue with send_key esc on machine with poor performance.

to check https://openqa.opensuse.org/tests/948888#live

https://openqa.opensuse.org/tests/948903
Assigned worker: openqa-aarch64:16

failed at consoletest_setup.
works fine:
https://openqa.opensuse.org/tests/948987
Assigned worker: openqa-aarch64:13

the difference is clear.

This is a typing issue appearing randomly. See: https://openqa.opensuse.org/tests/948903#step/consoletest_setup/3
The problem is known but no great fix have been found yet.

https://openqa.opensuse.org/tests/949870#next_previous shows that a couple of failures on gnucash for typing issue which is known and almost all the time for machines with performance issue in general: aarch64, ppc64le. But this kind of issue is not happened at tip windows here.

I agree on that we have typing issue in general, so a fix for this cannot be done by this ticket.

to check https://openqa.opensuse.org/tests/955942 later

https://openqa.opensuse.org/tests/956021#step/gnucash/33 failed at:

assert_screen([qw(generic-desktop gnucash-close-without-saving-changes)]);
assert_and_click('gnucash-close-without-saving-changes') if match_has_tag('gnucash-close-without-saving-changes');

It seems here like timeout issue.

I think that assert_and_click requires coordinates now. Maybe the problem here?

https://openqa.opensuse.org/tests/955977#step/gnucash/26 shows problem: Tip of Day cannot be closed

if (match_has_tag('gnucash-assistant-close')) {
assert_and_click 'gnucash-assistant-close';
assert_and_click 'gnucash-assistant-show-again-no';
assert_screen([qw(gnucash gnucash-tip-close)]);
if (match_has_tag('gnucash-tip-close')) {
send_key 'esc';
}
}

consider to replace send_key 'esc' by assert_and_click (create new needle).

this is sporadic issue however, it works in general with send_key esc or with assert_and_click.

https://openqa.opensuse.org/tests/latest?test=poo%2350492&distri=opensuse&flavor=DVD&version=Tumbleweed&machine=aarch64&arch=aarch64#next_previous

100 test runs shows 3 times failure for gnucash. What we can do is: replace send_key esc to reduce the failure due to performance issue.

WIP PR:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/7662

run
openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/7662 https://openqa.opensuse.org/tests/954695 TEST=poo#46703 _GROUP_ID="0" TEST=poo#50492 _GROUP_ID="0" --skip-deps --skip-download EXCLUDE_MODULES=vino_screensharing_available,rrdtool_x11,yast2_lan_restart,hwsim_wpa2_enterprise_setup,yast2_network_use_nm,NM_wpa2_enterprise,check_default_network_manager --host https://openqa.opensuse.org

to check:
https://openqa.opensuse.org/tests/956636

needle for assert_and_click created on o3 already.

https://openqa.opensuse.org/tests/956790#step/gnucash/27 still failed.

it seems that gnucash-assistant is not coming up at all, need to cover this as well.
to check:
https://openqa.opensuse.org/tests/957503#live

update PR and need to check new test verification runs (50 times):
https://openqa.opensuse.org/tests/957762

https://openqa.opensuse.org/tests/957820#step/gnucash/35 shows still problem with match needle for x11_start_program.

Increase match_timeout => 60 for new try.

check test results: https://openqa.opensuse.org/tests/958469#live

update test-gnucash-2 for match needle at gnucash tutorial.

looks better, but still failed sometimes because cancel from gnucash-assistant. check or update the needle now
see issue: https://openqa.opensuse.org/tests/958657#step/gnucash/33
2 dialogue and 1 window overlapped.

https://openqa.opensuse.org/tests/959003#settings

add step to check gnucash-overlapped and make it in focus and then run assert_and_click

openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/7662 https://openqa.opensuse.org/tests/954695 TEST=poo#50492 _GROUP_ID="0" --skip-deps --skip-download EXCLUDE_MODULES=vino_screensharing_available,rrdtool_x11,yast2_lan_restart,hwsim_wpa2_enterprise_setup,yast2_network_use_nm,NM_wpa2_enterprise,check_default_network_manager,multi_users_dm,keyboard_layout_gdm,seahorse,exiv2,prepare_test_data,consoletest_setup,hostname

to check:
https://openqa.opensuse.org/tests/959369#next_previous

https://openqa.opensuse.org/tests/959413/file/autoinst-log.txt shows still problem with timeout for needle match within more than 3 needles

increase:

x11_start_program('gnucash', target_match => [qw(gnucash gnucash-tip-close gnucash-assistant-close gnucash-overlapped)], match_timeout => 120);

update needle for assert-and-cick as well.

The issue is clear that worker performance caused failure.

to check: https://openqa.opensuse.org/tests/961399

https://openqa.opensuse.org/tests/961384#step/gnucash/41 shows now repo issue with snapshot!

the latest test on o3 https://openqa.opensuse.org/tests/961400 shows problem.
It means I cannot go further with clone job from o3. And remote worker on seattle10 is still not available.

Have to stop work on this ticket and set it as blocked for now.

as I already said that I cannot verify my code changes on o3, that comment #50492#note-31

He just mean that the status blocked requires a ticket.
This should be: #52850

foursixnine is investigating if the kernel feature hugepages is supported by seattle10.
He suggested to clone the job with empty setting: QEMU_HUGE_PAGES_PATH=

If this doesn't work, you can set the ticket to blocked again.

hm, I don't understand how seattle10 would block verification on o3. So far I have not seen any useful statistical investigation. It can be really simple as in:

• before any fixes it fails in 28/100 runs
• with assert_and_click rather than send_key it fails 3/100

or something similar. I recommend to look into the overall system performance. Could be helpful to know if the system is heavily loaded when failing.

ok, it doesn't block it because hugepages feature can be ignored by cloning with the empty setting: http://slindomansilla-vm.qa.suse.de/tests/1472

okurz wrote:

hm, I don't understand how seattle10 would block verification on o3. So far I have not seen any useful statistical investigation. It can be really simple as in:

• before any fixes it fails in 28/100 runs
• with assert_and_click rather than send_key it fails 3/100

or something similar. I recommend to look into the overall system performance. Could be helpful to know if the system is heavily loaded when failing.

assert_and_click hat a timetout and send_key seems to be a problem in general when worker has been heavily loaded. aarch64 workers are in general slow. failure statistic 28/100 is much worse than 3/100 runs.

And the problem is not single aarch64 is apparently weak or slow than other workers.

What I'm trying do is to reduce the failures, otherwise we should reject the ticket.

check now current status for PR:

https://openqa.opensuse.org/tests/972177#live

Rejected the other ticket: You were using an outdated build, please use more recent builds. They are however blocked by https://bugzilla.opensuse.org/show_bug.cgi?id=1138455 so if you want to be blocked, use that bug :)

Still I recommend to do sensible changes in test code which is independant of architecture, test the changes with whatever architecture is most easy for you and get it over with, no need to waste time waiting on aarch64 specific issues.

thanks @okurz
yes, I checked now: https://bugzilla.opensuse.org/show_bug.cgi?id=1138455