Actions
action #27020
closed[sle][functional][medium][sporadic] test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side)
Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
Start date:
2017-10-25
Due date:
2017-12-06
% Done:
0%
Estimated time:
Difficulty:
Description
Observation¶
openQA test in scenario sle-15-Installer-DVD-s390x-create_hdd_textmode@zkvm fails in
change_desktop
Reproducible¶
Fails since (at least) Build 318.1 that is three times already so seems to be reproducible
AC1: Improve robustness of this test-module
Expected result¶
Last good: 310.1
Problem¶
- REJECTED H1 Behaviour of product changed consistently -> need to adapt test
- REJECTED H2 Recent changes to our test? Not aware of any here
- CONFIRMED H3: race condition/timing issue i.e. keys pressed too fast (tab is pressed twice sometimes)
Further details¶
Always latest result in this scenario: latest
Updated by okurz over 6 years ago
- Due date set to 2017-11-22
- Target version set to Milestone 12
Updated by okurz over 6 years ago
recent example: https://openqa.suse.de/tests/1249955
Updated by okurz over 6 years ago
- Due date changed from 2017-11-22 to 2017-12-06
Updated by mgriessmeier over 6 years ago
- Subject changed from [sle][functional]test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side) to [sle][functional][medium] test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side)
- Status changed from New to In Progress
- Assignee set to mgriessmeier
Updated by mgriessmeier over 6 years ago
- Subject changed from [sle][functional][medium] test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side) to [sle][functional][medium][sporadic] test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side)
- Description updated (diff)
found out that this special behaviour is sporadic by looking at the previous results.
it first appeared in Build 318.1 and then happened only twice in the following 10 tests.
So I assume some kind of race-condition which results into an acceptance criteria of making the test more robust
Updated by mgriessmeier over 6 years ago
I've identified the real issue as sometimes 'send_key_until_needlematch' sometimes presses 'tab' twice instead of only once because the needle is probably not specific enough - will come up with a fix.
fail:
14:51:41.0405 Debug: /var/lib/openqa/cache/tests/sle/tests/installation/change_desktop.pm:48 called testapi::send_key_until_needlematch
14:51:41.0407 13374 <<< testapi::check_screen(mustmatch='patterns-list-selected', timeout=1)
14:51:41.6862 13376 no match: 0.9s
...
14:51:42.8387 13374 >>> testapi::_check_backend_response: match=patterns-list-selected timed out after 1
14:51:42.8755 Debug: /var/lib/openqa/cache/tests/sle/tests/installation/change_desktop.pm:48 called testapi::send_key_until_needlematch
14:51:42.8757 13374 <<< testapi::send_key(key='tab', do_wait=0)
14:51:43.0807 Debug: /var/lib/openqa/cache/tests/sle/tests/installation/change_desktop.pm:48 called testapi::send_key_until_needlematch
14:51:43.0809 13374 <<< testapi::check_screen(mustmatch='patterns-list-selected', timeout=1)
14:51:43.1365 13376 MATCH(patterns-list-selected-20141218:0.00)
...
14:51:43.8178 13374 >>> testapi::_handle_found_needle: found patterns-list-selected-20160121, similarity 1.00 @ 16/65
good:
18:28:51.0453 Debug: /var/lib/openqa/cache/tests/sle/tests/installation/change_desktop.pm:48 called testapi::send_key_until_needlematch
18:28:51.0455 16573 <<< testapi::check_screen(mustmatch='patterns-list-selected', timeout=1)
18:28:51.1083 16579 MATCH(patterns-list-selected-20141218:0.00)
18:28:51.1699 16579 MATCH(patterns-list-selected-20150721:0.49)
18:28:51.2248 16579 MATCH(patterns-list-selected-20160121:0.00)
18:28:51.2895 16579 MATCH(patterns-list-selected-20160504:0.08)
18:28:51.3507 16579 MATCH(patterns-list-selected-not-highligted-20160504:0.00)
18:28:51.4038 16579 MATCH(patterns-list-selected-ssh-20150212:0.00)
18:28:51.4552 16579 MATCH(patterns-list-selected-text-20141218:0.00)
18:28:51.5161 16579 MATCH(patterns-list-selected-20170802:0.00)
18:28:51.6529 16579 MATCH(patterns-list-selected-20170808:0.00)
18:28:51.7637 16579 MATCH(patterns-list-selected-sle15-20171011:0.00)
...
18:28:51.7643 16579 no match: 0.9s
18:28:52.1004 16579 MATCH(patterns-list-selected-20141218:0.00)
18:28:52.1529 16579 MATCH(patterns-list-selected-20150721:0.48)
18:28:52.1989 16579 MATCH(patterns-list-selected-20160121:1.00)
18:28:52.2537 16579 MATCH(patterns-list-selected-20160504:0.00)
18:28:52.3077 16579 MATCH(patterns-list-selected-not-highligted-20160504:0.00)
18:28:52.3549 16579 MATCH(patterns-list-selected-ssh-20150212:0.00)
18:28:52.4008 16579 MATCH(patterns-list-selected-text-20141218:0.00)
18:28:52.4524 16579 MATCH(patterns-list-selected-20170802:0.00)
18:28:52.5451 16579 MATCH(patterns-list-selected-20170808:0.00)
18:28:52.6393 16579 MATCH(patterns-list-selected-sle15-20171011:0.00)
18:28:52.7251 16573 >>> testapi::_handle_found_needle: found patterns-list-selected-20160121, similarity 1.00 @ 16/65
Updated by mgriessmeier over 6 years ago
- Status changed from In Progress to Feedback
I propose the following to prevent such cornercases:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3929
I've cloned the job in question 30 times to see if it still happens
setting to feedback
Updated by mgriessmeier over 6 years ago
- Status changed from Feedback to Resolved
30 verification runs passed, PR merged
-> please reopen if issue occurs again
Actions