Project

General

Profile

action #27020

[sle][functional][medium][sporadic] test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side)

Added by okurz about 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Start date:
2017-10-25
Due date:
2017-12-06
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-Installer-DVD-s390x-create_hdd_textmode@zkvm fails in
change_desktop

Reproducible

Fails since (at least) Build 318.1 that is three times already so seems to be reproducible

AC1: Improve robustness of this test-module

Expected result

Last good: 310.1

Problem

  • REJECTED H1 Behaviour of product changed consistently -> need to adapt test
  • REJECTED H2 Recent changes to our test? Not aware of any here
  • CONFIRMED H3: race condition/timing issue i.e. keys pressed too fast (tab is pressed twice sometimes)

Further details

Always latest result in this scenario: latest

History

#1 Updated by okurz about 4 years ago

  • Due date set to 2017-11-22
  • Target version set to Milestone 12

#3 Updated by okurz about 4 years ago

  • Due date changed from 2017-11-22 to 2017-12-06

#4 Updated by mgriessmeier about 4 years ago

  • Subject changed from [sle][functional]test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side) to [sle][functional][medium] test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side)
  • Status changed from New to In Progress
  • Assignee set to mgriessmeier

#5 Updated by mgriessmeier about 4 years ago

  • Subject changed from [sle][functional][medium] test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side) to [sle][functional][medium][sporadic] test fails in change_desktop on s390x zkvm, wrong screen section selected (package list, not pattern list on left side)
  • Description updated (diff)

found out that this special behaviour is sporadic by looking at the previous results.

it first appeared in Build 318.1 and then happened only twice in the following 10 tests.
So I assume some kind of race-condition which results into an acceptance criteria of making the test more robust

#6 Updated by mgriessmeier about 4 years ago

I've identified the real issue as sometimes 'send_key_until_needlematch' sometimes presses 'tab' twice instead of only once because the needle is probably not specific enough - will come up with a fix.

fail:

14:51:41.0405 Debug: /var/lib/openqa/cache/tests/sle/tests/installation/change_desktop.pm:48 called testapi::send_key_until_needlematch
14:51:41.0407 13374 <<< testapi::check_screen(mustmatch='patterns-list-selected', timeout=1)
14:51:41.6862 13376 no match: 0.9s
...
14:51:42.8387 13374 >>> testapi::_check_backend_response: match=patterns-list-selected timed out after 1
14:51:42.8755 Debug: /var/lib/openqa/cache/tests/sle/tests/installation/change_desktop.pm:48 called testapi::send_key_until_needlematch
14:51:42.8757 13374 <<< testapi::send_key(key='tab', do_wait=0)
14:51:43.0807 Debug: /var/lib/openqa/cache/tests/sle/tests/installation/change_desktop.pm:48 called testapi::send_key_until_needlematch
14:51:43.0809 13374 <<< testapi::check_screen(mustmatch='patterns-list-selected', timeout=1)
14:51:43.1365 13376 MATCH(patterns-list-selected-20141218:0.00)
...
14:51:43.8178 13374 >>> testapi::_handle_found_needle: found patterns-list-selected-20160121, similarity 1.00 @ 16/65

good:

18:28:51.0453 Debug: /var/lib/openqa/cache/tests/sle/tests/installation/change_desktop.pm:48 called testapi::send_key_until_needlematch
18:28:51.0455 16573 <<< testapi::check_screen(mustmatch='patterns-list-selected', timeout=1)
18:28:51.1083 16579 MATCH(patterns-list-selected-20141218:0.00)
18:28:51.1699 16579 MATCH(patterns-list-selected-20150721:0.49)
18:28:51.2248 16579 MATCH(patterns-list-selected-20160121:0.00)
18:28:51.2895 16579 MATCH(patterns-list-selected-20160504:0.08)
18:28:51.3507 16579 MATCH(patterns-list-selected-not-highligted-20160504:0.00)
18:28:51.4038 16579 MATCH(patterns-list-selected-ssh-20150212:0.00)
18:28:51.4552 16579 MATCH(patterns-list-selected-text-20141218:0.00)
18:28:51.5161 16579 MATCH(patterns-list-selected-20170802:0.00)
18:28:51.6529 16579 MATCH(patterns-list-selected-20170808:0.00)
18:28:51.7637 16579 MATCH(patterns-list-selected-sle15-20171011:0.00)
...
18:28:51.7643 16579 no match: 0.9s
18:28:52.1004 16579 MATCH(patterns-list-selected-20141218:0.00)
18:28:52.1529 16579 MATCH(patterns-list-selected-20150721:0.48)
18:28:52.1989 16579 MATCH(patterns-list-selected-20160121:1.00)
18:28:52.2537 16579 MATCH(patterns-list-selected-20160504:0.00)
18:28:52.3077 16579 MATCH(patterns-list-selected-not-highligted-20160504:0.00)
18:28:52.3549 16579 MATCH(patterns-list-selected-ssh-20150212:0.00)
18:28:52.4008 16579 MATCH(patterns-list-selected-text-20141218:0.00)
18:28:52.4524 16579 MATCH(patterns-list-selected-20170802:0.00)
18:28:52.5451 16579 MATCH(patterns-list-selected-20170808:0.00)
18:28:52.6393 16579 MATCH(patterns-list-selected-sle15-20171011:0.00)
18:28:52.7251 16573 >>> testapi::_handle_found_needle: found patterns-list-selected-20160121, similarity 1.00 @ 16/65

#7 Updated by mgriessmeier about 4 years ago

  • Description updated (diff)

#8 Updated by mgriessmeier about 4 years ago

  • Status changed from In Progress to Feedback

I propose the following to prevent such cornercases:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3929

I've cloned the job in question 30 times to see if it still happens

setting to feedback

#9 Updated by mgriessmeier about 4 years ago

  • Status changed from Feedback to Resolved

30 verification runs passed, PR merged

-> please reopen if issue occurs again

Also available in: Atom PDF