Project

General

Profile

action #27922

[functional][sporadic][hard]test fails in yast2_proxy - timeout issue?

Added by okurz about 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Bugs in existing tests
Start date:
2017-11-20
Due date:
2017-12-06
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-yast2_ncurses@64bit fails in
yast2_proxy
looks like a sporadic timeout related issue

Reproducible

Fails since (at least) Build 20170925

Expected result

Last good: previous job run was fine: https://openqa.opensuse.org/tests/536539#step/yast2_proxy/176

Acceptance criteria

  • AC1: yast2_proxy runs stable in 20/20 runs

Tasks

  • Investigate recent failures including logfiles (0.2-1h) -> DONE, see https://progress.opensuse.org/issues/27922#note-2
  • Find out in which step a potential timeout might be missing, wrong, too low (2-4h) -> PARTIALLY DONE, most likely not the wait_serial one
  • Verify with statistics (1-2h)

Further details

Always latest result in this scenario: latest

History

#1 Updated by mgriessmeier about 4 years ago

  • Assignee set to mgriessmeier

#2 Updated by mgriessmeier about 4 years ago

  • Subject changed from [functional][sporadic][medium]test fails in yast2_proxy - timeout issue? to [functional][sporadic][hard]test fails in yast2_proxy - timeout issue?
  • Description updated (diff)
  • Assignee deleted (mgriessmeier)

so first findings:

  • it failed in 3 out of the last 20 runs
  • wait_serial is called in the end of the module to look for the string 'yast2-squid-status-0'

Timeout is six minutes with a comment in the code that it might take long on sle12.
The weird thing is, that in cases where it works, it finds that string in between 4 and 8 seconds, so it's not even a cornercase which lead me to the assumption that this timeout is not the one to be tweaked

in bad cases we sometimes see 'VNC stalled' messages and sometimes not

good (found within 3 seconds) -> https://openqa.opensuse.org/tests/531465:

11:23:16.5019 28627 <<< testapi::wait_serial(timeout=360, regexp='yast2-squid-status-0')
11:23:19.5868 28627 >>> testapi::wait_serial: yast2-squid-status-0: ok

bad (simply times out after 360s) -> https://openqa.opensuse.org/tests/536670:

08:45:04.0976 31833 <<< testapi::wait_serial(timeout=360, regexp='yast2-squid-status-0')
08:51:05.3207 31833 >>> testapi::wait_serial: yast2-squid-status-0: fail

bad (with VNC stalls) -> https://openqa.opensuse.org/tests/535389:

23:15:12.7980 27044 <<< testapi::wait_serial(regexp='yast2-squid-status-0', timeout=360)
23:16:32.3292 27046 considering VNC stalled, no update for 4.00 seconds
23:16:32.5831 27046 pointer type 1 0 1024 768 -257
23:16:32.5833 27046 led state 0 1 1 -261
23:17:20.8797 27046 considering VNC stalled, no update for 4.00 seconds
23:17:21.1299 27046 pointer type 1 0 1024 768 -257
23:17:21.1300 27046 led state 0 1 1 -261
23:18:18.9423 27046 considering VNC stalled, no update for 4.00 seconds
23:18:19.1955 27046 pointer type 1 0 1024 768 -257
23:18:19.1955 27046 led state 0 1 1 -261
23:19:05.2512 27046 considering VNC stalled, no update for 4.00 seconds
23:19:05.5016 27046 pointer type 1 0 1024 768 -257
23:19:05.5017 27046 led state 0 1 1 -261
23:19:51.5551 27046 considering VNC stalled, no update for 4.01 seconds
23:19:51.8068 27046 pointer type 1 0 1024 768 -257
23:19:51.8069 27046 led state 0 1 1 -261
23:21:01.6518 27046 considering VNC stalled, no update for 4.00 seconds
23:21:01.9051 27046 pointer type 1 0 1024 768 -257
23:21:01.9053 27046 led state 0 1 1 -261
23:21:14.0934 27044 >>> testapi::wait_serial: yast2-squid-status-0: fail

3 out of 20 fails without some obvious reference point to start with, I consider this as hard
I'm unassigning myself because I won't find the time to work on that in the next few days

#3 Updated by mgriessmeier almost 4 years ago

  • Assignee set to mgriessmeier

taking again to do some more investigation

#4 Updated by mgriessmeier almost 4 years ago

  • Status changed from In Progress to Feedback

updated needle -> https://github.com/os-autoinst/os-autoinst-needles-opensuse/pull/293
and triggered 25 jobs on opeth.suse.de

#5 Updated by mgriessmeier almost 4 years ago

  • Status changed from Feedback to Resolved

25/25 passed yast2_proxy tests on opeth.suse.de, PR merged

Also available in: Atom PDF