Project

General

Profile

Actions
Copy link

action #93677

closed

t/full-stack.t runs into 9m make-level timeout exceeding test module internal timeout size:S

Added by okurz over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
dheidler
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2021-06-08
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://app.circleci.com/pipelines/github/os-autoinst/openQA/6619/workflows/f6ed6d22-8098-4e01-8a8d-4b3b4509954f/jobs/62263?invite=true#step-111-86 shows

RETRY=0 timeout -s SIGINT -k 5 -v $((9 * (0 + 1) ))m tools/retry prove -l --harness TAP::Harness::JUnit --timer t/full-stack.t
[15:32:02] t/full-stack.t .. 25/? timeout: sending signal INT to command ‘tools/retry’
make[2]: *** [Makefile:188: test-unit-and-integration] Error 124

so the overall test run is aborted on the level of Makefile after 9m. That means that t/full-stack.t was stuck for that time and also the test module internal timeout in https://github.com/os-autoinst/openQA/blob/master/t/full-stack.t#L59 of 90s with a scaling factor exceeds the 9m. Default scaling from external/os-autoinst-common/lib/OpenQA/Test/TimeLimit.pm is 3*2 so computing to 3*2*90s = 540s ~ 9m conflicting with the makefile time limit

Acceptance criteria

  • AC1: Time-based stability of full-stack test is confirmed
  • AC2: test module internal timeout is well below make-level timeout also in case of CI+cover

Suggestions

  • Check time-based stability of full-stack test locally, e.g. with make test FULLSTACK=1 STABILITY_TEST=1 RETRY=10 KEEP_DB=1 TESTS=t/full-stack.t
  • Bump timeout in Makefile

Related issues 1 (1 open — 0 closed)

Copied to openQA Project (public) - action #95132: t/full-stack.t can take significantly longer than previously 90s, performance regression or environment specific slowness?New

Actions
Actions
Copy link
 #1

Updated by livdywan over 3 years ago

  • Subject changed from t/full-stack.t runs into 9m make-level timeout exceeding test module internal timeout to t/full-stack.t runs into 9m make-level timeout exceeding test module internal timeout size:S
Actions
Copy link
 #2

Updated by osukup over 3 years ago

  • Description updated (diff)
Actions
Copy link
 #3

Updated by osukup over 3 years ago

wheen run locally:

Bailout called. Further testing stopped: test exceeds runtime limit of '90' seconds
Bail out! test exceeds runtime limit of '90' seconds
FAILED--Further testing stopped: test exceeds runtime limit of '90' seconds

randomly between test 160-180

Actions
Copy link
 #4

Updated by dheidler over 3 years ago

  • Assignee set to dheidler
Actions
Copy link
 #5

Updated by dheidler over 3 years ago

Hm - for me it is:

Bailout called.  Further testing stopped:  URL for os-autoinst cmd srv not available
Actions
Copy link
 #6

Updated by dheidler over 3 years ago

  • Status changed from Workable to Blocked

I think this needs to wait until the full-stack test is fixed for TW.

Actions
Copy link
 #7

Updated by okurz over 3 years ago

  • Status changed from Blocked to Feedback

Only use "Blocked" with a ticket reference for blocker

Actions
Copy link
 #10

Updated by okurz over 3 years ago

  • Copied to action #95132: t/full-stack.t can take significantly longer than previously 90s, performance regression or environment specific slowness? added
Actions
Copy link
 #11

Updated by dheidler over 3 years ago

  • Status changed from Feedback to Resolved
Actions
Copy link

Also available in: Atom PDF