Project

General

Profile

action #93677

t/full-stack.t runs into 9m make-level timeout exceeding test module internal timeout size:S

Added by okurz about 2 months ago. Updated 15 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2021-06-08
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

https://app.circleci.com/pipelines/github/os-autoinst/openQA/6619/workflows/f6ed6d22-8098-4e01-8a8d-4b3b4509954f/jobs/62263?invite=true#step-111-86 shows

RETRY=0 timeout -s SIGINT -k 5 -v $((9 * (0 + 1) ))m tools/retry prove -l --harness TAP::Harness::JUnit --timer t/full-stack.t
[15:32:02] t/full-stack.t .. 25/? timeout: sending signal INT to command ‘tools/retry’
make[2]: *** [Makefile:188: test-unit-and-integration] Error 124

so the overall test run is aborted on the level of Makefile after 9m. That means that t/full-stack.t was stuck for that time and also the test module internal timeout in https://github.com/os-autoinst/openQA/blob/master/t/full-stack.t#L59 of 90s with a scaling factor exceeds the 9m. Default scaling from external/os-autoinst-common/lib/OpenQA/Test/TimeLimit.pm is 3*2 so computing to 3*2*90s = 540s ~ 9m conflicting with the makefile time limit

Acceptance criteria

  • AC1: Time-based stability of full-stack test is confirmed
  • AC2: test module internal timeout is well below make-level timeout also in case of CI+cover

Suggestions

  • Check time-based stability of full-stack test locally, e.g. with make test FULLSTACK=1 STABILITY_TEST=1 RETRY=10 KEEP_DB=1 TESTS=t/full-stack.t
  • Bump timeout in Makefile

Related issues

Copied to openQA Project - action #95132: t/full-stack.t can take significantly longer than previously 90s, performance regression or environment specific slowness?New

History

#1 Updated by cdywan 26 days ago

  • Subject changed from t/full-stack.t runs into 9m make-level timeout exceeding test module internal timeout to t/full-stack.t runs into 9m make-level timeout exceeding test module internal timeout size:S

#2 Updated by osukup 25 days ago

  • Description updated (diff)

#3 Updated by osukup 25 days ago

wheen run locally:

Bailout called. Further testing stopped: test exceeds runtime limit of '90' seconds
Bail out! test exceeds runtime limit of '90' seconds
FAILED--Further testing stopped: test exceeds runtime limit of '90' seconds

randomly between test 160-180

#4 Updated by dheidler 24 days ago

  • Assignee set to dheidler

#5 Updated by dheidler 24 days ago

Hm - for me it is:

Bailout called.  Further testing stopped:  URL for os-autoinst cmd srv not available

#6 Updated by dheidler 23 days ago

  • Status changed from Workable to Blocked

I think this needs to wait until the full-stack test is fixed for TW.

#7 Updated by okurz 23 days ago

  • Status changed from Blocked to Feedback

Only use "Blocked" with a ticket reference for blocker

#10 Updated by okurz 18 days ago

  • Copied to action #95132: t/full-stack.t can take significantly longer than previously 90s, performance regression or environment specific slowness? added

#11 Updated by dheidler 15 days ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF