Project

General

Profile

Actions
Copy link

action #97241

closed

build failure of os-autoinst in 18-backend-qemu.t on OBS (timeout) size:S

Added by okurz over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
okurz
Category:
Regressions/Crashes
Target version:
Ready
Start date:
2021-08-19
Due date:
2021-09-09
% Done:

0%

Estimated time:

Description

Observation

from
Visit https://build.opensuse.org/package/live_build_log/devel:openQA:tested/os-autoinst/openSUSE_Tumbleweed/i586

[  120s] test 3
[  120s]     Start 3: test-perl-testsuite
[  120s] 
[  120s] 3: Test command: /home/abuild/rpmbuild/BUILD/os-autoinst-4.6.1625796159.4048f0f9/tools/invoke-tests "--prove-tool" "/usr/bin/prove" "--make-tool" "/usr/bin/ninja" "--build-directory" "/home/abuild/rpmbuild/BUILD/os-autoinst-4.6.1625796159.4048f0f9/build"
[  120s] 3: Test timeout computed to be: 10000000
[  206s] 3: [22:10:50] ./00-compile-check-all.t ................. ok    86270 ms ( 0.06 usr  0.00 sys + 78.22 cusr  8.16 csys = 86.44 CPU)
[  210s] 3: [22:10:54] ./01-test_needle.t ....................... ok     3903 ms ( 0.02 usr  0.02 sys +  3.58 cusr  0.54 csys =  4.16 CPU)
[  211s] 3: [22:10:55] ./02-test_ocr.t .......................... skipped: No tesseract installed
[  212s] 3: [22:10:56] ./03-testapi.t ........................... ok     1553 ms ( 0.04 usr  0.04 sys +  1.54 cusr  0.38 csys =  2.00 CPU)
[  213s] 3: [22:10:57] ./04-check_vars_docu.t ................... ok      248 ms ( 0.00 usr  0.00 sys +  0.24 cusr  0.04 csys =  0.28 CPU)
[  214s] 3: [22:10:58] ./04-testapi-python.t .................... ok      993 ms ( 0.00 usr  0.00 sys +  0.92 cusr  0.08 csys =  1.00 CPU)
[  215s] 3: [22:10:58] ./05-distribution.t ...................... ok      873 ms ( 0.00 usr  0.00 sys +  0.80 cusr  0.06 csys =  0.86 CPU)
[  216s] 3: [22:11:00] ./08-autotest.t .......................... ok     1105 ms ( 0.02 usr  0.00 sys +  0.96 cusr  0.14 csys =  1.12 CPU)
[  218s] 3: [22:11:02] ./10-terminal.t .......................... ok     2131 ms ( 0.02 usr  0.00 sys +  0.94 cusr  0.18 csys =  1.14 CPU)
[  222s] 3: [22:11:06] ./10-test-image-conversion-benchmark.t ... ok     4172 ms ( 0.02 usr  0.00 sys +  3.94 cusr  0.60 csys =  4.56 CPU)
[  223s] 3: [22:11:07] ./11-image-ppm.t ......................... ok      964 ms ( 0.00 usr  0.00 sys +  0.96 cusr  0.26 csys =  1.22 CPU)
[  224s] 3: [22:11:08] ./12-bmwqemu.t ........................... ok      754 ms ( 0.00 usr  0.00 sys +  0.68 cusr  0.08 csys =  0.76 CPU)
[  225s] 3: [22:11:08] ./15-logging.t ........................... ok      747 ms ( 0.00 usr  0.02 sys +  0.70 cusr  0.06 csys =  0.78 CPU)
[  225s] 3: [22:11:09] ./16-send_with_fd.t ...................... ok      865 ms ( 0.00 usr  0.00 sys +  0.88 cusr  0.16 csys =  1.04 CPU)
[  226s] 3: [22:11:10] ./17-basetest.t .......................... ok     1027 ms ( 0.00 usr  0.06 sys +  0.92 cusr  0.12 csys =  1.10 CPU)
[  267s] 3: Bailout called.  Further testing stopped:  test exceeds runtime limit of '40' seconds
[  267s] 3: FAILED--Further testing stopped: test exceeds runtime limit of '40' seconds
[  267s] 3/3 Test #3: test-perl-testsuite ..............***Failed  147.02 sec
[  267s]

the next test after 17-basetest.t is 18-backend-qemu.t which is most likely running into the timeout

Suggestion

  • Check if this occured before
  • Try and reproduce it locally
  • Increase the scaling factor for OBS - maybe OBS was just slow here
  • Should test exceeds runtime limit of '40' seconds have included the failing test name? ($0)
Actions
Copy link
 #1

Updated by livdywan over 3 years ago

  • Subject changed from build failure of os-autoinst in OBS to build failure of os-autoinst in 18-backend-qemu.t on OBS (timeout) size:S
  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #2

Updated by okurz over 3 years ago

  • Assignee set to okurz

https://build.opensuse.org/package/live_build_log/devel:openQA:tested/os-autoinst/openSUSE_Tumbleweed/i586 is currently fine and looks like the error did not reproduce often
Bumped OBS specific timeout scaling in https://github.com/os-autoinst/os-autoinst/pull/1754

Created draft https://github.com/os-autoinst/os-autoinst/pull/1755 not to be intended to be merged. This should artifically trigger timeouts in both github actions as well as in OBS.

Same for openQA in https://github.com/os-autoinst/openQA/pull/4149

Actions
Copy link
 #3

Updated by okurz over 3 years ago

  • Due date set to 2021-09-09
  • Status changed from Workable to In Progress
Actions
Copy link
 #4

Updated by okurz over 3 years ago

  • Status changed from In Progress to Feedback

https://github.com/os-autoinst/os-autoinst/pull/1754 with the fix merged.

For the feedback which test runs into timeout in openQA from https://github.com/os-autoinst/openQA/pull/4149#issuecomment-906402870 circleCI clearly reports the test that times out:

From circleCI:

[12:43:27] t/03-auth.t ............................................... 1/? Bailout called.  Further testing stopped:  test exceeds runtime limit of '18' seconds
FAILED--Further testing stopped: test exceeds runtime limit of '18' seconds
make[2]: *** [Makefile:191: test-unit-and-integration] Error 255

OBS does not:

[ 1007s] ./t/full-stack.t ............................................ skipped: set FULLSTACK=1 (be careful)
[ 1030s] Bailout called.  Further testing stopped:  test exceeds runtime limit of '20' seconds
[ 1030s] FAILED--Further testing stopped: test exceeds runtime limit of '20' seconds
[ 1030s] make[1]: *** [Makefile:195: test-unit-and-integration] Error 255
[ 1030s] make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/openQA-4.6.1629981680.d6b834c6e'
[ 1030s] make: *** [Makefile:190: test-with-database] Error 2
[ 1030s] error: Bad exit status from /var/tmp/rpm-tmp.IDX6QL (%check)

in os-autoinst both GHA and OBS from https://github.com/os-autoinst/os-autoinst/pull/1755#issuecomment-906404163 no test module is reported

from GHA:

8: [12:40:55] ./18-qemu-options.t ...................... ok   151857 ms ( 0.09 usr  0.01 sys + 147.03 cusr  7.20 csys = 154.33 CPU)
8: Bailout called.  Further testing stopped:  test exceeds runtime limit of '6' seconds
8: FAILED--Further testing stopped: test exceeds runtime limit of '6' seconds

from OBS:

[  219s] 3: [13:01:39] ./17-basetest.t .......................... ok     1179 ms ( 0.00 usr  0.01 sys +  1.08 cusr  0.10 csys =  1.19 CPU)
[  229s] 3: Bailout called.  Further testing stopped:  test exceeds runtime limit of '10' seconds
[  229s] 3: FAILED--Further testing stopped: test exceeds runtime limit of '10' seconds
[  229s] 3/3 Test #3: test-perl-testsuite ..............***Failed  120.20 sec
[  229s]

who has an idea how to improve that?

Actions
Copy link
 #7

Updated by okurz over 3 years ago

  • Status changed from Feedback to Resolved

all merged, latest build results are good

Actions
Copy link

Also available in: Atom PDF