action #93083
closed
unstable test in os-autoinst master
Added by okurz almost 3 years ago.
Updated almost 3 years ago.
Category:
Regressions/Crashes
Description
Observation¶
https://github.com/os-autoinst/os-autoinst/runs/2619306122?check_suite_focus=true#step:3:538 shows
6: [10:47:32] ./18-backend-qemu.t ...................... ok 5026 ms ( 0.00 usr 0.01 sys + 4.83 cusr 0.20 csys = 5.04 CPU)
6:
6: # Failed test 'execution time of isotovideo (24.3694479465485 s) within reasonable limits'
6: # at ./18-qemu-options.t line 87.
6: # '24.3694479465485'
6: # <
6: # '24'
6: # Looks like you failed 1 test of 16.
6:
6: # Failed test 'qemu_append_option'
6: # at ./18-qemu-options.t line 110.
6: # Looks like you failed 1 test of 4.
Acceptance criteria¶
Suggestions¶
- Bisect if recent changes in os-autoinst have made isotovideo execution slower
- Fix problem (or bump timeout)
- Ensure badge on repo is fine after merge
Related issues
1 (1 open — 0 closed)
- Status changed from Workable to In Progress
Without coverage the test takes locally 13 seconds to run and with it takes 146 so it is 11.2307692308 times slower with coverage. For the concrete check of isotovideo's runtime it is 1.50357818603516 s vs. 17.4516468048096 s so the factor is 11.606744 which is even worse than the overall. The test only scales the isotovideo timeout by a factor of 6 which seems a bit less. I'm changing that to 12. The uncaled timeout of 4 s seems reasonable so I'm keeping it as-is.
Bisect if recent changes in os-autoinst have made isotovideo execution slower
I've been going back within the history but didn't get different results. Likely isotovideo has not become slower. (I didn't get further than the introduction of CMake.)
PR: https://github.com/os-autoinst/os-autoinst/pull/1682
Merged, thx. Your test against the old state in git history is convincing :)
Plans to improve further?
- Due date set to 2021-06-12
Setting due date based on mean cycle time of SUSE QE Tools
- Status changed from In Progress to Resolved
PR has been merged.
I don't know how to improve it further. Trying to fix the underlying problem of slow coverage analysis when forking is likely out of scope here.
- Copied to action #93336: unstable test in os-autoinst master - slow coverage analysis when forking in t/18-backend-qemu.t added
Also available in: Atom
PDF