action #93083
closedunstable test in os-autoinst master
Description
Observation¶
https://github.com/os-autoinst/os-autoinst/runs/2619306122?check_suite_focus=true#step:3:538 shows
6: [10:47:32] ./18-backend-qemu.t ...................... ok 5026 ms ( 0.00 usr 0.01 sys + 4.83 cusr 0.20 csys = 5.04 CPU)
6:
6: # Failed test 'execution time of isotovideo (24.3694479465485 s) within reasonable limits'
6: # at ./18-qemu-options.t line 87.
6: # '24.3694479465485'
6: # <
6: # '24'
6: # Looks like you failed 1 test of 16.
6:
6: # Failed test 'qemu_append_option'
6: # at ./18-qemu-options.t line 110.
6: # Looks like you failed 1 test of 4.
Acceptance criteria¶
- AC1: https://github.com/os-autoinst/os-autoinst shows passed tests in badge
Suggestions¶
- Bisect if recent changes in os-autoinst have made isotovideo execution slower
- Fix problem (or bump timeout)
- Ensure badge on repo is fine after merge
Updated by mkittler almost 3 years ago
- Status changed from Workable to In Progress
Without coverage the test takes locally 13 seconds to run and with it takes 146 so it is 11.2307692308 times slower with coverage. For the concrete check of isotovideo's runtime it is 1.50357818603516 s vs. 17.4516468048096 s so the factor is 11.606744 which is even worse than the overall. The test only scales the isotovideo timeout by a factor of 6 which seems a bit less. I'm changing that to 12. The uncaled timeout of 4 s seems reasonable so I'm keeping it as-is.
Bisect if recent changes in os-autoinst have made isotovideo execution slower
I've been going back within the history but didn't get different results. Likely isotovideo has not become slower. (I didn't get further than the introduction of CMake.)
Updated by okurz almost 3 years ago
Merged, thx. Your test against the old state in git history is convincing :)
Plans to improve further?
Updated by openqa_review almost 3 years ago
- Due date set to 2021-06-12
Setting due date based on mean cycle time of SUSE QE Tools
Updated by mkittler almost 3 years ago
- Status changed from In Progress to Resolved
PR has been merged.
I don't know how to improve it further. Trying to fix the underlying problem of slow coverage analysis when forking is likely out of scope here.
Updated by okurz almost 3 years ago
- Copied to action #93336: unstable test in os-autoinst master - slow coverage analysis when forking in t/18-backend-qemu.t added