Project

General

Profile

action #93083

unstable test in os-autoinst master

Added by okurz 2 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2021-05-25
Due date:
2021-06-12
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

https://github.com/os-autoinst/os-autoinst/runs/2619306122?check_suite_focus=true#step:3:538 shows

6: [10:47:32] ./18-backend-qemu.t ...................... ok     5026 ms ( 0.00 usr  0.01 sys +  4.83 cusr  0.20 csys =  5.04 CPU)
6: 
6:     #   Failed test 'execution time of isotovideo (24.3694479465485 s) within reasonable limits'
6:     #   at ./18-qemu-options.t line 87.
6:     #     '24.3694479465485'
6:     #         <
6:     #     '24'
6:     # Looks like you failed 1 test of 16.
6: 
6: #   Failed test 'qemu_append_option'
6: #   at ./18-qemu-options.t line 110.
6: # Looks like you failed 1 test of 4.

Acceptance criteria

Suggestions

  • Bisect if recent changes in os-autoinst have made isotovideo execution slower
  • Fix problem (or bump timeout)
  • Ensure badge on repo is fine after merge

Related issues

Copied to openQA Project - action #93336: unstable test in os-autoinst master - slow coverage analysis when forking in t/18-backend-qemu.tNew

History

#1 Updated by mkittler about 2 months ago

  • Assignee set to mkittler

#2 Updated by mkittler about 2 months ago

  • Status changed from Workable to In Progress

Without coverage the test takes locally 13 seconds to run and with it takes 146 so it is 11.2307692308 times slower with coverage. For the concrete check of isotovideo's runtime it is 1.50357818603516 s vs. 17.4516468048096 s so the factor is 11.606744 which is even worse than the overall. The test only scales the isotovideo timeout by a factor of 6 which seems a bit less. I'm changing that to 12. The uncaled timeout of 4 s seems reasonable so I'm keeping it as-is.

Bisect if recent changes in os-autoinst have made isotovideo execution slower

I've been going back within the history but didn't get different results. Likely isotovideo has not become slower. (I didn't get further than the introduction of CMake.)


PR: https://github.com/os-autoinst/os-autoinst/pull/1682

#3 Updated by okurz about 2 months ago

Merged, thx. Your test against the old state in git history is convincing :)

Plans to improve further?

#4 Updated by openqa_review about 2 months ago

  • Due date set to 2021-06-12

Setting due date based on mean cycle time of SUSE QE Tools

#5 Updated by mkittler about 2 months ago

  • Status changed from In Progress to Resolved

PR has been merged.

I don't know how to improve it further. Trying to fix the underlying problem of slow coverage analysis when forking is likely out of scope here.

#6 Updated by okurz about 2 months ago

  • Copied to action #93336: unstable test in os-autoinst master - slow coverage analysis when forking in t/18-backend-qemu.t added

Also available in: Atom PDF