action #111254
closedcoordination #109668: [saga][epic] Stable and updated non-qemu backends for SLE validation
coordination #109740: [epic] Stable os-autoinst unit tests with good coverage
Cover code of os-autoinst path backend/ fully (statement coverage) size:M
Added by okurz almost 3 years ago. Updated over 2 years ago.
Description
Acceptance criteria¶
- AC1: the path backend/ is listed in codecov.yml under "fully_covered"
Suggestions¶
- Take a look into https://app.codecov.io/gh/os-autoinst/os-autoinst/ about what is currently covered.
- Add unit test statement coverage for that to bump the target
- Currently work is ongoing: https://github.com/os-autoinst/os-autoinst/pull/2048
Updated by okurz almost 3 years ago
- Copied from action #111251: Cover code of os-autoinst path OpenQA/ fully (statement coverage) size:M added
Updated by mkittler over 2 years ago
- Subject changed from Increase code coverage of os-autoinst path backend/ to Cover code of os-autoinst path backend/ fully (statement coverage) size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by livdywan over 2 years ago
- Blocked by action #109620: os-autoinst: Improve unit-test code coverage for backend::svirt size:M added
Updated by livdywan over 2 years ago
- Related to action #100464: [unit test][os-autoinst] create a test for backend::baseclass::clear_serial_buffer() added
Updated by livdywan over 2 years ago
Updated by mkittler over 2 years ago
- Status changed from Workable to In Progress
Updated by mkittler over 2 years ago
- Status changed from In Progress to Feedback
PR is ready to be merged.
Updated by mkittler over 2 years ago
The PR has been merged and codecov showed 100 % coverage for baseclass.pm. So I suppose the hardest part is over.
Updated by mkittler over 2 years ago
- Status changed from Feedback to In Progress
PR for general hardware backend: https://github.com/os-autoinst/os-autoinst/pull/2159
PR for s390x backend: https://github.com/os-autoinst/os-autoinst/pull/2161
PR for driver: https://github.com/os-autoinst/os-autoinst/pull/2160
Updated by mkittler over 2 years ago
- Status changed from In Progress to Feedback
Mainly waiting for reviews (but there are still pending changed to be implemented).
Updated by mkittler over 2 years ago
Looks like the test for t/29-backend-generalhw.t is unstable now:
[ 417s] 3: ok 1527 ms ( 0.02 usr 0.00 sys + 1.23 cusr 0.33 csys = 1.58 CPU)
[ 419s] 3: [12:09:00] ./t/29-backend-amt.t .......................
[ 419s] 3: ok 1 - backend can be created but is deprecated
[ 419s] 3: ok 2 - wsman_cmdline generated
[ 419s] 3: ok 3 - can call wsman
[ 419s] 3: ok 4 - can call enable_solider
[ 419s] 3: ok 5 - can call configure_vnc
[ 419s] 3: ok 6 - can call get_power_state
[ 419s] 3: ok 7 - can call is_shutdown
[ 419s] 3: ok 8 - can call set_power_state
[ 419s] 3: ok 9 - select_next_boot evaluates wsman command
[ 419s] 3: ok 10 - can call do_start_vm
[ 419s] 3: ok 11 - can call do_stop_vm
[ 419s] 3: ok 12 - no (unexpected) warnings (via done_testing)
[ 419s] 3: 1..12
[ 419s] 3: ok 2654 ms ( 0.00 usr 0.00 sys + 2.49 cusr 0.17 csys = 2.66 CPU)
[ 421s] 3: # No tests run!
[ 421s] 3:
[ 421s] 3: # Failed test 'No tests run for subtest "stop grabbing"'
[ 421s] 3: # at ./t/29-backend-generalhw.t line 174.
[ 421s] 3: # Looks like you failed 1 test of 2.
[ 421s] 3:
[ 421s] 3: # Failed test 'serial grab'
[ 421s] 3: # at ./t/29-backend-generalhw.t line 177.
[ 421s] 3: Can't kill('-TERM', '4096'): No such process at /home/abuild/rpmbuild/BUILD/os-autoinst-4.6.1663592796.abfa696/backend/generalhw.pm line 186
[ 421s] 3: # Tests were run but no plan was declared and done_testing() was not seen.
[ 421s] 3: # Looks like your test exited with 255 just after 10.
[ 421s] 3: [12:09:02] ./t/29-backend-generalhw.t .................
…
[ 421s] 3: # Subtest: serial grab
[ 421s] 3: # Subtest: capturing output
[ 421s] 3: ok 1 - serial PID assigned: 4095
[ 421s] 3: ok 2 - serial output captured
[ 421s] 3: 1..2
[ 421s] 3: ok 1 - capturing output
[ 421s] 3: # Subtest: stop grabbing
[ 421s] 3: 1..0
[ 421s] 3: not ok 2 - No tests run for subtest "stop grabbing"
[ 421s] 3: 1..2
[ 421s] 3: not ok 9 - serial grab
[ 421s] 3: ok 10 - no (unexpected) warnings (via END block)
[ 421s] 3: Dubious, test returned 255 (wstat 65280, 0xff00)
[ 421s] 3: Failed 1/10 subtests
Updated by mkittler over 2 years ago
Not sure where this code has a race condition. I won't investigate it further until it happens another time.
Updated by mkittler over 2 years ago
Another PR for the QEMU backend: https://github.com/os-autoinst/os-autoinst/pull/2183
That should leave only pvm and ikvm left.
Updated by okurz over 2 years ago
mkittler wrote:
Another PR for the QEMU backend: https://github.com/os-autoinst/os-autoinst/pull/2183
That should leave only pvm and ikvm left.
Yes, but we deprecated both those backends, see https://github.com/os-autoinst/os-autoinst/commit/2797e185b269cf453d09fed6350c16ea454b3c07 and https://github.com/os-autoinst/os-autoinst/commit/0918185fc4071b64a4147763726932e6679e9112 so IMHO not worth the effort to increase code coverage. How about marking them as not completely covered explicitly in the codecov config file and keep it for some months or years.
Updated by mkittler over 2 years ago
Sounds good - although it seems that our codecov config file is structured the other way around. We explicitly mention fully covered files.
Updated by mkittler over 2 years ago
- Status changed from Feedback to Resolved
All PR have been merged so we're at 100 % in backend/ (except for ikvm and pvm), see https://github.com/os-autoinst/os-autoinst/commit/12e02b136124ec0e21244b01b03e4e7f1969f019.
Updated by okurz 4 months ago
- Copied to action #167920: Cover code of os-autoinst path consoles/ fully (statement coverage) added