Project

General

Profile

Actions

action #111254

closed

coordination #109668: [saga][epic] Stable and updated non-qemu backends for SLE validation

coordination #109740: [epic] Stable os-autoinst unit tests with good coverage

Cover code of os-autoinst path backend/ fully (statement coverage) size:M

Added by okurz over 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Feature requests
Target version:
Start date:
2022-05-18
Due date:
% Done:

0%

Estimated time:

Description

Acceptance criteria

  • AC1: the path backend/ is listed in codecov.yml under "fully_covered"

Suggestions


Related issues 3 (1 open2 closed)

Blocked by openQA Project - action #109620: os-autoinst: Improve unit-test code coverage for backend::svirt size:MResolvedosukup

Actions
Copied from openQA Project - action #111251: Cover code of os-autoinst path OpenQA/ fully (statement coverage) size:MResolvedtinita2022-05-18

Actions
Copied to openQA Project - action #167920: Cover code of os-autoinst path consoles/ fully (statement coverage) size:MNew

Actions
Actions #1

Updated by okurz over 2 years ago

  • Copied from action #111251: Cover code of os-autoinst path OpenQA/ fully (statement coverage) size:M added
Actions #2

Updated by mkittler over 2 years ago

  • Subject changed from Increase code coverage of os-autoinst path backend/ to Cover code of os-autoinst path backend/ fully (statement coverage) size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #3

Updated by livdywan over 2 years ago

  • Blocked by action #109620: os-autoinst: Improve unit-test code coverage for backend::svirt size:M added
Actions #5

Updated by okurz over 2 years ago

  • Target version changed from Ready to future
Actions #6

Updated by okurz over 2 years ago

  • Target version changed from future to Ready
Actions #7

Updated by mkittler about 2 years ago

  • Assignee set to mkittler
Actions #9

Updated by mkittler about 2 years ago

  • Status changed from Workable to In Progress
Actions #10

Updated by mkittler about 2 years ago

  • Status changed from In Progress to Feedback

PR is ready to be merged.

Actions #11

Updated by mkittler about 2 years ago

The PR has been merged and codecov showed 100 % coverage for baseclass.pm. So I suppose the hardest part is over.

Actions #12

Updated by mkittler about 2 years ago

  • Status changed from Feedback to In Progress
Actions #13

Updated by mkittler about 2 years ago

  • Status changed from In Progress to Feedback

Mainly waiting for reviews (but there are still pending changed to be implemented).

Actions #14

Updated by mkittler about 2 years ago

Looks like the test for t/29-backend-generalhw.t is unstable now:

[  417s] 3: ok     1527 ms ( 0.02 usr  0.00 sys +  1.23 cusr  0.33 csys =  1.58 CPU)
[  419s] 3: [12:09:00] ./t/29-backend-amt.t ....................... 
[  419s] 3: ok 1 - backend can be created but is deprecated
[  419s] 3: ok 2 - wsman_cmdline generated
[  419s] 3: ok 3 - can call wsman
[  419s] 3: ok 4 - can call enable_solider
[  419s] 3: ok 5 - can call configure_vnc
[  419s] 3: ok 6 - can call get_power_state
[  419s] 3: ok 7 - can call is_shutdown
[  419s] 3: ok 8 - can call set_power_state
[  419s] 3: ok 9 - select_next_boot evaluates wsman command
[  419s] 3: ok 10 - can call do_start_vm
[  419s] 3: ok 11 - can call do_stop_vm
[  419s] 3: ok 12 - no (unexpected) warnings (via done_testing)
[  419s] 3: 1..12
[  419s] 3: ok     2654 ms ( 0.00 usr  0.00 sys +  2.49 cusr  0.17 csys =  2.66 CPU)
[  421s] 3:         # No tests run!
[  421s] 3: 
[  421s] 3:     #   Failed test 'No tests run for subtest "stop grabbing"'
[  421s] 3:     #   at ./t/29-backend-generalhw.t line 174.
[  421s] 3:     # Looks like you failed 1 test of 2.
[  421s] 3: 
[  421s] 3: #   Failed test 'serial grab'
[  421s] 3: #   at ./t/29-backend-generalhw.t line 177.
[  421s] 3: Can't kill('-TERM', '4096'): No such process at /home/abuild/rpmbuild/BUILD/os-autoinst-4.6.1663592796.abfa696/backend/generalhw.pm line 186
[  421s] 3: # Tests were run but no plan was declared and done_testing() was not seen.
[  421s] 3: # Looks like your test exited with 255 just after 10.
[  421s] 3: [12:09:02] ./t/29-backend-generalhw.t ................. 
…
[  421s] 3: # Subtest: serial grab
[  421s] 3:     # Subtest: capturing output
[  421s] 3:         ok 1 - serial PID assigned: 4095
[  421s] 3:         ok 2 - serial output captured
[  421s] 3:         1..2
[  421s] 3:     ok 1 - capturing output
[  421s] 3:     # Subtest: stop grabbing
[  421s] 3:         1..0
[  421s] 3:     not ok 2 - No tests run for subtest "stop grabbing"
[  421s] 3:     1..2
[  421s] 3: not ok 9 - serial grab
[  421s] 3: ok 10 - no (unexpected) warnings (via END block)
[  421s] 3: Dubious, test returned 255 (wstat 65280, 0xff00)
[  421s] 3: Failed 1/10 subtests
Actions #15

Updated by mkittler about 2 years ago

Not sure where this code has a race condition. I won't investigate it further until it happens another time.

Actions #16

Updated by mkittler about 2 years ago

Another PR for the QEMU backend: https://github.com/os-autoinst/os-autoinst/pull/2183

That should leave only pvm and ikvm left.

Actions #17

Updated by okurz about 2 years ago

mkittler wrote:

Another PR for the QEMU backend: https://github.com/os-autoinst/os-autoinst/pull/2183

That should leave only pvm and ikvm left.

Yes, but we deprecated both those backends, see https://github.com/os-autoinst/os-autoinst/commit/2797e185b269cf453d09fed6350c16ea454b3c07 and https://github.com/os-autoinst/os-autoinst/commit/0918185fc4071b64a4147763726932e6679e9112 so IMHO not worth the effort to increase code coverage. How about marking them as not completely covered explicitly in the codecov config file and keep it for some months or years.

Actions #18

Updated by mkittler about 2 years ago

Sounds good - although it seems that our codecov config file is structured the other way around. We explicitly mention fully covered files.

Actions #19

Updated by mkittler about 2 years ago

  • Status changed from Feedback to Resolved

All PR have been merged so we're at 100 % in backend/ (except for ikvm and pvm), see https://github.com/os-autoinst/os-autoinst/commit/12e02b136124ec0e21244b01b03e4e7f1969f019.

Actions #20

Updated by okurz 5 days ago

  • Copied to action #167920: Cover code of os-autoinst path consoles/ fully (statement coverage) size:M added
Actions

Also available in: Atom PDF