Project

General

Profile

action #80298

coordination #55364: [epic] Let's make codecov reports reliable

Fix flaky coverage - lib/OpenQA/WebSockets.pm

Added by okurz about 2 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-11-24
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

See #55364 : codecov reports often report about coverage changes which are obviously not related to the actual changes of a PR, e.g. when documentation is changed. We can already trust our coverage analysis more but should have only coverage changes reported for actual changes we introduced in a pull request.

Acceptance criteria

  • AC1: lib/OpenQA/WebSockets.pm does not appear anymore as changing code coverage in unrelated changes

Suggestions

  • One can start with rm -rf cover_db/ && make coverage KEEP_DB=1 TESTS=t/27-websockets.t but more than just this single test module is needed for complete coverage of lib/OpenQA/WebSockets.pm
  • check coverage details in generated html report
  • Fix uncovered lines with "uncoverable" statements or other means
  • retry multiple times to check for flakyness

Related issues

Copied from openQA Project - action #80274: Fix flaky coverage - t/lib/OpenQA/Test/Utils.pmIn Progress2020-11-242021-01-23

History

#1 Updated by okurz about 2 months ago

  • Copied from action #80274: Fix flaky coverage - t/lib/OpenQA/Test/Utils.pm added

#2 Updated by okurz about 2 months ago

  • Status changed from Workable to Feedback
  • Assignee set to okurz

Let's see if https://github.com/os-autoinst/openQA/pull/3588 is enough, otherwise someone else can try to apply a proper solution. I am merely adding "uncoverable" statements :)

#3 Updated by okurz about 1 month ago

  • Status changed from Feedback to Resolved

https://codecov.io/gh/os-autoinst/openQA/src/master/lib/OpenQA/WebSockets.pm shows that the file should have a consistent 100% statement coverage now.

Also available in: Atom PDF