Project

General

Profile

Actions

action #95662

closed

Codecov: Green/red markers are off by one or more lines sometimes size:S

Added by tinita over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2021-07-19
Due date:
2021-08-03
% Done:

0%

Estimated time:

Description

Observation

If you look at this report:
https://codecov.io/gh/os-autoinst/openQA/src/master/t/lib/OpenQA/Test/Utils.pm#L478

you will see that starting with these 3 uncoverable lines # uncoverable statement the rest of the markers are off by 3, resulting in empty lines being marked as covered for example.

Acceptance criteria

  • AC1: Our codecov reports in openQA show red/green markers on correct lines when using uncoverable … in a previous line

Problem

This seems to be a bug in Devel::Cover::Report::Codecovbash

Tasks

  • Take over maintainership for Devel::Cover::Report::Codecovbash as original author is not interested anymore
  • Fix the bug in the source repo of Devel::Cover::Report::Codecovbash
  • Provide an updated package on CPAN with updated source repo location + bugfix
  • Provide updated package of the above within openSUSE
  • Ensure that the updated, fixed package is used within our CI environment and reports are correct

Files


Related issues 1 (0 open1 closed)

Related to openQA Project (public) - action #80274: Fix flaky coverage - t/lib/OpenQA/Test/Utils.pm size:MResolvedkraih2020-11-24

Actions
Actions #1

Updated by tinita over 3 years ago

I created a bug report: https://github.com/houseabsolute/Devel-Cover-Report-Codecovbash/issues/1

edit: original repo was deleted, I appended the issue log below.
My fork is here: https://github.com/perlpunk/Devel-Cover-Report-Codecovbash

Actions #2

Updated by tinita over 3 years ago

  • Status changed from In Progress to Feedback
Actions #3

Updated by tinita over 3 years ago

  • Status changed from Feedback to In Progress

I got the offer to take over ownership of this module and will release a bugfix

Actions #4

Updated by openqa_review over 3 years ago

  • Due date set to 2021-08-03

Setting due date based on mean cycle time of SUSE QE Tools

Actions #5

Updated by okurz over 3 years ago

In the meantime https://github.com/houseabsolute/Devel-Cover-Report-Codecovbash/issues/1 is 404. Can you provide an updated link? Maybe you already plan to update the according CPAN and then openSUSE package with the new source repo plus a fix?

Actions #6

Updated by tinita over 3 years ago

  • Related to action #80274: Fix flaky coverage - t/lib/OpenQA/Test/Utils.pm size:M added
Actions #7

Updated by okurz over 3 years ago

  • Subject changed from Codecov: Green/red markers are off by one or more lines sometimes to Codecov: Green/red markers are off by one or more lines sometimes size:S
  • Description updated (diff)
Actions #8

Updated by tinita over 3 years ago

As Dave has already deleted the original repo after I forked it, here is the log of the github issue:

@perlpunk

We found that our Coverage reports on codecov.io are sometimes off by 1 or more lines. It starts at lines with # uncoverable statement.
We are using version 0.03

The Devel::Cover HTML report looks fine, but in the codecov.io HTML, the green/red markers move up, so that empty lines are marked as green for example:
https://codecov.io/gh/os-autoinst/openQA/src/master/t/lib/OpenQA/Test/Utils.pm#L652

I looked into the code and found that this is done on purpose:
https://github.com/houseabsolute/Devel-Cover-Report-Codecovbash/blob/master/lib/Devel/Cover/Report/Codecovbash.pm#L93

return if $statement->[0]->uncoverable;
But maybe what was wanted is:

return undef if $statement->[0]->uncoverable;
Because return in list context will just return an empty list, so nothing is pushed into the array at all.

======================================
@autarch

Member
autarch commented 15 hours ago
Thanks for the report.

If you look at the source for this module, you'll see that I just copied this from Devel-Cover-Report-Codecov. I don't actually know what most of this does! I'm happy to accept a patch, but maybe you'd like to take this distro over if you're actually using it. I only created it to test out codecov support for my Azure Pipelines helpers project.

======================================
@perlpunk

Author
perlpunk commented 14 hours ago
Thanks, I would take the module over :)
My CPAN handle: TINITA

======================================
@autarch

Member
autarch commented 14 hours ago
You are now the primary owner. We can also transfer this repo to another owner. I think the easiest way to do that is to add you as an owner of this repo and let you transfer it. Otherwise you'd need to give me admin access to an organization of your choice.

======================================
@perlpunk

Author
perlpunk commented 14 hours ago
Great, thanks.
From my experience, it's enough if I fork it (which I just did), and as soon as you delete your repo (after I made a new release), my fork will be seen as the origin.

======================================
@autarch

Member
autarch commented 14 hours ago
Alright, sounds good. I will delete the repo now.
Actions #9

Updated by tinita over 3 years ago

For verifying my fix, I pushed a branch with a local fixed copy of the module, and the codecov report looks correct now.

Actions #11

Updated by tinita over 3 years ago

Request for submit to factory: https://build.opensuse.org/request/show/907468

Actions #12

Updated by tinita over 3 years ago

Created screenshot of the current report

Actions #13

Updated by tinita over 3 years ago

  • Status changed from In Progress to Feedback

waiting for SR to be accepted

Actions #15

Updated by okurz over 3 years ago

https://build.opensuse.org/package/show/devel:openQA:Leap:15.2/perl-Devel-Cover-Report-Codecovbash has it now. There was a PR updating the dependencies but failing in multiple tests

Actions #18

Updated by tinita over 3 years ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF