action #95662
closedCodecov: Green/red markers are off by one or more lines sometimes size:S
Description
Observation¶
If you look at this report:
https://codecov.io/gh/os-autoinst/openQA/src/master/t/lib/OpenQA/Test/Utils.pm#L478
you will see that starting with these 3 uncoverable lines # uncoverable statement
the rest of the markers are off by 3, resulting in empty lines being marked as covered for example.
Acceptance criteria¶
- AC1: Our codecov reports in openQA show red/green markers on correct lines when using
uncoverable …
in a previous line
Problem¶
This seems to be a bug in Devel::Cover::Report::Codecovbash
Tasks¶
- Take over maintainership for Devel::Cover::Report::Codecovbash as original author is not interested anymore
- Fix the bug in the source repo of Devel::Cover::Report::Codecovbash
- Provide an updated package on CPAN with updated source repo location + bugfix
- Provide updated package of the above within openSUSE
- Ensure that the updated, fixed package is used within our CI environment and reports are correct
Files
Updated by tinita over 3 years ago
I created a bug report: https://github.com/houseabsolute/Devel-Cover-Report-Codecovbash/issues/1
edit: original repo was deleted, I appended the issue log below.
My fork is here: https://github.com/perlpunk/Devel-Cover-Report-Codecovbash
Updated by tinita over 3 years ago
- Status changed from Feedback to In Progress
I got the offer to take over ownership of this module and will release a bugfix
Updated by openqa_review over 3 years ago
- Due date set to 2021-08-03
Setting due date based on mean cycle time of SUSE QE Tools
Updated by okurz over 3 years ago
In the meantime https://github.com/houseabsolute/Devel-Cover-Report-Codecovbash/issues/1 is 404. Can you provide an updated link? Maybe you already plan to update the according CPAN and then openSUSE package with the new source repo plus a fix?
Updated by tinita over 3 years ago
- Related to action #80274: Fix flaky coverage - t/lib/OpenQA/Test/Utils.pm size:M added
Updated by okurz over 3 years ago
- Subject changed from Codecov: Green/red markers are off by one or more lines sometimes to Codecov: Green/red markers are off by one or more lines sometimes size:S
- Description updated (diff)
Updated by tinita over 3 years ago
As Dave has already deleted the original repo after I forked it, here is the log of the github issue:
@perlpunk
We found that our Coverage reports on codecov.io are sometimes off by 1 or more lines. It starts at lines with # uncoverable statement.
We are using version 0.03
The Devel::Cover HTML report looks fine, but in the codecov.io HTML, the green/red markers move up, so that empty lines are marked as green for example:
https://codecov.io/gh/os-autoinst/openQA/src/master/t/lib/OpenQA/Test/Utils.pm#L652
I looked into the code and found that this is done on purpose:
https://github.com/houseabsolute/Devel-Cover-Report-Codecovbash/blob/master/lib/Devel/Cover/Report/Codecovbash.pm#L93
return if $statement->[0]->uncoverable;
But maybe what was wanted is:
return undef if $statement->[0]->uncoverable;
Because return in list context will just return an empty list, so nothing is pushed into the array at all.
======================================
@autarch
Member
autarch commented 15 hours ago
Thanks for the report.
If you look at the source for this module, you'll see that I just copied this from Devel-Cover-Report-Codecov. I don't actually know what most of this does! I'm happy to accept a patch, but maybe you'd like to take this distro over if you're actually using it. I only created it to test out codecov support for my Azure Pipelines helpers project.
======================================
@perlpunk
Author
perlpunk commented 14 hours ago
Thanks, I would take the module over :)
My CPAN handle: TINITA
======================================
@autarch
Member
autarch commented 14 hours ago
You are now the primary owner. We can also transfer this repo to another owner. I think the easiest way to do that is to add you as an owner of this repo and let you transfer it. Otherwise you'd need to give me admin access to an organization of your choice.
======================================
@perlpunk
Author
perlpunk commented 14 hours ago
Great, thanks.
From my experience, it's enough if I fork it (which I just did), and as soon as you delete your repo (after I made a new release), my fork will be seen as the origin.
======================================
@autarch
Member
autarch commented 14 hours ago
Alright, sounds good. I will delete the repo now.
Updated by tinita over 3 years ago
For verifying my fix, I pushed a branch with a local fixed copy of the module, and the codecov report looks correct now.
Updated by tinita over 3 years ago
Released 0.04 to CPAN: https://metacpan.org/dist/Devel-Cover-Report-Codecovbash
Updated by tinita over 3 years ago
Request for submit to factory: https://build.opensuse.org/request/show/907468
Updated by tinita over 3 years ago
Created screenshot of the current report
Updated by tinita over 3 years ago
- Status changed from In Progress to Feedback
waiting for SR to be accepted
Updated by tinita over 3 years ago
SR accepted, but not yet in https://build.opensuse.org/project/show/devel:openQA:Leap:15.2
Updated by okurz over 3 years ago
https://build.opensuse.org/package/show/devel:openQA:Leap:15.2/perl-Devel-Cover-Report-Codecovbash has it now. There was a PR updating the dependencies but failing in multiple tests
Updated by tinita over 3 years ago
Dependency PR merged: https://github.com/os-autoinst/openQA/pull/4064
Updated by tinita over 3 years ago
Report from pull request looks good: https://codecov.io/gh/os-autoinst/openQA/src/a9da3486e7dee700d94e06d20863c6a770864b18/t/lib/OpenQA/Test/Utils.pm