Project

General

Profile

action #99597

Fix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm size:M

Added by okurz 2 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2021-10-01
Due date:
2021-10-29
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

Flaky coverage. No logs available.

PR where it happened: https://github.com/os-autoinst/openQA/pull/4256#issuecomment-932021197 (look into the github comment history)

Acceptance criteria

  • AC1: lib/OpenQA/Worker/WebUIConnection.pm is not showing up in codecov reports for multiple PRs with unrelated changes

Suggestions

  • Find an occurence and preserve the results
  • Try 24-worker-webui-connection.t
webuiconnection-coverage.png (67.5 KB) webuiconnection-coverage.png tinita, 2021-10-14 10:02
12038

Related issues

Copied from openQA Project - action #99594: Fix flaky coverage - t/lib/OpenQA/Test/FullstackUtils.pm size:MResolved2021-10-012021-10-27

History

#1 Updated by okurz 2 months ago

  • Copied from action #99594: Fix flaky coverage - t/lib/OpenQA/Test/FullstackUtils.pm size:M added

#2 Updated by cdywan about 2 months ago

  • Subject changed from Fix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm to Fix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm size:M
  • Description updated (diff)
  • Status changed from New to Workable

#3 Updated by tinita about 2 months ago

  • Description updated (diff)

#4 Updated by tinita about 2 months ago

  • Status changed from Workable to Feedback

Monitoring new pull requests to catch a new occurrence of this coverage decrease, in order to save the codecov.io report and see which code actually wasn't covered.

#5 Updated by tinita about 2 months ago

  • Assignee set to tinita

#6 Updated by tinita about 2 months ago

It happened again:
https://app.codecov.io/gh/os-autoinst/openQA/compare/4295/changes (note that the content of the url might change if the code in the PR is updated)

93.14% coverage, minus 0.98%

178     
179            my $command_handler = OpenQA::Worker::CommandHandler->new($self);
180            $tx->on(
181                json => sub {
182                    $command_handler->handle_command(@_);
183                });
184            $tx->on(
185 ---            finish => sub ($tx, $code, $reason = undef) {
186                    # uncoverable subroutine
187                    # https://progress.opensuse.org/issues/55364
188 ---                $reason //= 'no reason';

The lines marked with --- lost coverage

#8 Updated by tinita about 2 months ago

  • Status changed from Feedback to In Progress

#9 Updated by openqa_review about 2 months ago

  • Due date set to 2021-10-29

Setting due date based on mean cycle time of SUSE QE Tools

#10 Updated by tinita about 2 months ago

It also happened (coverage increased by 0.98%) in a PR with my retry hook: https://github.com/os-autoinst/openQA/pull/4299

Here we can see the fullstack test failed twice and the cover_db_fullstack folder was deleted:
https://app.circleci.com/pipelines/github/os-autoinst/openQA/8102/workflows/7c3a1746-c718-4f21-9a02-f0d895ba51a2/jobs/76627

So at least we know that the flaky coverage in WebUIConnection is not caused by the coverage of failed tests.

#11 Updated by tinita about 1 month ago

  • Status changed from In Progress to Feedback

I forgot to mention https://github.com/os-autoinst/openQA/pull/4318 t: Mark statements as uncoverable in WebUIConnection.pm

Will monitor new PRs for a bit to see if it still happens.

#12 Updated by tinita about 1 month ago

  • Status changed from Feedback to Resolved

I don't see WebUIConnection.pm changes in codecov reports anymore, so resolving.

Also available in: Atom PDF