Project

General

Profile

Actions

action #99597

closed

Fix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm size:M

Added by okurz about 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2021-10-01
Due date:
2021-10-29
% Done:

0%

Estimated time:

Description

Observation

Flaky coverage. No logs available.

PR where it happened: https://github.com/os-autoinst/openQA/pull/4256#issuecomment-932021197 (look into the github comment history)

Acceptance criteria

  • AC1: lib/OpenQA/Worker/WebUIConnection.pm is not showing up in codecov reports for multiple PRs with unrelated changes

Suggestions

  • Find an occurence and preserve the results
  • Try 24-worker-webui-connection.t

Files

webuiconnection-coverage.png (67.5 KB) webuiconnection-coverage.png tinita, 2021-10-14 10:02

Related issues 1 (0 open1 closed)

Copied from openQA Project - action #99594: Fix flaky coverage - t/lib/OpenQA/Test/FullstackUtils.pm size:MResolvedtinita2021-10-012021-10-27

Actions
Actions #1

Updated by okurz about 3 years ago

  • Copied from action #99594: Fix flaky coverage - t/lib/OpenQA/Test/FullstackUtils.pm size:M added
Actions #2

Updated by livdywan almost 3 years ago

  • Subject changed from Fix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm to Fix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #3

Updated by tinita almost 3 years ago

  • Description updated (diff)
Actions #4

Updated by tinita almost 3 years ago

  • Status changed from Workable to Feedback

Monitoring new pull requests to catch a new occurrence of this coverage decrease, in order to save the codecov.io report and see which code actually wasn't covered.

Actions #5

Updated by tinita almost 3 years ago

  • Assignee set to tinita
Actions #6

Updated by tinita almost 3 years ago

It happened again:
https://app.codecov.io/gh/os-autoinst/openQA/compare/4295/changes (note that the content of the url might change if the code in the PR is updated)

93.14% coverage, minus 0.98%

178     
179            my $command_handler = OpenQA::Worker::CommandHandler->new($self);
180            $tx->on(
181                json => sub {
182                    $command_handler->handle_command(@_);
183                });
184            $tx->on(
185 ---            finish => sub ($tx, $code, $reason = undef) {
186                    # uncoverable subroutine
187                    # https://progress.opensuse.org/issues/55364
188 ---                $reason //= 'no reason';

The lines marked with --- lost coverage

Actions #8

Updated by tinita almost 3 years ago

  • Status changed from Feedback to In Progress
Actions #9

Updated by openqa_review almost 3 years ago

  • Due date set to 2021-10-29

Setting due date based on mean cycle time of SUSE QE Tools

Actions #10

Updated by tinita almost 3 years ago

It also happened (coverage increased by 0.98%) in a PR with my retry hook: https://github.com/os-autoinst/openQA/pull/4299

Here we can see the fullstack test failed twice and the cover_db_fullstack folder was deleted:
https://app.circleci.com/pipelines/github/os-autoinst/openQA/8102/workflows/7c3a1746-c718-4f21-9a02-f0d895ba51a2/jobs/76627

So at least we know that the flaky coverage in WebUIConnection is not caused by the coverage of failed tests.

Actions #11

Updated by tinita almost 3 years ago

  • Status changed from In Progress to Feedback

I forgot to mention https://github.com/os-autoinst/openQA/pull/4318 t: Mark statements as uncoverable in WebUIConnection.pm

Will monitor new PRs for a bit to see if it still happens.

Actions #12

Updated by tinita almost 3 years ago

  • Status changed from Feedback to Resolved

I don't see WebUIConnection.pm changes in codecov reports anymore, so resolving.

Actions

Also available in: Atom PDF