action #99597
closedFix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm size:M
Description
Observation¶
Flaky coverage. No logs available.
PR where it happened: https://github.com/os-autoinst/openQA/pull/4256#issuecomment-932021197 (look into the github comment history)
Acceptance criteria¶
- AC1: lib/OpenQA/Worker/WebUIConnection.pm is not showing up in codecov reports for multiple PRs with unrelated changes
Suggestions¶
- Find an occurence and preserve the results
- Try
24-worker-webui-connection.t
Files
Updated by okurz about 3 years ago
- Copied from action #99594: Fix flaky coverage - t/lib/OpenQA/Test/FullstackUtils.pm size:M added
Updated by livdywan almost 3 years ago
- Subject changed from Fix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm to Fix flaky coverage - lib/OpenQA/Worker/WebUIConnection.pm size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by tinita almost 3 years ago
- Status changed from Workable to Feedback
Monitoring new pull requests to catch a new occurrence of this coverage decrease, in order to save the codecov.io report and see which code actually wasn't covered.
Updated by tinita almost 3 years ago
It happened again:
https://app.codecov.io/gh/os-autoinst/openQA/compare/4295/changes (note that the content of the url might change if the code in the PR is updated)
93.14% coverage, minus 0.98%
178
179 my $command_handler = OpenQA::Worker::CommandHandler->new($self);
180 $tx->on(
181 json => sub {
182 $command_handler->handle_command(@_);
183 });
184 $tx->on(
185 --- finish => sub ($tx, $code, $reason = undef) {
186 # uncoverable subroutine
187 # https://progress.opensuse.org/issues/55364
188 --- $reason //= 'no reason';
The lines marked with ---
lost coverage
Updated by tinita almost 3 years ago
Updated by tinita almost 3 years ago
- Status changed from Feedback to In Progress
Updated by openqa_review almost 3 years ago
- Due date set to 2021-10-29
Setting due date based on mean cycle time of SUSE QE Tools
Updated by tinita almost 3 years ago
It also happened (coverage increased by 0.98%) in a PR with my retry hook: https://github.com/os-autoinst/openQA/pull/4299
Here we can see the fullstack test failed twice and the cover_db_fullstack
folder was deleted:
https://app.circleci.com/pipelines/github/os-autoinst/openQA/8102/workflows/7c3a1746-c718-4f21-9a02-f0d895ba51a2/jobs/76627
So at least we know that the flaky coverage in WebUIConnection is not caused by the coverage of failed tests.
Updated by tinita almost 3 years ago
- Status changed from In Progress to Feedback
I forgot to mention https://github.com/os-autoinst/openQA/pull/4318 t: Mark statements as uncoverable in WebUIConnection.pm
Will monitor new PRs for a bit to see if it still happens.
Updated by tinita almost 3 years ago
- Status changed from Feedback to Resolved
I don't see WebUIConnection.pm changes in codecov reports anymore, so resolving.