QA » openQA Project

Likely the problem is older. Looking back further into the old history from https://app.circleci.com/pipelines/github/os-autoinst/openQA?branch=master I find e.g. https://app.circleci.com/pipelines/github/os-autoinst/openQA/3950/workflows/ce67fc66-a48f-4f31-a750-f99c5f2ece0f/jobs/37623/steps from 22 days ago and it shows the same problem. Did someone merge a change that had a big coverage drop and did not realize the problem?

EDIT: Digging even further the oldest I found so far is https://app.circleci.com/pipelines/github/os-autoinst/openQA/3806/workflows/a002e2f8-c9f3-4229-bbfb-66eb8258c243/jobs/36169/steps from 2020-08-11 commit 40c3d2c . https://app.circleci.com/pipelines/github/os-autoinst/openQA/3471/workflows/d603f1cc-6b9c-4610-9f4f-a2429fb99649/jobs/33218 from 2020-07-03 commit 46240bf,a010576 is still fine.

I think I may have found the commit where it started to fail:
https://github.com/os-autoinst/openQA/pull/3242 deps: Update Mojo-IOLoop-ReadWriteProcess (0.25 -> 0.27)

Last good: https://app.circleci.com/pipelines/github/os-autoinst/openQA/3484/workflows/367605f2-05ae-4666-bf63-f77f146078d6/jobs/33336
First bad: https://app.circleci.com/pipelines/github/os-autoinst/openQA/3503/workflows/d59af86e-378d-4150-ba44-b85c92f10e17/jobs/33453

great that you found this. I prepared https://github.com/os-autoinst/openQA/pull/3399 to analyze the code coverage of the folder "t/" as well. This should tell us which code is never executed.

I just tested this locally and was able to confirm.

% make test-with-database PROVE_ARGS="-v t/25-cache-service.t"
...
ok 22 - OpenQA::CacheService::Task::Sync
ok 23 - no (unexpected) warnings (via done_testing)
1..23
    # Worker cache service on port 9530 stopped
# Looks like your test exited with -1 just after 23.
Dubious, test returned 255 (wstat 65280, 0xff00)
All 23 subtests passed 

Test Summary Report
-------------------
t/25-cache-service.t (Wstat: 65280 Tests: 23 Failed: 0)
  Non-zero exit status: 255
Files=1, Tests=23, 132 wallclock secs ( 0.08 usr  0.00 sys + 17.74 cusr  1.52 csys = 19.34 CPU)
Result: FAIL
make[1]: *** [Makefile:174: test-unit-and-integration] Error 1
make[1]: Leaving directory '/home/tina/openqa-devel/repos/openQA'
make: *** [Makefile:169: test-with-database] Error 2
% echo $?
2

However, with the older version of the module, everything is fine:

% cpanm -l local Mojo::IOLoop::ReadWriteProcess@0.25
% export PERL5LIB=$PWD/local/lib/perl5
% make test-with-database PROVE_ARGS="-v t/25-cache-service.t"
...
ok 22 - OpenQA::CacheService::Task::Sync
ok 23 - no (unexpected) warnings (via done_testing)
1..23
    # Worker cache service on port 9530 stopped
ok
All tests successful.
Files=1, Tests=23, 130 wallclock secs ( 0.07 usr  0.01 sys + 16.29 cusr  1.39 csys = 17.76 CPU)
Result: PASS
make[1]: Leaving directory '/home/tina/openqa-devel/repos/openQA'
[ 0 = 1 ] || pg_ctl -D /dev/shm/tpg stop
waiting for server to shut down.... done
server stopped
% echo $?
0

merged https://github.com/os-autoinst/openQA/pull/3399 about "t/" code coverage. We could do the same for os-autoinst btw.

for os-autoinst trying with https://github.com/os-autoinst/os-autoinst/pull/1536