Project

General

Profile

Actions
Copy link

action #71536

closed

Unhandled perl warnings in t/ui/15-admin-workers.t, not failing tests as expected (possibly other test modules as well)

Added by okurz over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
mkittler
Category:
Feature requests
Target version:
Ready
Start date:
2020-09-18
Due date:
% Done:

0%

Estimated time:

Description

Observation

t/ui/15-admin-workers.t shows perl warnings but still succeeds, e.g.:

$ prove -l t/ui/15-admin-workers.t 
t/ui/15-admin-workers.t .. 1/? Use of uninitialized value in pattern match (m//) at /home/okurz/local/os-autoinst/openQA/lib/OpenQA/WebAPI/Controller/Admin/Workers.pm line 28.
Use of uninitialized value in pattern match (m//) at /home/okurz/local/os-autoinst/openQA/lib/OpenQA/WebAPI/Controller/Admin/Workers.pm line 28.
Use of uninitialized value in pattern match (m//) at /home/okurz/local/os-autoinst/openQA/lib/OpenQA/WebAPI/Controller/Admin/Workers.pm line 28.
t/ui/15-admin-workers.t .. 7/? Use of uninitialized value in pattern match (m//) at /home/okurz/local/os-autoinst/openQA/lib/OpenQA/WebAPI/Controller/Admin/Workers.pm line 28.
t/ui/15-admin-workers.t .. ok    
All tests successful.
Files=1, Tests=19, 13 wallclock secs ( 0.04 usr  0.00 sys +  1.98 cusr  0.22 csys =  2.24 CPU)
Result: PASS

This is also visible in circleci even though even less obvious. https://app.circleci.com/pipelines/github/os-autoinst/openQA/4264/workflows/c03c8397-a61e-481c-afbb-8c8839a69e9a/jobs/40889/steps is green and there is no warning visible in the log output however https://app.circleci.com/pipelines/github/os-autoinst/openQA/4264/workflows/c03c8397-a61e-481c-afbb-8c8839a69e9a/jobs/40889/artifacts references https://40889-20883829-gh.circle-artifacts.com/0/artifacts/ui/15-admin-workers.t which shows the same warnings as above.

Our test code uses use Test::Warnings ':report_warnings'; but to no avail. I also crosschecked if maybe Test::Warnings in a newer version is the problem itself but the problem reproduces with perl-Test-Warnings-0.026

Expected result

  • Tests should fail if there is any warning, regardless of the source

Problem

This could be related to the fact that we start the SeleniumDriver in a background process and the warning might come from there. Introducing an artificial warning in the main thread is correctly handled.

Suggestions

  • Try to narrow down the source of the problem and identify from which process the warning comes
  • Extend the existing approach to handle warnings or find a different approach for all background processes, e.g. compare to https://github.com/os-autoinst/os-autoinst/pull/1497
Actions
Copy link

Also available in: Atom PDF