Make warnings in tests fatal
As an openQA developer I want openQA unit and integration tests to fail when warnings are encountered to ensure not introduce regressions
- "make test" fails if new warnings are encountered
- the existing warnings are fixed or properly marked and skipped
- It should help if we first fix all existing warnings to make new ones more visible
- research how to make "prove" catch warnings and make them fatal
- why does "prove -W" ("") not work as I expect it?
- decide on an approach
- make warnings fatal in all tests
- fix existing warnings
#7 Updated by okurz over 4 years ago
- Assignee set to rpalethorpe
as mentioned in https://github.com/os-autoinst/os-autoinst/pull/716#discussion_r100562659 rpalethorphe stepped up to look into some warnings.
#8 Updated by rpalethorpe over 4 years ago
- Assignee deleted (
I just removed my custom code to catch warnings and added Test::Warnings to the virtio terminal tests. https://github.com/os-autoinst/os-autoinst/pull/722 Seems to work fine!
And now I am stepping down.
- Priority changed from Normal to Low
We use Test::Warnings in most test modules within os-autoinst as well as openQA with good success. We do not have a check that Test::Warnings is used everywhere and also we do not have good reporting about warnings from tests or so, this can still be done.
- Status changed from New to Feedback
- Assignee set to okurz
Created https://github.com/os-autoinst/os-autoinst/pull/1497 to make all perl warnings encountered from test distributions fatal with a switch to be able to disable that. created https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/10764 to fix a perl warning I found in very common jobs, e.g. https://openqa.opensuse.org/tests/1348002/file/autoinst-log.txt from the "gnome" test suite.
SLE tests look worse, e.g. https://openqa.suse.de/tests/4512233/file/autoinst-log.txt for "sle-15-SP2-Server-DVD-HPC-Incidents-x86_64-hpc_conman@64bit" showing problems in lib/main_common about qam incidents. Also covered in the above PR now.
kraih pointed out https://metacpan.org/pod/strictures which we have available in openSUSE as
perl-strictures already. This looks even better.
well, no action was done since the last comment and nothing changed. I could just unassign but I still have my ticket open and it's in "future" so I think no one else is more likely to pick it up when it's done and the ticket is "Low" so I don't think anything needs to be changed. Unassigning and eventually reassigning would just be noise. As long as you still consider myself a useful member of the team I guess we can keep it this way :)