action #93101
closed
investigation jobs scheduled against specific git hashes load wrong product dir files
Added by okurz over 3 years ago.
Updated over 3 years ago.
Category:
Regressions/Crashes
Description
Observation¶
https://openqa.opensuse.org/tests/1757493#comments shows investigation jobs, e.g. textmode:investigate:last_good_tests:f4a80a14155f874200833e833d30edac634ad9a1: http://openqa.opensuse.org/t1757550 showing the error:
[37m[2021-05-25T14:38:16.121 CEST] [debug] git hash in /var/lib/openqa/pool/1/os-autoinst-distri-opensuse: f4a80a14155f874200833e833d30edac634ad9a1
[0mBareword "load_pynfs_tests" not allowed while "strict subs" in use at /var/lib/openqa/share/tests/opensuse/products/opensuse/main.pm line 273.
Compilation failed in require at /usr/lib/os-autoinst/OpenQA/Isotovideo/Utils.pm line 211.
As the path /var/lib/openqa/share/tests/opensuse/products/opensuse/main.pm is mentioned it looks like the default path is preferred when instead products/opensuse/main.pm should be read from the local git checkout directory. This might be a regression from work on #80372 or a subticket, e.g. #67723 or #90302
Acceptance criteria¶
- AC1: openQA investigate tests against "last good tests" should read test code from the same git hash, i.e. from the local git checkout directory
Suggestions¶
- check if this is a regression introduced
- this likely can be solved within os-autoinst only
- Status changed from Workable to In Progress
I've just created a custom branch which prints Running custom version at the beginning of main.pm and it works as expected. However, in the same log a Perl warning appears similarly to the one mentioned in the ticket description:
Use of uninitialized value in pattern match (m//) at /hdd/openqa-devel/openqa/pool/1/os-autoinst-distri-opensuse/lib/main_common.pm line 697.
main_common::remove_common_needles() called at /hdd/openqa-devel/openqa/share/tests/opensuse/products/opensuse/main.pm line 36
OpenQA::Isotovideo::Utils::cleanup_needles() called at /hdd/openqa-devel/repos/os-autoinst/needle.pm line 329
needle::init() called at /hdd/openqa-devel/repos/os-autoinst/isotovideo line 270
The code sets unshift @INC, '.'; before the main.pm require. Apparently it is only half-effective.
The problem is actually that PRODUCTDIR is set to the default by the worker using an absolute path. So os-autoinst actually behaves as expected (test libraries are used from Git, main.pm also loaded from default products directory).
Draft to fix this on the worker-side: https://github.com/os-autoinst/openQA/pull/3920
- Due date set to 2021-06-10
Setting due date based on mean cycle time of SUSE QE Tools
- Status changed from In Progress to Feedback
- Due date changed from 2021-06-10 to 2021-06-15
- Status changed from Feedback to Resolved
It looks like variables are now set as expected in production, in particular PRODUCT_DIR is now a relative path (e.g. https://openqa.suse.de/tests/6232735). This should now work. Of course it is hard to tell from the logs which files have actually been loaded unless there's an error caused by loading the wrong file. I've checked some recent incompletes and didn't find such errors (only a lot with e.g. Could not find '24eddcb8631f572d64236898aaac3dcfe5a14b06' in complete history at /usr/lib/os-autoinst/OpenQA/Isotovideo/Utils.pm line 96..).
Also available in: Atom
PDF