action #93324
openException in log parser leads to failed job with hard to guess root cause
0%
Description
Observation¶
http://autobot.qa.suse.de/tests/2436 - this job has state failed , and one of modules has result Passed another has result None.
Actual reason of failure is that log parsing for External Result tab has failed because of changes in input log file for parsing . in os-autoinst log you may see following error :
[0mUse of uninitialized value $t_name in pattern match (m//) at /usr/share/openqa/lib/OpenQA/Parser/Format/IPA.pm line 42.
OpenQA::Parser::Format::IPA::parse(OpenQA::Parser::Format::IPA=HASH(0x55dc86d92e00), "{\"tests\": [{\"outcome\": \"passed\", \"test_index\": 0, \"nodeid\": \""...) called at /usr/share/openqa/lib/OpenQA/Parser.pm line 68
OpenQA::Parser::load(OpenQA::Parser::Format::IPA=HASH(0x55dc86d92e00), "ulogs/img_proof-20210518103037.results") called at /usr/share/openqa/lib/OpenQA/Parser.pm line 43
OpenQA::Parser::parser("IPA", "ulogs/img_proof-20210518103037.results") called at /usr/lib/os-autoinst/testapi.pm line 2108
eval {...} called at /usr/lib/os-autoinst/testapi.pm line 2105
testapi::parse_extra_log("IPA", "img_proof_results/ec2/ami-0ae7dcf6b1b1188dd/i-06bf1da3a81e2f0"...) called at sle/tests/publiccloud/img_proof.pm line 131
img_proof::run(img_proof=HASH(0x55dc84eecc58)) called at /usr/lib/os-autoinst/basetest.pm line 356
eval {...} called at /usr/lib/os-autoinst/basetest.pm line 350
basetest::runtest(img_proof=HASH(0x55dc84eecc58)) called at /usr/lib/os-autoinst/autotest.pm line 374
eval {...} called at /usr/lib/os-autoinst/autotest.pm line 374
autotest::runalltests() called at /usr/lib/os-autoinst/autotest.pm line 242
eval {...} called at /usr/lib/os-autoinst/autotest.pm line 242
autotest::run_all() called at /usr/lib/os-autoinst/autotest.pm line 298
autotest::__ANON__(Mojo::IOLoop::ReadWriteProcess=HASH(0x55dc858a43a0)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
eval {...} called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
Mojo::IOLoop::ReadWriteProcess::_fork(Mojo::IOLoop::ReadWriteProcess=HASH(0x55dc858a43a0), CODE(0x55dc8563ad20)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 477
Suggestion¶
- Check if img-proof 7.3.0 is broken or regressed somehow
- See what else may have changed wrt generation of .results
- Reproduce with other tests
Files
Updated by VANASTASIADIS over 3 years ago
- Category set to Regressions/Crashes
- Target version set to Ready
Updated by livdywan over 3 years ago
- I would also expect it to be Incomplete if the parser throws an exception.
- The parser should be able to parse img_proof-20210518103037.log
So that's two AC, maybe two tickets if it's more involved? I'd probably start the parser fix with a reproducer based on the log file
Updated by livdywan over 3 years ago
It seems like name is not contained in the results:
{"tests": [{"outcome": "passed", "test_index": 0, "nodeid": "test_soft_reboot"}, {"nodeid": "test_sles_license.py::test_sles_license[paramiko://18.192.182.175]", "lineno": 3, "outcome": "passed", "keywords": ["test_sles_license[paramiko://18.192.182.175]", "SLES", "paramiko://18.192.182.175", "test_sles_license.py"], "setup": {"duration":
I think we need to confirm what a successful run looks like. There's only 2 in total here and the previous one doesn't have a .results file The successful (but failed) run here has a name field in the JSON unlike the broken case: https://openqa.suse.de/tests/5988889
Also img-proof=7.0.0 vs img-proof=7.3.0 which presumably changed the format
Updated by livdywan over 3 years ago
- Copied to action #93342: Exception in log parser should Incomplete if the parser can't parse or throws an exception added
Updated by livdywan over 3 years ago
- Description updated (diff)
- Status changed from New to Workable
Updated by mkittler over 3 years ago
Note that the backtrace from the ticket description is actually just a warning. Of course the parser code shouldn't produce any warnings (regardless of the input). It should either cope with some data being missing or should emit a real error.