action #41696
closed[sle][functional][u][sporadic] test fails in install_test_suite - The test package runs successfully, but os-autoinst fails to detect it
0%
Description
Observation¶
Test shown as failed with missing screeshots
Comparing os-autoinst log with previous passed version, there are this error messages:
Use of uninitialized value $ts_category in substitution (s///) at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 58.
Use of uninitialized value $ts_category in substitution (s///) at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 60.
Use of uninitialized value $ts_name in substitution (s///) at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 61.
Use of uninitialized value $ts_name in substitution (s///) at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 62.
Use of uninitialized value in pattern match (m//) at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 63.
Use of uninitialized value $ts_category in concatenation (.) or string at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 103.
Use of uninitialized value $ts_name in concatenation (.) or string at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 103.
Use of uninitialized value $ts_category in concatenation (.) or string at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 103.
Use of uninitialized value $ts_name in concatenation (.) or string at /usr/share/openqa/lib/OpenQA/Parser/Format/JUnit.pm line 103.
Use of uninitialized value in join or string at /usr/share/openqa/lib/OpenQA/Parser/Result/OpenQA.pm line 62.
- H1 It is a product bug - REJECTED by E1
- H2 It is a problem of the test suite package on http://dist.nue.suse.com/ibs/QA:/Head/SLE-15/ - REJECTED by E1
H3 An update of os-autoinst is causing the problem. - Testing on E3
E1 After installing the package from http://dist.nue.suse.com/ibs/QA:/Head/SLE-15/ in a docker container, and running the test suite, the test suite runs properly.
E3 Try to reproduce it locally.
Reproducible¶
- openQA test in scenario sle-15-SP1-Installer-DVD-x86_64-qa_userspace_cracklib@64bit fails in install_test_suite
- Fails since (at least) Build 47.1
Expected result¶
Last good: 46.1 (or more recent)
Further details¶
Always latest result in this scenario: latest
Updated by okurz over 5 years ago
- Subject changed from [sle][functional] test fails in install_test_suite - The test package runs successfully, but os-autoinst fails to detect it to [sle][functional][u] test fails in install_test_suite - The test package runs successfully, but os-autoinst fails to detect it
@cfconrad could it be that your PR https://github.com/os-autoinst/openQA/pull/1731 caused this?
Updated by cfconrad over 5 years ago
okurz wrote:
@cfconrad could it be that your PR https://github.com/os-autoinst/openQA/pull/1731 caused this?
I don't see how. When I take a look into https://openqa.suse.de/tests/2096062/file/execute_test_run-output.xml it really don't have the attribute "package" in the testsuite dom-element.
Previous runs create a proper execute_test_run-output.xml, where these attributes exists.
For me it looks like the xml, which is created with generateXML(), is wrong. I found this error message in the os-autoinst.log
Can't exec "hostname": Permission denied at /var/lib/openqa/cache/tests/sle/lib/ctcs2_to_junit.pm line 70.
Could be a correlation, cause all elements after 'hostname' are missed.
Updated by okurz over 5 years ago
- Assignee set to EDiGiacinto
yes, good catch. This has been mentioned on chat. But I assume that should not cause the symptoms of undefined variable handling.
@EDiGiacinto, I think you did most of the work in the parser module, can you help?
Updated by EDiGiacinto over 5 years ago
- Assignee deleted (
EDiGiacinto)
To me looks like a malformed XML is fed into the parser, as basically it can't find the sections/attributes expected to be in the JUnit format (but please note, our JUnit implementation is more close to the one that we receive from Slenkins tests). But i doubt those warnings make the test fail, it's more likely that at the end of the process, the parsed file was interpreted as a failed one.
I don't know if the format generated by generateXML() is following closely JUnit, but I would suggest to use XUnit as it is more generic, and it should be even fully compatible with regards to JUnit. To use it it's a slightly small change: parse_extra_log( XUnit => 'file.xml' );
Is this still reproducible?
Updated by okurz over 5 years ago
- Target version changed from Milestone 20 to Milestone 21
Updated by SLindoMansilla over 5 years ago
- Subject changed from [sle][functional][u] test fails in install_test_suite - The test package runs successfully, but os-autoinst fails to detect it to [sle][functional][u][sporadic] test fails in install_test_suite - The test package runs successfully, but os-autoinst fails to detect it
- Assignee set to SLindoMansilla
Only seen 2 times in 3 months: https://openqa.suse.de/tests/2336917#next_previous
Statistical investigation: https://openqa.suse.de/tests/overview?build=126.1poo41696
Updated by SLindoMansilla over 5 years ago
- Assignee deleted (
SLindoMansilla)
Not reproducible on 100 of 100 jobs: https://openqa.suse.de/tests/overview?build=126.1poo41696
The last time it happened was 28 days ago. And that time it caused an incomplete. Logs are missing.
Updated by okurz over 5 years ago
- Status changed from New to Resolved
- Assignee set to okurz
well, if we can not reproduce it and we do not have more information let's assume that an update of os-autoinst might have fixed it.