QA » openQA Project

Observation¶

The serial failure detection code in basetest::parse_serial_output_qemu() unconditionally changes $self->{result} which can result in test failure being turned into softfail or even pass. Result should be changed only when the new value is a "higher level" of failure than the current value (e.g. softfail -> fail, but never fail -> softfail).
Example: https://openqa.suse.de/tests/9481132#step/nfs41_01/12

The nfs41_01 test in the example job failed with exit code 137 (SIGKILL) but the CPU soft lockup box at the end (serial failure with type info) changed the result to pass. This continued in all subsequent test modules until the job timed out.

Acceptance criteria¶

• AC1: Test module results are never reset to a lower level

Suggestions¶

• Extend tests in t/17-basetest.t with the updated expectation (The main bunch of tests was added in 2018 by riafarov who is not active anymore)
• Investigate the code path in basetest::parse_serial_output_qemu
• Update the implementation