Actions
action #119461
closedSerial failure autodetection overrides test result when it shouldn't size:M
Description
Observation¶
The serial failure detection code in basetest::parse_serial_output_qemu() unconditionally changes $self->{result}
which can result in test failure being turned into softfail or even pass. Result should be changed only when the new value is a "higher level" of failure than the current value (e.g. softfail
-> fail
, but never fail
-> softfail
).
Example: https://openqa.suse.de/tests/9481132#step/nfs41_01/12
The nfs41_01 test in the example job failed with exit code 137 (SIGKILL) but the CPU soft lockup box at the end (serial failure with type info
) changed the result to pass
. This continued in all subsequent test modules until the job timed out.
Acceptance criteria¶
- AC1: Test module results are never reset to a lower level
Suggestions¶
- Extend tests in t/17-basetest.t with the updated expectation (The main bunch of tests was added in 2018 by riafarov who is not active anymore)
- Investigate the code path in
basetest::parse_serial_output_qemu
- Update the implementation
Updated by livdywan almost 2 years ago
- Subject changed from Serial failure autodetection overrides test result when it shouldn't to Serial failure autodetection overrides test result when it shouldn't size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by dheidler almost 2 years ago
- Status changed from Workable to In Progress
Updated by dheidler almost 2 years ago
- Status changed from In Progress to Feedback
Actions