Project

General

Profile

Actions

action #119461

closed

Serial failure autodetection overrides test result when it shouldn't size:M

Added by MDoucha about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2022-10-26
Due date:
% Done:

0%

Estimated time:

Description

Observation

The serial failure detection code in basetest::parse_serial_output_qemu() unconditionally changes $self->{result} which can result in test failure being turned into softfail or even pass. Result should be changed only when the new value is a "higher level" of failure than the current value (e.g. softfail -> fail, but never fail -> softfail).
Example: https://openqa.suse.de/tests/9481132#step/nfs41_01/12

The nfs41_01 test in the example job failed with exit code 137 (SIGKILL) but the CPU soft lockup box at the end (serial failure with type info) changed the result to pass. This continued in all subsequent test modules until the job timed out.

Acceptance criteria

  • AC1: Test module results are never reset to a lower level

Suggestions

  • Extend tests in t/17-basetest.t with the updated expectation (The main bunch of tests was added in 2018 by riafarov who is not active anymore)
  • Investigate the code path in basetest::parse_serial_output_qemu
  • Update the implementation
Actions #1

Updated by okurz about 2 years ago

  • Target version set to Ready
Actions #2

Updated by livdywan about 2 years ago

  • Subject changed from Serial failure autodetection overrides test result when it shouldn't to Serial failure autodetection overrides test result when it shouldn't size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #3

Updated by dheidler about 2 years ago

  • Assignee set to dheidler
Actions #4

Updated by dheidler about 2 years ago

  • Status changed from Workable to In Progress
Actions #5

Updated by dheidler about 2 years ago

  • Status changed from In Progress to Feedback
Actions #6

Updated by dheidler about 2 years ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF