Project

General

Profile

action #119461

Serial failure autodetection overrides test result when it shouldn't size:M

Added by MDoucha 3 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Concrete Bugs
Target version:
Start date:
2022-10-26
Due date:
% Done:

0%

Estimated time:
Difficulty:
easy

Description

Observation

The serial failure detection code in basetest::parse_serial_output_qemu() unconditionally changes $self->{result} which can result in test failure being turned into softfail or even pass. Result should be changed only when the new value is a "higher level" of failure than the current value (e.g. softfail -> fail, but never fail -> softfail).
Example: https://openqa.suse.de/tests/9481132#step/nfs41_01/12

The nfs41_01 test in the example job failed with exit code 137 (SIGKILL) but the CPU soft lockup box at the end (serial failure with type info) changed the result to pass. This continued in all subsequent test modules until the job timed out.

Acceptance criteria

  • AC1: Test module results are never reset to a lower level

Suggestions

  • Extend tests in t/17-basetest.t with the updated expectation (The main bunch of tests was added in 2018 by riafarov who is not active anymore)
  • Investigate the code path in basetest::parse_serial_output_qemu
  • Update the implementation

History

#1 Updated by okurz 3 months ago

  • Target version set to Ready

#2 Updated by cdywan 3 months ago

  • Subject changed from Serial failure autodetection overrides test result when it shouldn't to Serial failure autodetection overrides test result when it shouldn't size:M
  • Description updated (diff)
  • Status changed from New to Workable

#3 Updated by dheidler 3 months ago

  • Assignee set to dheidler

#4 Updated by dheidler 3 months ago

  • Status changed from Workable to In Progress

#5 Updated by dheidler 3 months ago

  • Status changed from In Progress to Feedback

#6 Updated by dheidler 3 months ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF