Project

General

Profile

action #95455

[qac][kernel] Make record_soft_failure shows error as regression

Added by ybonatakis 2 months ago. Updated 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Refactor/Code Improvements
Target version:
-
Start date:
2021-07-13
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

The idea is that when there is a bug open we have record_soft_failure to workaround in the test level.
But when the bug is resolved there are two approach AFAIK. remove the softfail or leave it intact to find regressions.
however when the bug is gone and softfail still exists in the code, there is not an easy way to distinguish the regression.

I propose to enhance the record_soft_failure to show the regression. One idea is to introduce a flag(extra param?) which we can switch
when the bug has been resolved. Then we can either make the test fail*[1] or show the soft_fail with another colour.

I am not sure if this is the best approach so please comment below if something better comes in your mind.

*[1] maybe we can introduce a parameter to define severity. Then if(critical)->(fail)->else(pass)

History

#1 Updated by maritawerner 2 months ago

  • Subject changed from Make record_soft_failure shows error as regression to [tools] Make record_soft_failure shows error as regression

#2 Updated by okurz 2 months ago

  • Subject changed from [tools] Make record_soft_failure shows error as regression to [qac][kernel] Make record_soft_failure shows error as regression

ybonatakis I think for now this should be handled within os-autoinst-distri-opensuse, at best together with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12736 . So can you handle that within "qac" and together with mdoucha from "kernel"?

Also available in: Atom PDF