[qac][kernel] Make record_soft_failure shows error as regression
The idea is that when there is a bug open we have
record_soft_failure to workaround in the test level.
But when the bug is resolved there are two approach AFAIK. remove the softfail or leave it intact to find regressions.
however when the bug is gone and softfail still exists in the code, there is not an easy way to distinguish the regression.
I propose to enhance the
record_soft_failure to show the regression. One idea is to introduce a flag(extra param?) which we can switch
when the bug has been resolved. Then we can either make the test fail* or show the soft_fail with another colour.
I am not sure if this is the best approach so please comment below if something better comes in your mind.
* maybe we can introduce a parameter to define severity. Then if(critical)->(fail)->else(pass)
Updated by okurz about 2 years ago
- Subject changed from [tools] Make record_soft_failure shows error as regression to [qac][kernel] Make record_soft_failure shows error as regression
@ybonatakis I think for now this should be handled within os-autoinst-distri-opensuse, at best together with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12736 . So can you handle that within "qac" and together with mdoucha from "kernel"?