action #68413
open
De-duplicate definitions of needle soft-fail markers (was: screenshot shows soft-fail border for a needle with bug reference but the module turns out to be "passed", expected "softfailed")
Added by okurz almost 4 years ago.
Updated over 3 years ago.
Category:
Regressions/Crashes
Description
Observation¶
see https://openqa.opensuse.org/tests/1306944#step/wine/26
Reproducible¶
Reproduced, e.g. in the same scenario in other jobs the same
Expected result¶
The softfailed needle should make the module result turn "softfailed" as well
Problem¶
Likely a regression that was introduced within the last year with some changes regarding the handling of softfails
Further details¶
Always latest result in this scenario: latest
- Project changed from openQA Tests to openQA Project
- Category changed from Bugs in existing tests to Regressions/Crashes
- Status changed from New to Workable
- Assignee set to Xiaojing_liu
- Target version set to Ready
@Xiaojing_liu can you please read the ticket and state if you can explain what is happening here? Could this be related to work done by you some months ago about handling of softfailed/workaround needles?
- Status changed from Workable to In Progress
The PR is merged however I think the reason why we have this problem in the first place was os-autoinst commit ed3da7cc that introduced the regex /\S+\-(bsc|poo|bnc)(\d+)\-\S+/ within os-autoinst which we already have in openQA. We should try to find a way that we only have one place for the definition. E.g. we should be able to move the corresponding code from openQA to os-autoinst and just use it from there.
okurz wrote:
The PR is merged however I think the reason why we have this problem in the first place was os-autoinst commit ed3da7cc that introduced the regex /\S+\-(bsc|poo|bnc)(\d+)\-\S+/ within os-autoinst which we already have in openQA. We should try to find a way that we only have one place for the definition. E.g. we should be able to move the corresponding code from openQA to os-autoinst and just use it from there.
I could not find a way to define this in one place. Both os-autoinst and openQA need to parse the needle file name.
- Status changed from In Progress to Feedback
- Assignee deleted (
Xiaojing_liu)
I did not find a way to improve the code, so I delete the Assignee and set it to Feedback.
- Subject changed from screenshot shows soft-fail border for a needle with bug reference but the module turns out to be "passed", expected "softfailed" to De-duplicate definitions of needle soft-fail markers (was: screenshot shows soft-fail border for a needle with bug reference but the module turns out to be "passed", expected "softfailed")
- Status changed from Feedback to Workable
Ok, clear. Let's still try to use this ticket to address this issue
If the needle is marked as workaround needle¹, its match should be recorded as softfail (and the module result set to softfailed) by os-autoinst - regardless whether the bugref is recognized or not. Judging by the Git history of the needles repo I'm wondering whether the problem was that the needle has not been marked as workaround needle from the beginning. However, then it is not clear to me why the workaround property is present within details-wine.json of the mentioned job.
Regarding the duplication: We could create a Perl module which provides the regex. We could make it part of os-autoinst and install it into the global Perl prefix or we make it part of the new repository https://github.com/os-autoinst/os-autoinst-common and use it via Git subrepo.
¹ "properties": ["workaround"] appears in the JSON file
mkittler wrote:
[…] I'm wondering whether the problem was that the needle has not been marked as workaround needle from the beginning
Because a needle is also regarded as "workaround" based on a ticket reference in the needle's filename without the property being set.
- Target version changed from Ready to future
Also available in: Atom
PDF