action #68413
openDe-duplicate definitions of needle soft-fail markers (was: screenshot shows soft-fail border for a needle with bug reference but the module turns out to be "passed", expected "softfailed")
0%
Description
Observation¶
see https://openqa.opensuse.org/tests/1306944#step/wine/26
Reproducible¶
Reproduced, e.g. in the same scenario in other jobs the same
Expected result¶
The softfailed needle should make the module result turn "softfailed" as well
Problem¶
Likely a regression that was introduced within the last year with some changes regarding the handling of softfails
Further details¶
Always latest result in this scenario: latest
Updated by okurz over 4 years ago
- Project changed from openQA Tests (public) to openQA Project (public)
- Category changed from Bugs in existing tests to Regressions/Crashes
- Status changed from New to Workable
- Assignee set to Xiaojing_liu
- Target version set to Ready
@Xiaojing_liu can you please read the ticket and state if you can explain what is happening here? Could this be related to work done by you some months ago about handling of softfailed/workaround needles?
Updated by Xiaojing_liu over 4 years ago
- Status changed from Workable to In Progress
I check this issue, it is related to the pr that I worked some months ago. I did not consider the needle file contained boo.
Made a pull request to fix this: https://github.com/os-autoinst/os-autoinst/pull/1450
Updated by okurz over 4 years ago
The PR is merged however I think the reason why we have this problem in the first place was os-autoinst commit ed3da7cc that introduced the regex /\S+\-(bsc|poo|bnc)(\d+)\-\S+/ within os-autoinst which we already have in openQA. We should try to find a way that we only have one place for the definition. E.g. we should be able to move the corresponding code from openQA to os-autoinst and just use it from there.
Updated by Xiaojing_liu over 4 years ago
okurz wrote:
The PR is merged however I think the reason why we have this problem in the first place was os-autoinst commit ed3da7cc that introduced the regex
/\S+\-(bsc|poo|bnc)(\d+)\-\S+/within os-autoinst which we already have in openQA. We should try to find a way that we only have one place for the definition. E.g. we should be able to move the corresponding code from openQA to os-autoinst and just use it from there.
I could not find a way to define this in one place. Both os-autoinst and openQA need to parse the needle file name.
Updated by Xiaojing_liu over 4 years ago
- Status changed from In Progress to Feedback
- Assignee deleted (
Xiaojing_liu)
I did not find a way to improve the code, so I delete the Assignee and set it to Feedback.
Updated by okurz over 4 years ago
- Subject changed from screenshot shows soft-fail border for a needle with bug reference but the module turns out to be "passed", expected "softfailed" to De-duplicate definitions of needle soft-fail markers (was: screenshot shows soft-fail border for a needle with bug reference but the module turns out to be "passed", expected "softfailed")
- Status changed from Feedback to Workable
Ok, clear. Let's still try to use this ticket to address this issue
Updated by mkittler over 4 years ago
If the needle is marked as workaround needle¹, its match should be recorded as softfail (and the module result set to softfailed) by os-autoinst - regardless whether the bugref is recognized or not. Judging by the Git history of the needles repo I'm wondering whether the problem was that the needle has not been marked as workaround needle from the beginning. However, then it is not clear to me why the workaround property is present within details-wine.json of the mentioned job.
Regarding the duplication: We could create a Perl module which provides the regex. We could make it part of os-autoinst and install it into the global Perl prefix or we make it part of the new repository https://github.com/os-autoinst/os-autoinst-common and use it via Git subrepo.
¹ "properties": ["workaround"] appears in the JSON file
Updated by okurz over 4 years ago
mkittler wrote:
[…] I'm wondering whether the problem was that the needle has not been marked as workaround needle from the beginning
Because a needle is also regarded as "workaround" based on a ticket reference in the needle's filename without the property being set.