action #33706
closedParameterize workaround property
50%
Description
Right now we use the filename of the needle to mark the bug we're working around. This should be formalized into the workaround property, so the webui can show the Soft Failed explicitly in the overview
Files
Updated by Xiaojing_liu almost 5 years ago
In overview page, there is a list of failed needles when put cursor on a failed module. There is no list shown in Soft Failed module. Does this action mean that it should show a list of somethings likes Failed modules? I do some test, and check the code, there is no needle name recorded in result json file when result status is "softfailed". Does this action mean we should write somethings in result json file in order to show detail list? if it does, what need to be recorded?
Updated by coolo almost 5 years ago
No, this is something else. And I don't want that.
Currently workaround property is a boolean, so if a needle matched with this property, we mark the match and the test as softfailed. But the reviewer has no chance to know why. So this ticket asks to make workaround a string (with migration path treating boolean as 'unknown' I suppose).
Then the soft fail reason would become this string.
Updated by nicksinger almost 5 years ago
The functional team currently uses the following test to check for proper workaround tags: https://github.com/os-autoinst/os-autoinst-needles-tests/blob/a09a78a8687059f01b1765d7af4ff1efe32e6879/test.py
This should give you some insights what values are possible in the filename and how we're using it inside SUSE.
Updated by Xiaojing_liu almost 5 years ago
- Assignee set to Xiaojing_liu
- Target version changed from Ready to Current Sprint
Updated by Xiaojing_liu almost 5 years ago
- Status changed from New to In Progress
Updated by okurz almost 5 years ago
- Category changed from 124 to Feature requests
Updated by okurz over 4 years ago
@Xiaojing_liu I guess after three months this ticket deserves an update :)
Updated by Xiaojing_liu over 4 years ago
- % Done changed from 0 to 50
openQA PR: https://github.com/os-autoinst/openQA/pull/2070
os-autoinst PR: https://github.com/os-autoinst/os-autoinst/pull/1153
Updated by okurz over 4 years ago
- File Screenshot_20190818_162455_new_os-autoinst_old_openqa_question_mark.png Screenshot_20190818_162455_new_os-autoinst_old_openqa_question_mark.png added
- File Screenshot_20190818_162455_new_os-autoinst_old_openqa_soft_fail_needle.png Screenshot_20190818_162455_new_os-autoinst_old_openqa_soft_fail_needle.png added
both merged and deployed to o3. Explicit calls of record_soft_failure look fine, I could not find any soft-failed needles right now. On OSD with the change of os-autoinst deployed I can see that record_soft_failure in old job results causes question marks in a 5 month old job like in
and still correct handling of soft-fail needles as in
after the corresponding openQA change deployed to osd now as well this still looks the same.
Next steps can be to check if results are rendered correctly after new tests have been executed anytime after now. Also I guess – reading the ticket description – the soft-failures can be propagated to the display of failed modules in both /tests/overview as well as the "next&previous" list. @coolo can you confirm that this meets what you had in mind?
Updated by coolo over 4 years ago
I can't say for sure what I had in mind, but I don't think with 'in the overview' refers to /tests/overview - but merely that it doesn't require a look at the filename of specific needles to see the problem. So if it's visible like explict record_soft_fails, we're done here.
Updated by okurz over 4 years ago
- Status changed from In Progress to Resolved
ok, so I guess I can set it to "Resolved" then