Parameterize workaround property
|Target version:||Current Sprint|
Right now we use the filename of the needle to mark the bug we're working around. This should be formalized into the workaround property, so the webui can show the Soft Failed explicitly in the overview
#1 Updated by Xiaojing_liu 10 months ago
In overview page, there is a list of failed needles when put cursor on a failed module. There is no list shown in Soft Failed module. Does this action mean that it should show a list of somethings likes Failed modules? I do some test, and check the code, there is no needle name recorded in result json file when result status is "softfailed". Does this action mean we should write somethings in result json file in order to show detail list? if it does, what need to be recorded?
No, this is something else. And I don't want that.
Currently workaround property is a boolean, so if a needle matched with this property, we mark the match and the test as softfailed. But the reviewer has no chance to know why. So this ticket asks to make workaround a string (with migration path treating boolean as 'unknown' I suppose).
Then the soft fail reason would become this string.
#3 Updated by nicksinger 10 months ago
The functional team currently uses the following test to check for proper workaround tags: https://github.com/os-autoinst/os-autoinst-needles-tests/blob/a09a78a8687059f01b1765d7af4ff1efe32e6879/test.py
This should give you some insights what values are possible in the filename and how we're using it inside SUSE.
#8 Updated by Xiaojing_liu 7 months ago
- % Done changed from 0 to 50
openQA PR: https://github.com/os-autoinst/openQA/pull/2070
os-autoinst PR: https://github.com/os-autoinst/os-autoinst/pull/1153
- File Screenshot_20190818_162455_new_os-autoinst_old_openqa_question_mark.png added
- File Screenshot_20190818_162455_new_os-autoinst_old_openqa_soft_fail_needle.png added
both merged and deployed to o3. Explicit calls of
record_soft_failure look fine, I could not find any soft-failed needles right now. On OSD with the change of os-autoinst deployed I can see that
record_soft_failure in old job results causes question marks in a 5 month old job like in
and still correct handling of soft-fail needles as in
after the corresponding openQA change deployed to osd now as well this still looks the same.
Next steps can be to check if results are rendered correctly after new tests have been executed anytime after now. Also I guess – reading the ticket description – the soft-failures can be propagated to the display of failed modules in both /tests/overview as well as the "next&previous" list. @coolo can you confirm that this meets what you had in mind?
I can't say for sure what I had in mind, but I don't think with 'in the overview' refers to /tests/overview - but merely that it doesn't require a look at the filename of specific needles to see the problem. So if it's visible like explict record_soft_fails, we're done here.