action #33706

Parameterize workaround property

Added by coolo almost 2 years ago. Updated 6 months ago.

Status:ResolvedStart date:23/03/2018
Priority:NormalDue date:
Assignee:Xiaojing_liu% Done:

50%

Category:Feature requests
Target version:Current Sprint
Difficulty:
Duration:

Description

Right now we use the filename of the needle to mark the bug we're working around. This should be formalized into the workaround property, so the webui can show the Soft Failed explicitly in the overview

History

#1 Updated by Xiaojing_liu 10 months ago

In overview page, there is a list of failed needles when put cursor on a failed module. There is no list shown in Soft Failed module. Does this action mean that it should show a list of somethings likes Failed modules? I do some test, and check the code, there is no needle name recorded in result json file when result status is "softfailed". Does this action mean we should write somethings in result json file in order to show detail list? if it does, what need to be recorded?

#2 Updated by coolo 10 months ago

No, this is something else. And I don't want that.

Currently workaround property is a boolean, so if a needle matched with this property, we mark the match and the test as softfailed. But the reviewer has no chance to know why. So this ticket asks to make workaround a string (with migration path treating boolean as 'unknown' I suppose).

Then the soft fail reason would become this string.

#3 Updated by nicksinger 10 months ago

The functional team currently uses the following test to check for proper workaround tags: https://github.com/os-autoinst/os-autoinst-needles-tests/blob/a09a78a8687059f01b1765d7af4ff1efe32e6879/test.py
This should give you some insights what values are possible in the filename and how we're using it inside SUSE.

#4 Updated by Xiaojing_liu 10 months ago

  • Assignee set to Xiaojing_liu
  • Target version changed from Ready to Current Sprint

#5 Updated by Xiaojing_liu 10 months ago

  • Status changed from New to In Progress

#6 Updated by okurz 8 months ago

  • Category changed from 124 to Feature requests

#7 Updated by okurz 7 months ago

@Xiaojing_liu I guess after three months this ticket deserves an update :)

#9 Updated by okurz 6 months ago

both merged and deployed to o3. Explicit calls of record_soft_failure look fine, I could not find any soft-failed needles right now. On OSD with the change of os-autoinst deployed I can see that record_soft_failure in old job results causes question marks in a 5 month old job like in

Screenshot_20190818_162455_new_os-autoinst_old_openqa_question_mark.png

and still correct handling of soft-fail needles as in

Screenshot_20190818_162455_new_os-autoinst_old_openqa_soft_fail_needle.png

after the corresponding openQA change deployed to osd now as well this still looks the same.

Next steps can be to check if results are rendered correctly after new tests have been executed anytime after now. Also I guess – reading the ticket description – the soft-failures can be propagated to the display of failed modules in both /tests/overview as well as the "next&previous" list. @coolo can you confirm that this meets what you had in mind?

#10 Updated by coolo 6 months ago

I can't say for sure what I had in mind, but I don't think with 'in the overview' refers to /tests/overview - but merely that it doesn't require a look at the filename of specific needles to see the problem. So if it's visible like explict record_soft_fails, we're done here.

#11 Updated by okurz 6 months ago

  • Status changed from In Progress to Resolved

ok, so I guess I can set it to "Resolved" then

Also available in: Atom PDF