QA

Motivation¶

openqa-review puts reminder comments jobs even they are multiple months old. This can be confusing to some users and the existing options in https://github.com/os-autoinst/openqa_review/blob/master/openqa_review/openqa_review.py#L196 might not be good enough for some users.
Also see https://suse.slack.com/archives/C02CANHLANP/p1631255003169600

Ideas¶

• DONE Don't comment if the content of the comment would be the same with same job -> #100982 #101722
• DONE Maybe just extend the reminder comment with the option to specify a label instead of a bugref, e.g. label:wontfix:bsc1234 -> https://github.com/os-autoinst/openqa_review/pull/177
• DONE Change the reminder comments if there are already existing reminder comments:
  1. Compare the existing reminder comments with the template. If it's the same except for $name and $url then do not repeat the steps what can be done to prevent more reminders, just the starting line and $name and $url and maybe a pointer to the previous comment
  2. If there are already two (or more?) similar reminder comments as above vary the message even more
  3. If there are more than N reminder comments write a final notice and that openqa-review gives up to remind
  4. For now we will try exponential backoff, see #106907
• DONE Extend the reminder comments to state where exactly the ticket reference comes from, at best point the step detail in openQA or to a code location or the workaround needle that references a ticket
  • openQA test detail step covered in #106909
  • for reference, the autoinst-log.txt has a line [2022-02-08T11:34:07.525532+01:00] [debug] needle 'generic-desktop-test-20220206' is a workaround. The reason is bsc#123456
• DONE Additional suggestions by mdoucha: Don't remind on soft-fails: "Most soft-fails in kernel groups are WONTFIX issues and stuff that isn't important enough to fix immediately. We know about it, we'll fix what needs to be fixed eventually, we don't need a reminder". This seems to be a squad specific workflow which is not generally true. So here I would stick with the current behaviour -> It's suggested to use "record_info" instead of "record_soft_failure" for such issues. However we decided to support a specific "no reminder" pattern that is configurable and can be used to avoid reminders -> #106368

Discarded ideas¶

• Suggestion by foursixnine: Don't comment at all on a job result that is multiple months old. okurz sees a problem with that approach because if these results are still the "most recent" ones then users might be able to follow why such jobs would not be referenced in reminder comments -> It's suggested to move according job groups to "Released" or "EOL" or simply ensure that testing starts earlier, e.g. for Alpha0 of new product iterations