action #138299
closedcoordination #102915: [saga][epic] Automated classification of failures
QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests
Make the final aggregation messages from openqa-investigate more prominent size:S
Description
Motivation¶
#132272 already showed us that besides clear product issues we can now also clearly identify test issues. However the message is rather subtle, nestled behind the repetition of the "retry job evaluation" comment, e.g.
from https://openqa.suse.de/tests/12563482#comments
- first comment: all 4 investigation jobs triggered including nice light bulb with link to more information
- second comment: bisect jobs
- third comment: retry job result evaluation and just behind with a not so obvious message the text "Jobs including the last good test are ok, likely a test issue"
so likely many users will not see the very important result message about the "likely a test issue" and what it means. So we should make the final aggregation messages from openqa-investigate more prominent, for example in the very same comment blank line between for separation, nice unicode icon to catch the eye, maybe also combine the result comment as edit of the first comment.
Acceptance criteria¶
- AC1: The final evaluation comment is obviously more prominent
Suggestions¶
- Similar to the light bulb icon which we added just recently use a fancy unicode icon to catch the eye for that important message
- Take a look into https://github.com/os-autoinst/scripts/ where the code is, search for the string "Detailed explanation of this comment"
- Editing comments is already in our tooling. If it's feasible and easy enough to do then include the result message in the first comment but it's not necessarily mandatory
Updated by okurz about 1 year ago
- Copied from action #132272: Identify reproducible *TEST* issues (not product issues anymore) using openqa-investigate size:M added
Updated by okurz about 1 year ago
- Target version changed from Tools - Next to Ready
Updated by okurz 11 months ago
I just had a look at https://openqa.opensuse.org/tests/3751441#comments
I think the new t#123 syntax isn't recognized for creating the bubbles
Updated by okurz 11 months ago
- Status changed from Feedback to Workable
- Priority changed from Normal to Urgent
This seems to become more severe. Now comments like in https://openqa.opensuse.org/tests/3752529#comments just state "Likely not a sporadic failure.". I think we lost all about identifying test/product/infra issues.
Updated by tinita 11 months ago
Why do you think it's broken?
Here I have an example where it's identifying a test issue:
https://openqa.opensuse.org/tests/3751367#comments
And here we have a product issue:
https://openqa.opensuse.org/tests/3752623#comments
I think the missing bullets is a bit annoying though.
Updated by okurz 11 months ago
tinita wrote in #note-8:
Why do you think it's broken?
Here I have an example where it's identifying a test issue:
https://openqa.opensuse.org/tests/3751367#comments
And here we have a product issue:
https://openqa.opensuse.org/tests/3752623#comments
ok, fine. So maybe I misjudged due to the missing bullets :)
I think the missing bullets is a bit annoying though.
yes, that. That should be addressed today, at least due to reverting the original PR, or maybe just one commit out of that.
Updated by dheidler 11 months ago
- Status changed from Workable to Feedback
Fix for bullet points (also making them look nice): https://github.com/os-autoinst/openQA/pull/5375