Project

General

Profile

Actions

action #138299

closed

coordination #102915: [saga][epic] Automated classification of failures

QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests

Make the final aggregation messages from openqa-investigate more prominent size:S

Added by okurz about 1 year ago. Updated 11 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
Feature requests
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Motivation

#132272 already showed us that besides clear product issues we can now also clearly identify test issues. However the message is rather subtle, nestled behind the repetition of the "retry job evaluation" comment, e.g.
from https://openqa.suse.de/tests/12563482#comments

  • first comment: all 4 investigation jobs triggered including nice light bulb with link to more information
  • second comment: bisect jobs
  • third comment: retry job result evaluation and just behind with a not so obvious message the text "Jobs including the last good test are ok, likely a test issue"

so likely many users will not see the very important result message about the "likely a test issue" and what it means. So we should make the final aggregation messages from openqa-investigate more prominent, for example in the very same comment blank line between for separation, nice unicode icon to catch the eye, maybe also combine the result comment as edit of the first comment.

Acceptance criteria

  • AC1: The final evaluation comment is obviously more prominent

Suggestions

  • Similar to the light bulb icon which we added just recently use a fancy unicode icon to catch the eye for that important message
  • Take a look into https://github.com/os-autoinst/scripts/ where the code is, search for the string "Detailed explanation of this comment"
  • Editing comments is already in our tooling. If it's feasible and easy enough to do then include the result message in the first comment but it's not necessarily mandatory

Related issues 1 (0 open1 closed)

Copied from openQA Project - action #132272: Identify reproducible *TEST* issues (not product issues anymore) using openqa-investigate size:MResolvedtinita

Actions
Actions #1

Updated by okurz about 1 year ago

  • Copied from action #132272: Identify reproducible *TEST* issues (not product issues anymore) using openqa-investigate size:M added
Actions #2

Updated by okurz about 1 year ago

  • Target version changed from Tools - Next to Ready
Actions #3

Updated by okurz 12 months ago

  • Subject changed from Make the final aggregation messages from openqa-investigate more prominent to Make the final aggregation messages from openqa-investigate more prominent size:S
  • Description updated (diff)
  • Status changed from New to Workable
Actions #4

Updated by dheidler 12 months ago

  • Assignee set to dheidler
Actions #5

Updated by dheidler 11 months ago

  • Status changed from Workable to Feedback
Actions #6

Updated by okurz 11 months ago

Actions #7

Updated by okurz 11 months ago

  • Status changed from Feedback to Workable
  • Priority changed from Normal to Urgent

This seems to become more severe. Now comments like in https://openqa.opensuse.org/tests/3752529#comments just state "Likely not a sporadic failure.". I think we lost all about identifying test/product/infra issues.

Actions #8

Updated by tinita 11 months ago

Why do you think it's broken?
Here I have an example where it's identifying a test issue:
https://openqa.opensuse.org/tests/3751367#comments
And here we have a product issue:
https://openqa.opensuse.org/tests/3752623#comments

I think the missing bullets is a bit annoying though.

Actions #9

Updated by okurz 11 months ago

tinita wrote in #note-8:

Why do you think it's broken?
Here I have an example where it's identifying a test issue:
https://openqa.opensuse.org/tests/3751367#comments
And here we have a product issue:
https://openqa.opensuse.org/tests/3752623#comments

ok, fine. So maybe I misjudged due to the missing bullets :)

I think the missing bullets is a bit annoying though.

yes, that. That should be addressed today, at least due to reverting the original PR, or maybe just one commit out of that.

Actions #10

Updated by dheidler 11 months ago

  • Status changed from Workable to Feedback

Fix for bullet points (also making them look nice): https://github.com/os-autoinst/openQA/pull/5375

Actions #11

Updated by dheidler 11 months ago

  • Status changed from Feedback to Resolved

Looks fine on o3 now.

Actions

Also available in: Atom PDF