Project

General

Profile

Actions

action #152853

closed

coordination #102915: [saga][epic] Automated classification of failures

QA (public) - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests

Prevent faulty openQA workers causing wrong openqa-investigate conclusions size:M

Added by okurz 12 months ago. Updated 10 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Motivation

With #109920 openqa-investigate can identify clear PRODUCT REGRESSIONS and with #132272 we identify TEST REGRESSIONS and with #151399 we can already identify infrastructure issues. Sometimes openqa-investigate can come or lead to wrong conclusions when different workers on which tests have been executed have an impact. So we should see that openqa-investigate by default is not impacted by that, e.g. by executing investigation jobs only on the very same openQA worker.

Acceptance criteria

  • AC1: openqa-investigate last_good_test+last_good_build jobs run on the same worker as the original failure if uniquely identifyable by WORKER_CLASS
  • AC2: openqa-investigate provides a caution notice if no worker could be uniquely identified

Suggestions


Related issues 1 (0 open1 closed)

Copied from openQA Project (public) - action #151399: Identify reproducible *infrastructure* issues using openqa-investigate size:MResolvedtinita2023-11-24

Actions
Actions #1

Updated by okurz 12 months ago

  • Copied from action #151399: Identify reproducible *infrastructure* issues using openqa-investigate size:M added
Actions #2

Updated by tinita 12 months ago

  • Subject changed from Prevent faulty openQA workers causing wrong openqa-investigate conclusions to Prevent faulty openQA workers causing wrong openqa-investigate conclusions size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #3

Updated by livdywan 11 months ago ยท Edited

Let's give it another go (it wasn't picked up in a month). Brought it up on workadventu.re and it seems it was really a lack of time.

Actions #4

Updated by okurz 11 months ago

  • Target version changed from Ready to future
Actions #5

Updated by ybonatakis 11 months ago

  • Status changed from Workable to In Progress
  • Assignee set to ybonatakis
Actions #6

Updated by okurz 11 months ago

picking up a ticket that is in "future", i.e. not in our backlog should be the rare exception. How did you find this ticket?

Actions #7

Updated by livdywan 11 months ago

okurz wrote in #note-6:

picking up a ticket that is in "future", i.e. not in our backlog should be the rare exception. How did you find this ticket?

As mentioned in chat. The ticket was in the backlog until yesterday afternoon and we talked about how to approach it.

Actions #8

Updated by ybonatakis 11 months ago

I was reay to submit a draft before i notice that the requirements require WORKER_CLASS only for last_good_test+last_good_build. So i need to review the code tomorrow

Actions #9

Updated by okurz 11 months ago

  • Target version changed from future to Ready

As discussed in the unblock adding to backlog for now. Please commit your draft and then I suggest to continue with #151399

Actions #10

Updated by ybonatakis 11 months ago

  • Status changed from In Progress to Feedback
Actions #11

Updated by ybonatakis 11 months ago

  • Status changed from Feedback to In Progress
Actions #12

Updated by ybonatakis 11 months ago

  • Status changed from In Progress to Feedback
Actions #13

Updated by okurz 11 months ago

  • Due date set to 2024-02-07
  • Status changed from Feedback to In Progress

Feedback in PR provided. I suggest to keep tickets "In Progress" when we are just waiting for feedback within the team.

Actions #14

Updated by ybonatakis 11 months ago

  • Due date deleted (2024-02-07)

I have to update the changes as i think it sets wrongly name when it clones multimachine jobs

Actions #15

Updated by okurz 11 months ago

  • Due date set to 2024-02-07
Actions #16

Updated by ybonatakis 11 months ago

  • Status changed from In Progress to Resolved

writeup:
It is difficult to identify the actual worker which the test picked up so we clone the test with exactly list of workers which come from vars.json

Actions #17

Updated by okurz 11 months ago

  • Due date deleted (2024-02-07)

Ok, would be good to have a link to a production openQA investigate comment showing the feature in action

Actions #18

Updated by livdywan 10 months ago

okurz wrote in #note-17:

Ok, would be good to have a link to a production openQA investigate comment showing the feature in action

https://openqa.suse.de/tests/13580562#settings

Seems to work as intended. Although I did not see a counter example where it would fail to identify the worker class.

Actions

Also available in: Atom PDF