Project

General

Profile

Actions
Copy link

coordination #102915

open

[saga][epic] Automated classification of failures

Added by okurz over 2 years ago. Updated 8 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
QA - future
Start date:
2015-12-18
Due date:
% Done:

41%

Estimated time:
(Total: 40.00 h)

Description

User Story

As a reviewer of failed openQA tests I want failures of jobs to be automatically classified for example "test regression", "product regression", "infrastructure issue" to not waste time on bisecting failures

Acceptance criteria

  • AC1: If a job fails or incompletes a classification is provided so that at best no further "test review" effort is needed by human reviewers to triage the issue

Subtasks 29 (8 open — 21 closed)

coordination #9966: [epic] Be more robust about spurious errorsNew2015-12-18

Actions
QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed testsBlockedokurz2021-07-20

Actions
QA - action #95742: In openqa-investigate jobs add URL to original job as settingResolvedokurz2021-07-20

Actions
QA - action #95746: Identify likely "sporadic" openQA tests with "openqa-investigate" size:MResolvedXiaojing_liu2021-07-20

Actions
action #98862: Comment about intermittent/sporadic test issues on original job if openqa-investigate retry job passes size:MResolvedtinita2021-09-18

Actions
action #109920: Identify reproducible product issues using openqa-investigate size:MResolvedtinita2023-03-23

Actions
QA - action #110176: [spike solution] [timeboxed:10h] Restart hook script in delayed minion job based on exit code size:MResolvedkraih2022-06-15

Actions
action #110518: Call job_done_hooks if requested by test setting (not only openQA config as done so far) size:MResolvedmkittler2021-09-18

Actions
action #110530: Do NOT call job_done_hooks if requested by test settingResolvedmkittler2021-09-18

Actions
action #112523: Make hook scripts restartable with a special exit codeResolvedkraih2022-06-15

Actions
action #124991: Copy ids of other investigate jobs to retry jobRejectedokurz2023-02-23

Actions
action #126527: [spike] Parse comments to identify reproducible product issues using openqa-investigate size:MResolvedtinita2023-03-23

Actions
action #132272: Identify reproducible *TEST* issues (not product issues anymore) using openqa-investigate size:MResolvedtinita

Actions
action #132332: Multiple investigation comments for multimachine tests size:MResolvedtinita2023-03-23

Actions
action #138299: Make the final aggregation messages from openqa-investigate more prominent size:SResolveddheidler

Actions
action #151399: Identify reproducible *infrastructure* issues using openqa-investigate size:MResolvedtinita2023-11-24

Actions
action #151402: [spike solution][timeboxed:20h] Allow to search for tests by comment on the UI size:MResolvedmkittler2023-11-24

Actions
action #152281: Schedule openQA SLE maintenance bisect jobs with lower priority same as openqa-investigateResolvedmkittler2023-12-08

Actions
action #152851: Notify about reproducible *infrastructure* issues using openqa-investigateNew2023-11-24

Actions
action #152853: Prevent faulty openQA workers causing wrong openqa-investigate conclusions size:MResolvedybonatakis

Actions
action #154027: [UI][UX] Allow to search for tests by comment on the UI in /testsNew

Actions
action #154036: Allow to search for comments or tests by comment in the UI "search" barNew

Actions
QA - coordination #119338: [epic] SLO Automation for openqatests SLOsBlockedokurz2022-10-12

Actions
QA - action #116545: Automated alerts and reminders about SLO's for openqatests (only one reminder) size:MResolvedtinita

Actions
QA - action #118186: Prevent SLO alerts for tickets with open subtasks as they inherit priority and can not be changed directly size:MResolvedjbaier_cz2022-10-12

Actions
QA - action #119176: Automated alerts and reminders about SLO's for openqatests size:MIn Progressybonatakis2022-10-21

Actions
QA - action #157522: No ticket reminder comments about SLO's for openqatests size:MResolvedtinita

Actions
QA - action #157858: Repeated reminder comments about SLO's for openqatests size:SResolvedtinita2024-03-25

Actions
action #155731: [brainstorm] Be more robust about spurious errorsNew

Actions

Related issues 3 (3 open — 0 closed)

Related to openQA Project - action #45011: Allow detection of known failures at the autoinst-log.txtWorkable2018-12-11

Actions
Related to openQA Project - coordination #89095: [epic] Provide openqa-investigate information on job detailsNew2021-02-25

Actions
Copied from openQA Project - coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQANew2018-04-16

Actions
Actions
Copy link
 #1

Updated by okurz over 2 years ago

  • Copied from coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA added
Actions
Copy link
 #2

Updated by okurz over 2 years ago

  • Related to action #45011: Allow detection of known failures at the autoinst-log.txt added
Actions
Copy link
 #4

Updated by okurz over 1 year ago

  • Related to coordination #89095: [epic] Provide openqa-investigate information on job details added
Actions
Copy link
 #5

Updated by okurz 2 months ago

  • Subtask #9966 added
Actions
Copy link
 #6

Updated by okurz 2 months ago

  • Subtask #155731 added
Actions
Copy link
 #7

Updated by okurz about 1 month ago

  • Subtask #119338 added
Actions
Copy link

Also available in: Atom PDF