Project

General

Profile

Actions
Copy link

coordination #102915

open

[saga][epic] Automated classification of failures

Added by okurz about 3 years ago. Updated 22 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
QA (public) - future
Start date:
2015-12-18
Due date:
% Done:

46%

Estimated time:
(Total: 40.00 h)

Description

User Story

As a reviewer of failed openQA tests I want failures of jobs to be automatically classified for example "test regression", "product regression", "infrastructure issue" to not waste time on bisecting failures

Acceptance criteria

  • AC1: If a job fails or incompletes a classification is provided so that at best no further "test review" effort is needed by human reviewers to triage the issue

Subtasks 45 (9 open — 36 closed)

coordination #9966: [epic] Be more robust about spurious errorsNew2015-12-18

Actions
coordination #89095: [epic] Provide openqa-investigate information on job detailsNew2021-02-25

Actions
QA (public) - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed testsBlockedokurz2021-07-20

Actions
QA (public) - action #95742: In openqa-investigate jobs add URL to original job as settingResolvedokurz2021-07-20

Actions
QA (public) - action #95746: Identify likely "sporadic" openQA tests with "openqa-investigate" size:MResolvedXiaojing_liu2021-07-20

Actions
action #98862: Comment about intermittent/sporadic test issues on original job if openqa-investigate retry job passes size:MResolvedtinita2021-09-18

Actions
action #109920: Identify reproducible product issues using openqa-investigate size:MResolvedtinita2023-03-23

Actions
QA (public) - action #110176: [spike solution] [timeboxed:10h] Restart hook script in delayed minion job based on exit code size:MResolvedkraih2022-06-15

Actions
action #110518: Call job_done_hooks if requested by test setting (not only openQA config as done so far) size:MResolvedmkittler2021-09-18

Actions
action #110530: Do NOT call job_done_hooks if requested by test settingResolvedmkittler2021-09-18

Actions
action #112523: Make hook scripts restartable with a special exit codeResolvedkraih2022-06-15

Actions
action #124991: Copy ids of other investigate jobs to retry jobRejectedokurz2023-02-23

Actions
action #126527: [spike] Parse comments to identify reproducible product issues using openqa-investigate size:MResolvedtinita2023-03-23

Actions
action #132272: Identify reproducible *TEST* issues (not product issues anymore) using openqa-investigate size:MResolvedtinita

Actions
action #132332: Multiple investigation comments for multimachine tests size:MResolvedtinita2023-03-23

Actions
action #138299: Make the final aggregation messages from openqa-investigate more prominent size:SResolveddheidler

Actions
action #151399: Identify reproducible *infrastructure* issues using openqa-investigate size:MResolvedtinita2023-11-24

Actions
action #151402: [spike solution][timeboxed:20h] Allow to search for tests by comment on the UI size:MResolvedmkittler2023-11-24

Actions
action #152281: Schedule openQA SLE maintenance bisect jobs with lower priority same as openqa-investigateResolvedmkittler2023-12-08

Actions
action #152851: Notify about reproducible *infrastructure* issues using openqa-investigateNew2023-11-24

Actions
action #152853: Prevent faulty openQA workers causing wrong openqa-investigate conclusions size:MResolvedybonatakis

Actions
action #154027: [UI][UX] Allow to search for tests by comment on the UI in /testsNew

Actions
action #154036: Allow to search for comments or tests by comment in the UI "search" barNew

Actions
QA (public) - coordination #119338: [epic] SLO Automation for openqatests SLOsResolvedokurz2022-10-12

Actions
QA (public) - action #116545: Automated alerts and reminders about SLO's for openqatests (only one reminder) size:MResolvedtinita

Actions
QA (public) - action #118186: Prevent SLO alerts for tickets with open subtasks as they inherit priority and can not be changed directly size:MResolvedjbaier_cz2022-10-12

Actions
QA (public) - action #119176: Automated alerts and reminders about SLO's for openqatests size:MResolvedybonatakis2022-10-21

Actions
QA (public) - action #157522: No ticket reminder comments about SLO's for openqatests size:MResolvedtinita

Actions
QA (public) - action #157858: Repeated reminder comments about SLO's for openqatests size:SResolvedtinita2024-03-25

Actions
action #155731: [brainstorm] Be more robust about spurious errorsNew

Actions
coordination #166655: [epic] openqa-label-known-issuesNew2020-02-14

Actions
action #63451: Improve openqa-monitor-incompletes and openqa-label-known-issues to not report about incompletes with cloneResolvedlivdywan2020-02-14

Actions
action #69178: workaround for #64776 using https://github.com/os-autoinst/scripts/blob/master/openqa-label-known-issuesResolvedokurz2020-07-21

Actions
openQA Infrastructure (public) - action #76774: auto-review/openqa-label-known-issues does not conclude within GitLab CIResolvedokurz2020-10-29

Actions
openQA Infrastructure (public) - action #96713: Slow grep in openqa-label-known-issues leads to high CPU usageResolvedokurz

Actions
action #97508: openqa-label-known-issues not triggered correctly on o3 size:SResolvedmkittler2021-08-25

Actions
action #116614: openqa-label-known-issues might label jobs incorrectlyResolvedtinita2022-09-15

Actions
action #132665: [alert] openqa-label-known-issues-and-investigate minion hook failed on o3 size:SResolvedtinita2023-07-13

Actions
action #163769: Errors in gru journal: jq (173 /opt/os-autoinst-scripts/openqa-label-known-issues): parse error: Invalid numeric literal at line 1, column 8 (rc: 4 Input: >>>Waiting 1s until retry #12Resolvedtinita2024-07-11

Actions
action #164153: Errors in gru journal: openqa-label-known-issues error fetching JSON output size:SResolveddheidler2024-07-11

Actions
action #164296: openqa-label-known-issues does not look at known issues if autoinst-log.txt does not exist but reason could be looked at size:SResolvedybonatakis

Actions
action #165716: [o3] Munin - minion hook failed - /opt/os-autoinst-scripts/openqa-label-known-issues: ERROR: line 68 size:MResolvedybonatakis2024-08-23

Actions
action #166649: Rewrite openqa-label-known-issues in Python or another better maintainable languageNew

Actions
action #166772: openqa-label-known-issues overrides size:SResolvedtinita2024-09-13

Actions
action #169747: Multiple finalize_job_results and hook_script minion jobs per openQA job size:MResolvedmkittler2024-09-13

Actions

Related issues 4 (3 open — 1 closed)

Related to openQA Project (public) - action #45011: Allow detection of known failures at the autoinst-log.txtWorkable2018-12-11

Actions
Related to openQA Project (public) - coordination #159300: [saga][epic] AI integration with openQANew2024-04-19

Actions
Related to QA (public) - coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and releaseResolvedokurz2021-04-21

Actions
Copied from openQA Project (public) - coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQANew2018-04-16

Actions
Actions
Copy link
 #1

Updated by okurz about 3 years ago

  • Copied from coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA added
Actions
Copy link
 #2

Updated by okurz about 3 years ago

  • Related to action #45011: Allow detection of known failures at the autoinst-log.txt added
Actions
Copy link
 #4

Updated by okurz over 2 years ago

  • Related to coordination #89095: [epic] Provide openqa-investigate information on job details added
Actions
Copy link
 #5

Updated by okurz 10 months ago

  • Subtask #9966 added
Actions
Copy link
 #6

Updated by okurz 10 months ago

  • Subtask #155731 added
Actions
Copy link
 #7

Updated by okurz 9 months ago

  • Subtask #119338 added
Actions
Copy link
 #8

Updated by okurz 6 months ago

Actions
Copy link
 #10

Updated by okurz 3 months ago

  • Subtask #89095 added
Actions
Copy link
 #11

Updated by okurz 3 months ago

  • Subtask #166655 added
Actions
Copy link
 #12

Updated by okurz 3 months ago

  • Related to coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release added
Actions
Copy link

Also available in: Atom PDF