Project

General

Profile

coordination #102915

[saga][epic] Automated classification of failures

Added by okurz 9 months ago. Updated about 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
QA - future
Start date:
2021-07-20
Due date:
% Done:

75%

Estimated time:
(Total: 40.00 h)
Difficulty:

Description

User Story

As a reviewer of failed openQA tests I want failures of jobs to be automatically classified for example "test regression", "product regression", "infrastructure issue" to not waste time on bisecting failures

Acceptance criteria

  • AC1: If a job fails or incompletes a classification is provided so that at best no further "test review" effort is needed by human reviewers to triage the issue

Subtasks

QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed testsBlockedokurz

QA - action #95742: In openqa-investigate jobs add URL to original job as settingResolvedokurz

QA - action #95746: Identify likely "sporadic" openQA tests with "openqa-investigate" size:MResolvedXiaojing_liu

action #98862: Comment about intermittent/sporadic test issues on original job if openqa-investigate retry job passesNew

QA - action #109920: Identify reproducible product issues using openqa-investigate size:MBlockedokurz

QA - action #110176: [spike solution] [timeboxed:10h] Restart hook script in delayed minion job based on exit code size:MResolvedkraih

action #110518: Call job_done_hooks if requested by test setting (not only openQA config as done so far) size:MResolvedmkittler

action #110530: Do NOT call job_done_hooks if requested by test settingResolvedmkittler

action #112523: Make hook scripts restartable with a special exit codeResolvedkraih

Related issues

Related to openQA Project - action #45011: Allow detection of known failures at the autoinst-log.txtWorkable2018-12-11

Copied from openQA Project - coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQANew2018-04-16

History

#1 Updated by okurz 9 months ago

  • Copied from coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA added

#2 Updated by okurz 9 months ago

  • Related to action #45011: Allow detection of known failures at the autoinst-log.txt added

Also available in: Atom PDF