Project

General

Profile

Actions
Copy link

action #126527

closed

coordination #102915: [saga][epic] Automated classification of failures

QA (public) - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests

[spike] Parse comments to identify reproducible product issues using openqa-investigate size:M

Added by livdywan over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
tinita
Category:
Feature requests
Target version:
Ready
Start date:
2023-03-23
Due date:
% Done:

0%

Estimated time:

Description

Motivation

See #109920

Acceptance criteria

  • AC1: It is known that parsing comments is reliable
  • AC2: On failed openQA jobs with openqa-investigate info "retry X, last_good_test X, last_good_build V, last_good_test+build V" (X: failed, V: passed) a comment is written pointing to a likely product regression
  • AC3: No such comment is written on other jobs

Suggestions (based on #109920)

We already have those comments:

* systemd-networkd:investigate:retry: http://openqa.opensuse.org/t3071081
* systemd-networkd:investigate:last_good_tests:c2c7d0f5ef0e75043509bf7fe1324a81eee077e3: http://openqa.opensuse.org/t3071082
* systemd-networkd:investigate:last_good_build:369.2: http://openqa.opensuse.org/t3071083
* systemd-networkd:investigate:last_good_tests_and_build:c2c7d0f5ef0e75043509bf7fe1324a81eee077e3+369.2: http://openqa.opensuse.org/t3071084
  • Consider adding something like IDS: 3071081,3071082,3071083,3071084 to comments to simplify parsing
  • In the hook script go to the investigate origin job, fetch comments and parse the ids. Then check if all other jobs are finished. If yes, compare the results. If not, exit with 142 to rerun this hook later.
Actions
Copy link

Also available in: Atom PDF