Project

General

Profile

Actions

action #126527

closed

coordination #102915: [saga][epic] Automated classification of failures

QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests

[spike] Parse comments to identify reproducible product issues using openqa-investigate size:M

Added by livdywan about 1 year ago. Updated 12 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2023-03-23
Due date:
% Done:

0%

Estimated time:

Description

Motivation

See #109920

Acceptance criteria

  • AC1: It is known that parsing comments is reliable
  • AC2: On failed openQA jobs with openqa-investigate info "retry X, last_good_test X, last_good_build V, last_good_test+build V" (X: failed, V: passed) a comment is written pointing to a likely product regression
  • AC3: No such comment is written on other jobs

Suggestions (based on #109920)

We already have those comments:

* systemd-networkd:investigate:retry: http://openqa.opensuse.org/t3071081
* systemd-networkd:investigate:last_good_tests:c2c7d0f5ef0e75043509bf7fe1324a81eee077e3: http://openqa.opensuse.org/t3071082
* systemd-networkd:investigate:last_good_build:369.2: http://openqa.opensuse.org/t3071083
* systemd-networkd:investigate:last_good_tests_and_build:c2c7d0f5ef0e75043509bf7fe1324a81eee077e3+369.2: http://openqa.opensuse.org/t3071084
  • Consider adding something like IDS: 3071081,3071082,3071083,3071084 to comments to simplify parsing
  • In the hook script go to the investigate origin job, fetch comments and parse the ids. Then check if all other jobs are finished. If yes, compare the results. If not, exit with 142 to rerun this hook later.
Actions #1

Updated by livdywan about 1 year ago

  • Category set to Feature requests
  • Target version set to Ready
Actions #2

Updated by livdywan about 1 year ago

  • Subject changed from [spike] Parse comments to identify reproducible product issues using openqa-investigate to [spike] Parse comments to identify reproducible product issues using openqa-investigate size:M
Actions #3

Updated by tinita about 1 year ago

  • Description updated (diff)
Actions #4

Updated by tinita about 1 year ago

  • Status changed from Workable to In Progress
  • Assignee set to tinita
Actions #5

Updated by openqa_review about 1 year ago

  • Due date set to 2023-04-08

Setting due date based on mean cycle time of SUSE QE Tools

Actions #6

Updated by tinita about 1 year ago

Found an unrelated bug during working on this: https://github.com/os-autoinst/scripts/pull/222 Fix handling of case when no last good tests were found (merged)

Actions #7

Updated by tinita about 1 year ago

  • Status changed from In Progress to Feedback
Actions #8

Updated by tinita about 1 year ago

  • Due date changed from 2023-04-08 to 2023-04-15

Also wait for @okurz' feedback

Actions #9

Updated by tinita about 1 year ago

  • Status changed from Feedback to Resolved

Ok, spike solution shows that it is feasable.
Going back to #109920

Actions #10

Updated by okurz 12 months ago

  • Due date deleted (2023-04-15)
Actions #11

Updated by okurz 12 months ago

  • Parent task changed from #109920 to #94105
Actions

Also available in: Atom PDF