Project

General

Profile

action #126527

coordination #102915: [saga][epic] Automated classification of failures

QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests

[spike] Parse comments to identify reproducible product issues using openqa-investigate size:M

Added by cdywan 3 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
tinita
Category:
Feature requests
Target version:
Ready
Start date:
2023-03-23
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

See #109920

Acceptance criteria

  • AC1: It is known that parsing comments is reliable
  • AC2: On failed openQA jobs with openqa-investigate info "retry X, last_good_test X, last_good_build V, last_good_test+build V" (X: failed, V: passed) a comment is written pointing to a likely product regression
  • AC3: No such comment is written on other jobs

Suggestions (based on #109920)

We already have those comments:

* systemd-networkd:investigate:retry: http://openqa.opensuse.org/t3071081
* systemd-networkd:investigate:last_good_tests:c2c7d0f5ef0e75043509bf7fe1324a81eee077e3: http://openqa.opensuse.org/t3071082
* systemd-networkd:investigate:last_good_build:369.2: http://openqa.opensuse.org/t3071083
* systemd-networkd:investigate:last_good_tests_and_build:c2c7d0f5ef0e75043509bf7fe1324a81eee077e3+369.2: http://openqa.opensuse.org/t3071084
  • Consider adding something like IDS: 3071081,3071082,3071083,3071084 to comments to simplify parsing
  • In the hook script go to the investigate origin job, fetch comments and parse the ids. Then check if all other jobs are finished. If yes, compare the results. If not, exit with 142 to rerun this hook later.

History

#1 Updated by cdywan 3 months ago

  • Category set to Feature requests
  • Target version set to Ready

#2 Updated by cdywan 3 months ago

  • Subject changed from [spike] Parse comments to identify reproducible product issues using openqa-investigate to [spike] Parse comments to identify reproducible product issues using openqa-investigate size:M

#3 Updated by tinita 3 months ago

  • Description updated (diff)

#4 Updated by tinita 2 months ago

  • Status changed from Workable to In Progress
  • Assignee set to tinita

#5 Updated by openqa_review 2 months ago

  • Due date set to 2023-04-08

Setting due date based on mean cycle time of SUSE QE Tools

#6 Updated by tinita 2 months ago

Found an unrelated bug during working on this: https://github.com/os-autoinst/scripts/pull/222 Fix handling of case when no last good tests were found (merged)

#7 Updated by tinita 2 months ago

  • Status changed from In Progress to Feedback

#8 Updated by tinita 2 months ago

  • Due date changed from 2023-04-08 to 2023-04-15

Also wait for okurz' feedback

#9 Updated by tinita about 2 months ago

  • Status changed from Feedback to Resolved

Ok, spike solution shows that it is feasable.
Going back to #109920

#10 Updated by okurz about 2 months ago

  • Due date deleted (2023-04-15)

#11 Updated by okurz about 1 month ago

  • Parent task changed from #109920 to #94105

Also available in: Atom PDF