action #126527
coordination #102915: [saga][epic] Automated classification of failures
QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests
[spike] Parse comments to identify reproducible product issues using openqa-investigate size:M
Start date:
2023-03-23
Due date:
% Done:
0%
Estimated time:
Difficulty:
Description
Motivation¶
See #109920
Acceptance criteria¶
- AC1: It is known that parsing comments is reliable
- AC2: On failed openQA jobs with openqa-investigate info "retry X, last_good_test X, last_good_build V, last_good_test+build V" (X: failed, V: passed) a comment is written pointing to a likely product regression
- AC3: No such comment is written on other jobs
Suggestions (based on #109920)¶
We already have those comments:
* systemd-networkd:investigate:retry: http://openqa.opensuse.org/t3071081 * systemd-networkd:investigate:last_good_tests:c2c7d0f5ef0e75043509bf7fe1324a81eee077e3: http://openqa.opensuse.org/t3071082 * systemd-networkd:investigate:last_good_build:369.2: http://openqa.opensuse.org/t3071083 * systemd-networkd:investigate:last_good_tests_and_build:c2c7d0f5ef0e75043509bf7fe1324a81eee077e3+369.2: http://openqa.opensuse.org/t3071084
- Consider adding something like
IDS: 3071081,3071082,3071083,3071084
to comments to simplify parsing - In the hook script go to the investigate origin job, fetch comments and parse the ids. Then check if all other jobs are finished. If yes, compare the results. If not, exit with 142 to rerun this hook later.
History
#5
Updated by openqa_review 2 months ago
- Due date set to 2023-04-08
Setting due date based on mean cycle time of SUSE QE Tools
#6
Updated by tinita 2 months ago
Found an unrelated bug during working on this: https://github.com/os-autoinst/scripts/pull/222 Fix handling of case when no last good tests were found (merged)
#9
Updated by tinita about 2 months ago
- Status changed from Feedback to Resolved
Ok, spike solution shows that it is feasable.
Going back to #109920
#10
Updated by okurz about 2 months ago
- Due date deleted (
2023-04-15)
#11
Updated by okurz about 1 month ago
- Parent task changed from #109920 to #94105