action #132272
closed
coordination #102915: [saga][epic] Automated classification of failures
QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests
Identify reproducible *TEST* issues (not product issues anymore) using openqa-investigate size:M
Added by okurz about 1 year ago.
Updated 8 months ago.
Category:
Feature requests
Description
Motivation¶
With #109920 openqa-investigate can identify clear PRODUCT REGRESSIONS. We can extend that same concept to now clearly identify test regressions.
Acceptance criteria¶
- AC1: On failed openQA jobs with openqa-investigate info "retry X, last_good_test V, last_good_build X, last_good_test+build V" (X: failed, V: passed) a comment is written pointing to a likely test regression
- AC2: No such comment is written on other jobs
Suggestions¶
Steps¶
- Copied from action #109920: Identify reproducible product issues using openqa-investigate size:M added
- Subject changed from Identify reproducible test issues using openqa-investigate to Identify reproducible *TEST* issues (not product issues anymore) using openqa-investigate size:M
- Description updated (diff)
- Status changed from New to Workable
- Priority changed from High to Normal
- Target version changed from Ready to future
Our backlog is full with more urgent tasks
- Target version changed from future to Ready
- Status changed from Workable to In Progress
- Assignee set to tinita
- Due date set to 2023-10-31
Setting due date based on mean cycle time of SUSE QE Tools
- Description updated (diff)
- Status changed from In Progress to Feedback
- Status changed from Feedback to Resolved
- Due date deleted (
2023-10-31)
- Copied to action #138299: Make the final aggregation messages from openqa-investigate more prominent size:S added
- Status changed from Resolved to In Progress
There is an error in the code when there is no last good build or test:
Oct 20 12:36:13 new-ariel openqa-gru[12895]: /opt/os-autoinst-scripts/openqa-investigate: line 239: : command not found
Oct 20 12:36:13 new-ariel openqa-gru[12897]: /opt/os-autoinst-scripts/openqa-investigate: line 241: : command not found
if ([[ -z "$pass_lgtb" ]] || "$pass_lgtb") && ([[ -z "$pass_lgt" ]] || ! "$pass_lgt") && "$pass_lgb"; then
elif ([[ -z "$pass_lgtb" ]] || "$pass_lgtb") && "$pass_lgt" && ([[ -z "$pass_lgb" ]] || ! "$pass_lgb"); then
Not sure why it doesn't complain in the unit test...
- Due date set to 2023-11-04
Setting due date based on mean cycle time of SUSE QE Tools
- Status changed from In Progress to Resolved
The warning doesn't appear in the logs anymore
- Due date deleted (
2023-11-04)
- Copied to action #151399: Identify reproducible *infrastructure* issues using openqa-investigate size:M added
Also available in: Atom
PDF