Actions
coordination #80828
closedcoordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues
[epic] Trigger 'auto-review' and 'openqa-investigate' from within openQA when jobs incomplete or fail on o3+osd
Description
Motivation¶
auto-review does a good job but we could benefit from running auto-review more often. We should trigger it directly when openQA jobs fail as well
Acceptance criteria¶
- AC1: auto-review is triggered on o3 when jobs fails
Suggestions¶
- DONE: Same as in #80736 but for failed, enable for failed as well, first with label-known
- DONE: Review openQA documentation for the current support, e.g. to consider apparmor, config vs. env, etc. -> http://open.qa/docs/#_enable_custom_hook_scripts_on_job_done_based_on_result
- DONE: wait for deployment to osd and enable there as well for incomplete and failed
- DONE: monitor impact of "label-known-issues" on o3+osd as well as progress.o.o -> all good, no problems or overload observed
- DONE: After "label-known-issues", enable "investigate" jobs on o3+osd
- DONE: monitor impact on o3+osd
- DONE: Think about a better approach for apparmor -> https://github.com/os-autoinst/openQA/pull/3651
- As discussed in https://chat.suse.de/channel/testing?msg=jfuNwoCqDShNp5tbu and follow-up:
- Reflect status of "todo=1" from test overview in build badges -> solved in #91658
- Evaluate alternative where to store "investigation information" not in comments but elsewhere, e.g. show in "investigation" tab?
Updated by okurz over 3 years ago
- Copied from action #80736: Trigger 'auto-review' from within openQA when jobs incomplete (or fail) , for testing: auto_review:"tests died: unable to load main.pm, check the log for the cause" added
Updated by okurz over 3 years ago
- Copied to action #80830: Trigger 'openqa-investigate' from within openQA when jobs fail on o3 added
Updated by okurz over 3 years ago
- Tracker changed from action to coordination
- Subject changed from Trigger 'auto-review' from within openQA when jobs fail on o3 to [epic] Trigger 'auto-review' and 'openqa-investigate' from within openQA when jobs incomplete or fail on o3+osd
- Status changed from In Progress to Blocked
Updated by okurz over 3 years ago
- Description updated (diff)
I added a section https://progress.opensuse.org/projects/qa/wiki/Wiki#Change-announcements and announced
- https://progress.opensuse.org/projects/qa/wiki/Wiki#Change-announcements
- to openqa@suse.de (no link to archive was available at time of writing)
- https://one.suse.com/content/perma?id=48601
- https://chat.suse.de/channel/testing?msg=4bmgLqLtLyBt5rEn4
- https://matrix.to/#/!ZzdoIxkFyLxnxddLqM:matrix.org/$1607671930328692VbvoZ:matrix.org?via=matrix.org&via=opensuse.org&via=t2bot.io
EDIT: Also improved the README of os-autoinst/scripts that I point to: https://github.com/os-autoinst/scripts/pull/59
Updated by okurz over 3 years ago
- Description updated (diff)
Updated suggestions for what had been completed already and added two further suggestions from user feedback.
Actions