Project

General

Profile

coordination #80828

coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues

[epic] Trigger 'auto-review' and 'openqa-investigate' from within openQA when jobs incomplete or fail on o3+osd

Added by okurz 6 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-12-08
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Motivation

auto-review does a good job but we could benefit from running auto-review more often. We should trigger it directly when openQA jobs fail as well

Acceptance criteria

  • AC1: auto-review is triggered on o3 when jobs fails

Suggestions

  • DONE: Same as in #80736 but for failed, enable for failed as well, first with label-known
  • DONE: Review openQA documentation for the current support, e.g. to consider apparmor, config vs. env, etc. -> http://open.qa/docs/#_enable_custom_hook_scripts_on_job_done_based_on_result
  • DONE: wait for deployment to osd and enable there as well for incomplete and failed
  • DONE: monitor impact of "label-known-issues" on o3+osd as well as progress.o.o -> all good, no problems or overload observed
  • DONE: After "label-known-issues", enable "investigate" jobs on o3+osd
  • DONE: monitor impact on o3+osd
  • DONE: Think about a better approach for apparmor -> https://github.com/os-autoinst/openQA/pull/3651
  • As discussed in https://chat.suse.de/channel/testing?msg=jfuNwoCqDShNp5tbu and follow-up:
    • Reflect status of "todo=1" from test overview in build badges -> solved in #91658
    • Evaluate alternative where to store "investigation information" not in comments but elsewhere, e.g. show in "investigation" tab?

Subtasks

action #80736: Trigger 'auto-review' from within openQA when jobs incomplete (or fail) , for testing: auto_review:"tests died: unable to load main.pm, check the log for the cause"Resolvedokurz

action #80826: Trigger 'auto-review' from within openQA when jobs incomplete on osd as wellResolvedokurz

action #80830: Trigger 'openqa-investigate' from within openQA when jobs fail on o3Resolvedokurz

action #81206: Trigger 'openqa-investigate' from within openQA when jobs fail on osdResolvedokurz

action #81859: openqa-investigate triggers incomplete sets for multi-machine scenariosResolvedmkittler

History

#1 Updated by okurz 6 months ago

  • Copied from action #80736: Trigger 'auto-review' from within openQA when jobs incomplete (or fail) , for testing: auto_review:"tests died: unable to load main.pm, check the log for the cause" added

#2 Updated by okurz 6 months ago

  • Copied to action #80830: Trigger 'openqa-investigate' from within openQA when jobs fail on o3 added

#3 Updated by okurz 6 months ago

  • Tracker changed from action to coordination
  • Subject changed from Trigger 'auto-review' from within openQA when jobs fail on o3 to [epic] Trigger 'auto-review' and 'openqa-investigate' from within openQA when jobs incomplete or fail on o3+osd
  • Status changed from In Progress to Blocked

#5 Updated by okurz 5 months ago

  • Description updated (diff)

Updated suggestions for what had been completed already and added two further suggestions from user feedback.

#6 Updated by okurz 4 months ago

  • Description updated (diff)

#7 Updated by okurz about 1 month ago

  • Description updated (diff)
  • Status changed from Blocked to Resolved

Reflect status of "todo=1" from test overview in build badges -> solved in #91658

Also available in: Atom PDF