Project

General

Profile

action #81206

coordination #39719: [saga][epic] Detection of "known failures" for stable tests, easy test results review and easy tracking of known issues

coordination #80828: [epic] Trigger 'auto-review' and 'openqa-investigate' from within openQA when jobs incomplete or fail on o3+osd

Trigger 'openqa-investigate' from within openQA when jobs fail on osd

Added by okurz 6 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Acceptance criteria

  • AC1: openqa-investigate is triggered on osd when jobs fails

Suggestions

  • Same as in #80830 but for osd
  • monitor impact on osd

Related issues

Related to openQA Project - action #81859: openqa-investigate triggers incomplete sets for multi-machine scenariosResolved2021-01-07

Copied from openQA Project - action #80830: Trigger 'openqa-investigate' from within openQA when jobs fail on o3Resolved2020-12-08

History

#1 Updated by okurz 6 months ago

  • Copied from action #80830: Trigger 'openqa-investigate' from within openQA when jobs fail on o3 added

#2 Updated by okurz 6 months ago

  • Priority changed from Low to Normal

Trying out on osd with

env host=openqa.suse.de exclude_group_regex='.*(Development|Public Cloud|Released|Others|Kernel|Virtualization).*' /opt/os-autoinst-scripts/openqa-label-known-issues-and-investigate-hook

#3 Updated by openqa_review 6 months ago

  • Due date set to 2021-01-04

Setting due date based on mean cycle time of SUSE QE Tools

#4 Updated by okurz 6 months ago

have not encountered problem but config was reset, likely by salt. Making persistent with https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/420

#5 Updated by okurz 6 months ago

many unexpected jobs were triggered. Apparently the git update cron job on osd did not work since 2020-12-08 as the directory belongs to root but the cron job runs as geekotest.

https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/422

Did SQL delete from jobs where test ~ ':investigate:' and state ~ 'scheduled'; to repair for currently scheduled ones. And for running ones to delete nested ones:

for i in $(openqa-cli api --osd jobs state=running | jq '.jobs | .[] | select(.test|test(":investigate:.*:investigate:")) | .id'); do openqa-cli api --osd -X delete jobs/$i ; done

#6 Updated by okurz 5 months ago

  • Related to action #81859: openqa-investigate triggers incomplete sets for multi-machine scenarios added

#7 Updated by okurz 5 months ago

  • Status changed from In Progress to Resolved

With https://openqa.suse.de/tests?match=:investigate: we can see that investigation jobs are triggered. There was some discussion in https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/425 and we disabled openqa-investigate for "Maintenance", based on feedback as visible in related tickets.

#8 Updated by okurz 5 months ago

  • Due date deleted (2021-01-04)

Also available in: Atom PDF