Actions
action #77887
closed[tools][openqa] Enable automatic openQA investigation jobs for osd as well
Start date:
2020-11-14
Due date:
% Done:
0%
Estimated time:
Tags:
Description
Motivation¶
SUSE QEM is challenged by a relative high false-positive rate of openQA tests. This is due to objectively higher product quality of released products in comparison to products in development, i.e. pre-GM SLE including Tumbleweed snapshots before release. We already use "openqa-investigate" for o3 which has been running there for multiple months and I received positive feedback. We can now extend the solution to osd.
Acceptance criteria¶
- AC1: Automatic investigation jobs are triggered and commented on new, unlabeled failures within osd production groups, e.g. not development groups
- AC2: Automatic investigation jobs run within a reasonable time to provide useful feedback to reviewers in their regular review routines
- AC3: No harmful performance impact on infrastructure due to too many automatic investigation jobs
Suggestions¶
- There is a script based solution "openqa-investigate" which we already use automatically within o3, see https://gitlab.suse.de/openqa/auto-review/-/blob/master/.gitlab-ci.yml#L110
- Try out dry-runs of "openqa-investigate" against OSD and check for obvious things missing or going wrong, e.g. immediate errors or crashes or unreasonable results
- Extending to osd can be just as simple as applying a similar block in .gitlab-ci.yml for osd
- When activated monitor over couple of days for usefulness and performance impact
- Consider changing the schedule of the scheduled pipeline, e.g. trigger more often over the day, or even "continuous" :)
- Optional: Ensure that auto-review walks first over all issues, potentially even failed ones to detect known issues, and if unknown to auto-review, only then trigger investigation jobs
Updated by okurz almost 4 years ago
- Project changed from 46 to openQA Infrastructure
Updated by okurz almost 4 years ago
- Status changed from Workable to Resolved
- Assignee set to okurz
All this was covered with openqa-investigate triggered as part of job-done-hooks on osd: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/420
Actions