action #102374
closed
QA - coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release
coordination #89062: [epic] Simplify review for SUSE QAM
Support use of force_result via ticket title in auto-review size:M
Added by okurz over 2 years ago.
Updated over 2 years ago.
Category:
Feature requests
Description
Motivation¶
We have the new "force_result" openQA feature (see #95479) and there is auto-review which allows to automatically comment on jobs. As an alternative to retriggering on sporadic issues we can use auto-review to change the result of jobs showing known acceptable failures. This allows to mark failures that should be ignored for an auto-approval process, e.g. as used by qa-maintenance/bot-ng, while ensuring that there is an according open ticket to properly fix the failures. Additionally this allows to search easily for jobs related to tickets.
Acceptance criteria¶
- AC1: tickets with a special command in the subject line can change the job result of openQA jobs
- AC2: The "retry" command of auto-review still works
- AC3: jobs with changed results still have a reference to the according tickets
Suggestions¶
- Tracker changed from coordination to action
- Description updated (diff)
- Subject changed from Combine "force_result" with auto-review to Support use of "force_result" via ticket title in auto-review size:M
- Description updated (diff)
- Status changed from New to Workable
- Status changed from Workable to In Progress
- Assignee set to tinita
- Due date set to 2021-12-09
Setting due date based on mean cycle time of SUSE QE Tools
- Status changed from In Progress to Feedback
- Status changed from Feedback to In Progress
Still need to work on the PR
- Due date changed from 2021-12-09 to 2021-12-10
- Status changed from In Progress to Feedback
- Due date changed from 2021-12-10 to 2021-12-13
Still waiting for feedback
PR was merged, I will check logs on o3 / osd.
- Subject changed from Support use of "force_result" via ticket title in auto-review size:M to Support use of "force_result" via ticket title in auto-review size:M auto_review:"systemctl start docker' failed at .*/openqa_webui.pm":force_result:softfailed
Using this ticket as a live test
- Due date changed from 2021-12-13 to 2021-12-15
As there are unfortunately luckily no failures in openqa-in-openqa tests today, I couldn't check if the feature works live.
I will try to create a failing test myself.
Meanwhile, I don't see any newly introduced problems in the gru logs of o3 and openqa.
This should not have anything to do with openqa-in-openqa tests. But as we already discussed you can create a new test from your tests distri fork which fails early, e.g. a die "my message"
- Subject changed from Support use of "force_result" via ticket title in auto-review size:M auto_review:"systemctl start docker' failed at .*/openqa_webui.pm":force_result:softfailed to Support use of "force_result" via ticket title in auto-review size:M auto_review:"Test died: fail on purpose at .*/openqa_webui.pm":force_result:softfailed
- Subject changed from Support use of "force_result" via ticket title in auto-review size:M auto_review:"Test died: fail on purpose at .*/openqa_webui.pm":force_result:softfailed to Support use of force_result via ticket title in auto-review size:M auto_review:"Test died: fail on purpose at .*/openqa_webui.pm":force_result:softfailed
- Subject changed from Support use of force_result via ticket title in auto-review size:M auto_review:"Test died: fail on purpose at .*/openqa_webui.pm":force_result:softfailed to Support use of force_result via ticket title in auto-review size:M
- Status changed from Feedback to Resolved
- Due date deleted (
2021-12-15)
Also available in: Atom
PDF