action #110911
closedDo not trigger investigation jobs and notifications on jobs where openQA automatically retries size:M
Description
Motivation¶
In https://openqa.opensuse.org/tests/2335254#comments we observed that there are two comments:
system wrote 2022-05-09 11:39:58 +0000
Restarting because RETRY is set to 2 (and only restarted 0 times so far)
and then 8s later
geekotest wrote 2022-05-09 11:40:06 +0000
Automatic investigation jobs
so openQA automatically triggered a retry because we know of the presence of sporadic issues and signal the intent to retry to mitigate hence the investigation and also the notification should be avoided.
Acceptance criteria¶
- AC1: No investigation jobs and notifications are triggered on jobs that are automatically retried by openQA
- AC2: investigation jobs and notifications are still triggered on the last retried job if it fails with an unknown issue
Suggestions¶
- Understand when and where we trigger the job done hooks and retrying
- retry is handled in https://github.com/os-autoinst/openQA/blob/master/lib/OpenQA/Schema/Result/Jobs.pm#L2032
- job_done_hooks are triggered as part of https://github.com/os-autoinst/openQA/blob/master/lib/OpenQA/Schema/Result/Jobs.pm#L2022
- It needs to be checked if we can simply move the retry call further above so that it happens before we call any job_done_hooks or if this breaks something else. If it works, good, if not then we need to find a different way to ensure that openqa-investigate only handles jobs that would not be restarted anyway.
Updated by okurz over 2 years ago
- Copied from action #110803: [tools] openqa_from_containers: test fails in setup_env size:S added
Updated by okurz over 2 years ago
- Assignee set to okurz
Let me check if it is as simple as triggering a retry before the job_done_hooks.
Updated by okurz over 2 years ago
trying out in https://github.com/os-autoinst/openQA/pull/4646
Updated by okurz over 2 years ago
- Subject changed from Investigation jobs and notifications are not triggered on jobs where openQA automatically retries to Investigation jobs and notifications are not triggered on jobs where openQA automatically retries size:M
- Status changed from New to In Progress
Updated by okurz over 2 years ago
- Subject changed from Investigation jobs and notifications are not triggered on jobs where openQA automatically retries size:M to Do not trigger investigation jobs and notifications on jobs where openQA automatically retries size:M
- Due date set to 2022-05-26
- Status changed from In Progress to Feedback
Both PRs merged
Updated by okurz over 2 years ago
- Due date deleted (
2022-05-26) - Status changed from Feedback to Resolved
Deployed the changes on o3. At least I can confirm that openqa-investigation jobs itself are still triggered correctly, see https://openqa.opensuse.org/tests/2344589#comments , and also in general jobs seem to work in general. So I assume this is done.