action #98862
coordination #102915: [saga][epic] Automated classification of failures
QA - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests
Comment about intermittent/sporadic test issues on original job if openqa-investigate retry job passes
0%
Description
Comment about intermittent/sporadic test issues on original job if openqa-investigate retry job passes
User story¶
When a job fails, we will investigate the job by triggering new job with different conditions, such as rerun, re-run with last good build, or last commit in the test distribution.
If a retry job passes, the original job may be a intermittent/sporadic case.
We should provide that assessment as a comment back on the original job.
Acceptance criteria¶
- AC1: If an openqa-investigate retry job passes a comment is created on the original job with the assessment that the issue is intermittent or a sporadic test issue
Suggestions¶
- Take a look how we identify likely sporadic issues as a result of failed "retry" jobs in https://github.com/os-autoinst/scripts/blob/master/openqa-investigate#L136=
- Extend to react according on passed retry
- Ensure that hook scripts in github.com/os-autoinst/scripts/ act as one should expect when called on passed jobs
- Use
- Alternative: Extend o3 config to also run job_done_hooks on passed jobs and monitor performance impact
- Same on osd as on o3
Related issues
History
#1
Updated by Xiaojing_liu 9 months ago
- Related to action #95746: Identify likely "sporadic" openQA tests with "openqa-investigate" size:M added
#2
Updated by okurz 9 months ago
- Subject changed from Comment back to the origin job according to the investigate job' result to Comment back to the origin job according to the investigate job's result
- Category set to Feature requests
- Target version set to future
- Parent task set to #94105
thanks. Let's collect a bit of experience first with reacting on failed jobs as done in #95746 before coming back here reacting on passed jobs.
#3
Updated by okurz 2 months ago
- Subject changed from Comment back to the origin job according to the investigate job's result to Comment about intermittent/sporadic test issues on original job if openqa-investigate retry job passes
- Description updated (diff)
- Target version changed from future to Ready
#4
Updated by okurz about 2 months ago
- Copied to action #110518: Call job_done_hooks if requested by test setting (not only openQA config as done so far) size:M added
#5
Updated by okurz about 2 months ago
- Description updated (diff)
- Status changed from New to Blocked
- Assignee set to okurz
blocked by #110518
#6
Updated by mkittler about 1 month ago
- Status changed from Blocked to New
- Assignee deleted (
okurz)
#7
Updated by okurz about 1 month ago
With #110518 resolved we can now trigger investigation jobs within openqa-investigate with the new test setting to trigger hooks on itself so that we can evaluate the result and comment accordingly on the original job.