Project

General

Profile

Actions

action #181421

open

coordination #102915: [saga][epic] Automated classification of failures

QA (public) - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests

Investigate hook_script minion jobs should detect when jobs are cancelled

Added by tinita 30 days ago. Updated 26 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Start date:
2025-04-25
Due date:
% Done:

0%

Estimated time:

Description

Observation

When there are investigation jobs that are cancelled or obsoleted then the investigate:retry job is retrying 1440 times with a delay of one minute before giving up:
https://openqa.suse.de/minion/jobs?id=15293289

If many of those jobs exist, Minion will spend a lot of time retrying those instead of working on other jobs that could run and do useful stuff.

Maybe openqa-investigate can detect that jobs are cancelled.

  • AC1: If investigate jobs are cancelled, openqa-investigate detects that and creates the final comment without waiting any longer
---
args:
- env ...
  grep_timeout=60 nice ionice -c idle /opt/os-autoinst-scripts/openqa-label-known-issues-and-investigate-hook
- 17444939
- delay: 60
  kill_timeout: 10s
  retries: 1440
  skip_rc: 142
  timeout: 10m
attempts: 1
children: []
created: 2025-04-24T15:48:34.951746Z
delayed: 2025-04-25T10:20:42.619188Z
expires: ~
finished: ~
id: 15293289
lax: 0
notes:
  hook_cmd: env ...
    grep_timeout=60 nice ionice -c idle /opt/os-autoinst-scripts/openqa-label-known-issues-and-investigate-hook
  hook_rc: 142
  hook_result: ''
parents: []
priority: 0
queue: default
result: ~
retried: 2025-04-25T10:19:42.619188Z
retries: 921
started: 2025-04-25T10:19:35.150145Z
state: inactive
task: hook_script
time: 2025-04-25T10:20:24.275635Z
worker: 1989

Suggestions

  • Note that the main investigate:retry job might be cancelled itself, or one of the other jobs. Handling both cases might require different changes to the code

Related issues 1 (0 open1 closed)

Copied to openQA Project (public) - action #181427: [easy][beginner] Run investigation hook_script minion jobs with linear backoff size:SResolvedybonatakis2025-04-25

Actions
Actions #1

Updated by okurz 30 days ago

  • Parent task set to #94105
Actions #2

Updated by tinita 29 days ago

  • Description updated (diff)
Actions #3

Updated by tinita 29 days ago

  • Copied to action #181427: [easy][beginner] Run investigation hook_script minion jobs with linear backoff size:S added
Actions #4

Updated by tinita 29 days ago

  • Description updated (diff)

I split out the backoff: #181427

Actions #5

Updated by tinita 29 days ago

  • Subject changed from Investigate minion jobs should detect when jobs are cancelled to Investigate hook_script minion jobs should detect when jobs are cancelled
Actions #6

Updated by okurz 26 days ago

  • Target version changed from Ready to Tools - Next

We will try with #181427 first which should already help

Actions

Also available in: Atom PDF