Project

General

Profile

Actions

action #164988

open

coordination #92323: [saga][epic] Scale up: Fine-grained control over use and removal of results, assets, test data

Better accounting for openqa-investigation jobs size:S

Added by okurz 4 months ago. Updated 16 days ago.

Status:
Workable
Priority:
Low
Assignee:
-
Category:
Feature requests
Start date:
2024-08-06
Due date:
% Done:

0%

Estimated time:

Description

Motivation

#164979 alerted us about /results being nearly full. We found that groupless jobs are now the biggest offender linked to heavy jobs failing often triggering also heavy openqa-investigate jobs.

Acceptance criteria

  • AC1: Big investigation jobs will not fill up our disk space; we would instead just keep less of them.

Suggestions

  • Count investigation jobs towards the group of the original job
    • Investigation jobs are groupless to avoid being considered for the result of the according group
    • It is probably also not wanted by users; investigation jobs should not cause normal jobs to be stored less long but still be kept for a short time.
    • The way the cleanup algorithm currently works makes this also hard to implement. It goes though jobs group by group and factoring in groupless jobs here without good relations in the database is not straight forward / efficient.
  • Use a dedicated group for all investigation jobs
    • Sounds most promising - just create a new group and schedule investigation jobs to be part of it.
    • There is a caveat: Having all investigation jobs in one group does not solve the problem that investigation jobs for a particular scenario become very big. If we put everything in one group one scenario might cause other investigation jobs to be stored only very shortly.

Related issues 1 (0 open1 closed)

Copied from openQA Infrastructure (public) - action #164979: [alert][grafana] File systems alert for WebUI /results size:SResolvedmkittler2024-08-21

Actions
Actions #1

Updated by okurz 4 months ago

  • Copied from action #164979: [alert][grafana] File systems alert for WebUI /results size:S added
Actions #2

Updated by okurz 4 months ago

  • Tags deleted (alert, infra)
  • Description updated (diff)
Actions #3

Updated by tinita 4 months ago

  • Target version changed from Tools - Next to Ready
Actions #4

Updated by livdywan 3 months ago

  • Subject changed from Better accounting for openqa-investigation jobs to Better accounting for openqa-investigation jobs size:S
  • Description updated (diff)
  • Status changed from New to Workable
Actions #5

Updated by okurz 3 months ago

  • Priority changed from Normal to Low
Actions #6

Updated by okurz 2 months ago

  • Parent task set to #92323
Actions #7

Updated by okurz 16 days ago

  • Target version changed from Ready to Tools - Next
Actions

Also available in: Atom PDF