Project

General

Profile

Actions
Copy link

action #97763

closed

coordination #103941: [saga][epic] Scale up: Efficient, event-based handling of storage on new, clean instances

coordination #64881: [epic] Reconsider triggering cleanup jobs

Event-based cleanup jobs triggered based on quota size:M

Added by livdywan over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
livdywan
Category:
Feature requests
Target version:
Ready
Start date:
2021-08-31
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Idle instances of openQA, e.g. personal single-user developer instances, only trigger cleanup jobs when quota usage is likely to change, e.g. when new builds or jobs are scheduled or jobs complete.

Acceptance criteria

  • AC1: Cleanup is triggered based on job finished event
  • AC2: No extra minion job for every single finished job

Suggestion

  • Look into existing check for available space
  • Extend the minion job that already exists to check for "assets" quota, see Limit.pm
Actions
Copy link
 #1

Updated by livdywan over 2 years ago

  • Subject changed from Event-based cleanup jobs triggered based on quota to Event-based cleanup jobs triggered based on quota size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #2

Updated by livdywan over 2 years ago

  • Status changed from Workable to In Progress
Actions
Copy link
 #3

Updated by livdywan over 2 years ago

In the PR I'm introducing a new option misc_limits/trigger_cleanup_on_job_done which is 0 by default. Kinda wondering if we shouldn't just configure this through hooks without a new feature? Although we'd have to extend existing hook commands or the scripts we call, the code doesn't do anything special.

Actions
Copy link
 #4

Updated by livdywan over 2 years ago

  • Assignee set to livdywan
Actions
Copy link
 #5

Updated by VANASTASIADIS over 2 years ago

  • Due date set to 2021-09-23

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #6

Updated by livdywan over 2 years ago

Here's a more generic version that supports different minion tasks e.g. if we wanted to have both assets and results cleaned up in the future, we could change that option easily: https://github.com/os-autoinst/openQA/pull/4186

Actions
Copy link
 #7

Updated by livdywan over 2 years ago

  • Status changed from In Progress to Feedback

Waiting for reviews

Actions
Copy link
 #8

Updated by livdywan over 2 years ago

cdywan wrote:

Waiting for reviews

Got my reviews. Still found an issue with tests. Apparently conditionally parsing a trigger into an array means it's a string when it's not set at all and leads to Can't use string ("") as an ARRAY ref, see also my comment on the PR.

Actions
Copy link
 #9

Updated by livdywan over 2 years ago

  • Status changed from Feedback to Resolved

PR merged. Since the user story assumes a "personal development machine" I'm resolving this ticket.

Actions
Copy link
 #10

Updated by okurz over 2 years ago

  • Due date deleted (2021-09-23)
Actions
Copy link

Also available in: Atom PDF