Project

General

Profile

action #101376

coordination #103941: [saga][epic] Scale up: Efficient, event-based handling of storage on new, clean instances

coordination #64881: [epic] Reconsider triggering cleanup jobs

Use cleanup triggers on finished jobs by default

Added by cdywan over 1 year ago. Updated over 1 year ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
QA - future
Start date:
2021-08-31
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

#97763 implemented triggers on finished jobs via minion_task_triggers/on_job_done=. This would be great to enable by default, especially since currently no clean-up will ever be performed by default.

Acceptance criteria

  • AC1: Use cleanup triggers on finished jobs by default

Suggestions

  • Consider removing the dependency on the systemd timer and ensure that production instances still run cleanup after the update
    • Research how to properly communicate these changes based on systemd files, if no better way is found, just do it and tell our usual users, e.g. ping in pull request -> #101602
  • Get familiar with timers and triggers
  • Test the impact on performance, e.g. trigger 1k tiny jobs which trigger cleanup on done
  • Ensure that the df-check is enabled along with enabling the config
  • Add a default config for minion_task_triggers/on_job_done= in etc/openqa/openqa.ini (not in code so that existing instances are not changed)
  • Ensure that the configured cleanup jobs cover the same as the existing systemd timer

Related issues

Copied to openQA Project - action #101602: Research how to properly communicate these changes based on systemd files size:SWorkable

History

#1 Updated by cdywan over 1 year ago

#2 Updated by cdywan over 1 year ago

  • Parent task changed from #64746 to #64881

#3 Updated by okurz over 1 year ago

  • Tracker changed from coordination to action
  • Priority changed from Normal to Low

#4 Updated by okurz over 1 year ago

  • Copied to action #101602: Research how to properly communicate these changes based on systemd files size:S added

#5 Updated by okurz over 1 year ago

  • Subject changed from Use cleanup triggers on finished jobs by default to Use cleanup triggers on finished jobs by default size:M
  • Description updated (diff)
  • Status changed from New to Workable

#6 Updated by okurz over 1 year ago

  • Subject changed from Use cleanup triggers on finished jobs by default size:M to Use cleanup triggers on finished jobs by default
  • Status changed from Workable to New
  • % Done changed from 50 to 0

Discussed during estimation meeting 2021-10-28. mkittler and kraih are not sure if the new default would be that beneficial. Maybe there is still a too big overhead when triggering the cleanup events. We could run the "test" of about 1k jobs first to gather some performance data. Another possible test: Trigger cleanup times on o3+osd more frequently.

#7 Updated by okurz over 1 year ago

  • Target version changed from Ready to future

Let's keep it on hold until we come to new conclusions

Also available in: Atom PDF