Project

General

Profile

Actions

action #181154

open

openQA Infrastructure (public) - coordination #151582: [epic] Future improvements for QE infrastructure salt management

Automatic cleanup of kvm assets produced by the s390x kvm backend (svirt) in os-autoinst size:S

Added by livdywan about 1 month ago. Updated 18 days ago.

Status:
Workable
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2025-04-17
Due date:
% Done:

0%

Estimated time:
Tags:

Description

Motivation

In the context of #154180 the question came up if old kvm assets should be cleaned up as part of (osd) production infrastructure or perhaps this could be an upstream openQA feature.

For reference, what we do at the time of this writing:
https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/libvirt/cleanup-openqa-assets

 if [[ $(df | grep "/var/lib/libvirt/images" | awk '{print $5}' | sed "s/\%//") -gt 70 ]] ; then
    find /var/lib/libvirt/images/*.qcow2 ! -exec fuser -s "{}" \; -delete 2>/dev/null

Acceptance criteria

  • AC1: s390x kvm assets are prepared and deleted as part of openQA job scheduling without additional handling

Suggestions

  • Lookup how the s390x kvm backend works from https://github.com/os-autoinst/os-autoinst/blob/master/backend/svirt.pm
  • Confirm if the backend code actually prepares these images and decides where to store them
    • In which case it would make sense to clean them automatically
    • The script in salt cleans "some" files by a pattern, not necessarily the ones created by the backend
    • Can this be fully controlled by os-autoinst or openQA clean-up jobs?
    • This would be a new openQA feature in this case?
  • Consider to clean up assets when no longer in use, or even deleted after they have been used, or reuse the same files over and over again to not fill up
  • Note this is concerning the libvirt host, meaning different scope to what openQA cleanup usually manages - do we want openQA to care about this case, too?
    • Maybe want a "worker cache" but not on workers
  • Keep it simple? Just delete the oldest file?

Related issues 1 (1 open0 closed)

Copied from openQA Infrastructure (public) - action #154180: Proper kvm asset cleanup for s390x kvm backend (svirt) and tests size:SBlockedlivdywan

Actions
Actions #1

Updated by livdywan about 1 month ago

  • Copied from action #154180: Proper kvm asset cleanup for s390x kvm backend (svirt) and tests size:S added
Actions #2

Updated by okurz about 1 month ago

  • Category set to Feature requests
  • Target version changed from future to Ready
Actions #3

Updated by livdywan about 1 month ago

  • Subject changed from Automatic cleanup of kvm assets produced by the s390x kvm backend (svirt) in os-autoinst to Automatic cleanup of kvm assets produced by the s390x kvm backend (svirt) in os-autoinst size:S
  • Description updated (diff)
  • Status changed from New to Workable
Actions #4

Updated by livdywan 19 days ago

  • Priority changed from Normal to High

I'm raising prio since we found that the current implementation is flaky and usage of space goes up quite a bit unless it runs very frequently.

Actions #5

Updated by okurz 18 days ago

  • Priority changed from High to Low

Yes, the current implementation is not optimal but has been running for years. Unfortunately we other other priorities preventing us from handling this with higher priority, e.g. #180626

Actions

Also available in: Atom PDF