action #100988
closedask for OSD /srv to be enlarged (was: investigate how to optimize /srv data utilization on OSD) size:S
0%
Description
Motivation¶
Over 80% space of /srv on OSD are used up. Most of this data is used by our postgresql database.
I raised this concern in slack where some possible reasons where discussed. One of them is to figure out why postgresql uses so much data. @mkittler mentioned that a fresh database import lowers disk space consumption drastically.
Also see poo#89821 for some history about the alert itself.
Acceptance Criteria¶
AC1: Alert does not trigger any longer
AC2: Understand why our production database uses the space it uses
Suggestions¶
- Enlarge partition by opening an eng-infra ticket and ask for some more space for /dev/vdb
- Figure out if the disk utilization of our database can be optimized
- DONE by coolo: Try if the disk utilization can be reduced. E.g. by running the postgresql vaccum
See if an auto vaccum can be configured or if thresholds can be lowered (https://suse.slack.com/archives/C02AJ1E568M/p1634033225193500?thread_ts=1634030652.186600&cid=C02AJ1E568M)-> #100979better alerts-> #100976
Updated by okurz over 3 years ago
- Copied from action #100985: Come up with a way to regularly check job group configs for outliers and misconfiguration, e.g. overly long result retention periods added
Updated by okurz over 3 years ago
- Subject changed from investigate how to optimize /srv data utilization on OSD size:S to ask for OSD /srv to be enlarged (was: investigate how to optimize /srv data utilization on OSD) size:S
Updated by okurz over 3 years ago
- Priority changed from High to Low
- Target version changed from Ready to future
with recent manual cleanup work by coolo /srv went to 62% (62G) usage so this task is less important right now
Updated by okurz over 3 years ago
With vacuuming the top-2 database tables we are now on 44% usage of /srv, i.e. 44G used, 57G free
Updated by okurz about 2 years ago
- Tags set to infra
- Status changed from Workable to Resolved
- Assignee set to okurz
meanwhile space was increased, /srv is now 200G, currently 37% used