Project

General

Profile

action #103380

Configure retention/downsampling policy for specific monitoring data stored within InfluxDB

Added by mkittler about 2 months ago. Updated about 2 months ago.

Status:
New
Priority:
Low
Assignee:
-
Target version:
Start date:
2021-12-01
Due date:
% Done:

0%

Estimated time:

Description

motivation

See #94492 - as part of this ticket a retention period for the entire telegraf database has been set (to remove data older than two years). It would still be desirable to setup more fine grained retention policies and down-sampling for specific monitoring data. This was not done because it seems InfluxDB only let's one create retention policies per database (and we store all of our monitoring data in the single database telegraf). Additionally, downsampling needs to be taken care of per field so setting it up for the entire database would be a lot of work (and one needed to extend it every time a new field is added).

suggestions

  • Read comments in #94492 (but with a grain of salt since I might have missed some possibilities).
  • Likely we needed to move data we want to retain shorter than two years or downsample into another database (and ensure we use that database for reading/writing data).

Related issues

Related to openQA Infrastructure - action #94492: Configure retention/downsampling policy for monitoring data stored within InfluxDB size:MResolved2021-06-22

History

#1 Updated by mkittler about 2 months ago

  • Related to action #94492: Configure retention/downsampling policy for monitoring data stored within InfluxDB size:M added

#2 Updated by okurz about 2 months ago

  • Priority changed from Normal to Low
  • Target version set to Ready

#3 Updated by okurz about 2 months ago

  • Target version changed from Ready to future

Discussed in SUSE QE Tools weekly meeting and we decided that our current approach with a global retention period might be good enough for the following years. Likely one could improve by starting with a fresh database or multiple ones and properly configure specific downsampling policies for each measurement.

Also available in: Atom PDF