action #103380
opencoordination #161414: [epic] Improved salt based infrastructure management
Configure retention/downsampling policy for specific monitoring data stored within InfluxDB
0%
Description
Motivation¶
See #94492 - as part of this ticket a retention period for the entire telegraf
database has been set (to remove data older than two years). It would still be desirable to setup more fine grained retention policies and down-sampling for specific monitoring data. This was not done because it seems InfluxDB only let's one create retention policies per database (and we store all of our monitoring data in the single database telegraf
). Additionally, downsampling needs to be taken care of per field so setting it up for the entire database would be a lot of work (and one needed to extend it every time a new field is added).
Suggestions¶
- Read comments in #94492 (but with a grain of salt since I might have missed some possibilities).
- Likely we needed to move data we want to retain shorter than two years or downsample into another database (and ensure we use that database for reading/writing data).
- Check if we have global retention enabled
- Look into if maybe influxdb (v.1) vs. influxdb2 (separate package) is of any help here
Updated by mkittler almost 3 years ago
- Related to action #94492: Configure retention/downsampling policy for monitoring data stored within InfluxDB size:M added
Updated by okurz almost 3 years ago
- Priority changed from Normal to Low
- Target version set to Ready
Updated by okurz almost 3 years ago
- Target version changed from Ready to future
Discussed in SUSE QE Tools weekly meeting and we decided that our current approach with a global retention period might be good enough for the following years. Likely one could improve by starting with a fresh database or multiple ones and properly configure specific downsampling policies for each measurement.
Updated by okurz almost 3 years ago
https://www.influxdata.com/influxdb-templates/downsampling-influxdb/ sounds promising
Updated by okurz about 1 month ago
- Related to action #167722: Efficient use of monitoring data within influxdb on monitor.qe.nue2.suse.org size:M added