Project

General

Profile

action #94492

Updated by mkittler almost 3 years ago

## Motivation 

 Currently the default retention period is in-place: 

 ``` 
 > use telegraf 
 Using database telegraf 
 > show retention policies 
 name      duration shardGroupDuration replicaN default 
 ----      -------- ------------------ -------- ------- 
 autogen 0s         168h0m0s             1          true 
 ``` 

 According to the documentation that means we store the data infinitely¹. 

 It would be possible to configure a time-based retention policy per database (we store all our monitoring data in one database called `telegraf`): https://docs.influxdata.com/influxdb/v1.8/query_language/manage-database/#create-retention-policies-with-create-retention-policy 
 This raises the question which durations we'd like to configure. It would also make sense to "[downsample and retain data](https://docs.influxdata.com/influxdb/v1.8/guides/downsample_and_retain/)" to store data with less granularity over time. 

 I haven't found anything about disk-space-based retention policies within the documentation of InfluxDB so I **assume** this is not an option. 

 --- 

 ¹ 
 > When you create a database, InfluxDB creates a retention policy called autogen with an infinite duration… 

 ## Acceptance Criteria 

 * **AC1:** **AC 1:** The directory monitor.qa.suse.de:/var/lib/influxdb has significantly less than the current 120GB 
 * **AC2:** Grafana is able to show at least 6 months worth of (optionally downsampled) data 

 ## Suggestions 

 * Ensure we have a backup before the process remove data 
 * Check what the current InfluxDB version 1.8 provides 
 * If we have downsampling, we should aim for at least 1 month of full-sampled data data 
 * Checkout previous attempts to export data 
     * dump is still lying around at `/var/lib/influxdb/dumps/postgresql`; it can be deleted if the disk space is needed 
     * see #93922#note-10 but do *not* make the same mistake of using `grep` again for filtering

Back