action #61094
closedo3: cleanup of snapshot-changes
100%
Description
Observation¶
ariel:/var/lib/openqa/share/factory # du -BM -d1
1M ./tmp
377585M ./hdd
460845M ./iso
564647M ./repo
31157M ./snapshot-changes
69747M ./other
1503980M
so currently we have 31G for snapshot changes. We should have a cleanup strategy, asked DimStar in #opensuse-factory
Updated by okurz over 4 years ago
- Status changed from New to Feedback
- Assignee set to okurz
- Target version set to Current Sprint
[17/12/2019 14:59:10] <DimStar> okurz: I'd say yes - certainly everything < 2019 is no longer needed (ever)
[17/12/2019 14:59:39] <DimStar> generally, it doesn't even make sense to keep more than 10 of the 'before the current' link ones
[17/12/2019 15:00:00] <DimStar> everything older is already as .txt on the download server
[17/12/2019 15:04:22] <okurz> cool, thx for the answer. So I will clean out manually once and we can then add a `find -mtime … -delete` in the scripts
did a manual rm $(ls | head -n -10).
https://gitlab.suse.de/openqa/scripts/merge_requests/425 for an automatic cleanup.
Updated by okurz about 4 years ago
this triggered the problem that newer snapshot changes were not recorded anymore. Reverted for now with https://gitlab.suse.de/openqa/scripts/merge_requests/433
Instead of calling rm $(ls … | head -n -10) from top level I should probably do that when the perl code decends into the individual directories.
Updated by okurz about 4 years ago
- Assignee changed from okurz to andriinikitin
@andriinikitin as you are currently working on covering o3 with ObsRsync I guess it makes sense if you also take this ticket to keep in mind that "snapshot-changes" need to be covered as well which is currently triggered within rsync.pl . The task for this ticket is to ensure that older changelog files are eventually pruned.
Updated by andriinikitin about 4 years ago
- Status changed from Feedback to In Progress
Updated by livdywan about 4 years ago
- Priority changed from Normal to Low
This can be considered low prio I think. The number is already down from the original report and most likely the changelogs won't make a huge difference:
cdywan@ariel:~> du -BM -d1 /var/lib/openqa/share/factory/snapshot-changes
2692M /var/lib/openqa/share/factory/snapshot-changes
Updated by andriinikitin about 4 years ago
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
This should be covered by new cron job:
$ cat /etc/cron.d/factory-news-maintenance
# remove files excluding 60 the newest for each arch and flavor
# weekly
0 1 * * SUN find /var/lib/snapshot-changes/opensuse/Tumbleweed/ -maxdepth 1 -mindepth 1 -type f -printf '%T@\t%p\n' | sort -t $'\t' -g | head -n -60 | cut -d $'\t' -f 2- | xargs rm
15 1 * * SUN find /var/lib/snapshot-changes/opensuse-aarch64/Tumbleweed/ -maxdepth 1 -mindepth 1 -type f -printf '%T@\t%p\n' | sort -t $'\t' -g | head -n -60 | cut -d $'\t' -f 2- | xargs rm
30 1 * * SUN find /var/lib/snapshot-changes/opensuse/15.2/ -maxdepth 1 -mindepth 1 -type f -printf '%T@\t%p\n' | sort -t $'\t' -g | head -n -60 | cut -d $'\t' -f 2- | xargs rm
45 1 * * SUN find /var/lib/snapshot-changes/opensuse-aarch64/15.2/ -maxdepth 1 -mindepth 1 -type f -printf '%T@\t%p\n' | sort -t $'\t' -g | head -n -60 | cut -d $'\t' -f 2- | xargs rm
# monthly
0 2 1 * * find /var/lib/snapshot-changes/microos/Tumbleweed/ -maxdepth 1 -mindepth 1 -type f -printf '%T@\t%p\n' | sort -t $'\t' -g | head -n -60 | cut -d $'\t' -f 2- | xargs rm
15 2 1 * * find /var/lib/snapshot-changes/microos-aarch64/Tumbleweed/ -maxdepth 1 -mindepth 1 -type f -printf '%T@\t%p\n' | sort -t $'\t' -g | head -n -60 | cut -d $'\t' -f 2- | xargs rm
0 2 1 * * find /var/lib/snapshot-changes/kubic/Tumbleweed/ -maxdepth 1 -mindepth 1 -type f -printf '%T@\t%p\n' | sort -t $'\t' -g | head -n -60 | cut -d $'\t' -f 2- | xargs rm
30 2 1 * * find /var/lib/snapshot-changes/kubic-aarch64/Tumbleweed/ -maxdepth 1 -mindepth 1 -type f -printf '%T@\t%p\n' | sort -t $'\t' -g | head -n -60 | cut -d $'\t' -f 2- | xargs rm