tickets #119935
closed15.4/appliances causes high traffic to mirrors
0%
Description
/distribution/leap/15.4/appliances contains 17GB of files that are updated quite often and get synced to many mirrors.
When I checked today, 31 of 40 rsyncs were for 15.4/appliances. This causes quite some disk-IO and network-IO.
We should find some way to publish these less often, e.g. weekly or monthly. Or exclude them from being synced to all mirrors.
Files
| stage.o.o-if_external-week.png (39.6 KB) stage.o.o-if_external-week.png | pontifex2 rsync network traffic |
Updated by favogt over 1 year ago
How often is "quite often"? In the last two weeks it was updated four times.
It's intentional that the media is updated as often as possible, currently every few days.
Updated by bmwiedemann over 1 year ago
I had seen updates on 2022-11-02, 2022-11-03, 2022-11-04 that are visible as extra spikes in the attached munin graph.
Since then it has been rather silent (only the daily Tumbleweed spike).
If there was a way to bunch these updates together in future, that would be appreciated.
Each spike means, download.o.o will be slower for some hours on these days.
Updated by favogt over 1 year ago
15.4:Images:ToTest was published on 11-03 and 11-04, but not 11-02.
We could potentially cut the size in half (-8GiB) by just not doing CR DVDs for s390x and ppc64le.
I'm surprised that it's only about 17GiB, it should be > 30GiB (4x 4G CR DVD + 6x 1G Live CD + 17x 1G JeOS). Are the other files somehow excluded?
Updated by bmwiedemann over 1 year ago
Maybe I counted wrong. It is 51TB:
du -sm /srv/ftp-stage/pub/opensuse/distribution/leap/15.4/appliances/
51296 /srv/ftp-stage/pub/opensuse/distribution/leap/15.4/appliances/
Updated by bmwiedemann 5 months ago
- Status changed from New to Resolved
Current image files are ~2 weeks old, so not a big problem anymore.