Project

General

Profile

tickets #119935

15.4/appliances causes high traffic to mirrors

Added by bmwiedemann 3 months ago. Updated 3 months ago.

Status:
New
Priority:
Normal
Assignee:
opensuse-admin-obs
Category:
OBS
Target version:
-
Start date:
2022-11-04
Due date:
% Done:

0%

Estimated time:

Description

/distribution/leap/15.4/appliances contains 17GB of files that are updated quite often and get synced to many mirrors.

When I checked today, 31 of 40 rsyncs were for 15.4/appliances. This causes quite some disk-IO and network-IO.

We should find some way to publish these less often, e.g. weekly or monthly. Or exclude them from being synced to all mirrors.

stage.o.o-if_external-week.png (39.6 KB) stage.o.o-if_external-week.png pontifex2 rsync network traffic bmwiedemann, 2022-11-08 03:55
14077

History

#1 Updated by bmwiedemann 3 months ago

  • Private changed from Yes to No

#2 Updated by favogt 3 months ago

How often is "quite often"? In the last two weeks it was updated four times.

It's intentional that the media is updated as often as possible, currently every few days.

#3 Updated by bmwiedemann 3 months ago

14077

I had seen updates on 2022-11-02, 2022-11-03, 2022-11-04 that are visible as extra spikes in the attached munin graph.
Since then it has been rather silent (only the daily Tumbleweed spike).
If there was a way to bunch these updates together in future, that would be appreciated.
Each spike means, download.o.o will be slower for some hours on these days.

#4 Updated by favogt 3 months ago

15.4:Images:ToTest was published on 11-03 and 11-04, but not 11-02.

We could potentially cut the size in half (-8GiB) by just not doing CR DVDs for s390x and ppc64le.

I'm surprised that it's only about 17GiB, it should be > 30GiB (4x 4G CR DVD + 6x 1G Live CD + 17x 1G JeOS). Are the other files somehow excluded?

#5 Updated by bmwiedemann 3 months ago

Maybe I counted wrong. It is 51TB:

du -sm /srv/ftp-stage/pub/opensuse/distribution/leap/15.4/appliances/
51296 /srv/ftp-stage/pub/opensuse/distribution/leap/15.4/appliances/

Also available in: Atom PDF