communication #20740
closedserious problem with mirror database
100%
Description
As reported via several channels the mirror database has a serious problem. Several paths either have no mirrors listed at all or not enough.
Updated by lnussel over 6 years ago
- Assignee changed from 17572 to Anonymous
Lars was working on it last night, assigning to him.
Updated by Anonymous over 6 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 10
Updated by lnussel over 6 years ago
regarding missing mirrors in the update repo looks like mb scan could help. Eg.g http://download.opensuse.org/update/leap/42.2/oss/repodata/0318827fcbb2c69d5d43ad8f58ad93c74c77962cf844051f7fbfbb809ffaab52-appdata.xml.gz.mirrorlist the file is mirrored e.g on ftp.uni-stuttgart.de
Updated by lnussel over 6 years ago
potentially related to the mirrordb issue, the size of the rsync modules is too low:
http://www.poeml.de/~poeml/rsyncinfo-stage.o.o.txt
The mirror db still lists several old distributions, e.g. 13.1. They do not exist on pontifex anymore though. The log files still record access to them and redirect to gwdg. So the rsync modules take them into consideration but there are actually no matching files on pontifex' disk. Therefore on disk there's actually less content than calculated.
Updated by Anonymous over 6 years ago
- % Done changed from 10 to 20
lnussel wrote:
potentially related to the mirrordb issue, the size of the rsync modules is too low:
http://www.poeml.de/~poeml/rsyncinfo-stage.o.o.txtThe mirror db still lists several old distributions, e.g. 13.1. They do not exist on pontifex anymore though. The log files still record access to them and redirect to gwdg. So the rsync modules take them into consideration but there are actually no matching files on pontifex' disk. Therefore on disk there's actually less content than calculated.
I'm currently running a vacuum on the database - plus as many scans in parallel as possible (which could/can still be done by anyone with access to scanner-opensuse, btw).
The size of the rsync modules is another topic - but this is something for the afternoon coffee needed
Regards,
Lars
Updated by Anonymous over 6 years ago
- Status changed from In Progress to Closed
- % Done changed from 20 to 100
Result: the two database servers behind mirrorbrain ran out of sync (slave lags behind master)...
While the scanner and all processes with write access are using the master DB Server, the Apache behind download.opensuse.org used the slave DB server - as this slave was not in sync any more, there were no mirrors listed.
Implemented a workaround for the moment by using the master DB server also for apache. This should help us for the moment, but needs a proper fix later.
Updated by mrmazda over 6 years ago
I am in FL USA. DOO offers 8 "closest" mirrors for 42.3 isos, the closest of which is ~6661km distant (somewhere in Ireland), 173% of the beeline distance to
https://sfo-korg-mirror.kernel.org/opensuse/distribution/leap/42.3/iso/openSUSE-Leap-42.3-DVD-x86_64.iso on the opposite side of CONUS.
http://mirrors.opensuse.org/ claims 17 North American mirrors for 42.3, yet DOO offers up none of them. Of those 17, at least 4 produced 404 trying to contact with Firefox.
The closest NA mirror I could identify http://www.gtlib.gatech.edu/pub/opensuse/distribution/leap/42.3/iso/ is 16% of the distance to Ireland, though is empty.