Project

General

Profile

communication #20740

serious problem with mirror database

Added by lnussel over 3 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Immediate
Assignee:
-
Category:
Mirrors
Target version:
-
Start date:
2017-07-25
Due date:
% Done:

100%

Estimated time:

Description

As reported via several channels the mirror database has a serious problem. Several paths either have no mirrors listed at all or not enough.

History

#1 Updated by lnussel over 3 years ago

  • Private changed from Yes to No

#2 Updated by lnussel over 3 years ago

  • Assignee changed from 17572 to Anonymous

Lars was working on it last night, assigning to him.

#3 Updated by Anonymous over 3 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 10

#4 Updated by lnussel over 3 years ago

regarding missing mirrors in the update repo looks like mb scan could help. Eg.g http://download.opensuse.org/update/leap/42.2/oss/repodata/0318827fcbb2c69d5d43ad8f58ad93c74c77962cf844051f7fbfbb809ffaab52-appdata.xml.gz.mirrorlist the file is mirrored e.g on ftp.uni-stuttgart.de

#5 Updated by lnussel over 3 years ago

potentially related to the mirrordb issue, the size of the rsync modules is too low:
http://www.poeml.de/~poeml/rsyncinfo-stage.o.o.txt

The mirror db still lists several old distributions, e.g. 13.1. They do not exist on pontifex anymore though. The log files still record access to them and redirect to gwdg. So the rsync modules take them into consideration but there are actually no matching files on pontifex' disk. Therefore on disk there's actually less content than calculated.

#6 Updated by Anonymous over 3 years ago

  • % Done changed from 10 to 20

lnussel wrote:

potentially related to the mirrordb issue, the size of the rsync modules is too low:
http://www.poeml.de/~poeml/rsyncinfo-stage.o.o.txt

The mirror db still lists several old distributions, e.g. 13.1. They do not exist on pontifex anymore though. The log files still record access to them and redirect to gwdg. So the rsync modules take them into consideration but there are actually no matching files on pontifex' disk. Therefore on disk there's actually less content than calculated.

I'm currently running a vacuum on the database - plus as many scans in parallel as possible (which could/can still be done by anyone with access to scanner-opensuse, btw).

The size of the rsync modules is another topic - but this is something for the afternoon coffee needed

Regards,
Lars

#7 Updated by Anonymous over 3 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 20 to 100

Result: the two database servers behind mirrorbrain ran out of sync (slave lags behind master)...

While the scanner and all processes with write access are using the master DB Server, the Apache behind download.opensuse.org used the slave DB server - as this slave was not in sync any more, there were no mirrors listed.

Implemented a workaround for the moment by using the master DB server also for apache. This should help us for the moment, but needs a proper fix later.

#8 Updated by mrmazda over 3 years ago

I am in FL USA. DOO offers 8 "closest" mirrors for 42.3 isos, the closest of which is ~6661km distant (somewhere in Ireland), 173% of the beeline distance to
https://sfo-korg-mirror.kernel.org/opensuse/distribution/leap/42.3/iso/openSUSE-Leap-42.3-DVD-x86_64.iso on the opposite side of CONUS.

http://mirrors.opensuse.org/ claims 17 North American mirrors for 42.3, yet DOO offers up none of them. Of those 17, at least 4 produced 404 trying to contact with Firefox.

The closest NA mirror I could identify http://www.gtlib.gatech.edu/pub/opensuse/distribution/leap/42.3/iso/ is 16% of the distance to Ireland, though is empty.

Also available in: Atom PDF