tickets #116239
closed.TR is directed to Asia instead of Europe
Added by ilgaz.nsa@gmail.com over 1 year ago. Updated 11 months ago.
0%
Description
Hello,
Congratulations for this new improved system to pick mirrors.
Turkey's (TR) country backbone is optimized for European connections and
Asian connections have rather horrible ping rates and/or loss.
For example,
ping mirrorcache-eu.opensuse.org
PING proxy-nue.opensuse.org (195.135.221.140) 56(84) bytes of data.
64 bytes from proxy-nue.opensuse.org (195.135.221.140): icmp_seq=1 ttl=52
time=54.1 ms
64 bytes from proxy-nue.opensuse.org (195.135.221.140): icmp_seq=2 ttl=52
time=54.6 ms
64 bytes from proxy-nue.opensuse.org (195.135.221.140): icmp_seq=3 ttl=52
time=56.2 ms
C
--- proxy-nue.opensuse.org ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 54.055/54.945/56.197/0.911 ms
The mirror I suggested/enabled verinomi.com can also serve great to EU
users rather than Asia. It should be in European pool. Right now, when I
check my IP (which is fixed, 31.223.0.183 ) it shows in Asia.
{"country":"TR","ip":"31.223.0.183","region":"AS"}
Have a nice day
Ilgaz Öcal
Updated by pjessen over 1 year ago
- Category set to Mirrors
- Private changed from Yes to No
Fwiw, retrieving a mirrorlist with COUNTRY=tr, I don't get any mirrors in Turkey ?
Updated by avicenzi over 1 year ago
pjessen wrote:
Fwiw, retrieving a mirrorlist with COUNTRY=tr, I don't get any mirrors in Turkey ?
The problem is not a mirror in Turkey, but rather that Turkey's requests should be sent to Europe instead of Asia since the internet backbone in Turkey has a better link to Europe according to Ilgaz.
Turkey is classified as being in Asia in the MaxMind DB and also by Wikipedia. That's why requests are sent to mirrors in Asia.
Is it possible to override this behavior in MirrorCache? Selected countries to override the default continent they are sent to.
Countries in South/Central America may have the same issue, as some have better backbones to Brazil or the USA instead of Europe.
Updated by pjessen over 1 year ago
avicenzi wrote:
pjessen wrote:
Fwiw, retrieving a mirrorlist with COUNTRY=tr, I don't get any mirrors in Turkey ?
The problem is not a mirror in Turkey, but rather that Turkey's requests should be sent to Europe instead of Asia since the internet backbone in Turkey has a better link to Europe according to Ilgaz.
a) the above is clearly a problem - surely I should be getting a list of our mirrors in Turkey when I specify COUNTRY=tr.
b) Ilgaz did some good work on getting more mirrors in Turkey, personally I wouldn't go creating a custom config, but it's none of my business.
We used to have ASN support (for network 'proximity') in mirrorbrain, but I guess that wasn't migrated to mirrorcache.
Updated by avicenzi over 1 year ago
It seems to be working, at least one mirror in Turkey returned in this case:
$ curl -H 'X-Forwarded-For: 31.223.0.183' http://download.opensuse.org/tumbleweed/iso/README.FIRST.metalink
<?xml version="1.0" encoding="UTF-8"?>
<metalink version="3.0" xmlns="http://www.metalinker.org/" type="dynamic" origin="http://download.opensuse.org/tumbleweed/iso/README.FIRST.metalink" generator="MirrorCache" pubdate="2022-09-22 08:40:36 UTC">
<publisher>
<name>openSUSE</name>
<url>download.opensuse.org</url>
</publisher>
<files>
<file name="README.FIRST">
<size>340</size>
<!-- <mtime>1663765680</mtime> -->
<hash type="md5">ac20b0990012314f1463bd8beda8d67e</hash>
<hash type="sha-1">2577b0abca57bf117e04ca34c9dcc7d887e08891</hash>
<hash type="sha-256">7d30e2fbb33cca14376829a76e7c298c91ff50b82389c63da942477d6bfe3fc1</hash>
<resources>
<!-- Mirrors which handle this country (TR): -->
<url type="http" location="TR" preference="100">http://ftp.linux.org.tr/opensuse/tumbleweed/iso/README.FIRST</url>
<!-- Mirrors in the same continent (AS): -->
<url type="http" location="SG" preference="99">http://twrepo.opensuse.id//tumbleweed/iso/README.FIRST</url>
<url type="http" location="TW" preference="98">http://free.nchc.org.tw/opensuse/tumbleweed/iso/README.FIRST</url>
<url type="http" location="IL" preference="97">http://mirror.isoc.org.il/pub/opensuse/tumbleweed/iso/README.FIRST</url>
<url type="http" location="AM" preference="96">http://opensuse.ucom.am//tumbleweed/iso/README.FIRST</url>
<url type="http" location="CN" preference="95">http://mirrors.bfsu.edu.cn/opensuse/tumbleweed/iso/README.FIRST</url>
<url type="http" location="CN" preference="94">http://mirrors.nju.edu.cn/opensuse/tumbleweed/iso/README.FIRST</url>
<url type="http" location="CN" preference="93">http://mirrors.tuna.tsinghua.edu.cn/opensuse/tumbleweed/iso/README.FIRST</url>
<url type="http" location="CN" preference="92">http://mirrors.cqu.edu.cn/opensuse/tumbleweed/iso/README.FIRST</url>
<url type="http" location="CN" preference="91">http://mirror.lzu.edu.cn/opensuse/tumbleweed/iso/README.FIRST</url>
<url type="http" location="UZ" preference="90">http://mirror.dc.uz/opensuse/tumbleweed/iso/README.FIRST</url>
<!-- Mirrors in other parts of the world: -->
<!-- File origin location: -->
<url type="http" location="" preference="89">http://downloadcontent2.opensuse.org/tumbleweed/iso/README.FIRST</url>
</resources>
</file>
</files>
</metalink>
MirrorCache is returning mirrors in the same country first, not by proximity, then it adds mirrors in the same continent.
Maybe MirrorCache should do:
- Mirrors in the same country (all mirrors)
- Mirrors close to the country (top 5 or 10 by proximity)
- Mirrors in the same continent (top 5 or 10)
- Mirrors in other parts of the world
This still does not guarantee that Europe mirrors are close to Turkey if compared to Asia.
Turkey has only two mirrors it seems, it is very likely that they don't have all the data, and requests could still be sent to Asia.
Updated by pjessen over 1 year ago
avicenzi wrote:
It seems to be working, at least one mirror in Turney returned in this case:
$ curl -H 'X-Forwarded-For: 31.223.0.183' http://download.opensuse.org/tumbleweed/iso/README.FIRST.metalink
Maybe the issue is specific to the mirrorlist, with COUNTRY=cc specified?
I still see:
Mirrors
List of best mirrors for IP address 185.85.248.1, located at 47.1449,8.1551 in (TR)
Mirrors which handle this country:
None
MirrorCache is returning mirrors in the same country first, not by proximity, then it adds mirrors in the same continent.
Right, but mirrorbrain would also order those by network proximity, also within the same country.
Maybe MirrorCache should do:
- Mirrors in the same country (all mirrors)
- Mirrors close to the country (top 5 or 10 by proximity)
- Mirrors in the same continent (top 5 or 10)
- Mirrors in other parts of the world
+1 I think that is pretty much what mirrorbrain did, it would just combine 2 and 3.
This still does not guarantee that Europe mirrors are close to Turkey if compared to Asia.
I think the issue is - should we really be customising our setup depending on which countries have the "best" connections to other countries?
I agree the initial idea - that a "country will have the best connections on its own continent" - may not always hold true anymore, but what do we replace it with?
Updated by andriinikitin over 1 year ago
- Status changed from New to In Progress
I agree the initial idea - that a "country will have the best connections on its own continent" - may not always hold true anymore, but what do we replace it with?
I am thinking about adding a feature, which allows some good mirrors to serve other regions. Then e.g. those mirrors will appear in the second list, which will have meaning "Mirrors in the same continent or nearby".
So, e.g. because of distance:
- for requests from TR: those specially marked mirrors from DE will have higher priority than mirrors from CN;
- for requests from MY: those specially marked mirrors from DE will have lower priority than mirrors from CN.
And opposite: the TR mirror will appear in requests from EU if it is marked in DB accordingly.
It is not exactly what is requested in this ticket, but I think being redirected to several good mirrors in EU should be good enough for users from TR.
Additionally - I wanted to highlight some difference in purpose of .metalink versus .mirrorlist :
- metalink is used by download clients (like libzypp or aria2c). The list doesn't need to be big, let's say 10 the best mirrors should be enough to perform download in parallel. (It also may save some time for creating list: e.g. if the country has enough mirrors, then no need to search DB for more; also save resources for transfer : https://github.com/openSUSE/mirrorbrain/issues/36 ). The sort is heuristics, which includes mirror's stability in last 24h and physical distance according to GeoIP database. Don't mention mirrors which don't support http/https protocol of the user's request.
- mirrorlist is usually used by human (admin) for troubleshooting. Sort is alphabetical. Need to properly list all mirrors without regard for http / https support.
I definitely want to apply this new feature to .metalink (maybe with some comment explaining why mirrors from other region have high priority), but not sure if it is needed for .mirrorlist
Let's assume I can deploy it next Thursday, or will update the ticket about the progress.
Regards,
Andrii Nikitin
Updated by andriinikitin over 1 year ago
- Status changed from In Progress to Feedback
This should be resolved, let us know if you still see any problem.