openSUSE admin

Hi there Vicent,

First of all, I'd like to point out that download.opensuse.org is basically a redirector.
So, those logs of CI runs won't get us much closer to the truth. And taking a look at our
monitor metrics for the last 48 hours for the download.opensuse.org host doesn't show
anything unusual to me. Thus, in order to get (next) to the bottom of this we'd need to
see the Zypper log files (/var/log/zypper.log) of those failed runs -- which I understand
might not be possible due to the CI be using containers.

That said, we also have regional redirectors around the world -- mirrorcache-eu.opensuse.org
in EU, and there might be other secondary redirectors around EU that I'm not aware of
(yet) -- that send people to mirrors of such regions. This means that nowadays there
won't be that many requests reaching download.opensuse.org effectively as there were in
the past.

Looking at those CI runs I can see that all attempts failed at package download step of
the package installation, except attempt number 4 where it failed at repository metadata
refreshing. And they all fail with "Timeout exceeded when accessing ...", which makes me
think that the mirror those requests ended up had stability issues that caused those
connection timeout -- Zypper can timeout either establishing a connection to a server or
during downloads that takes too much time.

So, my best guess is the CI runs were hitting a bad mirror. Usually, depending on the
issue, bad mirrors are (temporarily) removed from the list of mirrors to avoid people
being redirected to them until their issues are sorted out. However, I imagine there
are cases where a bad mirror can't be distinguished from a good one, by our checks.

Unfortunately in this case I don't think there's much you can do to mitigate the
situation, since you people are using GitHub Actions. Otherwise, I'd advise to temporarily
use an alternate mirror directly instead of download.opensuse.org in the "baseurl" of
Zypper repo files under /etc/zypp/repos.d/. To find out what mirror you get redirected
to you can use an URL of a failing package download, such as:

curl -IL 'http://download.opensuse.org/distribution/leap/15.4/repo/oss/x86_64/gettext-runtime-0.20.2-1.43.x86_64.rpm'

And you can see a list of mirrors serving that package by visiting the URL:

http://download.opensuse.org/distribution/leap/15.4/repo/oss/x86_64/gettext-runtime-0.20.2-1.43.x86_64.rpm.mirrorlist

Kind regards,
Luciano

Interestingly enough, I've ran across the same issue, but in my case I strongly believe it happened because I was using a USB WiFi adapter with not good enough support by the in-tree Linux driver, which cripples severely my download speed.

And the "Timeout exceeded when accessing" output is a bit misleading as it time outs trying to download the rpm binary, really (Timeout exceeded when accessing 'http://mirrorcache-br-2.opensuse.org/repositories/GNOME:/Next/openSUSE_Factory/x86_64/lib
webkit2gtk-4_1-0-2.42.1-1038.1.x86_64.rpm'). What's more, I have "download.transfer_timeout = 300" in /etc/zypp/zypp.conf, instead of the default 180. However, from what I see Zypper is not honoring
that configuration, and is still timing out after only 3 minutes.

Regards,
Luciano