Project

General

Profile

Actions

tickets #159195

open

Repo server IP address not in DNS

Added by dnl028@gmail.com about 1 month ago. Updated 3 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Mirrors
Target version:
-
Start date:
2024-04-17
Due date:
% Done:

0%

Estimated time:

Description

Hi,

Have a system that firewall restricts all outbound traffic except a few
certain allowances, notably opensuse tumbleweed repos/mirrors.
Accordingly, have resolved all the repo related domains to their
respective ip addresses, but lately system updates fail to retrieve some
packages from cdn.opensuse.org, seemingly due to failure to reach it.
A dig on cdn.opensuse.org consistently resolves to these four, which
have been allowed already by the firewall...
151.101.129.91
151.101.1.91
151.101.65.91
151.101.193.91

However, these failed retrievals are apparently because cannot reach
146.75.9.91, and when manually adding it for firewall allowance, the
problem is resolved.
Although, this ip address is never found by a dig on the domain, and a
hard-coded / manual solution will not suffice here.
Is it some mistake that this ip is not in cdn.opensuse.org dns records,
and will perhaps be added? What do you advise on this?

This pertains to the the ticket...
https://progress.opensuse.org/issues/115142
...although, assume there's no need to actually look at said ticket to
understand what's above.

Thank you

Actions #1

Updated by crameleon about 1 month ago

  • Category set to Mirrors
  • Status changed from New to Closed
  • Private changed from Yes to No

Hi,

cdn.opensuse.org is a CNAME to a domain operated by Fastly, the company hosting the CDN. We do not control the Fastly domain and the IP addresses behind it - they could change at any time, and the HTTP server behind the initial domain might redirect you to other domains (and IP addresses), which is likely what you are observing.

For filtering outbound traffic I recommend choosing a static mirror close to you from https://mirrors.opensuse.org/ and configuring that both on your clients and in your firewall policy. Of course, the mirror providers might change their IP addresses at some point as well - but it is less common since they are usually bound to whitelisting on our origin server.

Best,
Georg

Actions #2

Updated by dnl028@gmail.com 15 days ago

  • Status changed from Closed to New

Yes, a static mirror (ip address) should work fine in the address of the
configured repos on the system in question. However, since the update
process seems to use more servers than just that of the configured repos
addresses, more server ip addresses will need to be allowed egress thru
the system firewall. That's at least one reason for doing this
dynamically in the first place, and allowing egress for all the mirror's
ip addresses. Would it seem there's just no dynamic way to accommodate
cdn.opensuse.org? That is the only problem with this approach that
otherwise works quite nicely.

Thanks,
Dan

On 4/17/2024 5:09 PM, crameleon wrote:

recommend choosing a static mirror close to you

Actions #3

Updated by crameleon 15 days ago

  • Status changed from New to Closed

Zypper will only connect to the configured download servers, and in case of redirection servers (for example download.opensuse.org), also the redirect targets.
You can use zypper lr -d to inspect which servers you have configured and curl -I <URL to .rpm file on configured mirror> to check whether a redirection to a different host is taking place.
Generally third party mirrors do not implement any redirection like download.opensuse.org does, hence my recommendation to use a consistent local mirror.

Actions #4

Updated by dnl028@gmail.com 14 days ago

  • Status changed from Closed to New

Ok. Although, setting the repos URIs to a mirror instead of
download.opensuse.org, was previously considered, and eventually opted
for the current approach.

For one, it seems that not all mirrors have all packages up-to-date,
which is apparently one reason why redirection will use multiple
mirrors. Thus, it would seem that constricting a tumbleweed dist-upgrade
to just one mirror might risk a less than current system state, or
perhaps even an inconsistent system?

While not necessarily drawing this conclusion, this snippet from the old
thread at https://progress.opensuse.org/issues/115142 has andriinikitin
at 2022-08-15 08:40 eluding to the possible cause...
"...a scenario where slightly outdated closest mirror may be used to
download 90% of packages, and some remote up-to-date mirrors will be
used to download the rest."

On 5/6/2024 8:05 PM, crameleon wrote:

[openSUSE Tracker]
Issue #159195 has been updated by crameleon.

Status changed from New to Closed

Zypper will only connect to the configured download servers, and in case of redirection servers (for example download.opensuse.org), also the redirect targets.
You can use zypper lr -d to inspect which servers you have configured and curl -I <URL to .rpm file on configured mirror> to check whether a redirection to a different host is taking place.
Generally third party mirrors do not implement any redirection like download.opensuse.org does, hence my recommendation to use a consistent local mirror.


tickets #159195: Repo server IP address not in DNS
https://progress.opensuse.org/issues/159195#change-794979

* Start date: 2024-04-17

Hi,

Have a system that firewall restricts all outbound traffic except a few
certain allowances, notably opensuse tumbleweed repos/mirrors.
Accordingly, have resolved all the repo related domains to their
respective ip addresses, but lately system updates fail to retrieve some
packages from cdn.opensuse.org, seemingly due to failure to reach it.
A dig on cdn.opensuse.org consistently resolves to these four, which
have been allowed already by the firewall...
151.101.129.91
151.101.1.91
151.101.65.91
151.101.193.91

However, these failed retrievals are apparently because cannot reach
146.75.9.91, and when manually adding it for firewall allowance, the
problem is resolved.
Although, this ip address is never found by a dig on the domain, and a
hard-coded / manual solution will not suffice here.
Is it some mistake that this ip is not in cdn.opensuse.org dns records,
and will perhaps be added? What do you advise on this?

This pertains to the the ticket...
https://progress.opensuse.org/issues/115142
...although, assume there's no need to actually look at said ticket to
understand what's above.

Thank you

Actions #5

Updated by crameleon 14 days ago

  • Status changed from New to Closed

Mirrors which are shown as up to date on the website I linked are are up to date and no discrepancies with the available packages or package versions are to be expected.
If you do not want to enter a potential risk, then you have to use download.opensuse.org, which, as explained, does not offer predictable IP addresses.
You have to decide what you value more or run your own mirror.

Actions #6

Updated by dnl028@gmail.com 12 days ago

  • Status changed from Closed to New

If would choose a mirror from https://mirrors.opensuse.org...
but the mirror would perhaps later not be fully current, would then a
tumbleweed dist-upgrade behave atomically regarding the non-current
packages and thus leave alot of the system non-current, or is it
non-atomic and only a small part would be left non-current?

From where are mirrors propagated? Is there a master repo server from
which all the mirrors are propagated? if so, can it function as a repo
to be used directly, in place of download.opensuse.org?

If would setup and deploy a mirror, can it be available only for private
use, or must it be available to the general public? Wouldn't mean to be
stingy, but just couldn't afford the bandwidth.

Thanks for all your help

On 5/7/2024 5:08 PM, crameleon wrote:

[openSUSE Tracker]
Issue #159195 has been updated by crameleon.

Status changed from New to Closed

Mirrors which are shown as up to date on the website I linked are are up to date and no discrepancies with the available packages or package versions are to be expected.
If you do not want to enter a potential risk, then you have to use download.opensuse.org, which, as explained, does not offer predictable IP addresses.
You have to decide what you value more or run your own mirror.


tickets #159195: Repo server IP address not in DNS
https://progress.opensuse.org/issues/159195#change-795372

* Start date: 2024-04-17

Hi,

Have a system that firewall restricts all outbound traffic except a few
certain allowances, notably opensuse tumbleweed repos/mirrors.
Accordingly, have resolved all the repo related domains to their
respective ip addresses, but lately system updates fail to retrieve some
packages from cdn.opensuse.org, seemingly due to failure to reach it.
A dig on cdn.opensuse.org consistently resolves to these four, which
have been allowed already by the firewall...
151.101.129.91
151.101.1.91
151.101.65.91
151.101.193.91

However, these failed retrievals are apparently because cannot reach
146.75.9.91, and when manually adding it for firewall allowance, the
problem is resolved.
Although, this ip address is never found by a dig on the domain, and a
hard-coded / manual solution will not suffice here.
Is it some mistake that this ip is not in cdn.opensuse.org dns records,
and will perhaps be added? What do you advise on this?

This pertains to the the ticket...
https://progress.opensuse.org/issues/115142
...although, assume there's no need to actually look at said ticket to
understand what's above.

Thank you

Actions #7

Updated by crameleon 3 days ago

would then a tumbleweed dist-upgrade behave atomically regarding the non-current packages and thus leave alot of the system non-current, or is it non-atomic and only a small part would be left non-current?

It's not fully atomic as far as I know, but there is some mechanism to have mirrors sync in time. I also do not see why you would have this issue when no one else does. Many users use mirrors directly.

Is there a master repo server from which all the mirrors are propagated? if so, can it function as a repo to be used directly, in place of download.opensuse.org?

There is, and it does offer a HTTP server at https://downloadcontent.opensuse.org/ which can be used directly, but it's not generally supported. Registered mirrors preferably sync from the restricted rsync://stage.opensuse.org.

If would setup and deploy a mirror, can it be available only for private use, or must it be available to the general public?

Of course it can be private.
https://en.opensuse.org/openSUSE:Mirror_infrastructure#Rsync_servers

Actions

Also available in: Atom PDF