Project

General

Profile

Actions

tickets #99366

closed

Fwd: [sysadmin] problem with bumps to https disabling open-source + mirrors

Added by lkml@tlinx.org over 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Mirrors
Target version:
-
Start date:
2021-09-28
Due date:
% Done:

100%

Estimated time:

Description

FWDing this to opensuse person in charge of mirrors, as richard brown
told me this was the address to forward mirror problems to.

I received an email from someone at kernel.org claiming no prob, but
since I complained they may drop opensuse (!!!), I explained the problem
-- they are violating the Web RFC's and that their violations have nothing
to do with opensuse's mirror use, would apply to any mirror usage
going through kernel.org's mirror system (because they overload 301
to use to permanently upgrade a page, but if part of a mirror chain,
they need to use 302 as a response code).

-------- Original Message --------
Subject: Re: [sysadmin] problem with bumps to https disabling
open-source + mirrors
Date: Mon, 27 Sep 2021 19:27:12 -0700
From: L A Walsh lkml@tlinx.org
To: Konstantin Ryabitsev konstantin@linuxfoundation.org
CC: webmaster@kernel.org, ftpadmin@kernel.org
References:
DM6PR04MB479652B1C6C9CA7C93B7CC0386A39@DM6PR04MB4796.namprd04.prod.outlook.com
614EEF10.9090507@tlinx.org
20210927191507.gwf7djkvs46knq3h@meerkat.local

On 2021/09/27 12:15, Konstantin Ryabitsev wrote:

On Sat, Sep 25, 2021 at 02:42:40AM -0700, L A Walsh wrote:

Please stop the https upgrades, because when they fail, open source becomes closed source.

I appreciate your concerns, but upgrading a connection from http to https does not result in "open source" becoming "closed source".


It does result in terminating a mirror search that was otherwise
started with 'http'. Breaking the mirror system is specifically
the part I am referring to (see below). There are other issues
I've personally had, regarding clients that were too old to work with
upgraded https, but please regard that as a separate topic.
That said, we will probably drop opensuse from our mirrors,
as they have been a continued source of mirroring issues. In a sense, this will fix your problem because we'll return 404 before an
upgrade to https -- if I understood your description of the problem correctly.


First the symptom:

As my (squid) proxy stepped through the mirrors, it received a
Web Response 302 == Found and is temporarily under a
different URI. [RFC 7231 6.2.3].

At, the edge of the kernel-mirror system, (
http://sfo.korg-mirror.kernel.org), the proxy is receiving a
301 response (Moved_Permanently) referring the proxy to
the server @ https://mirrors.edge.kernel.org. The
301 response provides the new permanent address to use
for the URI

[RFC7231:6.4.2 says:]
The server "SHOULD" generate a new location header that contains
the new "preferred URI reference for the permanent URI". "The user
agent MAY use the Location field for automatic redirection".

This means that if/when the new address at
"https//mirrors.edge.kernel.org"
doesn't have content, the "user agent" (proxy in my case) stops
searching for a copy of the content and returns a 404 (Not Found).

sfo.korg-mirror.kernel.org must return a 302 response when it
refers to mirrors.edge.kernel.org. It can still use 'https', but
unless it is known that mirrors.edge.kernel.org has content, it cannot
use 301.

Use of 301 instead of 302 will terminate the searching for content,
disabling search through the mirror system.

I would prefer that the kernel mirrors NOT stop carrying the suse
content for this problem, as it isn't limited to opensuse, but to
any systems using this as a mirror -- it is a fault of the http->https
system using response code 301 when, as a mirror, it can only use
'302'.

Actions #1

Updated by pjessen over 2 years ago

  • Category set to Mirrors
  • Assignee set to pjessen
  • Private changed from Yes to No

FWDing this to opensuse person in charge of mirrors, as richard brown
told me this was the address to forward mirror problems to.

Yes, admin@o.o is the place to report openSUSE infrastructure problems, and I am in charge of our mirror setup.

I don't know what I can add - you have already spoken to Konstantin. If http://sfo.korg-mirror.kernel.org is permanently moved to mirrors.edge.kernel.org, maybe we should just remove the former as a mirror.

Konstantin said:

That said, we will probably drop opensuse from our mirrors, as they have been a continued source of mirroring issues.

I am not aware of any issues. I'll write to Konstantin and ask.

Actions #2

Updated by suse@tlinx.org over 2 years ago

On 2021/09/28 02:07, redmine@opensuse.org wrote:

[openSUSE Tracker]
Issue #99366 has been updated by pjessen.

Category set to Mirrors
Assignee set to pjessen
Private changed from Yes to No

FWDing this to opensuse person in charge of mirrors, as richard brown
told me this was the address to forward mirror problems to.

Yes, admin@o.o is the place to report openSUSE infrastructure problems, and I am in charge of our mirror setup.

I don't know what I can add - you have already spoken to Konstantin. If http://sfo.korg-mirror.kernel.org is permanently moved to mirrors.edge.kernel.org, maybe we should just remove the former as a mirror.

What do you mean by "if server'a' is permanently moved to
server'b'" ? Are you speaking w/regards to the meaning of the '301'
result code?

If this results in, say the server that referred, originally me
to korg.mirror, instead, referring me to edge with a "302", but
using https, I think that should work, but given the chaos, can't
really say unless I see it.

Actions #3

Updated by pjessen over 2 years ago

suse@tlinx.org wrote:

What do you mean by "if server'a' is permanently moved to
server'b'" ? Are you speaking w/regards to the meaning of the '301'
result code?

When I go to http://sfo.korg-mirror.kernel.org/opensuse I am always redirected to https://mirrors.edge.kernel.org/opensuse, which seems to suggest the former has been deprecated in favour of the latter.

Actions #4

Updated by pjessen over 2 years ago

We currently have five kernel.org mirrors defined:

ams.edge.kernel.org
ewr.edge.kernel.org
nrt.edge.kernel.org
sjc.edge.kernel.org
sfo-korg-mirror.kernel.org -> mirror.edge.kernel.org

mirror.edge.kernel.org works with some DNS anycast setup, from my location here in Switzerland, it gives me the Amsterdam mirror.

Actions #5

Updated by Astara over 2 years ago

1) when you hit sfo-korg-mirror.kernel.org, are you using http or https?
2) when it forwards you to the next mirror, does it do so with a 302 response or a
301 response?

It's when korg-mirror forwards you from an http addr, to an https addr that does not have what
you requested.

If the referred to server has the content, then it isn't exercising the same
conditions as in the original bug. If you are forwarded to korg.mirror within https, you
likely won't see what the res code is as you are forwarded to the next member, but if you
could, you wouldn't see 301, as they are only using that to upgrade connectee's from
http->https.

Actions #6

Updated by pjessen over 2 years ago

  • Status changed from New to Feedback

Sorry about taking so long, I dropped the ball.

Astara wrote:

1) when you hit sfo-korg-mirror.kernel.org, are you using http or https?

http://

2) when it forwards you to the next mirror, does it do so with a 302 response or a
301 response?

I get a 301 redirect to http://mirrors.edge.kernel.org/opensuse/

per@toshiba1:~> wget -nd http://sfo-korg-mirror.kernel.org/opensuse/
--2021-12-13 12:48:07--  http://sfo-korg-mirror.kernel.org/opensuse/
Resolving sfo-korg-mirror.kernel.org (sfo-korg-mirror.kernel.org)... 149.20.37.36, 2001:4f8:4:6f:0:1994:3:14
Connecting to sfo-korg-mirror.kernel.org (sfo-korg-mirror.kernel.org)|149.20.37.36|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://mirrors.edge.kernel.org/opensuse/ [following]
--2021-12-13 12:48:08--  http://mirrors.edge.kernel.org/opensuse/
Resolving mirrors.edge.kernel.org (mirrors.edge.kernel.org)... 147.75.101.1, 2604:1380:2001:3900::1
Connecting to mirrors.edge.kernel.org (mirrors.edge.kernel.org)|147.75.101.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Actions #7

Updated by suse@tlinx.org over 2 years ago

I may or may not be able to get back to this...my ball was dropped on me

I had a stroke and came back to a fairly well (seems self) sabotaged
computer...not
sure what my involvement will be or how I will recover.

What a pain.

On 2021/12/13 03:50, redmine@opensuse.org wrote:

[openSUSE Tracker]
Issue #99366 has been updated by pjessen.

Status changed from New to Feedback

Sorry about taking so long, I dropped the ball.

Actions #8

Updated by pjessen almost 2 years ago

  • Status changed from Feedback to Resolved
  • % Done changed from 0 to 100

I'm sorry to hear that, I hope you are doing better.
For now, I'll close this as resolved, there has been a few changes to the kernel.org setup. Definitely feel free to re-open.

Actions

Also available in: Atom PDF