tickets #41933

Mirror redirector is down

Added by hellcp@opensuse.org 6 months ago. Updated 6 months ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:servers hosted in NBG
Target version:-
Duration:

Description

Hi,

Repositories are unreachable atm, seems that mirror redirector went
down.

LCP [Stasiek]
https://lcp.world


Subtasks

History

#1 Updated by cboltz 6 months ago

  • Category set to servers hosted in NBG
  • Assignee set to TBro
  • Priority changed from Normal to Urgent
  • Private changed from Yes to No

Thanks for the notice!

I had a quick look at the server, and dmesg indicates a filesystem issue with /srv :-( I have to hand over to Thorsten and Theo for fixing it. Both are probably sleeping already, so please be patient ;-)

I also updated status.opensuse.org.

#3 Updated by cboltz 6 months ago

I have some good news - I found a workaround to "hide" the downtime. See my mail to heroes@ for the details.

#4 Updated by Knurpht 6 months ago

cboltz wrote:

I have some good news - I found a workaround to "hide" the downtime. See my mail to heroes@ for the details.

Eh, Crhristian, I'm now getting a 500 Internal server error

#5 Updated by jberry 6 months ago

The disk is full. Likely why filesystem is complaining.

#6 Updated by pjessen 6 months ago

Last I saw this, it was fairly quickly fixed. Tidying up and clearing out stuff took much longer. Is anyone working on this problem?

#7 Updated by matwey 6 months ago

provo-mirror.opensuse.org is dead for me now.

1?: [LOCALHOST]                                         pmtu 1500
1:  93.180.26.1                                           0.422ms 
1:  93.180.26.1                                           0.413ms 
2:  93.180.21.2                                           1.139ms 
3:  188.44.33.42                                          0.821ms 
4:  93.180.0.190                                          1.130ms 
5:  194.190.254.117                                       1.207ms 
6:  185.141.124.145                                      10.567ms 
7:  185.141.124.61                                      104.332ms 
8:  10.32.67.209                                        143.176ms 
9:  80.249.208.122                                       60.476ms 
10:  64.125.31.104                                       145.471ms asymm 23 
11:  64.125.29.77                                        144.781ms asymm 22 
12:  64.125.29.126                                       145.955ms asymm 21 
13:  64.125.29.187                                       173.056ms asymm 20 
14:  64.125.29.213                                       154.133ms asymm 19 
15:  64.125.30.245                                       143.694ms 
16:  64.125.31.83                                        143.280ms 
17:  64.125.14.122                                       153.347ms 
18:  209.63.99.246                                       198.617ms asymm 20 
19:  70.103.180.172                                      199.280ms asymm 22 
20:  no reply
21:  no reply
22:  no reply
23:  no reply
24:  no reply
25:  no reply
26:  no reply
27:  no reply
28:  no reply
29:  no reply
30:  no reply
31:  no reply
Too many hops: pmtu 1500
Resume: pmtu 1500

#8 Updated by cboltz 6 months ago

provo-mirror.opensuse.org is dead for me now.

Should work again (I had to adjust a sysctl limit), but poor provo-mirror has a terribly outdated mirrorbrain database and therefore has to serve most files itsself. Please don't be too surprised if it's slow or overloaded ;-)

#9 Updated by juliogonzalezgil 6 months ago

I am able to connect to download.opensuse.org now, but I get SSL errors:

$ curl -v https://download.opensuse.org/repositories/systemsmanagement:/Uyuni:/Master/images-openSUSE_Leap_42.3/
*   Trying 195.135.221.134...
* TCP_NODELAY set
* Connected to download.opensuse.org (195.135.221.134) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* stopped the pause stream!
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Not sure if this is related, but at Feenode, #suse channel, people reported similar problems when using software.opensuse.org.

In this case the website itself works, but as soon as you try to look for a package you get:

SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: unknown protocol

For example: https://software.opensuse.org/search?utf8=%E2%9C%93&q=python

#10 Updated by fkrueger 6 months ago

The repos seem to be back and working, at least for me. Thx!

#11 Updated by matwey 6 months ago

Now I am being redirected to downloadcontent.opensuse.org which is alive but doesn't respond by HTTP

#12 Updated by juliogonzalezgil 6 months ago

Same here. I can browse the repositories, but as soon as I try to download a RPM, I get redirected to downloadcontent.opensuse that does not respond:

$ curl https://download.opensuse.org/repositories/systemsmanagement:/Uyuni:/Master/images-openSUSE_Leap_42.3/repo/Uyuni-Server-4.0-POOL-x86_64-Media1/x86_64/patterns-
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://downloadcontent.opensuse.org/repositories/systemsmanagement:/Uyuni:/Master/images-openSUSE_Leap_42.3/repo/Uyuni-Server-4.0-POOL-x86_64-Build0124-Media1/x86_64/patterns-uyuni_server-4.0-4.1.x86_64.rpm">here</a>.</p>
<hr>
<address>Apache/2.4.23 (Linux/SUSE) Server at download.opensuse.org Port 443</address>
</body></html>

$ curl -v http://downloadcontent.opensuse.org/repositories/systemsmanagement:/Uyuni:/Master/images-openSUSE_Leap_42.3/repo/Uyuni-Server-4.0-POOL-x86_64-Build0124-Media1/x86_64/patterns-uyuni_server-4.0-4.1.x86_64.rpm
*   Trying 195.135.221.157...
* TCP_NODELAY set
* connect to 195.135.221.157 port 80 failed: Connection refused
*   Trying 2620:113:80c0:8::27...
* TCP_NODELAY set
* Immediate connect fail for 2620:113:80c0:8::27: Network is unreachable
*   Trying 2620:113:80c0:8::27...
* TCP_NODELAY set
* Immediate connect fail for 2620:113:80c0:8::27: Network is unreachable
* Failed to connect to downloadcontent.opensuse.org port 80: Connection refused
* Closing connection 0
curl: (7) Failed to connect to downloadcontent.opensuse.org port 80: Connection refused

(I am totally ignorant about this, but downloadcontent.opensuse.org doesn't seem to be the Provo mirror, or at least it doesn't share the same public IP)

#13 Updated by tampakrap 6 months ago

  • Status changed from New to Closed
  • Assignee deleted (TBro)

fixed. I gave 1TB extra to /srv, ran xfs_repair and all is green again. Special thanks to darix for giving me precious pointers, and to mcaj for increasing the disk on the storage and helping me with the overall procedure

Also available in: Atom PDF