Project

General

Profile

Actions

tickets #104544

closed

New mirror mirror.aardsoft.fi

Added by bwachter-mirror@aardsoft.fi about 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Mirrors
Target version:
-
Start date:
2021-12-31
Due date:
2022-02-01
% Done:

100%

Estimated time:

Description

Admin Name: Bernd Wachter
Admin Email: bwachter-mirror@aardsoft.fi
Subscribed to mirror@lists.opensuse.org: YES
Sponsor Name: Aardsoft Oy
Sponsor URL: https://aardsoft.fi
RSYNC URL: mirror.aardsoft.fi::opensuse
RSYNC allowed only for scans: YES
HTTP URL: http://mirror.aardsoft.fi
FTP URL: -
Limitations: Finland (or, if possible, Nordics, comments below)
IPs and DNS for the whitelist on stage.opensuse.org: mirror.aardsoft.fi

I'm currently syncing opensuse-full-with-factory, and will add
repositories (and request pushing) once that's done. I assume switching
to the staging server might speed up the initial fill as well.

I want to watch traffic patterns for a while (especially to see if
people start using this mirror directly) before deciding on easing
restrictions, so initially a limit to the Nordics or - if that's not
possible - just to Finland would be good.

Initially rsync will be not public, I might revisit this depending on
traffic usage.

Actions #1

Updated by pjessen about 2 years ago

  • Category set to Mirrors
  • Assignee set to pjessen
  • Private changed from Yes to No

Hello Bernd
many thanks for volunteering your time and resources to help support our mirroring infrastructure!
We can easily do a restriction on Finland, Sweden, Norway and Denmark. How about the Baltic states?
I'll get back to this tomorrow, so until then Happy New Year!

Actions #2

Updated by bwachter-mirror@aardsoft.fi about 2 years ago

Hi,

Baltics is fine as well.

Bernd

Actions #3

Updated by pjessen about 2 years ago

  • Status changed from New to In Progress

Okay, I have created the database entry, and it is being scanned right now.
Some internal notes - despite this mirror being located at Hetzner (I presume in Finland), the initial geolocation said 'us'. I corrected that manually to 'fi', and added dk,se,no,lt,lv,ee as 'other countries'.
The IPv4 address is also reverse mapped, good stuff! I have added 'mirror.aardsoft.fi' to the ACL for stage.o.o so you can sync directly.

Actions #4

Updated by bwachter-mirror@aardsoft.fi about 2 years ago

Hi,

doesn't seem to be active yet:

@ERROR: access denied to opensuse-full-with-factory from mirror.aardsoft.fi (65.21.143.134)
rsync error: error starting client-server protocol (code 5) at main.c(1672) [Receiver=3.1.3]

Also, any chance you could also allow mirror-src.aardsoft.fi? We'll
probably experiment with splitting mirroring and mirror access to allow
for better/easier traffic control in late Q1, that'd allow us doing so
without having to contact you again. Currently it's just pointing to the
same IP address - when we start with that it'd change and also gain a
reverse mapping.

Bernd

Actions #5

Updated by pjessen about 2 years ago

Mirror is now active:

pontifex2 (download.o.o):~ # mb scan -e mirror.aardsoft.fi
Sun Jan  2 10:01:33 2022 mirror.aardsoft.fi: starting
Sun Jan  2 10:05:23 2022 mirror.aardsoft.fi: total files before scan: 0
Sun Jan  2 10:58:10 2022 mirror.aardsoft.fi: scanned 1975969 files (623/s) in 3167s
Sun Jan  2 10:58:11 2022 mirror.aardsoft.fi: files to be purged: 0
Sun Jan  2 11:02:39 2022 mirror.aardsoft.fi: total files after scan: 1975965 (delta: 1975965)
Sun Jan  2 11:02:39 2022 mirror.aardsoft.fi: purged old files in 269s.
Sun Jan  2 11:02:39 2022 mirror.aardsoft.fi: now enabled.
Sun Jan  2 11:02:39 2022 mirror.aardsoft.fi: done.
Sun Jan  2 11:02:39 2022 mirror.aardsoft.fi: testing status of base URL...
Sun Jan  2 11:02:39 2022 mirror.aardsoft.fi: OK. Mirror is online now.
Completed in 1.0 hours

First new mirror of 2022 and only the 2nd mirror in Finland!

bwachter-mirror@aardsoft.fi wrote:

doesn't seem to be active yet:

@ERROR: access denied to opensuse-full-with-factory from mirror.aardsoft.fi (65.21.143.134)
rsync error: error starting client-server protocol (code 5) at main.c(1672) [Receiver=3.1.3]

Yeah, I was a little too quick and I expected to have it added by the time you read this :-)

Also, any chance you could also allow mirror-src.aardsoft.fi? We'll
probably experiment with splitting mirroring and mirror access to allow
for better/easier traffic control in late Q1, that'd allow us doing so
without having to contact you again. Currently it's just pointing to the
same IP address - when we start with that it'd change and also gain a
reverse mapping.

I have added both mirror and mirror-src.

Actions #6

Updated by pjessen about 2 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

https://mirrors.opensuse.org was also updated and now shows mirror.aardsoft.fi.

Actions #7

Updated by pjessen about 2 years ago

bwachter-mirror@aardsoft.fi wrote:

I'm currently syncing opensuse-full-with-factory, and will add
repositories (and request pushing) once that's done.

When you're ready, just reopen this ticket and we'll get you set up for a repo push.
Be aware, you will need quite a lot of space.

Actions #8

Updated by bwachter-mirror@aardsoft.fi about 2 years ago

  • Status changed from Resolved to New

redmine@opensuse.org writes:

[openSUSE Tracker]
Issue #104544 has been updated by pjessen.

bwachter-mirror@aardsoft.fi wrote:

I'm currently syncing opensuse-full-with-factory, and will add
repositories (and request pushing) once that's done.

When you're ready, just reopen this ticket and we'll get you set up for a repo push.
Be aware, you will need quite a lot of space.

I've just verified everything works and seen a bunch of requests come
in without issues, so I was able to disable general access logging now.

I'm currently re-syncing opensuse-full-with-factory, and after that will
start pulling the repos. Is the size estimate ~16T in the documentation
still accurate? I have a bit over 20T currently available, and the
mirror should move to larger hardare somewhen in Q2 when the rest of our
infrastructure is up.

Bernd

Actions #9

Updated by pjessen about 2 years ago

  • Status changed from New to In Progress
  • % Done changed from 100 to 0

bwachter-mirror@aardsoft.fi wrote:

I'm currently re-syncing opensuse-full-with-factory, and after that will
start pulling the repos. Is the size estimate ~16T in the documentation
still accurate? I have a bit over 20T currently available, and the
mirror should move to larger hardare somewhen in Q2 when the rest of our
infrastructure is up.

I think repositories might be closer to 18Tb.

Actions #10

Updated by pjessen about 2 years ago

  • Due date set to 2022-02-01
  • Status changed from In Progress to Feedback
Actions #11

Updated by bwachter-mirror@aardsoft.fi about 2 years ago

redmine@opensuse.org writes:

[openSUSE Tracker]
Issue #104544 has been updated by pjessen.

Status changed from New to In Progress
% Done changed from 100 to 0

bwachter-mirror@aardsoft.fi wrote:

I'm currently re-syncing opensuse-full-with-factory, and after that will
start pulling the repos. Is the size estimate ~16T in the documentation
still accurate? I have a bit over 20T currently available, and the
mirror should move to larger hardare somewhen in Q2 when the rest of our
infrastructure is up.

I think repositories might be closer to 18Tb.

Syncing stage.opensuse.org::buildservice-repos with only ".~tmp~" quite
consitently now keeps the repositories directory at 7.2-7.4TB. Resync is
~10 minutes without errors logged - so I guess it'd make sense to switch
to pushing now. In case the missing 10TB come in that way - the volume
currently has 19TB available.

mirror.aardsoft.fi::repositories-push has write access to the
repositories directory for stage.opensuse.org.

Bernd

Actions #12

Updated by pjessen about 2 years ago

bwachter-mirror@aardsoft.fi wrote:

mirror.aardsoft.fi::repositories-push has write access to the
repositories directory for stage.opensuse.org.

Sounds good, I have just now created the push config for mirror.aardsoft.fi, it takes immediate effect.

Thanks again, more mirrors for repositories/ is always good!!

Actions #13

Updated by pjessen about 2 years ago

  • Status changed from Feedback to In Progress
  • % Done changed from 0 to 80

Hi Bernd
on my side it is all looking good, except your mirror is not yet marked as having 'repositories'. I think that is just our scanner running a little late, I'll have to check on that.

Actions #14

Updated by bwachter-mirror@aardsoft.fi about 2 years ago

redmine@opensuse.org writes:

[openSUSE Tracker]
Issue #104544 has been updated by pjessen.

Status changed from Feedback to In Progress
% Done changed from 0 to 80

Hi Bernd
on my side it is all looking good, except your mirror is not yet
marked as having 'repositories'. I think that is just our scanner
running a little late, I'll have to check on that.

also looks fine from my side - it also is now slowly filling up to the
expected levels, currently at 11TB.

I've also now noticed that ports is a separate module, so will also
add that in a bit.

Bernd

Actions #15

Updated by pjessen about 2 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 80 to 100

Closing as resolved, I think we're done here.
Thanks again for volunteering your time and resources!

Actions

Also available in: Atom PDF