tickets #135437
openUpdate openSUSE mirroring documentation/processes
0%
Description
I want to make sure our documentation on mirroring is up to date -- or at the very least, decent enough -- for both the one setting up a mirror and the one registering/managing it. My motivation comes from seeing many mirror-related tickets around where we basically assign it to Andrii and forget about it. Sometimes I see Alex giving a hand, but still that doesn't scale well long term. Of course I don't expect to see a boom on people wanting to register, or make changes to their mirrors. However, the bus factor is lurking around the corner.
And in order to getting people aboard on helping with such tasks, having decent documentation is the corner stone, in my opinion. I'm basically going to need to borrow "trained eyes" when I update bits and pieces of the documentation, just to make sure I didn't misunderstand something or maybe even update a part of the process that's deprecated nowadays.
This would be a good opportunity to also "retire" all that MirrorBrain stuff and put more emphasis on MirrorCache, I hope.
Checklist
- https://en.opensuse.org/openSUSE:Mirror_infrastructure
- https://en.opensuse.org/openSUSE:Mirror_howto
- https://progress.opensuse.org/projects/opensuse-admin-wiki/wiki#Mirrors
- https://progress.opensuse.org/projects/opensuse-admin-wiki/wiki/MirrorCache
- https://progress.opensuse.org/projects/opensuse-admin-wiki/wiki/Downloadcontent2
Updated by pjessen 8 months ago
Completely agree with your initiative.
The new CDN solution might also have an impact: https://news.opensuse.org/2023/07/31/try-out-cdn-with-opensuse-repos/
Updated by crameleon 8 months ago
I made a small bit of progress with this via https://progress.opensuse.org/issues/101191. Definitely needs more work than this.
Updated by luc14n0 8 months ago
pjessen wrote in #note-2:
Completely agree with your initiative.
The new CDN solution might also have an impact: https://news.opensuse.org/2023/07/31/try-out-cdn-with-opensuse-repos/
Agreed, it might at some point. For now, though, for mirror admins/operators, not so much AFAICT.
Updated by luc14n0 8 months ago
crameleon wrote in #note-3:
I made a small bit of progress with this via https://progress.opensuse.org/issues/101191. Definitely needs more work than this.
Very nice. So we don't have to worry about FTP-related stuff -- or if I/we bump into any, just get rid of it.
Updated by luc14n0 8 months ago
I believe we could merge both openSUSE:Mirror_infrastructure and openSUSE:Mirror_howto Wiki pages. They seem to have similar scope: help those who want to setup an openSUSE mirror. Or at the very least we should define separate scopes for them. Any inputs are welcomed.
And before I go on and start editing those two pages I'm going to create a separate page -- probably a GitHub gist, or something like that -- so it's easier to compare them.
Updated by pjessen 8 months ago
luc14n0 wrote in #note-4:
pjessen wrote in #note-2:
Completely agree with your initiative.
The new CDN solution might also have an impact: https://news.opensuse.org/2023/07/31/try-out-cdn-with-opensuse-repos/Agreed, it might at some point. For now, though, for mirror admins/operators, not so much AFAICT.
Agree, not much direct impact, except it might make them superfluous at some point?
Updated by pjessen 8 months ago
luc14n0 wrote in #note-5:
crameleon wrote in #note-3:
I made a small bit of progress with this via https://progress.opensuse.org/issues/101191. Definitely needs more work than this.
Very nice. So we don't have to worry about FTP-related stuff -- or if I/we bump into any, just get rid of it.
I would wonder about rsync support too - I'm not sure if mirrorcache even supports it?
I don't think it is used for scanning any more either. Looking at my own mirror's rsync logs, the last accesses were in June, from downloadcontent.o.o and from stage.o.o.
Updated by pjessen 8 months ago
luc14n0 wrote in #note-6:
I believe we could merge both openSUSE:Mirror_infrastructure and openSUSE:Mirror_howto Wiki pages. They seem to have similar scope: help those who want to setup an openSUSE mirror. Or at the very least we should define separate scopes for them. Any inputs are welcomed.
https://en.opensuse.org/openSUSE:Mirror_howto - IMHO, this is largely superfluous. Anyone who has bandwidth+storage+server+cooling will likely be a reasonably experienced sysadmin. The page doesn't hurt, but I would not put too much effort into it.
https://en.opensuse.org/openSUSE:Mirror_infrastructure - far more important, but woefully out of date. For instance, those hotstuff modules don't exist any more. The suggestion that 40-60Gb disk space is sufficient - try comparing to the numbers on https://mirrors.opensuse.org/list/rsyncinfo-stage.o.o.txt 😱
I would keep those pages separate - they are really describing two very different topics. One is the local setup, the other is how to fit into our infrastructure.
Updated by avicenzi 3 months ago
I did a minor update once, but there's a lot to clean in the mirror wiki.
If you need help ping me, we can work on the cleanup.
I also host a mirror, but I run everything with Docker (https://github.com/opensuse-brasil/mirror-swarm).
There are a few things that could be reused, or perhaps convert this project into a more official way to run a mirror.