Project

General

Profile

Actions

tickets #119293

closed

Request for https://manpages.opensuse.org machine

Added by kukuk over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Servers hosted in NBG
Target version:
-
Start date:
2022-10-24
Due date:
% Done:

100%

Estimated time:
Tags:

Description

Hi,

openSUSE MicroOS does not install documentation by default, so I was
looking for a way to provide the manual pages to end users and rememberd
the good old "docs.sun.com" idea of having a central documentation
server providing this online.

There are already similar services by other distributions:
https://manpages.debian.org/
https://manpages.ubuntu.com/
https://man.archlinux.org/
https://www.freebsd.org/cgi/man.cgi
https://man.openbsd.org/

and many more.
Since we wanted this for SLE Micro, too, I developed a tool to extract
and convert the manual pages from RPMs and build a more or less static
webpage from it.
If you have access to the SUSE internal developer network, you can look
at the prototype: http://dhcp216.future.suse.de/

If not, you can use the container for openSUSE MicroOS:
podman run -it --rm --name docserv -p 80:80 -p 443:443 opensuse/microos-docserv

since many people think this would make sense for openSUSE in general,
we need a machine to run nginx and the docserv daemon to serve this.
Idea would be https://manpages.opensuse.org

The data beside the installation are currently 2GB, but if we want to
add Leap and others later, we need more disk space for this.
And we need a certificate for the nginx webserver.

How can I get this?

Thanks,
Thorsten

--
Thorsten Kukuk, Distinguished Engineer, Senior Architect
SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany
Managing Director: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)

Actions #1

Updated by crameleon over 1 year ago

  • Tags set to vm
  • Category set to Servers hosted in NBG
  • Private changed from Yes to No

Very nice job, I always wanted a service like this for openSUSE. :-)

I suppose the service would be fine to run behind our HAProxy?
What about CPU cores, memory and openSUSE flavor for your backend VM?
Do you have access to the Heroes VPN already?

Actions #2

Updated by kukuk over 1 year ago

On Mon, Oct 24, redmine@opensuse.org wrote:

I suppose the service would be fine to run behind our HAProxy?

Since it is plain https, should be Ok. I don't know the openSUSE network
setup.

What about CPU cores, memory and openSUSE flavor for your backend VM?

My test machine runs with 2 CPUs and 1GB RAM, and only 350MB RAM are
used...
No idea what nginx needs in production, I have no experience here.
And how many really use that website later.
Tumbleweed or MicroOS are both fine for me. While I normally use
everywhere MicroOS on my private servers, not sure if Tumbleweed
wouldn't be better uptime wise.
An absolut minimal installation to just run nginx is enough, no
other fancy stuff except docserv-auxserver and docserv-config-nginx,
both are in Factory.

The data has to be generated somewhere else were we have access to the
Tumbleweed full tree and then pushed or pulled to this machine.
openSUSE release managers are looking into this.

Do you have access to the Heroes VPN already?

No.

--
Thorsten Kukuk, Distinguished Engineer, Senior Architect
SUSE Software Solutions Germany GmbH, Frankenstraße 146, 90461 Nuernberg, Germany
Managing Director: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
(HRB 36809, AG Nürnberg)

Actions #3

Updated by cboltz over 1 year ago

I'm not familiar with docserv, so let me ask a maybe silly question ;-)

You say that the data gets generated "somewhere else", and will then be synced to the webserver. Does that mean you copy over the manpages (in troff format), or does it mean you copy over HTML files that can simply be served by the webserver?

Background of my question: If you copy over the final HTML files (and don't need anything else), you could simply use our existing servers/VMs behind static.o.o (and some other domains) instead of having to maintain your own VM. (The static.o.o servers are quite boring - they run a plain nginx and serve whatever lives in the document root.)

Actions #4

Updated by kukuk over 1 year ago

On Mon, Oct 24, redmine@opensuse.org wrote:

[openSUSE Tracker]
Issue #119293 has been updated by cboltz.

I'm not familiar with docserv, so let me ask a maybe silly question ;-)

You say that the data gets generated "somewhere else", and will then be synced to the webserver. Does that mean you copy over the manpages (in troff format), or does it mean you copy over HTML files that can simply be served by the webserver?

Background of my question: If you copy over the final HTML files (and don't need anything else), you could simply use our existing servers/VMs behind static.o.o (and some other domains) instead of having to maintain your own VM. (The static.o.o servers are quite boring - they run a plain nginx and serve whatever lives in the document root.)

Would be nice if it would work with a plain nginx server, but it
doesn't. Beside the static html data there is a second daemon in the
background (docserv-auxserver), which does the mapping of the search to
the files. And for this, a special nginx configuration is required.

Thorsten

Actions #5

Updated by hellcp over 1 year ago

Would it be viable to do configuration for it in salt? I can probably help out with that since I had some experience with docserv already ;)

Actions #6

Updated by kukuk over 1 year ago

On Tue, Oct 25, redmine@opensuse.org wrote:

Would it be viable to do configuration for it in salt? I can probably help out with that since I had some experience with docserv already ;)

No idea what you want to solve with this. I used salt in the past, my
problem is, you need to have the same salt version on the machine from
where you call it. Another thing is, it's python, and quite some of my
work machines don't have python installed. So do we have a maintained
salt container?

Actions #7

Updated by hellcp over 1 year ago

We have an existing salt setup (https://code.opensuse.org/heroes/salt), and the majority of our infrastructure set up is in it, so we just like keeping it that way. That being said, it's not a requirement. It requires openSUSE infrastructure login, since it's still developed on our internal Gitlab.

Actions #8

Updated by crameleon over 1 year ago

If I prepare a VM, it will be set up with and managed by our Salt anyways, hence it would make sense for your service to be configured using a nicely written Salt profile as well. On your workstation machine you only need a text editor and git, just for development/testing purposes you might want a local VM with a Salt minion ...

Actions #9

Updated by crameleon over 1 year ago

Either way though, if you don't have access to our infrastructure yet, it would be cool if you'd make a second ticket requesting OpenVPN access and a FreeIPA account. Please attach your public GPG key. Also feel free to join #opensuse-admin for easier chatting.

Actions #10

Updated by crameleon over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to crameleon
Actions #11

Updated by crameleon over 1 year ago

  • % Done changed from 0 to 30
Actions #12

Updated by crameleon over 1 year ago

  • % Done changed from 30 to 60

Prepared virtual machine: man.infra.opensuse.org
IP: 192.168.47.29
Salt ID: https://gitlab.infra.opensuse.org/infra/salt/-/blob/production/pillar/id/man_infra_opensuse_org.sls
RackTables (SUSE internal): https://racktables.nue.suse.com/index.php?page=object&object_id=19354

SSH host key fingerprints:
SHA256:iWY3rkEajW89T1irCKrnUaCetA6+DtNzIclLVxfAJ3U (ECDSA)
SHA256:5wfgn0YVDlgOlkIMvXHOO0cN9tbFDJpVu7eGl9Dm7RE (ED25519)
SHA256:AvBkkdZ4qRUAO0j+/L7mrakYwIolkSed4R52NsL5RIk (RSA)

Actions #13

Updated by crameleon over 1 year ago

  • % Done changed from 60 to 80

Upgraded Leap 15.4 to Tumbleweed.

Actions #14

Updated by crameleon over 1 year ago

  • Status changed from In Progress to Blocked
  • % Done changed from 80 to 90

Added HAProxy rules and public DNS records. Took the liberty to add a redirect man.opensuse.org -> manpages.opensuse.org as well.

Waiting for #119485 to arrange access to the machine.

Actions #15

Updated by pjessen over 1 year ago

crameleon wrote:

Upgraded Leap 15.4 to Tumbleweed.

Afaik, we usually don't run TW unless it is absolutely necessary. Something to do with an SLA with SUSE ?

Actions #16

Updated by kukuk over 1 year ago

On Fri, Oct 28, redmine@opensuse.org wrote:

[openSUSE Tracker]
Issue #119293 has been updated by pjessen.

crameleon wrote:

Upgraded Leap 15.4 to Tumbleweed.

Afaik, we usually don't run TW unless it is absolutely necessary. Something to do with an SLA with SUSE ?

But we run MicroOS. So I'm fine with that, too.
Not sure if Leap is current enough, at least it does not contain any of
the required tools.

Actions #17

Updated by crameleon over 1 year ago

Afaik, we usually don't run TW unless it is absolutely necessary. Something to do with an SLA with SUSE ?

I don't like it either, but it's what was requested here and many machines have it for no apparent reason already. And it is the responsibility of the service owner to keep it updated/patched.

Actions #18

Updated by crameleon over 1 year ago

But we run MicroOS. So I'm fine with that, too.

Unfortunately we don't have an image for that, and I don't want to manually configure it.

Actions #19

Updated by crameleon over 1 year ago

  • Status changed from Blocked to In Progress
Actions #20

Updated by crameleon over 1 year ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100
  • Added group manpages-admins with user kukuk in FreeIPA (thank you, @hellcp)
  • Added and assigned manpages role in Salt along with a sudoers entry for manpages-admins and a basic nginx state

With this, the task is done from my end. Feel free to reach out if you have any trouble with accessing or configuring the machine.

Actions

Also available in: Atom PDF