openSUSE admin

2018-08-07 heroes meeting

[20:01:39] <cboltz> is everybody ready for the meeting?

[20:01:49] <bmwiedemann> totally

[20:01:51] <tampakrap> yes sir

[20:02:03] <cboltz> ok, then let's start the meeting ;-)

[20:02:18] <cboltz> the topics are on https://progress.opensuse.org/issues/38162

[20:02:40] <cboltz> let's start with Q&A - does someone from the community have a question?

[20:03:20] <lcp> not yet ;)

[20:04:24] <cboltz> ok, then let's continue with status reports

[20:04:47] <pjessen> status is hot and dry

[20:05:15] <cboltz> yeah, but I meant stuff running on our servers ;-)

[20:05:23] <tampakrap> status report from me: https://progress.opensuse.org/news/68 written here

[20:05:36] <tampakrap> there are 5 static websites that are running in kubernetes/cloudfoundry

[20:06:00] <tampakrap> also, yey!!

[20:06:41] <bmwiedemann> nice

[20:07:03] * cboltz still has to learn what all this cloudy stuff does

[20:07:24] <tampakrap> second: https://gitlab.infra.opensuse.org/infra/salt/merge_requests/211 this MR improved the testsuite a lot

[20:07:33] <tampakrap> from both quality and speed

[20:07:50] <tampakrap> so now if we give more build power to the test workers, we will also get faster results

[20:08:07] <tampakrap> and third: I worked with darix on the database full disk issue

[20:08:19] <bmwiedemann> what 'build workers' are these?

[20:08:20] <tampakrap> first step we gave extra 50GB to each node

[20:08:36] <tampakrap> second step we compressed the postgresql logs

[20:08:59] <tampakrap> third we removed a lot of expired session entries on the weblate database, which also fixed the pg_dump issue

[20:09:29] <cboltz> "third" sounds like a possible cleanup cronjob?

[20:09:33] <tampakrap> so things to be done: 1) add a cronjob to compress the logs 2) tell the weblate admin to put a cronjob to remove the sessions, or do it in any other way with django

[20:09:49] <bmwiedemann> tampakrap: 1) use logrotate

[20:09:51] <tampakrap> bmwiedemann: the gitlab-ci runners

[20:10:04] <tampakrap> we can't, postgresql is rotating the logs every day

[20:10:09] <tampakrap> we just need to compress them

[20:10:54] <tampakrap> that's it from me

[20:10:57] <bmwiedemann> OK. strange. never had that problem

[20:12:19] <cboltz> tampakrap: please either do the things to be done *now*, or at least open a tickets for them ;-)

[20:12:51] <tampakrap> okay

[20:13:13] <cboltz> I have a report from the "I'm only the messenger" category

[20:13:27] <cboltz> you might have seen the "Disallowed Key Characters." error on paste.opensuse.org

[20:13:49] <cboltz> I found out that this is caused by one of the cookies from news.o.o and lizards.o.o

[20:14:19] <cboltz> (no idea why paste.o.o looks at cookies that are not relevant for it, but obviously it does)

[20:14:29] * Son_Goku waves

[20:14:55] <cboltz> I mailed Jared and asked to limit the news.o.o and lizards.o.o cookies to the respective domain instead of *.o.o

[20:15:12] <cboltz> (no response yet)

[20:15:39] <cboltz> any other status report?

[20:16:14] <pjessen> been away on holiday. hot and dry :-)

[20:16:41] <cboltz> you don't need to be away for hot and dry, I have that here ;-)

[20:17:19] <pjessen> whats the story with mirrordb3 ? disk space okay now?

[20:17:37] <tampakrap> yes, read above

[20:17:53] <pjessen> ah got it

[20:18:41] <cboltz> let's continue with the next topic:

[20:18:49] <cboltz> FreeIPA or Æ-DIR

[20:19:07] <Son_Goku> :P

[20:19:19] <cboltz> Son_Goku and mstroeder - do you want to say a few words? ;-)

[20:19:23] <bmwiedemann> is any of them using kerberos? Then I'd vote for the other

[20:19:31] <mstroeder> Yeah, those strange Unicode stuff...

[20:19:49] * plinnell waves too

[20:19:50] <Son_Goku> FreeIPA uses 389ds, MIT Kerberos, and Dogtag

[20:21:14] <mstroeder> Well, my proposal was triggered by FreeIPA still running on Fedora. Æ-DIR is quite different compared to FreeIPA. Please look at the front page https://ae-dir.com

[20:21:23] <bmwiedemann> (because the kerberos design is from a time before public-key crypto was available and there are way too many CVEs for it)

[20:23:28] <mstroeder> Æ-DIR deliberately does not support Kerberos. For SSH logins I prefer keys, in a recent setup in the form of temp. SSH certs.

[20:24:04] <Son_Goku> you don't really have to use the kerberos stuff much if you don't want to, but I find it valuable for integration with SSO systems

[20:24:20] <mstroeder> tampakrap expressed interest to see an installation. If a couple of VMs are ready I can install it quickly for you to test it.

[20:25:09] <bmwiedemann> so if kerberos is optional, either should be fine

[20:25:35] <mstroeder> WebSSO systems are something different anyway and for SSH I prefer OpenSSH certs (after doing MFA). Yes, I know SPNEGO. BTDT but AFAICS it's of no interest for o.o

[20:26:13] <Son_Goku> anyway, my offer is to help make FreeIPA work natively on openSUSE

[20:26:36] <Son_Goku> I've been doing Fedora<->openSUSE packaging for a couple of years now

[20:27:10] <Son_Goku> and if there's interest in continuing to use FreeIPA, I can start looking into building functional openSUSE packages for FreeIPA based on the Fedora ones

[20:27:49] <Son_Goku> the other advantage of FreeIPA (if this is something that you care about) is that since it uses 389ds, it'll work on SLE 15

[20:28:17] <mstroeder> BTW: I'm also maintaining the OpenLDAP packages.

[20:28:34] <tampakrap> Son_Goku: you can go on with packaging freeipa for opensuse, checking an alternative shouldn't stop you

[20:28:58] <tampakrap> I won't guarantee that we're going to switch to any other solution, because the decision is not only mine to make

[20:29:31] <Son_Goku> tampakrap, well, if you guys aren't going to continue using FreeIPA, there wouldn't be much value in it

[20:30:00] <Son_Goku> I had heard from sysrich before that you had been using FreeIPA on Fedora, and I figured you guys usually like to have your infra on SUSE distributions

[20:30:18] <Son_Goku> (while I do love Fedora, I definitely understand the concept of self-hosting infra :) )

[20:30:37] <Son_Goku> it's the same reason I'm working (slowly) to port OBS to be packaged and run properly on Fedora

[20:30:39] <tampakrap> Son_Goku: we are going to continue to use opensuse until we have a better replacement, which we don't have yet, we have to evaluate the replacement first

[20:30:43] <tampakrap> which didn't happen yet

[20:30:44] <mstroeder> As said: I know that this switch is not easy for you.

[20:31:12] <cboltz> mstroeder: do you have an idea how difficult the migration (user accounts, DNS entries etc.) from freeipa to Æ-DIR would be?

[20:31:29] <cboltz> ("simple export/import" or "will be interesting[tm]"?

[20:32:01] <mstroeder> I'd like to have a look at your user data first. I'm pretty optimitic that there is a migration path. I would help with that.

[20:32:30] <mstroeder> I have some experience writing LDIF filter programs in Python.

[20:32:53] <bmwiedemann> Son_Goku: I think there was also interest in freeipa for internal SUSE IT

[20:33:52] <mstroeder> How are you maintaining the e-mail accounts @opensuse.org? Also in FreeIPA? AE-DIR has support for mail accounts.

[20:34:13] <tampakrap> we don't have mail accounts, they are aliases

[20:34:42] <cboltz> at the moment they are maintained in connect.opensuse.org, but we are looking for a replacement (not only for the mail aliases)

[20:34:57] <mstroeder> Also I proposed to switch to PowerDNS with LDAP backend and native LDAP replication. I'd also help with that. It's fairly easy.

[20:35:04] <cboltz> however, note that freeipa only has _admin_ accounts

[20:35:13] <mstroeder> There is also support for simple mail groups in AE-DIR.

[20:35:36] <mstroeder> Example configs for postfix and dovecot are in the git repo.

[20:35:38] <cboltz> so managing the @opensuse.org aliases would mean to add a new "category" of data

[20:36:03] <cboltz> (and probably also to add accounts for all openSUSE Members)

[20:38:09] <mstroeder> In AE-DIR the authorative mail accounts are aeUser entries augmented with a mail account object class. (I'm bad at typing fast so I'd suggest to collect question I'll answer in detail on the mailing list.)

[20:38:31] <Son_Goku> bmwiedemann, well, if for nothing else, I'll take a look for that ;)

[20:38:36] <bmwiedemann> mstroeder: DNS-> LDAP seems a bit overkill, or do we need dyndns?

[20:39:05] <mstroeder> I still believe in the "L" in LDAP. ;-)

[20:39:27] <mstroeder> My own VMs running PowerDNS and OpenLDAP are very small.

[20:40:34] <mstroeder> IMHO the FreeIPA / DNS / OpenDNSSEC integration is not light-weight too.

[20:40:51] * cboltz has to leave for a few minutes

[20:42:15] <mstroeder> Any more questions about AE-DIR now?

[20:43:40] <mstroeder> Son_Goku: You proposed to use Ipsilon for WebSSO. This sounds interesting though I'm not sure about its current project activity. From my understanding RedHat is endorsing KeyCloak now.

[20:44:04] * cboltz is back

[20:44:13] <Son_Goku> yeah, Keycloak is the project for Red Hat SSO

[20:44:17] <Son_Goku> but Fedora uses Ipsilon

[20:44:26] <Son_Goku> and it's actively developed and maintained

[20:44:38] <Son_Goku> and it's (IMO) easier to hack on because it's Python rather than Java

[20:45:00] <mstroeder> Full ack for the Python vs. Java statement! ;-)

[20:45:11] * cboltz also prefers python over java

[20:46:26] <mstroeder> I've looked at it but it uses lots of C wrapper modules for XML-DSIG etc. Unfortunately there also has not been a upstream release since quite a while.

[20:47:00] * lcp is interested in SSO talk

[20:48:01] <mstroeder> My own plans are outlined here: https://www.ae-dir.com/todo.html#sso

[20:48:22] <mstroeder> Of course I'm very much interested in ready-to-use solutions without doing all this work.

[20:48:44] <Son_Goku> ipsilon is also very themeable, so someone can make it look very Geeko ;)

[20:50:05] <mstroeder> In any case it would be good to replace the currently used SSO solution.

[20:50:52] <mstroeder> How many openSUSE users are registered? And how many services and systems are currently in use? Rough numbers are sufficient.

[20:51:25] <lcp> Son_Goku: looking around, no idea who that would be

[20:52:15] <plinnell> mstroeder: im guessing 500-700 offical members

[20:52:24] <plinnell> connect.o.o should know

[20:52:52] <plinnell> as for non @opensuse.org people in SSO via MF, could be several thousand over time

[20:53:11] <plinnell> none of them get flushed that i know of

[20:53:56] <cboltz> we currently have 42 *.infra.opensuse.org systems in salt - but that's only what the heroes manage

[20:54:45] <lcp> 406 members https://connect.opensuse.org/pg/groups/111/opensuse-members/

[20:54:49] <mstroeder> What is the schedule regarding transition of MF SSO to whatever?

[20:55:32] <lcp> 18692 users? https://connect.opensuse.org//pg/members/all/

[20:55:43] <tampakrap> there's no such plan, I wrote it also on the mailing list

[20:56:30] <cboltz> lcp: that number means 18692 users opened connect.o.o while being logged in, so the real number is higher

[20:56:45] <lcp> yeah, I know

[20:57:01] <lcp> it's the best estimate I have >:D

[20:57:42] <mstroeder> Rough numbers are sufficient.

[20:57:58] <plinnell> tampakrap: which list ?

[20:58:11] <mstroeder> What is the user backend of MF SSO users? eDirectory?

[20:58:44] <plinnell> yes and probably some tie in to AD on the legacy MF side

[20:59:28] <plinnell> AFAIk, some of the o.o sites are tied to it, I know OBS does, I have seen the code

[20:59:40] <mstroeder> So MF employees are in their AD and those are synced to eDirectory?

[20:59:41] <tampakrap> plinnell: https://lists.opensuse.org/heroes/2018-08/msg00009.html

[20:59:58] <plinnell> thanks

[21:00:36] <plinnell> oic

[21:02:31] <mstroeder> oic == OpenID Connect ? ;-)

[21:03:03] <plinnell> oh I see

[21:05:24] <cboltz> speaking about openID - www.opensuse.org/openid/ is sort of a blocker to move www.o.o to a VM in Nuremberg

[21:05:37] <cboltz> so if someone is familiar with openID and wants to help, that's more than welcome ;-)

[21:05:46] <tampakrap> s/to a VM/to cloudfoundry/

[21:06:26] <cboltz> that's a technical detail ;-)

[21:09:22] <mstroeder> It says it's only a test consumer. Still needed for something? BTW: "OpenID Connect" is not "OpenID".

[21:10:10] <cboltz> my guess that nobody removed the word "Test" ;-)

[21:10:15] <mstroeder> https://openid.net/connect/

[21:10:55] <lcp> >old SUSE logo

[21:11:08] <cboltz> and I'm quite sure that it gets used by some people (for example, openSUSE Asia used it for logging in to their logo vote, and in theory you can use it to login *anywhere*)

[21:11:09] <mstroeder> OpenID is nowadays considered rather obsolete and should be replaced by OpenID Connect.

[21:13:13] <tampakrap> so can we move? there is nothing to discuss if we don't have the freeipa packages or a test instance of ae-dir to compare i'd say

[21:14:13] <lcp> do you know traffic there, it might be easier to let it go if we know that nobody uses it

[21:15:50] <mstroeder> Yeah, I need root access to a couple of VMs and then I'll start with a PoC installation. We have to consider whether and how to migrate data. Would be nice to have complete read access to the 389-DS minus userPassword attribute of other users. Then you all play with it and decide.

[21:16:20] <tampakrap> cool

[21:16:52] <cboltz> lcp: www.o.o still runs in Provo, which means it can be a bit hard (and slow) to get logs

[21:16:58] <Son_Goku> tampakrap, I can try to have FreeIPA packaged for openSUSE in a few weeks or so

[21:17:12] <tampakrap> that would be highly appreciated

[21:17:13] <lcp> ah, yeaaah

[21:20:08] <lcp> but would be cool to replce oi with oic *because hopefully that would mean Novelless frontend*

[21:20:56] <mstroeder> It would be nice if somebody (3rd-party) would prepare a check list for comparing AE-DIR and FreeIPA. I could do it myself, but aiming for world-domination I'm biased of course.

[21:21:40] <mstroeder> forgot ;-)

[21:21:54] <cboltz> lcp: whoever is familiar with openID is more than welcome to work on it ;-)

[21:21:54] <lcp> mstroeder: everybody's a shill

[21:22:19] <lcp> I will look into it, not promising anything

[21:22:47] <cboltz> :-)

[21:23:41] <cboltz> if possible (without breaking backwards compability) it would be nice to have the openID stuff on a separate VM (or container) which then gets served unter openid.opensuse.org

[21:23:56] <cboltz> this would make www.o.o a completely static page

[21:26:43] <tampakrap> cboltz: anything else or can we close the meeting?

[21:27:02] <cboltz> well, there's the usual "review old tickets" topic ;-)

[21:27:06] <pjessen> nothing from me

[21:27:38] <plinnell> cboltz: i came here to discuss IRC

[21:27:41] <Son_Goku> after FreeIPA, I'll take a look at bringing Ipsilon

[21:28:06] <cboltz> plinnell: what exactly do you want to discuss?

[21:28:25] <plinnell> freenode has been getting hammered with spammers

[21:28:51] * lcp hopes to hear Matrix

[21:28:54] <plinnell> and we have quite a few channels where the owners or ops are never online

[21:30:57] <plinnell> so what we need IMO 1: some new folks elevated to op some of the channels

[21:31:18] <plinnell> 2. we need active folks elevated to owner level to add other ops

[21:31:39] <plinnell> mostly, so we can protect the channels better

[21:32:24] <plinnell> lcp: I seriously doubt we would drop freenode IRC

[21:32:46] <lcp> Well, we could just setup bridges between both

[21:32:50] <lcp> for now

[21:32:52] <plinnell> a gateway to Matrix and/or Discord would be a nice enhancement

[21:33:16] <lcp> I'm admin of our Discord server, tell me about it >:D

[21:33:52] <cboltz> plinnell: adding more poeple as owners and ops sounds like a good idea

[21:34:13] <plinnell> if you use the 'list access' command on IRC, you can see who are ops and owners

[21:34:33] <cboltz> do we need to ask the current (partially vanished) channel owners to do that?

[21:34:53] <tampakrap> we have also opensuse.slack.com

[21:34:57] <plinnell> darix and henne, I think setup most of the active channels

[21:35:11] <plinnell> slack 0_O

[21:35:26] <lcp> we have slack? why?

[21:35:44] <tampakrap> because people use it :)

[21:35:52] <plinnell> I have in mind a group of members who are active on IRC and know how it works in depth

[21:36:01] <tampakrap> I'm just saying, I don't want to start a flamewar

[21:36:02] <plinnell> one even wrote his own bot

[21:36:19] <plinnell> tampakrap: +1

[21:36:45] <plinnell> but bridges are a good thing... we have in the past had lots of siloed communication channels

[21:36:54] <lcp> honestly setting up Matrix server would be a good idea if it was faster than it currently is

[21:36:58] <Son_Goku> also, before I forget, I've started taking a look at packaging mailman3

[21:37:19] <plinnell> i'd like to avoid that we make the same mistake with all these new chat platforms

[21:37:24] <lcp> I would wait for Go version of it, because python one is sooooo sloooooow

[21:37:30] <Son_Goku> :/

[21:37:35] <Son_Goku> Go is evil

[21:38:06] <lcp> maybe it is, but it saves us from slow python in case of federated platforms >:D

[21:38:21] <plinnell> so thoughts ? file a ticket ?

[21:38:32] <plinnell> brb

[21:38:33] <cboltz> plinnell: your plan sounds good :-)

[21:38:34] <plinnell> bbi 5

[21:39:03] <cboltz> since you already have some people in mind, the easiest way would be to get in touch with darix and those people to get them added

[21:39:06] <plinnell> we will definitely need some help from daric and henne

[21:40:07] <plinnell> and i like adding the bridges where we can

[21:40:15] <plinnell> I see that as a separate task

[21:41:43] <cboltz> right - adding these bridges (Matrix etc.) is a separate task, and basically "only" needs someone who does it

[21:42:16] <mstroeder> I strongly doubt that Matrix is slow because of Python.

[21:43:11] <lcp> well, protocol is also kinda issue in this case

[21:48:58] <plinnell> i'll put in two tickets for this

[21:49:07] <plinnell> can I self assign myself the ticket

[21:50:25] <cboltz> not yet - you'll need to login on progress.o.o once to get your user account created

[21:50:48] <cboltz> after that, we can add you to the opensuse-admin project

[21:51:10] <plinnell> doing now

[21:51:58] <cboltz> same username as on IRC, right?

[21:53:00] <plinnell> yes

[21:53:20] <cboltz> I just added you

[21:53:48] <cboltz> (no idea if progress.o.o sees this "on the fly" or if you need to re-login)

[21:54:44] <tampakrap> on the fly

[21:55:13] <lcp> I'm happy to report that slack is almost empty

[21:55:15] <plinnell> i'm on a slow machine,so not logged in

[21:55:22] <plinnell> yes

[21:56:12] <plinnell> ok i'm in

[21:58:20] <plinnell> now how do i create a ticket

[21:58:24] <plinnell> relogin ?

[21:58:37] <cboltz> https://progress.opensuse.org/projects/opensuse-admin/issues/new

[21:59:39] <plinnell> thanks.. figured it out

[22:04:08] <plinnell> https://progress.opensuse.org/issues/39287

[22:10:12] <cboltz> thanks!

[22:10:27] <cboltz> does someone have anything else, or can we close the meeting?

[22:10:34] <Son_Goku> I think we're good

[22:11:22] <tampakrap> let's close

[22:11:27] <cboltz> ok, so I'll officially close the meeting

[22:11:27] <Fraser_Bell> Goody. Spammer is back.

[22:11:31] <tampakrap> thnx everyone!

[22:11:43] <cboltz> thanks everybody for joining!