Project

General

Profile

2020-04-07-heroes-meeting.txt

IRC meeting log - cboltz, 2020-04-07 21:43

 
1
2020-04-07 #opensuse-admin - heroes meeting
2

    
3
[19:58:59] * Eighth_Doctor waves
4
[20:00:31] <pjessen> good evening all
5
[20:00:34] <cboltz> hi everybody, and welcome to the heroes meeting!
6
[20:00:46] <tuanpembual> hi all,
7
[20:01:14] <cboltz> so far, we only have the "usual" topics on https://progress.opensuse.org/issues/64168 - but we can add things if needed
8
[20:02:15] <cboltz> does someone from the community have a question?
9
[20:04:25] <pjessen> i guess the community is busy elsewhere ?
10
[20:04:55] <cboltz> or people are too shy to ask us ;-)  (hint: we don't bite!)
11
[20:05:07] <Eighth_Doctor> eh, well, at least I'm working on ipsilon and noggin atm
12
[20:05:13] <klein> I just want to say that Lars asked me to build 2 new gitlab runners... and I dind't had time yet :-( (/me in shame)
13
[20:05:18] <Eighth_Doctor> (their packaging, that is)
14
[20:05:25] <cboltz> anyway - let's continue with status reports
15
[20:05:29] <klein> but... I fixed the keepalived upstream formula on the Saltstack repo :-)
16
[20:06:52] <tuanpembual> I have small update.
17
[20:07:02] <tuanpembual> from new progress.
18
[20:07:35] <Eighth_Doctor> new progress?
19
[20:07:37] <tuanpembual> still working to fix patch for create ticket as private with send email to redmine@o.o
20
[20:07:47] <cboltz> Eighth_Doctor: progress-test.o.o
21
[20:07:49] <tuanpembual> https://progress-test.opensuse.org
22
[20:08:24] <tuanpembual> that all
23
[20:09:36] <lcp> that's pretty cool, I have had a look at redmine theming and it will require quite a bit of css so I didn't touch it yet
24
[20:09:49] <Eighth_Doctor> it looks fancy :D
25
[20:11:57] <pjessen> okay let me continue - last week I enabled TLS for outound on anna and else.
26
[20:12:21] <tuanpembual> thanks
27
[20:12:25] <pjessen> I'll learn to type, honestly. one of these days
28
[20:13:07] <pjessen> TLS for outbound should have no big impact on anything, I doubt if anyone has noticed anything
29
[20:13:45] <pjessen> I've also been doing some more mirror cleanup, but only minor stuff.
30
[20:14:11] <cboltz> I've seen some admin-auto mails from pontifex (for example a missing database column) - any idea what's wrong?
31
[20:14:35] <pjessen> cboltz: I've not seen those, will have to check
32
[20:14:55] <pjessen> maybe someone is working on the mirroring setup ? otherwise it has not changed
33
[20:15:43] <klein> Darix/Lars team made changes on mirrorbrain last week
34
[20:16:30] <cboltz> that might explain it ;-)
35
[20:16:52] <pjessen> lets forward those mails to lars :-)
36
[20:17:34] <pjessen> anyway, from me, that's it - no progress on forums, still need someone who understand how to integrate it with our authenticatrion setup
37
[20:18:48] <lcp> pjessen: I hope to help you with that, but that will entirely depend on how much we actually know ;)
38
[20:19:17] <pjessen> lcp: together maybe we'll make it to 0 ? 0+0 ?  :-)
39
[20:19:48] <pjessen> I admit, I have no idea how it works.
40
[20:20:01] <lcp> I think we might be adding negatives ;)
41
[20:20:18] <pjessen> lcp: LOL
42
[20:21:04] <lcp> joking aside, it's saml, so we can set it up with the previous auth that mf had in their infra, since it's not dependent on the local server setup mostly
43
[20:21:31] <lcp> however that depends on what address the current site has so testing might suck
44
[20:22:02] <pjessen> that is sort of what I was hoping for, yeah. I just see some missing bits in the apache config, I suspect.
45
[20:22:07] <lcp> in any case, until we have some local saml setup, either with our own login system or SUSE login system, we kinda have to use that
46
[20:22:43] <Eighth_Doctor> I'm hoping to get a basic noggin package ready in the coming days
47
[20:23:28] <cboltz> sounds good, but - IMHO that should be a separate step and not block the forum move ;-)
48
[20:23:34] <lcp> I didn't setup the freeipa2, because we had the salt issue, and now that salt in the repo was updated, we have more salt issues there
49
[20:23:44] <cboltz> I'd start with whatever we have already available ;-)
50
[20:23:58] <lcp> and I also split myself in half and did a bunch of work on porting freeipa to openSUSE out of frustration obviously
51
[20:24:36] <pjessen> cboltz: agree
52
[20:25:41] <cboltz> in worst (?) case, it might even be an idea to write a small plugin that works with our auth proxy
53
[20:26:21] <cboltz> it's just a guess, but I wouldn't be surprised if that's easier than reverse-engeneering the needed saml config
54
[20:26:43] <pjessen> my main problem remains - I simply don't know how it works today.
55
[20:27:07] <cboltz> same for me
56
[20:27:12] <bmwiedemann> Good evening. What is the topic? The new bugzilla auth backend coming in May?
57
[20:27:30] <lcp> for testing I would probably request we had it pointing entirely at forums.o.o instead of forums-nbg.o.o, so we can test auth at least locally
58
[20:27:43] <lcp> and I also would request some access so I can take a look
59
[20:27:45] <cboltz> bmwiedemann: no, we are talking about forum move, especially authentification
60
[20:28:00] <cboltz> but what you mention also sounds interesting - can you tell us some details?
61
[20:28:30] <bmwiedemann> yes. I didnt mean to disturb your topic, though.
62
[20:28:36] <pjessen> lcp: I dont think we can update the DNS until we're ready to move.
63
[20:29:11] <cboltz> lcp: with "request some access", do you mean sudo on the forum-nbg server?
64
[20:29:45] <lcp> pjessen: not dns, but juggling forums.opensuse.org to the ip in the /etc/hosts should work
65
[20:29:51] <lcp> cboltz: yes
66
[20:30:37] <pjessen> lcp: afaik, the setup does not work on forums-ngb yet, but you should certainly have access
67
[20:31:10] <lcp> pjessen: unless we request it with provo to add forums-nbg to their saml setup for testing ;)
68
[20:32:11] <lcp> also I don't really need fully working as long as I can mess with auth plugin and the dump we got from provo
69
[20:32:51] <pjessen> I probably only need to key, then I can set it up for you.
70
[20:32:58] <pjessen> I probably only need your key, then I can set it up for you.
71
[20:33:27] <lcp> I believe it's in freeipa
72
[20:33:41] <cboltz> pjessen:   fetch_freeipa_ldap_sshpubkey.sh hellcp
73
[20:33:56] <pjessen> okay
74
[20:34:12] <lcp> oh yeah, there is a script for that
75
[20:34:44] <cboltz> normally it's used by sshd, but nobody will stop you from using it manually ;-)
76
[20:35:23] <lcp> maybe I should status report then now 
77
[20:36:12] <lcp> I have made a bunch of progress on matrix setup, and in salt there is a huuuge PR that does a lot of things like set up integrations and riot and some other stuff
78
[20:36:23] <lcp> at this point what's left are some pretty cosmetic things
79
[20:37:13] <lcp> basically only waiting for gitlab runners, and for that issue with firewall and haproxy setup at this point
80
[20:37:29] <pjessen> lcp: you should have root access on forum.i.o.o now.
81
[20:37:37] <lcp> thanks
82
[20:38:02] <lcp> outside of that, you might have noticed we transitioned to news-o-o and planet-o-o this last month
83
[20:39:08] <bmwiedemann> and discontinued lizards.o.o
84
[20:39:15] <bmwiedemann> so no more wordpress?
85
[20:39:16] <cboltz> yes, that was a "boring" (as in: no serious problems) move ;-)
86
[20:39:45] <cboltz> right, no more wordpress :-)
87
[20:40:01] <Eighth_Doctor> 
88
[20:40:07] <lcp> ah yeah, there are also formulas for postman3 and ipsilon and some other stuff waiting for gitlab-runners, but aren't PRs yet
89
[20:40:31] <tuanpembual> yay.
90
[20:40:34] * bmwiedemann loves getting rid of another php application.
91
[20:40:34] <lcp> just the moment gitlab-runners happen there is a bunch of stuff to review ;)
92
[20:41:40] <cboltz> well, reviews can be done without the gitlab-runners ;-)
93
[20:42:11] <cboltz> (but especially for the bigger MRs having them checked by the CI would be nice)
94
[20:42:25] <lcp> yup
95
[20:42:33] <lcp> and those are big prs, adding services
96
[20:43:24] <cboltz> yes, but then - the CI might not be too useful for this because it doesn't check everything
97
[20:44:34] <lcp> well, I feel like the only issues with those PRs might be CI complaining, because the rest looks fine, and there just might be small issues I didn't notice otherwise
98
[20:45:24] <cboltz> if you only worry about the CI, that's not enough reason to stop you ;-)
99
[20:45:51] <cboltz> I'll do a review later, and if everything looks good, I can merge it manually
100
[20:45:57] <cboltz> so that you can continue to work on the server
101
[20:46:05] <lcp> alrighty
102
[20:46:42] <cboltz> in worst case, we'll have to do a little follow-up fix once the CI works again ;-) - but if highstate works, it's unlikely that the CI will complain
103
[20:47:33] <lcp> at this point with matrix tho, I could use the firewall/haproxy more than merging the PR with the contents, since that is much more pressing for testing, because it's testing the base ;)
104
[20:48:45] <klein> cboltz: are we impacted because the CI is not working, so, my lack of time to finish the gitlab runners setup is the cause, right?
105
[20:49:40] * klein thinking to install at least one manually, so the work is not stoped because of this
106
[20:49:46] <cboltz> since you ask this way - I'm afraid the answer is yes
107
[20:50:48] <lcp> also in case the PR to matrix gets merged, I will need the corresponding setup on pgsql server
108
[20:50:50] <cboltz> I can work around by doing manual merges, but having the CI back would be nice ;-)
109
[20:51:43] <pjessen> bmwiedemann: bugzilla - was that why I saw a 2nd email from bugzilla-devel_noreply@suse.de  ?
110
[20:51:49] <bmwiedemann> yes
111
[20:52:00] <pjessen> bmwiedemann: very cool!
112
[20:52:02] <bmwiedemann> As you know, auth for bugzilla, forums, wiki, OBS etc is provided by MicroFocus right now, but there is a carve-out process ongoing since SUSE was sold by them. That means, that MF will stop providing these services and the deadline is already 2020-05-18.
113
[20:52:09] <bmwiedemann> So we are setting up our own bugizlla right now using a DB dump. And we are also setting up our own LDAP auth backend to serve the login-proxies in front of OBS.
114
[20:52:09] <bmwiedemann> The same ldap should also serve for the other public openSUSE services that need auth.
115
[20:52:18] <bmwiedemann> As you might know, the design of the login-proxies is so that the user-password is only ever seen by them and services hidden behind them just receive the authenticated user name in a HTTP header.
116
[20:52:18] <bmwiedemann> There will also be some kind of migration necessary, because we will not receive user passwords as part of MF's DB dump. Easiest would probably a password-reset email for every user.
117
[20:53:10] <lcp> bmwiedemann: that is a huge problem then, since we will need a saml server from SUSE side for forums in that case
118
[20:53:39] <bmwiedemann> forum software cannot be patched to accept http-header as auth?
119
[20:53:52] <bmwiedemann> I heard, this kind of patch is usually just a few lines
120
[20:53:58] <lcp> we don't know, we only have saml plugin
121
[20:54:26] <bmwiedemann> well, it exists: https://wiki.univention.de/index.php/SAML_Identity_Provider
122
[20:54:34] <pjessen> bmwiedemann: I would expect so, but nbody has much experience with the forums software
123
[20:55:05] <bmwiedemann> as long as it does not expose user passwords to the forum software, it should be fine.
124
[20:55:32] <pjessen> bmwiedemann: how can we test this?
125
[20:56:10] <bmwiedemann> once it is up and running - still takes some more days.
126
[20:56:29] <pjessen> okay, no hurry
127
[20:57:00] <pjessen> bmwiedemann: i'll be in touch by email tomorrow.
128
[20:57:35] <cboltz> bmwiedemann: somewhat related - do you also have a replacement for https://www.opensuse.org/openid/ on your radar?
129
[20:57:55] <cboltz> (moving the website itsself is easy, but so far /openid/ is a blocker)
130
[20:57:56] <bmwiedemann> I guess, there could also be an app for it.
131
[20:58:42] <cboltz> can you please check and report back as soon as you know?
132
[20:59:19] <bmwiedemann> is 'openid connect' good enough?
133
[20:59:52] <lcp> not the same thing I'm afraid
134
[21:03:28] <bmwiedemann> do we know, who consumes this openid endpoint?
135
[21:03:57] <lcp> we cannot know, since that's auth for users, not for specific services
136
[21:03:59] <bmwiedemann> I know, I installed an internal dyndns and it is using it for auth
137
[21:04:13] <lcp> that means that they can auth with any service using that id
138
[21:04:32] <lcp> any service that supports openid that is
139
[21:06:17] <lcp> bmwiedemann: alright, let's maybe approach it from this angle, we were planning splitting auth away from SUSE anyway, and we could try to synchronize the efforts from ours and your side so we split away from MF and each other at the same time
140
[21:06:35] <lcp> we do have software to do that selected, and it supports the things we more or less need
141
[21:07:27] <lcp> however, we don't have a strategy for getting the list of all of the users we would need to transition, which we would need as you need ;)
142
[21:07:48] <bmwiedemann> so would there be an openSUSE account separate from the bugzilla account then?
143
[21:08:22] <lcp> bugzilla is something we are discussing with redhat at this moment, to get their system for auth with multiple account systems
144
[21:08:41] * Eighth_Doctor has a note to ping Jim about this again
145
[21:09:27] <lcp> bugzilla would be unique in that it would be the one service where we would have both accounts be able to login
146
[21:09:51] <lcp> everything else is pretty obvious if it's openSUSE or SUSE
147
[21:10:06] <pjessen> lcp: afaict, we have the user list in the databaee
148
[21:10:40] <bmwiedemann> but for users it can still be confusing to have 2 different SSO systems then
149
[21:11:13] <lcp> pjessen: yeah, I kinda wanted to ask service admins to export the list of people registered in every service, but it's hard to get to everybody everywhere ;)
150
[21:11:31] <pjessen> 2 SSO systems is an oxymoron ....
151
[21:11:53] <lcp> bmwiedemann: well, I hope they will be branded sufficiently well, so people can tell which one they need to use
152
[21:12:37] <pjessen> lcp: i'm blinkered, I just want the forums migration out of the way.
153
[21:13:17] <bmwiedemann> yes, I guess, they will have green chameleons :-P
154
[21:13:22] <lcp> pjessen: yeeeeah, me too, we really need to set it up at MF right now
155
[21:14:24] <pjessen> lcp: if the auth system is going to go away in 6 weeks ?
156
[21:14:57] <lcp> pjessen:  so forums can work on this account system for 6 weeks
157
[21:15:01] <lcp> that's a lot of time
158
[21:16:09] <pjessen> lcp: true - if we can get it working in less than a week. Might be better to aim for the new auth system. We will also have to send a reset-your-pwd to all forums users.
159
[21:16:15] <lcp> if we rush through it, I and Conan Kudo can get the login system ready to test with forums-nbg next week/ in two weeks tho
160
[21:17:10] <Eighth_Doctor> yeah...
161
[21:17:10] <lcp> for testing is the keyword, because it certainly won't work a 100% ;)
162
[21:17:22] <pjessen> lcp: by all means have a go at it.
163
[21:18:45] <lcp> pjessen: I will create an issue for exporting usernames from all of the software then
164
[21:19:41] <lcp> then we will ask bmwiedemann or some contact if you prefer to give us the passwordless dump of those users' accounts
165
[21:19:43] <pjessen> lcp: okay, no prob
166
[21:21:17] <bmwiedemann> lcp: with email addrs and real names, it will count as PII and there are pretty strict rules about those in Germany.
167
[21:21:29] <bmwiedemann> just usernames would be easy, though.
168
[21:22:01] <pjessen> bmwiedemann: i can do usernames only.
169
[21:22:28] <lcp> bmwiedemann: hm, but what will the dump you get from MF contain then?
170
[21:23:26] <lcp> also, considering that openSUSE Project at this moment is not a split entity from SUSE, those things should be able to move around inside SUSE without much problem (but add that to the problems that board has to figure out for foundation)
171
[21:24:16] <lcp> bmwiedemann: ah yeah, any idea who to contact about deployments of openQA/OBS/OSEM and other stuff hosted by SUSE so we can export usernames there too?
172
[21:24:33] <bmwiedemann> lcp: the dump we get, should contain everything except passwords. But that also means that we need to be extra careful in handling that data. E.g. we need a workers council agreement.
173
[21:26:18] <lcp> bmwiedemann: eh, I will discuss that with board then too, I see that is going to be really problematic
174
[21:27:32] <bmwiedemann> sometimes I wonder if using github accounts for auth wouldnt be easier (for openSUSE stuff)
175
[21:28:18] <lcp> well, I am hoping to setup git forge right after we get openSUSE Account system, so no
176
[21:29:43] <cboltz> bmwiedemann: that would mean openSUSE uses Microsoft for authentification, and I'd expect some ;-) funny discussions if we do that *g,d&r*
177
[21:30:04] <pjessen> cboltz: yup.
178
[21:30:21] <bmwiedemann> Google does. And Microsoft uses chromium code as base for their browser... not so strange?
179
[21:30:42] <pjessen> code is one thing, data another
180
[21:31:07] <lcp> that principle still applies even here, amazing
181
[21:34:56] <bmwiedemann> so far, I added a reminder to export usernames for the heroes when we have it (hopefully end of April)
182
[21:36:53] <lcp> bmwiedemann: https://progress.opensuse.org/issues/65405
183
[21:36:54] <bmwiedemann> I guess, when openSUSE has a working separate auth, we could also send out password-reset mails to users with a link to that new auth system.
184
[21:37:35] <lcp> yup, that should be easy enough to do
185
[21:38:14] <bmwiedemann> still will need some effort, because we want to handle email bounces gracefully. And I expect there will be a lot of those.
186
[21:38:43] <lcp> as an additional point, since that was a request, we do want to allow for change of the username for users that want it, but we will handle that manually, since I don't expect that to be popular
187
[21:38:53] <lcp> at this point we have 2 requests like that
188
[21:39:28] <bmwiedemann> wow. Will that even work with all connected services? Or will it just be like a new acocunt for those?
189
[21:39:30] <lcp> so that's also why I'm asking for contact point for the OBS/openQA/OSEM deployments, since that's something that will have to be covered too
190
[21:39:47] <lcp> bmwiedemann: it will have to be done per service unfortunately
191
[21:39:58] <bmwiedemann> OBS => adrian , openQA=> coolo , OSEM dont know
192
[21:39:58] <lcp> if we can move accounts there, then yes
193
[21:40:24] <lcp> thanks! that's helpful
194
[21:40:35] <cboltz> I'm afraid changing usernames will open a can of worms because we'll need to change the username at lots of places (starting with > 20 wikis)
195
[21:41:01] <cboltz> I slightly ;-) doubt that we want to do this
196
[21:41:05] <bmwiedemann> lcp: and as we discussed in the openid topic, there can be any number of consumers we dont know about
197
[21:41:10] <lcp> cboltz: yeah, I do hope they don't just login into everything in our infra
198
[21:41:35] <lcp> bmwiedemann: we will be able to gracefully redirect to our new openid solution easily
199
[21:41:36] <cboltz> I don't even want to _check_ 20 wikis if a specific user ever logged in there ;-)
200
[21:41:52] <lcp> cboltz: I am afraid I will have to then ;)
201
[21:42:16] <lcp> somebody will have to gather info on who logged into every single service in our infra
202
[21:42:30] <bmwiedemann> automation could
203
[21:42:31] <lcp> only username tho
204
[21:42:42] <lcp> yes, of course
205
[21:42:58] <cboltz> I won't stop you, but that's not the point ;-) - allowing to change the username causes us lots of work without a big gain
206
[21:42:59] <lcp> I will automate as far as I can
207
[21:43:15] <lcp> then we will have to crossreference who was where if they request username change
208
[21:43:38] <lcp> cboltz: I call that being way too nice 
209
[21:43:38] <cboltz> the "old way" (just register a new username, and stop using the old one) is probably good enough, and doesn't cause work
210
[21:44:20] <cboltz> and even if we allow changing the username, we should IMHO block the old name from being re-used
211
[21:44:32] <lcp> of course
212
[21:45:07] <bmwiedemann> I agree with cboltz there
213
[21:46:49] <lcp> I know it will be a lot of work, but I think it's worth it as an option
214
[21:46:57] <lcp> one time offer for the transition
215
[21:49:03] <lcp> I will regret it later, but oh well
216
[21:49:19] <bmwiedemann> :-D
217
[21:49:21] <cboltz> I'm sure you'll regret it ;-)
218
[21:49:24] <lcp> alternatively we could offer it to the two poor souls that want it >:D
219
[21:49:52] <lcp> because I know both of them quite well
220
[21:49:58] <lcp> well, well enough
221
[21:50:54] <cboltz> still, I'd recommend that these poor souls first register a new account with the username they want
222
[21:51:21] <cboltz> that's easier than having to block the old username "manually"
223
[21:51:47] <cboltz> after that, we can chown ;-) their data on specific services
224
[21:51:49] <lcp> eh, it is a lot of effort to move stuff arround tho
225
[21:52:18] <bmwiedemann> maybe they have a list of services where they are interested in keeping stuff.
226
[21:52:33] <bmwiedemann> old forum posts might not be as interesting as OBS package maintainership
227
[21:52:44] <lcp> yup
228
[21:53:12] <lcp> I will ask then, I will handle this in https://progress.opensuse.org/issues/30970 and with Conan Kudo later ;)
229
[21:54:47] <lcp> alright, I think that's enough of accounts talk for now, we have a lot of work for the next few weeks >:D
230
[21:54:54] <Eighth_Doctor> :D
231
[21:55:34] <bmwiedemann> so we meet again on 2020-05-05 ?
232
[21:55:41] <lcp> I hate deadlines, but here we are ;)
233
[21:55:48] <pjessen> sounds like a plan
234
[21:56:07] <lcp> hopefully with a working accounts system too
235
[21:56:28] <lcp> unless there are more subjects we didn't discuss
236
[21:56:43] <bmwiedemann> If there is something needed - I'm probably reading emails more often than IRC on many days.
237
[21:58:16] <lcp> alright, got it
238
[21:58:26] <pjessen> ditto
239
[21:59:50] <cboltz> looks like that's it for today ;-)
240
[22:00:00] <cboltz> thanks everybody for joining, and for all the work you do!
241
[22:01:09] <pjessen> stay healthy everone
242
[22:02:27] <lcp> thanks, you too!
243
[22:05:07] <bmwiedemann> have a good night
244
[22:05:38] <tuanpembual> thanks all
245
[22:05:49] <tuanpembual> good morning :)
246
[22:06:02] <lcp> night!
247