2018-02-06 #opensuse-admin Heroes meeting [20:01:51] meeting time ;-) [20:01:52] its 20.01 ... tampakrap want to start? [20:01:54] :d [20:01:56] :D [20:02:05] meeting time! [20:02:12] STOP! Meeting time! [20:02:15] cboltz: taking over as usual? [20:02:29] no objections ;-) [20:02:51] does someone from the community have any questions? [20:03:14] one short info ... https://en.opensuse.org/openSUSE:Heroes i updated this page and this page https://en.opensuse.org/openSUSE:Heroes/Meetings [20:03:24] as it still said we meet in #opensuse-project [20:03:28] so that is fixed now [20:03:53] thanks! [20:04:27] not questions, but issues-questions that I see in our ticketing system [20:04:33] We should speak about a new Coordinator for the team after Lars left. [20:04:57] one is widhat doesn't have working ipv6, I plan to work on this tomorrow [20:05:23] Ada_Lovelace: how do you see this role in future? what are the tasks you think? [20:05:51] Answering questions like that by Douglas last week. [20:06:15] what was the question by douglas? about hardware sponsoring? [20:06:19] about the sponsoring? [20:06:34] Responsible for answers for hardware sponsorship and anything else. [20:06:38] i already took of this responsibility, as I promised on last offsite in december [20:06:51] and I'm directly in contact with Doug regarding this [20:06:57] Ok [20:07:19] actually i want to know "what is the rest of the stuff?" [20:07:33] organizing an offsite, i had no real big problems with doing it [20:07:44] but i don't want to be the projects babysitter [20:08:23] how about to discuss this topic at our offsite? [20:08:29] Having an eye on tickets. We don't need any babysitter. The main job is answering emails. [20:08:30] yes, we can do [20:08:41] We can do. [20:09:28] Ada_Lovelace: Exactly, answering E-Mails is something I don't see in a "coordinators" role... usually, everybody should be able to do it ... if it is about an "internal contact person" for questions which you cannot solve outside [20:09:33] speak up [20:09:37] and we can discuss it here or via mail [20:10:05] I got a request from a guy about handling mirror requests, juergen loehel [20:10:10] but I really like, that heroes are now more capable of working on their own, as we did the infrastructure move etc... [20:10:17] thomic: ^^ mind adding him to the list of the people to be invited to the offsite please? [20:10:18] however sorry for noise^^ offsite topic [20:10:35] tampakrap: i dont mind ;) [20:10:36] he started already practising [20:10:45] ticket number? [20:10:58] jloehel his username, works for suse in the nuremberg office [20:11:05] i know [20:11:06] :) [20:11:17] which ticket you mean? [20:11:20] the offsite ticket? [20:11:22] i just wanted to know if there is a track of pre-communication :) [20:11:26] https://progress.opensuse.org/issues/28908 [20:11:35] you said you had mails with him [20:11:38] about mirroring [20:11:39] no, it has been through irc [20:11:42] ah [20:13:16] anything else or should we move to the status report? [20:13:19] thomic: for hardware sponsoring, can you please add https://www.thomas-krenn.com/de/unternehmen/kampagnen/thomas-krenn-award.html to your TODO list? [20:13:31] tampakrap: he got mail [20:14:17] thank you! [20:14:21] cboltz: yes [20:14:28] i already saw this [20:14:36] but i need to talk to doug what to write there [20:14:37] :) [20:14:53] ok, thanks! [20:14:54] i'm not the best writer regarding sponsoring stuff.. I just support with hardware details etc [20:15:05] if somebody feels enlightened to help out [20:15:06] feel free [20:15:45] aehm wait [20:15:46] cboltz: [20:15:52] i think i skipped this one [20:15:57] for a big and valid reason [20:16:05] JURY Lenz Grimmer, Engineering Team Lead, SUSE LINUX GmbH [20:16:29] i don't think it's a nice idea to take part in a sponsorship were SUSE decides ... [20:16:44] that might be something like "Angehörige von der Teilnahme ausgeschlossen" :) [20:17:10] indeed, it might look strange if we win ;-) [20:17:17] yeah makes sense [20:17:18] yay [20:18:40] ok, so let's continue with status reports about everything [20:18:46] I'll start with elections.o.o [20:19:04] it "works", but I'd prefer if I wouldn't need quotes around "works" [20:19:36] ideally we should re-setup the machine with salt, but I've been too busy too do it [20:19:37] I owe you a uwsgi configuration, didn't forget about it [20:20:26] I'll ping you when I have time - either in the next 3 days or after carnival [20:20:31] what's our deadline for it? [20:21:02] we need it for the board election, so "as soon as the election starts" [20:21:14] do we have date for the elections? [20:21:32] nothing exact yet [20:21:33] It depends on the system. ^^ [20:21:49] Richard thought about May. [20:21:50] based on the testing you did so far, is this the system to go for? [20:21:51] that, and we also need some candidates ;-) [20:22:01] or should we start investing on something else? [20:22:18] well, we did the small test election about our favorite editor, and it worked [20:22:28] so I hope it will also work for the board voting [20:22:37] cool [20:22:56] what candidates you mean? [20:23:11] candidates for the election committee or candidates for the board? [20:23:15] Does it works with Communiy Members "only"? [20:23:28] for the board, we already have some people for the election comitee [20:23:55] okay I suppose it's up to them then to find the candidates and set the timeline :) [20:24:03] Ada_Lovelace: helios requires to upload a list of voters (= openSUSE Members) [20:24:06] I can give them the "go" if possible. [20:24:43] well, in worst case, we can use the current machine that "works" (with quotes) [20:25:02] that's not perfect, but also not a reason to delay the election [20:26:44] so feel free to give them the "go" (maybe first ask on board@ if someone disagrees, but IMHO there's no showstopper) [20:26:58] Then I'll write to our election committee, that they can start with their plan. [20:27:08] cool [20:27:48] more status reports? [20:27:53] sure [20:28:03] static.o.o first [20:28:26] there are three webpages there: studioexpress.o.o, html5test.o.o and static.o.o itself [20:28:34] the first two are in git repos, static.o.o is a mess [20:28:58] so I plan to gather the sources and put them also in a public git repo, as people wanted to do changes already [20:29:18] and since we're talking about static.o.o, I suppose there is no objection to put it in a public git repo [20:29:32] I'm pretty sure there are no generated files either [20:29:34] agreed? [20:29:45] where is the static.o.o stored currently? nowhere (except on the server, obviously) or spread over several git repos? [20:29:48] related ticket https://progress.opensuse.org/issues/30322 [20:30:11] some parts local only, some parts in git repos that are not automatically deployed [20:30:17] so they are outdated on the server [20:30:24] and I think I saw SVN there [20:30:47] SVN content might indeed be slightly ;-) outdated [20:30:52] as soon as we gather the sources somewhere, then we can automate the deployment via salt better, so we can finally redeploy the machines from sle11 to leap [20:31:04] and also create a static.o.o proper cluster in provo as well [20:31:17] in general, no objections to moving everything to a git repo [20:31:34] but before you move or merge stuff that is already in another git repo, speak with the people managing it [20:31:37] it would be cool if I manage to automate the deployment via a salt reactor and a CI [20:31:52] there might be cases where a separate repo for a subdirectory might make sense [20:31:55] but I'll have to find a good way to mirror the git repo from github to gitlab.i.o.o, as gitlab-ci doesn't work with github [20:32:23] cboltz: how about the other way around? gather them and see if anybody objects [20:32:33] I am pretty sure all that stuff are totally unmaintained right now [20:32:57] I'd ask nevertheless [20:33:06] there haven't been any server changes for almost three years [20:33:28] Do you know the owner of the content? [20:33:29] okay, ask it is [20:33:30] the risk with "just doing it" is that someone commits to the "old" repo without noticing that it isn't used anymore [20:33:39] I don't, I'll try to find them [20:33:49] Thanks! [20:34:17] well, I'm pretty sure that in at least two git repos people committed stuff that never hit production :) [20:34:35] either way, I'll ask around and see what happens [20:34:44] thanks! [20:35:07] next is community.i.o.o [20:35:27] I took a look at the websites hosted there and I filed tickets for all of them to move them away [20:35:37] to static.o.o if they are static or to a separate VM if they are not [20:35:45] I'll contact maintainers as well [20:36:05] the only thing is that there was an ancient non-working search.o.o instance that I disabled [20:36:11] and I put a redirect to software.o.o/search [20:36:44] meanwhile someone created a search.o.o replacement which is static, and I told him to gather feedback from the community and as soon as he is ready to file a ticket so we can deploy it [20:37:46] https://opensuse.github.io/search-o-o/ [20:37:49] this is the website [20:38:15] looks nice to me, even has a dark/light theme [20:38:23] I believe Ludwig created this repo... [20:38:50] nope, this repo was moved recently by me to the opensuse namespace from his personal one [20:39:18] probably you mean https://github.com/openSUSE/searchPage [20:39:28] Sorry! I saw search-o-o and thought to the github repo. [20:40:15] other webpages that are in community.i.o.o are fontinfo.o.o, shop.o.o and I don't remember what else [20:40:30] if we move them away it will be a huge step to offload the machine from stuff [20:40:54] it still has the bugbot irc bot and it is used by the docs people for the release-notes, but one step at a time [20:41:27] any comments here or should I move on? [20:41:38] one comment for the search page: [20:42:20] since we have two search pages now/soon (searchPage and search-o-o), we should decide if we want both or if switching to search-o-o makes more sense [20:42:44] (I'd vote for search-o-o, and maybe add the funny geeko back to not disappoint our firefox users) [20:42:55] agreed, that's one reason why I didn't deploy it (to avoid confusion) [20:43:47] you're free to send your feedback to the developer I'd say :) [20:44:15] yes, I'll do that later ;-) [20:44:15] I don't want to take the decision on which one should be our search page, I will just deploy one [20:45:08] okay moving on to mirrors: [20:45:17] indeed, that should be discussed with the community (in the clever way - "here's a new search page to replace search.o.o and the firefox search page, how do you like it?") *eg* [20:45:41] I mentioned already there is an ipv6 issue on widehat that I plan to take care tomorrow [20:46:27] also I mentioned already jloehel volunteered to start doing mirrors work, so I gave him access to scanner.o.o and gave him the task to add a new mirror to gain some experience with mirrordb and with communicating to mirror-admins [20:47:12] still many mirrors tickets open though, and I'm trying hard to find time to touch them but I'm failing so far [20:47:23] so if anyone is interested in doing mirrors work, feel free to ping me [20:48:26] next: my MF-IT tickets from the past two months are still unanswered, and I still have no responsible MF-IT person to handle provo tickets in progress.o.o [20:48:46] the issue is escalated to my managers, no reply yet [20:48:53] I'll keep you posted if there are news [20:48:56] nice[tm] [20:49:22] next: monitoring [20:49:33] Ada_Lovelace: would you have time to take a look at some issues regarding monitoring? [20:49:48] eg https://progress.opensuse.org/issues/30298 remove the ntp checks and replace them with chrony on the leap hosts [20:50:54] I will do it in March. I received a new attack after Lars's leaving with the reason "Max" in December. I said that I don't do any operative tasks for 3 moths now, because that's enough. [20:51:19] After the next attack I extend it to 6 months. [20:51:30] okay [20:52:12] I seriously hope you'll never need to extend it ;-) [20:52:13] next: I spoke with lnussel during my last nuremberg trip about two topics: his mirrors/release scripts on pontifex and about software.o.o ownership [20:52:41] I'll contribute again, but I have the same problem like Lars. I receive escalations via the Board without any reasons. [20:52:45] for the mirrors scripts he has, we agreed to meet at some point soon online to move them to salt, and I'll restore his pontifex access as well [20:52:52] Sorry! [20:53:56] about software.o.o tickets: ancorgs doesn't respond any more to them, I'm waiting for a new responsible person to assign him the tickets [20:54:03] meanwhile they will remain like this [20:54:52] and my last status update is about salt [20:55:21] we have extended the testsuite, apart from code validation tests we now have also tests that validate the generated content [20:55:37] we started with the nginx configs, where for all but three machines are now in salt [20:55:45] next is haproxy [20:56:01] also, the testsuite now has colors to better separate the errors from the info messages [20:56:15] and to better see which one of the conditions actually failed and which one passed [20:57:05] example https://gitlab.infra.opensuse.org/tampakrap/salt/-/jobs/2549 [20:57:24] I hope my reviews didn't shock you too much ;-) [20:57:36] your reviews are amazing as always <3 [20:58:07] I think that's all from me, the rest at the offsite :) [20:58:46] I have a little success message from Provo - KC Wise fixed the news.o.o newsfeed (which breaks on every wordpress update) today [20:59:09] yey! [20:59:38] does someone else have status reports? [21:01:00] aehm [21:01:06] did you talked about education? [21:01:17] sorry i was a bit "In the deep" [21:01:39] I don't have anything to say about this, there is a volunteer fully handling it now [21:01:46] yay [21:01:46] even replying to other volunteers [21:01:54] so I don't really know what's happening there [21:01:54] as we had the topic last month [21:02:04] i just want to tell this to everybody^^ [21:02:10] somebody cares about domain + project now [21:02:13] Aksem: did I give you VPN access already or I forgot? [21:02:14] we are out of the way :) [21:02:33] I think I forgot his vpn :/ [21:02:35] shame on me [21:05:29] anyways [21:05:37] i just clicked 3 baskets @ thomaskrenn [21:05:45] more than kick us from the list they can't do [21:05:53] my 3 ideas i configured in the budget [21:06:24] 1.5k euro -> LES small machine with monitor as "USB stick creator / Showcase for openSUSE booz" [21:06:48] 2k euro -> 1U server as standalone monitoring host for the openSUSE infra, if KVM cluster would go down in VLAN42 [21:07:20] 3k euro -> 2U server, small extra KVM host, which can be used as monitoring server and maybe an extra machine for something like status on a different cluster [21:07:55] any objections^^? cboltz tampakrap Ada_Lovelace [21:08:02] sounds good, thanks! [21:08:08] Sounds good. [21:08:27] +1 [21:09:04] can you share the links? I have no idea about datacenter grade hardware, so maybe I could learn a bit about it by viewing what you selected ;-) [21:10:02] cboltz: this is so non-datacenter grade :D :D [21:10:06] it needed to be cheap [21:10:23] ;-) [21:10:25] if you want to know, we can do a sub-/side-session with a laptop and a beer at offsite [21:10:38] it's no big deal :) [21:10:49] but i will put you on CC [21:10:57] sounds good - but only if you don't force me to drink beer ;-) [21:12:46] ah no we have boxbeutel as well cboltz :P [21:13:12] s/boxbeutel/bocksbeutel [21:13:24] however you spell it - you can't if you drank one :P [21:13:39] ;-) [21:14:39] more status reports? [21:15:41] doesn't look so, so let's continue with [21:15:44] offsite meeting planning and topics [21:16:12] https://progress.opensuse.org/issues/28908 is the ticket for the offsite meeting [21:16:55] does someone already have some topics we should discuss? [21:17:06] one is the coordinator that Ada_Lovelace proposed [21:17:22] right, I'll add it [21:17:26] a big and complex topic I'd like to discuss is vpn in provo and bridging the networks [21:18:23] also added [21:18:54] mirrors [21:19:01] our database clusters [21:20:07] also added [21:20:07] salt and automation in general (next steps, moving from pets to cattle etc) [21:20:47] rbrown is invited to come at some point to discuss board - infra topics [21:21:06] lnussel is invited to discuss release management topics [21:21:10] he also has the Kubic VM now, so I'd say he's invited all the time ;-) [21:21:21] getting rid of sle11 hosts asap [21:21:27] upcoming leap15 [21:22:05] roland is also invited to discuss management issues as well [21:22:35] thomic: we'll need to get definite timeslots from rbrown, rhaidl and lnussel I'd say [21:22:55] not to overlap with each other, and to give enough time to each one of them to discuss [21:23:14] ah also the mailing setup and the upcoming heinlein migration [21:24:09] monitoring and status.o.o could be a topic [21:25:05] all added, but I'll wait a few minutes before saving [21:25:41] do we need to discuss about any specific webapps? [21:25:48] thanks for the big list, even if it makes it hard to come up with more topics ;-) [21:25:52] the sle11 / old hosts topic covers some of them [21:26:19] you'll hate that topic, but - what about connect.o.o? [21:26:28] it's covered on the sle11 hosts [21:26:41] connect.o.o and progress.o.o are the first two that came to my mind as well [21:26:44] we still need it for membership management, but at the same time, it's terribly old [21:27:35] I'll list connect as separate topic because it needs more than moving to a new machine [21:27:36] so someone from the membership team should take care of it then now that miska went away [21:27:47] yeah do so please [21:28:01] progress.o.o was something that I discussed already with lnussel as well [21:28:25] do you want progress as separate topic as well? [21:28:32] the plan is to set up a leap machine with all plugins disabled, and then give it to lnussel and okurz since their teams (along with our team) are the heaviest users [21:28:39] and if it is fine, then move on [21:29:08] if plugins are missing, then we'll need to discuss if we're moving with missing functionality or if we're blocked until we find a ruby dev [21:29:20] yes add it, but it might be resolved until then [21:29:57] vpn in provo and bridging the networks are the same topic [21:30:01] that would be even better ;-) [21:30:39] well, I thought about "admin VPN Provo" vs. "bridging the network" as different topics [21:31:36] do you still think I should merge this into one topic? ;-) [21:32:03] well, setting up a vpn in provo is not a topic, it's an action item [21:32:13] bridging the two vpn networks is the topic for discussion [21:33:10] I could also call that an action item - I'm sure nobody is against bridging the networks ;-) [21:33:42] perhaps Provo admins? [21:33:49] how to do it is the topic here [21:34:44] tampakrap: ok, you convinced me [21:35:34] Ada_Lovelace: what do you mean? [21:35:34] Ada_Lovelace: you mean how to make them more responsive? [21:36:06] Yes and whether we are allowed to do it with the VPN bridge. [21:36:13] sounds like a subtopic for the management topics to discuss with Roland [21:36:32] the vpn bridge will not happen with the MF-IT network [21:36:44] that will remain as it is, we can't access it [21:36:56] Ok [21:37:49] right, the "old" machines there (AFAIK news.o.o and www.o.o) won't be reachable over VPN [21:38:04] see heroes@ [21:38:05] and lizards [21:39:11] thomic: it's in german, I'll use google translate [21:39:23] tampakrap: I'm very sorry :) [21:39:57] hrm [21:40:04] PROBLEM: HAProxy on mufasa.infra.opensuse.org - HAPROXY CRITICAL - Active service riesling is DOWN on riesling proxy ! Active service narwal4 is DOWN on static proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mufasa.infra.opensuse.org&service=HAProxy [21:40:07] bridging VPNs o.O .. yes that is a topic [21:40:08] :) [21:40:17] oh nice^^ what happened? [21:40:52] thomic: nothing ignore it [21:40:54] note "mufasa" - the bot is talking about the VMs we used in Provo during the power outage [21:41:06] ah yay [21:41:12] we should fix this :D [21:41:21] i always get scared and saw it several times [21:41:32] true [21:41:46] so.. [21:41:52] moar topics for offsite? [21:41:56] I wonder about narwal4 (aka "another static.o.o") - should it also be down? [21:42:24] no [21:42:29] any preferences regarding Dinner on Friday / Lunch / Dinner on Saturday? [21:42:34] but I'm not fixing it now :) [21:42:45] any stuff I need to remark, like, no meat :D [21:42:52] FYI: I just saved https://progress.opensuse.org/issues/28908 so that everybody can see the topics [21:42:56] otherwise I would just reserve :D [21:43:23] I'd say surprise us ;-) [21:44:08] oh kay :D [21:44:09] thomic: for lunch I'd say let's go somewhere around, nothing fancy [21:44:17] yay [21:44:19] for dinners surprise us yes [21:44:36] anyways I will go for a solution below 30 euro [21:44:44] that's not too expensive for everybody :) [21:45:02] we could order some pizza or something as well - if we don't want to go out :) [21:45:05] just in case [21:45:06] or invite someone with a company credit card ;-) [21:45:21] cboltz: i can't guarantee on that one^^ :( [21:45:33] but I'll talk to $people [21:45:34] (not saying that I want/need food 30 €) [21:45:44] maybe there is an expense possible [21:45:59] s/30/> 30/ [21:48:46] anything else for the offsite meeting? [21:48:57] yes [21:49:01] looking forward to it! [21:49:11] indeed! [21:49:16] me too? [21:49:17] :D [21:49:31] you too? [21:49:33] you don't know? [21:49:40] "me too!" [21:49:45] i'm looking forward [21:49:56] but i don't know if i'm allowed to use the term anymore [21:49:58] :D :D [21:50:18] -v please ;-) [21:50:43] you want me to be verbose, or why do you call me with -v? [21:50:43] :D [21:51:06] I wondered about "I don't know if I'm allowed to use the term anymore" [21:51:15] yay [21:51:26] however [21:51:38] let's call it an end? [21:51:47] yes [21:52:04] if someone comes up with another topic later, feel free to add it to the ticket [21:52:19] we can of course also add topics during the offsite, so nothing is set into stone ;-) [21:52:47] thanks everybody for joining the meeting and all the work you did! [21:52:48] yay. chaos as always ;) [21:53:51] yey!!