[19:54:21] hello partypeople [19:56:18] hello partyman [20:00:01] so meeting time! [20:00:04] who is here? [20:00:12] hi everybody! [20:00:17] I'm here [20:01:00] Ada_Lovelace: ping? [20:01:44] hi [20:02:16] Hi [20:02:30] cboltz: do you have the ticket handy? [20:02:44] https://progress.opensuse.org/issues/28911 [20:03:13] thanks [20:03:14] wanna chair? [20:03:29] why not ;-) [20:03:41] so first topic - questions and answers from the community [20:04:10] does someone have a question? [20:04:24] We had the topic election system in our Board meeting yesterday. [20:04:36] I have one question: Why is there so much spam in admin tickets? [20:04:45] tampakrap: It seems you want to pick it up? [20:05:07] okurz: because the admin@ are too common mail addresses [20:05:17] could be [20:05:20] sorry i', late [20:05:28] okurz: I'm afraid the answer is "because mx*.suse.de isn't really good at blocking spam" [20:05:35] Ada_Lovelace: well, tomas told me that you and cboltz offered to take over from where we left [20:06:12] I missed yesterday's board meeting, so - good to know ;-)) [20:06:26] ticket number for the election system is https://progress.opensuse.org/issues/25486 [20:06:32] cboltz: You are a Python expert. Do you want to do it? [20:06:33] and update from my side: Sorry, because of many computer problems (and SLE bugs to test) today I did not find the time to check my openSUSE admin VPN and won't be able to do tomorrow, maybe thursday but I consider it also not urgent from my point of view. I want to start with understanding who progress.o.o is setup and then at best help tampakrap to update it and optionally extend with addons [20:06:46] last two comments list what needs to be done [20:07:23] Thanks, okurz! :) [20:08:05] Ada_Lovelace: I have a lot in my plate these days so I'd like to avoid having to do this as well, my initial agreement would be to setup the python instance and give it to somebody to complete it [20:08:28] so if possible I'd like to avoid doing it and focus on some security issues, like the progress.o.o upgrade that okurz mentioned [20:08:47] and continue updating or retiring the rest of the sle11 services we have [20:08:51] I was surprised, that it is assigned to you, tampakrap. [20:09:23] it is assigned to me because I was working on it with tomas [20:09:30] Ah... [20:09:54] I never looked at helios on the technical side, therefore I hesitate to say yes instantly ;-) [20:09:55] but I'd really appreciate it if someone could take over from here, the instance is somehow running either way [20:10:14] well me neither :P [20:10:43] I would say take a look at the current stuff we did and the remaining tasks and try to solve them, and as always feel free to come back to me for help [20:10:51] cboltz: packages are finished. The connection for Members is missing. [20:10:53] is that acceptable? [20:11:35] Ada_Lovelace: in other words: the boring part is done, and "just" the difficult part remains ;-) [20:11:46] ;-) [20:11:54] it was quite hard to package it actually [20:12:09] sounds interesting[tm] [20:12:49] I'll have a look at it later - but that's all I'll promise for now ;-) [20:13:05] Ok. [20:13:32] okurz: Do you need any support by us or do you receive it at SUSE? [20:14:41] okay but keep in mind that it is important and urgent as the elections are coming [20:14:51] but you are both board members and you know that already :) [20:15:01] indeed ;-) [20:15:14] Ada_Lovelace: thank you I have enough support and want to *give* something myself :) [20:15:30] I'll announce a Call for election commitee this week. And Richard meant April would be a good time for elections. [20:16:08] okurz: Great! :) [20:16:16] okurz: so after you finish your vpn, we can have a call one day so I can walk you around the current instance and we can work on the upgrade together [20:16:40] okurz: the plugins are challenging as it will be a major upgrade, so we'll have to disable a lot of them, maybe all [20:17:03] we'll have to ask various teams which ones of those plugins are still using etc, so it's going to be quite some work [20:17:11] Sure, don't expect much from my side in the upcoming week though. My main goal is to understand the current setup of progress.o.o *readonly*, later one potentially *change* it [20:17:31] and I'm still not sure how the db migration in such a big upgrade is going to be, so we'll need to give it to people for testing before we do the final dump [20:18:01] okay [20:18:05] Sure, at best I can rebuild a setup locally - potentially with a reduced database portion - and try it out first [20:18:24] we have a staging host already waiting to be set up [20:18:56] so... while reading here i was looking into the question ... as far as i see here... mx1 and mx2 just forward the mails to lists5-opensuse.suse.de for admin@ without filtering anything, because we don't do filtering on mx, we do it in later steps and/or rely on the spamfilters our users have in place... which means ... lists5 maybe should add a filtering for the admin@ address? [20:19:03] keep in mind though that the admin work will not be a major part of my work so don't get your hopes up ;) I will just want to help solve "our own problems" instead of just hammering you with tickets [20:19:04] sorry for writing offtopic =) [20:20:04] thomic: we had a large discussion one/two meetings ago about setting up our own mx/relays, but it is frozen due to the heinlein agreement [20:20:13] yay... [20:20:14] thomic: might be an option, but not the best one - it will cause backscatter to whoever is the poor owner of the mail address the spammers use [20:20:27] ^^ correct [20:20:42] spam blocking should be done on mx*.suse.de to be able to _reject_ the spam instead of bouncing it [20:20:59] that is not easy to do. [20:21:26] you need to scan the email whilst it is inflight, i.e. not queued. [20:21:45] cboltz: nope... it is not done on mx [20:21:53] for no domains [20:22:05] as far as i see .. i dont have the full view yet.. [20:22:13] pjessen: right, but that's not really hard [20:22:16] but afaik we do spam filtering in a later step [20:23:27] thomic: "in a later step" sounds like a) "spam folder" or b) backscatter, and IMHO both are a bad idea (and a) is useless for admin@) [20:23:54] i guess it is a topic for another day, but spending 10-15 seconds in spamassassin before the email is queued means holding up the client with NOOPs etc. [20:24:56] cboltz: indeed we drop the mails one step later and/or have a filtering there.. but anyways I agree for admin@ it would not help there [20:25:08] i just wanted to answer the question "why" [20:25:14] it is because of the mailsetup atm :) [20:26:26] next topic then? [20:26:32] yes [20:26:34] Yes [20:26:39] status reports about everything [20:27:19] tampakrap: maybe you should start with the galera cluster ;-) [20:27:23] the salt and monitoring hosts lists are deviating, any volunteers to help me check them one by one and add them to the one or the other? [20:27:57] too late, I started with something else :) [20:28:32] silence, I take that as a no, I'll do it alone [20:29:33] so galera cluster: we still need to do the logrotate.timer (currently off on galera1) and enable backups on galera3 [20:29:47] cboltz: you want me to replicate here the whole story that I sent on the mailing list? [20:30:08] that would be superfluous ;-) [20:30:30] The solution should be enough. ;-) [20:30:52] I'm still surprised that logrotate (which might mean restarting mysql) causes the problem [20:31:19] so any takers for those two tasks? [20:31:22] I assume you compared the logrotate time with the time when mysql broke, but it still sounds strange [20:31:47] I can help you with logrotate.timer [20:32:12] what sounds strange? [20:32:12] (maybe after the meeting) [20:32:22] I guess logrotate was not to blame? [20:32:39] that it only takes a log rotation (and restarting mysql) to break the cluster [20:32:50] I'd expect it to be more stable [20:33:31] why are we logrotating mysql?? [20:34:13] why not? [20:34:28] if we don't, one day we will end up with a log that fills up the disk space :) [20:34:35] because mysql doesn't write any logs. [20:34:48] the default logrotate.d/mariadb rotates /var/log/mysql/mysqld.log [20:34:54] I never rotate any mysql logs. [20:35:18] galera maybe does? there is definitely a /var/log/mysql/mysqld.log [20:35:28] the logs written are usually query logs, but they have to turned on and off [20:35:39] yes, the log is there, but is it growing? [20:36:14] if it grows by e.g. 1kb a day, it should of course be rotated. [20:36:34] logrotate uses /usr/bin/mysqladmin --local flush-error-log flush-engine-log flush-general-log flush-slow-log so mysql doesn't even get restarted [20:36:43] which makes it even more surprising that the cluster breaks [20:36:49] so apparently the logroate is not the issue, there is no logrotate running right now on the cluster [20:36:54] and it went down an hour ago [20:36:56] :) [20:36:57] (all assuming that we are talking about the mariadb packages from 42.3) [20:37:24] I'll fix it after the meeting [20:37:41] PROBLEM: HTTP progress on redmine.infra.opensuse.org - HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 968 bytes in 6.020 second response time ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=redmine.infra.opensuse.org&service=HTTP%20progress [20:37:46] ah no wait [20:37:53] it logrotated again [20:37:58] but how? logrotate.timer is down [20:38:14] paste(bin) systemctl status logrotate.timer please [20:38:26] let's move with the meeting and do that after? [20:38:32] ok [20:39:34] so next topic? [20:40:10] depends - does someone else have a status report? [20:41:05] doesn't look so, so next topic [20:41:10] opensuse-education.org https://progress.opensuse.org/issues/29784 [20:41:34] ignore the link while progress is down ;-) [20:42:07] basically the question is a) do we want to take over the opensuse-education.org domain and b) what do we want to do with it? [20:42:13] PROBLEM: HTTP on conference.infra.opensuse.org - HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 922 bytes in 6.034 second response time ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=conference.infra.opensuse.org&service=HTTP [20:43:01] The content currently provided via that domain will also be removed [20:43:04] (Webpage as well as ISO images and RPMs provided via [20:43:06] http://files.opensuse-education.org/ ). [20:43:32] The ISO images and other files are currently mirrored to a host called [20:43:35] community.opensuse.org (community.infra.opensuse.org). I leave it up to [20:43:37] you to decide what should happen with the content on that mirror [20:43:39] server. Feel free to delete it as well or leave it as it is for [20:43:41] reference. [20:43:42] The source of the webpage is currently available at GitHub: [20:43:45] https://github.com/openSUSE-Education/www.opensuse-education.org [20:43:47] Please tell me until 2018-01-31 if I should add someone as new Owner of [20:43:49] the repository. [20:43:51] this is the mail [20:44:09] maybe the content could be copied to education.o.o ? [20:44:22] it is outdated since 2015 [20:44:29] I see two options [20:44:38] either somebody wants to take the project and care about the content [20:44:46] or we don't deliver ISOs from 2015 [20:45:13] we can't deliver insecure, oldstable under official .o.o domain, as debian is doing ;) [20:45:20] that's not our style i think [20:45:41] saving the content somewhere (non-accessible) if somebody asks ... works for me [20:45:52] but I won't publish it anywhere if it is outdated like this [20:45:56] okay, I assumed somebody cared about the content. [20:46:02] yes lars :) [20:46:14] Somebody by Ubuntu wanted to have the Life name. openSUSE education staied at openSUSE. [20:46:21] thomic: agreed, even if I'm not sure if I get your poke at debian ;-) [20:46:48] cboltz: was unfair, at least they security-bugfix oldstable ;) [20:46:56] so let it die? [20:47:21] If nobody wants to maintain it... [20:47:28] so we have time until end of this month [20:47:36] do we go down the road asking for somebody? [20:47:46] like on opensuse@ or opensuse-announce@? [20:47:55] opensuse-project I'd say [20:47:58] or do we just kill it? [20:48:14] opensuse-project and opensuse should be right lists for that. [20:48:30] unless someone creates updated ISOs, one option would be to make the domain a redirect to https://en.opensuse.org/Education (assuming that and the package repo are less outdated) [20:48:33] okay any takers? [20:48:41] * cboltz hates handing out domains to spammers [20:49:16] ah interesting note [20:49:18] the domain [20:49:22] do we want to keep it? [20:49:26] it belongs to lars privately [20:49:34] if it dies, the domain dies as well [20:49:37] just fyi [20:49:53] two domains [20:49:58] I didn't copy the first part sorry [20:50:04] The domains "opensuse-education.org" and "opensuse-education.de" will [20:50:07] be free after 2018-01-31. If you like, I can open a transfer request to [20:50:09] anyone who wants to take over. [20:50:30] for the .de address, you need an address in Germany [20:50:51] or a registrar with a german proxy [20:51:06] if we want to keep them [20:51:09] i would take them over [20:51:16] I'm a domain collecter anyways :D [20:51:31] but I don't mind if we transfer them to the "official" pool [20:51:36] i would even prefer that [20:51:41] but no clue how to do that ;) [20:52:03] to add them to freeipa you mean? [20:52:07] my vote: if we are not going use them, we dont need them. [20:52:30] tampakrap: maybe :) [20:52:38] there is internet-x listed as sponsor on the education web page [20:52:42] what do you mean by official pool then? [20:52:48] we could ask them kindly to give us a free account [20:53:06] tampakrap: yes, freeipa, but freeipa is only managing dns ... the domains belong to SUSE / Novell afaik [20:53:19] which means "the official by company managed pool" [20:53:29] it is more about law than about technology [20:54:02] Is anyone feeling mad, if these domains die? [20:54:07] If yes speak up now :) [20:54:24] If not, I would say, lets ask once more on the opensuse-project [20:54:25] okay my proposal: someone send a mail to the mailing lists asking for a new maintainer till 29/1 [20:54:30] if nobody replies, let it die [20:54:32] if somebody wants to maintain [20:54:46] if somebody replies, thomic takes the domains and the service remains alive for another 2 years [20:54:47] i would ask internet-x for sponsorships or kick people in provo [20:54:48] :) [20:54:57] I'd keep the domains nevertheless [20:55:10] ok so.. i start kicking anyways [20:55:28] i will write them a very polite german mail ... :) [20:55:52] thanks! [20:56:19] next topic? [20:56:28] yes and no [20:56:38] next topic would be the new election website [20:56:46] which we already discussed [20:57:01] so anything else from anyone or can we close the meeting? [20:58:18] well, maybe I have something [20:58:40] now that Lars left the heroes, who will organize the offsite meeting? [20:58:56] thomic: is this something you could do? [20:59:27] cboltz: I'll let you know before the next month's team meeting on the mailing list [20:59:45] ok [21:00:04] PROBLEM: HAProxy on mufasa.infra.opensuse.org - HAPROXY CRITICAL - Active service riesling is DOWN on riesling proxy ! Active service narwal4 is DOWN on static proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mufasa.infra.opensuse.org&service=HAProxy [21:01:02] anything else? [21:01:36] not from here [21:02:05] looks like we can close the meeting [21:02:12] thanks everybody! [21:02:12] good, thanks everybody