2017-07-04 #opensuse-admin - Heroes meeting [20:02:56] is everybody ready for the meeting? [20:03:07] my clock says you should be, so let's start ;-) [20:03:16] yes [20:03:55] does someone from the community have questions? [20:04:46] Seems: no [20:05:07] ok, so let me continue with the wiki update [20:05:11] Then: wiki update [20:05:47] Max fetched the latest files and database dump of the english wiki last week [20:06:10] Theo did the VM setup (+ several "smaller" things like creating databases, nameserver settings etc.) [20:06:29] the result is that en-test.opensuse.org is now running on Rieslingschorle [20:06:51] (that is, the riesling VM with Apache, PHP, Mediawiki - and the water VM with elasticsearch) [20:07:06] :D [20:07:15] so we have an up-to-date test wiki now [20:07:26] and it is running on the VMs that will go into production [20:07:32] (of course completely salted ;-) [20:07:43] my plan is: [20:08:01] - today: send out another call for testing [20:08:31] - next week: migrate and update the english wiki (unless testing brings up a showstopper) [20:08:56] - another week later: migrate and update the other wikis [20:09:38] I'm quite sure testing will mostly happen _after_ updating the english wiki - this is why I don't want to update all wikis at once ;-) [20:09:41] You read today that we shouldn't do a lot with the openSUSE infrastructure during next weeks (until release)? [20:09:59] yes, I did ;-) [20:10:25] we'll see how far we get with updating the non-english wikis [20:10:47] Yes. :_) [20:10:52] but I really want to have the english wiki updated before the 42.3 release [20:11:11] if something breaks, we still have enough time before the release to fix it ;-) [20:11:58] and from the marketing perspective, having the main wiki already running on 42.3 sounds like something we want to have ;-) [20:12:28] "look how much we trust our new release" ;-) [20:13:47] BTW: with "next week", I hope for the beginning of the week, which is 2 weeks before the release [20:14:23] fine by me [20:14:40] sounds like a solid plan [20:14:53] :-) [20:14:59] By me, too [20:15:10] how many days before the release should we stop to migrate non-english wikis? [20:16:09] (and: should we make that dependent on the size of the wiki?) [20:16:40] I would say 3 or 4 days [20:17:15] I would say ask the release managers and the marketing guys [20:18:09] That's a good option [20:18:38] I'll write a mail to the heroes ML about it anyway, so Ludwig should see it. CC'ing Doug is easy ;-) [20:19:57] BTW: if the update works without problems (it did for en-test), it means that we have to make the wiki read-only between a) packaging all files and getting a database dump and b) switching over the en.o.o DNS entry [20:20:33] so a few hours [20:20:45] which means you need to announce a downtime [20:21:06] "just" a read-only time, but yes [20:21:35] what do you mean? [20:21:57] during the update, you can still _read_ the wiki (it will still be served by the Provo server) [20:22:12] ah [20:22:12] but we have to make it read-only before making the database dump [20:22:17] the dns change will not happen instantly [20:22:25] so still announce it [20:22:37] and since it is a big change and it happens before the release [20:22:44] That should be a message on oensuse-project [20:22:46] I would say to involve doug to write a news.o.o article [20:23:11] it should be on news.o.o, on our blog (before AND after) and on opensuse-announce [20:23:27] with a reply-to opensuse-project [20:24:08] I had planned for a news.o.o article _after_ updating the english wiki [20:24:36] but I get your point - since we are close to the release, it makes sense to tell people in advance when they can relax instead of updating the wiki ;-) [20:24:43] no, it needs one before, and edit that page when it is finished [20:25:02] They would ask on the wiki list and opensuse-project, why they can have only read-only mode at this moment. [20:25:39] there will be a message in the wiki while it is read-only of course ;-) [20:25:54] sure, someone will ask even if you put big giant banners in front of everyone's face [20:26:10] (something like the "This is a test wiki" we have in en-test.o.o, but of course with a different text) [20:27:36] for the news.o.o article - [20:27:44] - should I ask Doug to handle it (which will be interesting for the "done now" update which will happen in the evening)? [20:27:58] - does one of us (Lars?) have access to news.o.o and can do it or [20:27:59] prepare a draft and send it to him [20:28:06] - should I ask for a news.o.o account? [20:28:18] there is a mailing list [20:28:22] news@o.o I think [20:28:28] prepare a draft and send it there [20:28:48] ok [20:29:05] even if you have your own account there, one extra eyeball pair wouldn't hurt to review [20:29:24] indeed ;-) [20:30:31] I think that's it for the wiki update [20:30:43] we'll see if the testing brings up something [20:31:05] if everything works without complaints, I'll coordinate the update day/time with tampakrap and mmaher_home [20:31:21] (probably beginning of next week) [20:32:03] tampakrap: a last question - can you reduce the TTL of the en.o.o DNS entry? [20:32:35] how much is it now? [20:32:56] and how much you want it? [20:33:17] current value is 86400 [20:34:31] my usual TTL for migration is 5 minutes ;-) [20:34:36] but maybe we should use an hour so that we don't get the load with a "big bang" and have at least a few minutes to provide additional resources [20:36:27] I'll ask MF-IT if it is possible [20:36:31] but I think it isn't [20:37:31] correct me if I'm wrong, but AFAIK the DNS zone gets managed in FreeIPA and nsprv*.novell.com are DNS slaves [20:37:41] so - shouldn't it be possible to set the TTL in FreeIPA? [20:38:47] I don't know if we transfer the TTL value as well to them [20:39:05] maybe ask darix, as i know, he was talking about that topic today with lars^^ [20:39:41] ok [20:40:12] so - anything else for the wiki update? [20:41:12] doesn''t look so ;-) [20:41:22] next topic - status reports [20:41:46] what happens outside the wiki? ;-) [20:42:54] Stanislav upgraded Weblate [20:43:16] l10n moved to new VM, now running postgres and with updated version [20:43:30] So it is. [20:43:45] that doesn't show statistics on the front page to do a bunch of sql queries by saying good morning and look like a slow webpage finally [20:44:12] the migration from mysql to postgres was totally pointless in my opinion, but oh well, he did the job not me [20:44:31] also I did some stuff to prepare per's mailman machine [20:44:38] and max did some mirror tickets [20:44:41] that's all I think [20:45:08] yeah, i got today to the point where i covered every mirror ticket where a costumer is involved so green [20:45:26] He told me the reason that most SUSE databases are running on postgres now [20:45:42] who told you that? [20:45:47] Stanislav [20:46:05] nope [20:46:26] the reason he decided that is because back then the mysql cluster had some issues that caused downtime [20:47:23] https://lists.opensuse.org/opensuse-translation/2017-06/msg00007.html [20:47:53] # [20:47:54] Also openSUSE infrastructure people prefer postgres, as it already [20:47:56] serves large part of the infrastructure. [20:48:28] But unimportant now... [20:48:48] I do prefer postgres personally, but when something is already running and a long migration effort is involved, then you balance your choices usually [20:48:54] anyway, whatever [20:49:43] the migration is done, looks good, congrats to him and thanks for all of his efforts [20:50:01] Yes [20:51:25] does someone have more status reports, or should we go to the next topic? [20:52:17] ok, next topic then [20:52:27] handling of missing tickets in the future [20:52:41] that might also translate to "handling of spam tickets" [20:53:23] with the amount of spam we get, it was only a question of time that a ticket gets accidently deleted [20:53:36] the options I see are: [20:53:42] Do we want to think about a new ticketsystem? [20:54:07] - continue the current workflow, with the risk of accidently deleting a non-spam ticket [20:54:47] - close (instead of delete) spam tickets, with the disadvantage that this will send out a mail to the (probably faked) sender address, and the advantage that we could disable the "delete" feature [20:55:41] this is not an option [20:55:54] - "fix" redmine (lots of people will be happy about this!) [20:55:55] close is no option that works for me [20:55:58] it may end up in a bounce war [20:56:06] - switch to another ticket system [20:56:59] and migrate content to the new ticketing system or not? [20:56:59] the perfect solution would obviously be not to receive spam tickets, but I'm afraid reality disagrees ;-) [20:57:06] because migration will be almost impossible [20:57:42] can't we install at least a spam filter to reduce the spam tickets? [20:58:19] - using a script for automated restoring of missing tickets via dumps [20:59:11] we have an offer from Heinlein to give all openSUSE Members a mailbox.org mailbox [20:59:34] this includes a spam filter (actually blocker) for the other @opensuse.org addresses like admin@ [20:59:37] cboltz: yeah this solution would also make the spam situation a lot better. but... whats the status about it? [21:00:11] it's waiting for the "cleanup" of inactive members [21:00:46] I don't remember the exact grace time we gave them to respond with an "I'm still active" mail [21:01:07] how long will it take to make the cleanup? just that i have a idea [21:01:13] am back. [21:01:13] IIRC it should be done in one or two months [21:01:52] wait? what? [21:01:56] 20:59:10< cboltz> we have an offer from Heinlein to give all openSUSE Members a mailbox.org mailbox [21:01:59] ??? [21:02:13] two months of looking a little bit deeper in spam tickets before killing them should be ok for us then. we can endure this ;) [21:02:36] careful with a spam filter for everyone@o.o it has to be very configurable. [21:02:42] https://media.ccc.de/v/gpn16-7582-run_your_own_fucking_infrastructure what is this channel about? [21:03:03] about moving infrastructure to "the cloud" [21:03:08] ok [21:03:11] .... -.- [21:03:49] what is heinlein? [21:04:02] tampakrap: heinlein is the provider of mailbox.org [21:04:04] thomic: I get your point, but it is a big improvement over the current forward-only mail address we offer [21:04:05] an IT provider [21:04:10] Our sponsor heinlein [21:04:11] https://heinlein-support.de/ [21:04:20] yay one of our sponsors [21:04:31] fine if it's about money or hardware [21:04:39] but their admins control our mailboxes? [21:04:42] hrm [21:04:47] for admin@o.o I think a challenge-respnse process would be the easiest [21:05:08] thomic: if I had to choose _one_ company in germany to do mail, it would probably be heinlein ;-) [21:05:19] yay [21:05:20] goodie [21:05:22] (and I say that as someone who runs several mailservers ;-) [21:05:24] yes [21:05:40] i also can provide other providers which are cool [21:05:42] cboltz: if I had to choose _one_ company in germany to do opensuse.org mail, it would definitely be suse [21:05:48] :D [21:06:01] agree with theo [21:06:12] the thing with heinlein was decided by the board right? [21:06:22] mails are sensitive, I don't want them out [21:06:29] decided?? [21:06:31] tampakrap: I see your point, but reality shows that suse doesn't have a working spamfilter ;-) [21:07:01] wait wot? cboltz? [21:07:06] i have a suse mailaccount [21:07:12] which has a really nice spamfilter [21:07:13] :D [21:07:24] maybe we should setup something similar for o.o [21:07:32] instead of complaining about stuff :) [21:08:08] opensuse.org mails go through mx2.suse.de [21:08:15] however - if it is a boards' decision - boardmembers should now raise hands [21:08:17] so I'd expect that the spamfilter also works for admin@o.o [21:08:43] cboltz: mx is the first step - before mail reaches my mailbox - there is a lot more behind [21:09:09] we heard about heinlein at the admin meeting last December, so before Sarah and I were elected to the board [21:09:18] ah k [21:09:19] so [21:09:21] nevertheless, I can bring up your concerns at the next board meeting [21:09:24] please clarify this [21:09:30] with rich & others [21:09:43] if boards wants to "outsource" mail services - i'm fine [21:10:00] but than .. heinlein SUPPORT should as well take the ticket queue for this ;) [21:10:16] because - nothing sucks more than playing proxy-admin to an external company [21:10:18] yes please. because i also got the information when i started on the opensuse topic that i have not to worry about spam b.c we go to heinlein. i also know that richard knows about that [21:10:21] of course [21:10:30] ;-) [21:10:46] i mean its fine [21:10:54] but maybe somebody should talk about that [21:10:59] in case of DNS foo [21:11:05] in case of - how to setup forwardings [21:11:09] in case of migration [21:11:11] etc. [21:11:24] and maybe somebody should inform the community and talk to lawyers [21:11:38] if we (as opensuse project) have licenses and agreements with our users [21:11:57] we need -by german law- an extra agreement if we give out userdata to 3rd parties [21:12:00] like heinlein [21:12:10] has board ever taken considerations like that? :D [21:12:21] it takes month to get this clarified correctly [21:12:48] * cboltz has a feeling that forwarding this meeting log to the board ML is better than just talking about it in the next meeting [21:13:08] and now i don't start talking about - what is with SUSE employees which are as well community member - does there is there an agreement by work council needed when we have a new 3rd party provider - and a security check? [21:13:13] :D [21:13:25] cboltz: feel free [21:13:33] I started a conversation with the board regarding an updated privacy policy to cover the mailing lists and the future mail [21:13:36] I don't want to be the "no-go" man [21:13:41] and I have no idea about the status [21:13:58] but there needs to be proper preparation to get such a big change handled by the organization [21:14:33] thomic: you brought up valid concerns, nobody will be mad at you ;-) [21:14:49] (actually it's a good thing to discuss this now instead of "too late") [21:15:13] *happy* to work in a bug-report friendly community :) - ah .. reminds me i need to write another bug report ... *please go on with meeting* [21:15:20] so since there is so much other stuff on the topic: spam for now -> still manually [21:15:58] i would say - for now ... keep the workflow going .. but consider to read (even 'spam') a bit more careful before deleting [21:16:08] second - clarify mail situation with board [21:16:16] yes but restoring deleted tickets is something I would like to have as Ada_Lovelace proposed [21:16:32] third - if mail is decided - maybe A) problem solves by heinlein-Awesome-Filterness B) think about further steps [21:16:49] if a tickets get deleted and we are aware of it, we can get it back because the mail is still in the admin.o.o inbox. right? [21:16:55] tampakrap: i would like to see this fixed in upstream before scripting on restore-scripts [21:17:08] upstream redmine is slow [21:17:13] mmaher_home: not the full conversation inside the ticket - thats the problem [21:17:14] we need to take actions here [21:17:23] thomic: the upstream feature request is about 10 years old IIRC... [21:17:33] thomic: just the initial one. thats enough? [21:17:44] cboltz: yup, i posted it first on the ML :D [21:18:00] mmaher_home: the initial one is not enough [21:18:06] think about following situation [21:18:11] There is a perl script for ticket restoring in the feature request. [21:18:44] what about somebody opening tickets on the page [21:18:53] or an admin does modification or comments [21:18:58] and then it gets deleted [21:19:41] IMHO the best solution is to subscribe to all tickets (there's an option on the "my account" page to set mail notifications "for all events in all my projects") [21:19:51] you'll get quite some mails, so filter them into a folder [21:20:09] if a ticket gets accidently deleted, this mail folder is your personal backup ;-) [21:20:23] not a bad idea [21:21:26] I have that subscription since December, just in case you need a backup of an older ticket ;-) [21:21:27] ok .. is an idea [21:23:34] so, to sum it up: [21:23:56] - we'll keep our workflow and are more careful before/when deleting a spam ticket [21:24:13] - I'll bring up the mail concerns to the board [21:25:26] anything else on this topic? [21:25:30] yes and if someone wants to play on a deletion recovery system/script, feel free [21:25:34] I'd love to have it [21:26:12] agreed [21:26:22] + [21:26:25] agreed [21:26:32] +1 [21:27:03] having a "real" undelete in redmine would be even better, but that's probably a bigger change - otherwise someone would already have done it ;-) [21:27:15] (but still, if someone is looking for his/her next hackweek project... ;-) [21:28:13] The base is available... [21:30:38] any other topic? [21:30:53] http://www.redmine.org/issues/1380#note-24 [21:31:20] That should be all for today... [21:31:51] gotta go. I won't make the next weeting, only just back from Greece and 1 August is our national holiday. Will likely be busy drinking beer .... [21:32:44] how sad... ;-) [21:33:18] thanks to everybody who joined the meeting, and also thanks to everybody who helped and will help with the wiki update!