2018-06-05 heroes meeting [20:03:50] so let's start [20:03:59] I took some notes for today's meeting [20:04:18] I went through all of the tickets, I closed many obsolete, I assigned a few unassigned [20:04:26] and there are a few that I'd like to bring up for discussion today [20:04:27] i noticed [20:04:45] any RED once tickets ? [20:04:48] * cboltz also enjoyed the mail flood [20:04:51] so if there's nothing else from anybody to talk, I'd like to start with the tickets cause they are plenty [20:04:58] mcaj_nb: they are of all colors [20:05:06] :D [20:05:28] so anything else to talk before I start with the tickets? [20:05:58] what about the Q&A from the community? [20:06:29] so if someone has a question, please ask *now* ;-) [20:06:47] Q: Why do I always stumble across this channel during meetings? [20:06:52] apart from knurpht's today's mail, which is also a ticket, I idn't see anything else [20:07:34] lcp: meeting has just started, do you want to ask something? [20:07:42] lcp: maybe it's a sign that you should join the heroes? ;-) [20:08:12] I don't think I have any questions yet, maybe I will come up with something later :P [20:08:14] (once your have everything re-designed and get bored) [20:08:18] okay [20:08:27] I'm starting with the older ones [20:08:29] https://progress.opensuse.org/issues/20154 [20:08:31] Oh, no worries, I found more stuff to redesign ;) [20:08:45] * mcaj_nb checking ... [20:08:59] ah sorry I closed that one [20:09:22] https://progress.opensuse.org/issues/20916 anybody willing to set up a backup server? [20:10:20] nobody, I'll take it [20:10:21] backap server doing what exactly? [20:10:22] next https://progress.opensuse.org/issues/23414 [20:10:23] tampakrap: backup bakckup .. [20:10:35] rsnapshot should be easy ;-) [20:10:46] what about one in SUSE - Prague site ? [20:11:08] we don't have dmz in prague [20:12:11] then we should have it and ask for it as "heroes team" ... [20:12:17] sounds like something we should fix, because having the backup at another location would be great [20:13:06] dmz in prague is a big project and it's not going to happen soon [20:13:36] tampakrap: can we push MF-IF a bit ^^ ? [20:13:37] dmz = serfer room ? [20:13:45] What do yu need besides of special firewall rules and a subnet? [20:13:56] disk space [20:14:19] we need an extra internet line and possibly MF-IT will need a new set of networking equipment, so not an easy task [20:14:22] That's the same like in Nuremberg (unimportant which place). [20:15:23] what's the exact problem you need to fix? [20:15:37] a) getting it into the heroes VPN (shouldn't be too hard) or [20:15:48] b) make sure to separate it from the SUSE network [20:15:48] ? [20:15:53] It seems like money for network hardware (switches)... [20:16:30] a) is probably a boring task - just attach it to our VPN [20:16:36] b) is more interesting[tm] [20:16:40] well in Prague SUSE we have one server and one storage (11TB) ready for opensuse ... [20:17:15] whan we need is DMZ to set by MF-IT [20:17:56] okay let's discuss it some other time, we can't solve this issue now [20:18:00] So servers are not used at the moment? [20:18:02] I'll look into it and get back to you [20:18:07] traffics, if we are talking about backup rsync I think will be "not so big" ... [20:18:35] let's move please [20:18:47] https://progress.opensuse.org/issues/23414 mirror ticket, pjessen wanna handle it? [20:18:53] sure [20:19:03] tampakrap: can you contact MF-IT and ask them move forward DMZ project in Prague ... then I'm happy ;) [20:19:34] perfect, assigned to you [20:19:52] https://progress.opensuse.org/issues/25178 this one I don't really understand [20:19:53] * mcaj_nb mcaj has only 15% battery ;) [20:20:00] he wants to register a mirror? [20:20:11] * mcaj_nb checking [20:20:31] yes I think so [20:20:36] "Shibboleth" rings a bell - didn't we have a similar, more verbose ticket for that? [20:20:40] It seems, that he wants to have a mirror. [20:20:51] I think I closed one just like it today or yesterday. [20:21:13] he only wants to mirror , he can use rsync.o.o [20:21:34] indeed, https://progress.opensuse.org/issues/36288 [20:22:13] it's duplicate? [20:22:26] IMHO yes [20:22:46] different people, but same thing [20:23:15] okay can someone mark them as duplicate then please? [20:23:35] okay [20:23:55] next https://progress.opensuse.org/issues/27132 was handled by lars and no feedback from the requester, pjessen wanna take over? [20:24:07] or would you prefer me to close it for lack of feedback? [20:24:09] another one? ha,okay [20:25:06] tampakrap, 6 months, let's close it [20:26:04] I'll let Per decide [20:26:14] next https://progress.opensuse.org/issues/29152 nobody looked at it [20:26:19] pjessen: guess what [20:26:28] hehe [20:26:29] it's a mirroring ticket! [20:26:45] I've not got much, give it me .... [20:26:47] IonutVan_: would you like to do some mirror work to offload pjessen a bit maybe? [20:27:22] tampakrap, I was thinking about the backup server if I have more details [20:27:42] I think I created the Armenia mirror a while back anyway [20:28:00] * mcaj_nb 5% and going sleep soon there is no changer on the garden pub ... [20:28:56] IonutVan_: okay let's discuss after the meeting [20:29:54] next is https://progress.opensuse.org/issues/29817 it was handled by a suse colleague who expressed interest in joining the team, but he vanished [20:30:22] so I'm reassigning to pjessen as well :) [20:30:37] next and more tricky is https://progress.opensuse.org/issues/30904 [20:30:47] anybody willing to investigate it? [20:32:38] * mcaj_nb ups ... no battery ... I will check the log record letter CU ... [20:32:55] It would be grat to have somebody outside from Europe for this ticket... [20:33:42] you mean, someone with his/her ssh key on status2.o.o? ;-) [20:34:06] If we have somebody, but it seemsnot... [20:34:27] * cboltz can add someone's key easily [20:37:03] no volunteers, let's leave it for now [20:37:13] I'll take a look if time permits [20:37:22] next https://progress.opensuse.org/issues/31261 [20:37:26] that's a new mirror? [20:37:59] yes [20:38:01] dunno, havent even seen it [20:38:09] queue it up [20:38:14] * cboltz wonders if this will become a "assign everything to pjessen" meeting [20:38:20] okay [20:38:25] me too [20:38:26] next https://progress.opensuse.org/issues/31936 [20:38:37] anyone willing to debug keyserver? [20:38:37] didnt know where so many open mirror tickets [20:39:05] it is reproducible [20:39:51] none, goes to me as well [20:40:33] next is https://progress.opensuse.org/issues/32557 and I don't really understand it [20:41:24] As requested, I marked the above two as duplicate (shibbolet ...) #25178 and #36288 [20:42:02] #32557 looks like rsync.o.o serves outdated content [20:42:20] can you check if it is still happening? [20:42:27] on a probably related note - monitoring says that /srv/ftp is 100% full [20:43:29] on rsync.o.o or on provo-mirror? [20:43:43] both :-( [20:44:12] sigh [20:44:23] for rsync.o.o, rsync still only lists things up to v5.0, so the ticket is still valid (rsync.o.o doesn't have v5.1) [20:45:48] okay [20:45:50] I'll take it [20:46:06] next is https://progress.opensuse.org/issues/36706 that I also don't understand [20:47:08] wrong checksum of the mirror... [20:47:41] I have at least one other ticket with these L3T guys. I don't understand the checksum problm either [20:48:20] network issue in the company? [20:49:02] what's the other ticket? [20:49:10] they mention SMT - isn't that the "local SLE mirror" tool? [20:49:33] if so, I wonder if "Checksum mismatch" could translate to "unknown key"? [20:49:38] is subscription management tool [20:49:53] (just a wild guess, the only thing I know about SMT is its name ;-) [20:49:53] yes thus my confusion, they confuse different stuff [20:50:43] pjessen: do you have ticket number of the other ticket from them? [20:50:54] I thing the interesting questions to start with are a) which repo do they mirror and b) how do they mirror it? [20:51:03] not right now, no. [20:51:28] okay send it to me when you have it and I'll try to make some sense out of it [20:51:49] okay [20:52:16] maybe https://progress.opensuse.org/issues/36661 - but it's about a (local?) mirror [20:52:24] https://progress.opensuse.org/issues/32407 and https://progress.opensuse.org/issues/36598 [20:52:28] those two are duplicates? [20:52:43] not sure, just from the same organistion [20:52:54] wait [20:53:44] they dont look like dupes [20:54:11] tampakrap: the two provo-mirror tickets are completely different - one is about disk full + outdated, the other one reports problems with a specific user agent [20:54:31] I also see them as different :) [20:55:46] okay I took the disk full one [20:55:49] any takers for the other? [20:57:11] I can reproduce the hanging request, but I'm afraid I'm too busy to debug it [20:57:46] okay [20:57:49] I'll take it [20:57:55] three monitoring tickets are next [20:57:57] https://progress.opensuse.org/issues/30036 [20:57:59] https://progress.opensuse.org/issues/32905 [20:58:01] https://progress.opensuse.org/issues/32914 [20:58:03] Ada_Lovelace: ^^ yours? [20:59:57] until she replies [21:00:00] https://progress.opensuse.org/issues/35719 cboltz yours? [21:00:10] you already started discussion about it [21:00:11] I had to do a lot of things for the university in the last months. [21:00:20] Yes. They are my ones. [21:01:11] tampakrap: the problem with the favicon.ico on www.o.o is that it already exists, but isn't in git [21:01:34] so whenever the new favicon gets added to git, someone will probably have to resolve the git conflict [21:01:50] I'm not sure if I can login on that server [21:02:22] https://github.com/openSUSE/landing-page/blob/master/assets/images/favicon.png [21:02:31] is that one the file you mention? [21:02:45] we are talking about favicon.*ico* which is a different file [21:02:55] (not in git, but exists on the www.o.o server) [21:02:55] That has to be a favicon.ico [21:03:12] is not in git, but it is on server [21:03:37] cboltz: then, could you handle adding it to git? I'll then talk to mf-it to remove it from the local machine and fix the conflict [21:03:53] or did I get it wrong? [21:04:09] tampakrap: ideally you should have/get access to that server yourself ;-) [21:04:24] cboltz: I already requested, not sure how long it will take [21:05:18] try asking Richard or Max - AFAIK their ssh keys shoud be on that server (unless they got removed again) [21:06:10] the problem here is timing - when the favicon.ico gets added to git, it will probably break the git pull cronjob [21:06:11] I did, it didn't help [21:06:32] so having to wait for MF-IT to resolve the git conflict doesn't sound like the best solution to me :-( [21:06:36] so I filed a ticket and now I'm waiting [21:06:54] IMHO we should wait with the favicon until you have access to the server [21:07:07] okay [21:07:54] I'll add a short notice to the ticket and assign it for you - please add a note as soon as you have access to the server [21:08:18] cboltz: I did already [21:08:33] next https://progress.opensuse.org/issues/36213 cboltz: I don't know anything more than you about this module [21:08:35] even better :_) [21:08:52] the only thing I could tell you is that your link points to 2.2 doc, while we have 2.4 if I'm not mistaken [21:09:01] i doubt it's going to make much of a difference though [21:09:22] no difference, I get the same result on a 42.3 test VM [21:10:02] so - is someone here who knows apache mod_negotiation better than I know it? [21:10:27] i know it pretty well, but im busy :-) [21:10:37] is this ab out the release notes? [21:10:50] it's about countdown.o.o [21:10:55] oh [21:11:04] The counter was used for the release day. [21:11:29] I know, that Victor had the task to change it last year... [21:11:41] the problem is basically that if your browser sends Accept-Language: de,en and you do an explicit request for https://countdown.opensuse.org/small-label.pl.png you'll get 406 Not Acceptable [21:12:55] So Polnish does not work... [21:13:17] the problem is that "pl" is not part of "de,en" [21:13:47] pl is just an example, you'll see the same for other languages that are not in your Accept-Language header [21:15:24] there is an apache config for default language order [21:16:04] https://countdown.opensuse.org/picker shows them all [21:16:11] pjessen, I think there should be as well a LanguagePriority somewhere [21:16:27] LanguagePriority, exactly [21:16:49] and ForceLanguagePriority Prefer Fallbac [21:17:25] https://httpd.apache.org/docs/2.4/mod/mod_negotiation.html bottom of this page [21:18:13] our apache config is in https://gitlab.infra.opensuse.org/infra/salt/merge_requests/197/diffs#e95a3f1c283cc076fdb307f53e29ae9e33c1caba [21:18:19] "ForceLanguagePriority Fallback uses LanguagePriority to serve a valid result, rather than returning an HTTP result 406 (NOT ACCEPTABLE)." [21:18:50] cboltz, I don't have the VPN setup yet, I can not access gitlab.infra :) [21:19:36] try http://paste.opensuse.org/0c231696 ;-) [21:21:45] IonutVan_: wanna take the ticket? [21:22:27] tampakrap, I don't know where to start :), but is not something critical if I can not do it, so I will try [21:23:04] cool [21:23:19] IonutVan_: if you want, I can give you a fully setup test VM for testing [21:23:42] Release Day is the past. So you can't damage anything for a new release at the moment. [21:23:55] Ada_Lovelace, exactly :) [21:24:12] next is https://progress.opensuse.org/issues/36766 [21:24:36] cboltz, I will keep in mind and I will reach out to you when I work on it [21:28:11] for the tumbleweed snapshots, the basic questions are already in the ticket: [21:28:32] - hosted on pontifex (possibly with mirrorbrain integration) or on a separate VM? [21:28:47] - with or without debug repo (-> disk space requirements) [21:28:54] - use hardlinks or redirects? [21:28:59] Knurpht told something about AWS... [21:29:56] right, at the moment it's hosted on AWS [21:31:01] So it is using EC2 or based on it with docker. [21:31:54] I don't know the technical details ;-) [21:32:27] the more interesting question IMHO is if pontifex can survive another 500 GB (or 1 TB if we include debug repos) [21:32:47] You can choose between those technologies on AWS. In addition S3 as storage and hopefully a backup technology in this case. [21:33:23] So that we can migrate away from AWS. We don't want to have 1x AWS - all the time AWS... [21:33:51] I gave an AWS workshop last month... [21:34:43] I don't know either if we can handle the network load and the disk space [21:34:47] I'll need to investigate [21:35:13] Yes. snapshots are on S3... [21:35:21] We should sync it. [21:35:30] so I'd say let's skip it for now, and let's resume on the next team meeting with more info [21:35:39] I'll have a chat with him as well meanwhile [21:35:42] sounds good? [21:35:54] yes [21:36:20] Yes. It seems, snapshots exist on S3. Our info. But you need access. ;) [21:36:30] okay last thing from me is that in case you didn't know Christian Mueller left from SUSE [21:36:41] he had the ticket regarding the cdn77 service [21:36:47] anyone willing to investigate that service? [21:37:16] silly question - what exactly does "investigate" mean? [21:37:36] find out if it might be useful? [21:37:42] AFAIK cdn77 is a CDN, so basically we'll have to add them as mirror and upload our files to them? [21:37:52] we wanted to see how it works and if it covers our needs to replace static.o.o at first, and even some mirrors stuff for later [21:38:41] I'm not sure if it's worth the effort for static.o.o - does it really cause that much traffic that it needs a CDN? [21:38:55] I'd focus on mirror usage [21:38:57] i was thinking the same [21:40:27] it's still one service less for us though [21:40:50] for free [21:41:05] the only "difficult" part for static.o.o is to get everything into one git repo [21:41:30] when this is done, setting up a VM with salt is a boring job, and I'll happily do that ;-) [21:42:40] yes [21:42:44] so cdn77 goes to me as well [21:42:49] that's all from my side [21:43:12] there are more unassigned tickets but not that urgent [21:43:41] @cboltz and I have to change to the Board Meeting in 15 minutes. ;) [21:44:29] since you'll do that, please check if there are news regarding the privacy policy and the heinlein [21:44:30] getting heroes tickets done would be a good excuse for being late ;-) [21:45:04] We have news about that from the last meeting. [21:45:19] A SUSE lawyer should review it... [21:46:29] right, I still have to write that mail - thanks for the reminder! [21:46:48] cool [21:46:52] about heinlein no news? [21:47:37] Richard recommended asking a lawyer for that too because handling mails externally has possible privacy implications [21:47:46] That was a SUSE internal discussion after the attack against me because of Max's mistakes... [21:48:35] cool [21:48:46] so anything else or can we close the meeting? [21:48:50] feel free to assign the Heinlein ticket to me - I'll ask Ciaran about the legal stuff [21:48:57] We can close it. [21:49:09] I don't think we have a ticket [21:50:17] ok, I'll handle it anyway and report back once I have an answer [21:50:18] lcp: would you like to handle this? https://progress.opensuse.org/issues/9342 [21:50:26] www.o.o is missing a 404 page [21:50:48] I would like to make all at once [21:50:57] and I would like to do that after static is up [21:52:04] okay [21:58:32] i guess we are done for tonight? [21:58:42] sure [21:59:33] okay, I guess I have some tickets to work tomorrow. see you later. [22:00:33] see you [22:00:40] * cboltz switches over to the board meeting [22:16:45] lcp / cboltz: https://github.com/tampakrap/static.opensuse.org please review before we move it to the official github repo and to production [22:17:08] lcp / cboltz: there's a NOTES file inside with the repos that it had on the server, I ran git pull in all of them [22:17:33] `This repository is empty` [22:17:39] Is it privated? [22:17:53] it is still uploading :) [22:18:09] ah [22:18:12] I was too quick [22:18:16] now it's done [22:22:13] cboltz / lcp: and let's continue the discussion on https://progress.opensuse.org/issues/30322 please [22:23:06] this is amazing [22:23:09] I had no clue so much was still on static [22:23:13] Like original openSUSE Wiki theme and stuff [22:33:32] tampakrap: It's fine, my guess is that majority of that stuff is no longer used anywhere on the internet and will have to be removed, but good thing is that we will be able to start from current state :D [22:35:33] I would guess that some of this stuff should be git modules and not stuff downloaded like that [22:46:21] git submodules are a pain to maintain, I would prefer a large repository instead