2018-04-03 #opensuse-admin Heroes meeting [19:59:13] * tampakrap will brb in 5 mins [20:01:46] Hi everybody! [20:01:56] Hi [20:02:38] while waiting for tampakrap, let's start the Heroes meeting with a topic where we don't necessarily need him ;-) [20:02:42] Questions and answers from the community [20:03:06] does someone have a question? ;-) [20:03:37] hi people [20:03:43] thomic is on vacation [20:03:48] pjessen: around? [20:04:25] yup [20:04:33] i'm here [20:04:39] cool [20:05:20] cboltz: link from the ticket please? [20:05:26] https://progress.opensuse.org/issues/34135 [20:05:52] so one question we have from the community is the question from Ana regarding planet [20:06:30] https://github.com/openSUSE/planet.opensuse.org/issues [20:06:47] and also this PR https://github.com/openSUSE/planet.opensuse.org/pull/67 [20:07:48] that's all my "fault" ;-) [20:07:55] and yet another issue I saw is that rawdog is still python2, we're going to have issue when we'll want to update the machine to leap15 [20:07:57] I only asked her why her blog isn't on planet, and that's the result ;_) [20:08:37] nice :) [20:09:12] tampakrap: yes, still/again here [20:09:26] DimStar: pm to not interrupt the meeting [20:09:31] it seems Leap 15 still contains python 2, so I'd hope that rawdog will continue to work [20:10:42] it will contain full python2? [20:10:49] then we're fine [20:10:52] at least in theory [20:11:06] I only checked osc ls openSUSE:Leap:15.0 python ;-) [20:11:48] but if the core is there, I'd expect that all the python2-* packages will also be there [20:12:28] okay [20:12:42] https://github.com/openSUSE/planet.opensuse.org/issues/69 this one is interesting where lcp created already a new design [20:13:24] so from a quick look, you replied to the local setup question (waiting for feedback) [20:13:25] He is creating designs for all systems. [20:13:44] cboltz has a easter egg in our wiki with that. ;) [20:13:47] the question about removing inactive blogs is pending replies to your mail to the community [20:13:57] and for the design related questions I don't have an opinion [20:14:12] so it should be fine from our side for now, correct? [20:14:12] that easter egg is from Guo Yunhe ;-) [20:14:37] Sorry [20:14:52] no problem ;-) [20:15:18] *I wish there was no python2 there :/* [20:15:25] It's such a pain [20:16:09] I'll be happy if we don't need it (and everythong works with py3) [20:16:15] lcp: I'm not aware of a python3 planet software though [20:16:32] but if rawdog still needs py2, I prefer the pain over breaking planet.o.o ;-) [20:16:38] I'm not aware of any other good planet software in facr [20:16:42] fact* [20:16:45] I looked around :/ [20:16:52] https://github.com/rubys/venus this is what I was using in gentoo, and I see it is still python2 as well [20:17:09] and there are alts with php and stuff [20:17:15] no :/ [20:17:36] as much as I hate python 2, I prefer it over php [20:17:53] yep so let's keep it I'd say [20:18:06] for the design I personally have no opinion at all [20:18:20] ditto [20:18:22] oh come on [20:18:25] be harsh [20:18:42] it's easier to develop with critisizm [20:19:17] ah, that I can do [20:19:25] do you have it uploaded somewhere? [20:19:38] not yet, it barely works :/ [20:19:53] I will have it later when I get back to it [20:20:01] okay ping me if you want to have a planet-test.o.o instance at some point [20:20:39] when you put it online, make sure to mention that in the ticket - if you are lucky, Ana has some time to help ;-) [20:21:26] great :D [20:21:55] yeah I'd say collaborate with each other already [20:21:55] looking at the scresnshot you added to the ticket - maybe reduce the height of the header a bit? [20:22:18] whitespace can be important, but too much whitespace can also mean "wasting" space ;-) [20:24:13] that said - I'm looking forward to have the new design on planet :-) [20:25:17] can/should we continue with the next topic? [20:25:23] status reports about everything [20:25:35] I'll start with a small one [20:26:16] I tested our sponsors.o.o idea with nginx (redirecting to a random image/sponsor logo in a directory), and it works as expected :-) [20:26:48] (actually I already have the salt code for it because that was easier than doing a manual setup ;-) [20:26:56] oh cool! [20:27:04] double cool for salt! [20:28:14] I also did some very early steps for a Leap 15 JeOS image, but at the moment it fails with "unresolvable" [20:28:39] so I'll need to spend some more time on it ;-) [20:28:59] if someone is interested nevertheless - https://build.opensuse.org/package/show/home:cboltz:openSUSE:infrastructure:Images:openSUSE_Leap_15.0/jeos [20:29:35] I'll check it yes [20:30:11] that's it from me ;-) [20:30:16] who else has status reports? [20:30:45] nothing fascinating from my side [20:30:54] not much from me either. [20:31:16] mailman test setup is in place, but i have not had time to continue [20:31:39] pjessen: any idea about those rsync.o.o tickets? [20:31:51] the ones about emtpy hotstuff? [20:31:53] haven't checked them, I just recall getting tickets again [20:32:05] maybe, let me check [20:32:19] I did look at it before easter, but I got distracted [20:32:33] https://progress.opensuse.org/issues/34030 [20:32:33] basically all hotstuff packages are 0 size. [20:32:42] that's different [20:33:07] havent even seen that one [20:34:36] okay [20:35:00] mirroring - I nee dhelp with figuring out an issue wrt tumbleweed [20:35:20] what's the issue? [20:35:51] our own mirror (hostsuise) is distributing lots of tumblweed, but mirror.o.o says we dont' [20:36:02] don't have it [20:36:32] mb mirrorlist says no, but mirrorbrain still redirects to us [20:36:56] smells like ipv6 glitch [20:37:35] yes, that is what my rsync log says too. It fails, then works a minute later, when the scanner tries the ipv4 address [20:38:09] okay I'll try my best [20:38:17] lets discuss it tomorrow [20:38:49] but do send me some hints about what you think might be worth looking at. [20:38:56] sure [20:39:49] so anything else or should we move? [20:39:58] otherwise- external monitoring - a new guest is on my TODO [20:40:03] cboltz: tall header is gonna be there only on first page I hope [20:40:28] still on my TODO. that's all from me for now. [20:41:35] lcp: ok, I'll check when you have something working so that I can check the whole picture ;-) [20:41:50] https://imgur.com/a/enORI [20:42:09] yeeeah, I need to start it for real :/ [20:44:01] does someone dare to close https://progress.opensuse.org/issues/34051 ? (or, better question, can someone load it without getting a gateway timeout? ;-) [20:44:18] is that the one with 500 mails? [20:44:23] right ;-) [20:44:48] yeah, sorry about that. took me a little too long to figure out. [20:44:54] checking [20:45:37] pjessen: no problem, such things happen ;-) [20:46:00] actually it's somewhat entertaining, at least if you [try to] look at it a few days later ;-) [20:46:00] it times out [20:46:35] hmm, maybe it's possible to close it with the mass handling on the ticket list page? let me try... ;-) [20:47:29] looks like it worked [20:47:36] wow [20:47:43] we'll see how many mails this triggers *eg* [20:48:09] both lists have the right control settings now, [20:49:16] delete it, don't close it please [20:50:00] maybe I should just parse output from rawdog to jekyll, my life would be so much easier :/ [20:50:08] tampakrap: too late, I already closed it ;-) - but it looks pjessen's changed settings helped [20:50:19] at least I don't see a bounce "war" ;-) [20:50:40] phew. [20:50:42] yeah the trick usually is to remove the mailing list from the watchers' list [20:51:31] ... which won't work if loading the ticket times out ;-) [20:51:44] yep [20:53:14] so anythin else or we can move? [20:53:26] lets move on [20:53:37] we got some tickets about spam on paste.o.o [20:54:09] yes, I mailed michal about this, and I escalated the issue to sysrich as well [20:54:09] who has access to it to delete that spam? [20:54:15] no reply so far [20:54:24] only michal has access, it is hosted in his private server [20:55:24] we should probably change that - AFAIK michal is very busy [20:55:47] so moving it to a VM where multiple people have access would be helpful ;-) [20:56:01] might be good with an improved auth mechanism. spammers will continue. [20:56:22] that's a long term solution [20:56:25] right, but one step after the other ;-) [20:56:29] the short term solutions that we proposed are: [20:56:39] 1) remove the sign-up button from top right corner [20:56:57] 2) give me admin access [20:57:15] then we can discuss about moving the service or even replacing the backend with something better [20:57:26] this one is a custom made from michal [20:58:12] according to the footer, it's based on Stikked, so at least not completely custom made [20:58:32] (I don't know how much is upstream and how much customized) [20:58:34] ah didn't notice that [20:58:56] thorsten found another one [20:59:04] I'll try to find it, sec [20:59:14] https://github.com/openSUSE/paste latest update to backend was 10 years ago from what I recall [20:59:40] I looked into it few months back [21:00:04] I think it was this one https://github.com/PrivateBin/PrivateBin [21:01:01] lcp: wow, sounds like up to date code :-/ (to be fair, the latest commit was 6 years ago, but that's not really better) [21:01:04] any changes to it will have to mean changing susepaste and susepaste-screenshot packages in openSUSE :P [21:02:12] that, and we'll also have to ensure that existing content will still be available under the known URLs [21:03:17] doesnt the content expire fairly quickly? [21:03:39] depends on what you choose when creating a paste [21:03:47] the current default is a week [21:03:51] https://susepaste.org/lists [21:04:01] here are all the current pastes [21:04:04] but IIRC the old default (like two years ago) was to keep forever [21:04:13] well "all" some might be private [21:05:16] a quick search shows https://github.com/claudehohl/Stikked so if we are lucky, we can "just" upgrade to the latest version [21:05:27] (yes, I know why I put quotes around "just" ;-) [21:07:08] It really should get new looks :P [21:08:11] the first step is to get admin access, then we can start with moving and updating it, and also give it a new look [21:08:54] *I have planet to take care of anyway* [21:09:13] ;-) [21:09:49] should we continue with the next topic? [21:09:53] oSC 18 [21:10:04] yes who is visiting me? [21:10:26] at home, or at oSC? ;-) [21:10:46] yes [21:11:30] * cboltz assumes the "yes" is the answer for "at home", and hopes tampakrap will have some cake ready [21:11:45] cake? [21:11:58] and cookies [21:12:36] I m not yet certain I'll attend, but this is beginning to sound tempting. [21:12:39] my girlfriend will make them, I don't want ot poison you [21:13:05] I have submitte my presentation about the salt structure and testing [21:13:24] so nobody will come? you'll leave me all alone there? [21:13:55] do you seriously think I'll miss oSC? ;-) [21:14:31] perfect [21:14:38] anybody else? [21:14:39] I know you'll need someone to ask evil questions in your talk, and I might be that one ;-) [21:14:51] cboltz: any presentation? [21:15:21] I didn't have a good idea for a Heroes-related presentation yet [21:15:50] I could come up with some funny stories about the wiki move, but IMHO it would be a year too late [21:16:09] (unfortunately we had too many proposals last year, so it was declined) [21:17:10] so if you have a good idea what I could talk about, tell me ;-) [21:17:28] I don't think it is late, because last year it wasn't even done [21:18:06] right, but all the "funny" things about the spam attack, getting access in Provo etc. are quite old [21:18:28] still, it's a story [21:18:31] and a talk describing "just" the move would be too technical and probably boring for most people [21:18:34] it's up to you though [21:18:51] it can be a short talk for 30 mins [21:18:57] mine will be short as well [21:21:06] hmm, just checking grepsig opensuse-admin maybe I should do a lightning talk about "what the Heroes would prefer not to tell you" ;-)) [21:21:37] like what? [21:22:00] like "nobody knows why it is down or when it will be up again"? [21:22:07] basically some funny failures [21:22:31] and/or funny quotes [21:23:06] yes I approve [21:23:28] you will regret that... [21:23:53] challenge accepted [21:25:43] so let's move, I don't see much enthusiasm about osc :) [21:26:14] there will surely be much more enthusiasm about the 10 oldest tickets [21:26:21] as I said, you will regret that - but since you are probably the main "victim" (you did quite some stuff, so of course you also broke some things) I'll check what I find, and will probably submit that talk ;-) [21:27:34] pjessen: indeed, you'll for example enjoy https://progress.opensuse.org/issues/4314 ;-) [21:28:28] this will need to redesign the search system [21:28:37] it's not an easy task [21:28:37] quite remarkably, I almost recruited a volunteer for that. [21:29:00] yeah. and I still wonder if google isn't good enough/ [21:29:12] or _why_ google isn't good enough [21:30:18] the local search might have some advantages if you want to search only in one list and in a gived date range (basically "I'm sure this was discussed on $list in $month $year") [21:30:32] but besides that, I agree that in most cases google will do the job [21:31:44] my feeling too. that's why I have not been trying to do much. [21:32:43] maybe it would be better (for next time), if someone would prepare a quick status of the 10 oldest tickets. [21:33:13] going through them one by one is a bit tedious [21:33:26] well, the plan/hope was that we can get some of the oldest tickets fixed/closed during the meeting ;-) [21:33:38] correct but next time I'll be on vacation, I won't join the meeting [21:33:49] (that obviously won't work for a big task like the ML search) [21:34:02] and BTW: this is the third-oldest ticket, and the good thing is that it's "only" a year old. When I got access to progress, there were more tickets with quite some dust on them [21:34:23] I'm looking at them now, and it takes me too long to grasp each issue enough to have an opinion/ [21:34:44] https://progress.opensuse.org/issues/9342 looks like MF-IT issue [21:35:50] https://progress.opensuse.org/issues/15268 is assigned to max - who was going to look into CDN77 ? [21:36:09] sorry, not the 10-oldest [21:36:27] tampakrap: that, or move www.o.o to Heroes infrastructure. The "problem" is that it includes openID auth, otherwise it would be an easy task [21:37:34] pjessen: at the offsite, cmueller picked up that task, so please reassign [21:38:08] (and don't worry too much about ignoring the 10-oldest restriction ;-) [21:38:34] thanks, done. [21:38:48] hi guys, again late to the game. 2000 couldn't work for me. I still want to be a valuable heroes member but so far I do not consider myself one. I never found the motivation to activate the openSUSE VPN access on my computer. I will ask asmorodskyi to help me with that - the motivation part as well as the technical ;) cwh made me an admin of the redmine instance on progress.o.o and I found some interesting settings but did [21:38:49] not change anything. [21:39:19] tampakrap: I still plan to help with progress.o.o updating and a bit of redmine administration but it has to go with low prio for me [21:39:55] sure [21:40:23] btw, anyone of you that can help with openqaworker1 being down? [21:40:43] it should be 192.168.112.100 if "ping openqaworker1" from ariel is correct [21:42:49] cboltz: what about this? https://progress.opensuse.org/issues/13296 [21:43:02] and because python2 vs. python3 has been mentioned: The plan for SLE15 was to move all core tools to python3 so that python2 can be moved to the legacy module. For Leap this means: python2 will for sure still be around for the time being but it's a good time to care about python3 support :) [21:43:18] okurz: spoke with DimStar already, I don't have access either [21:43:41] do you know who has? Should I raise a ticket elsewhere? [21:44:00] okurz: DimStar will do it [21:45:13] tampakrap: waiting for the new design everywhere will automagically solve it ;-) but if you point me to a git repo that hosts the navbar with the outdated link, I can fix it [21:45:54] (the actual fix is easy, I just need to know where the file hides ;-) [21:46:34] where is that bar? [21:46:42] I'm on www.o.o and can't see any tube link [21:47:26] same for software.o.o [21:47:30] right, www.o.o already has the new design, which means no more outdated links [21:47:34] they only have links to opensusetv [21:47:34] try for example the wiki [21:48:17] or paste.o.o [21:48:25] well [21:48:26] guys, I have to leave, need time to resolve some family matters. see you later. [21:48:32] the easiest would be to fix it in the proxy [21:48:41] put a redirect of tube.o.o to the opensusetv [21:48:56] I have no idea where that bar comes from [21:49:04] pjessen: have a nice evening [21:49:17] pjessen: enjoy the evening with your family! [21:49:30] tampa, I'll ping you tomorrow wrt the mirror issue [21:49:36] sure [21:49:45] tampakrap: AFAIK the bar is hosted on static.o.o as part of the bento theme [21:50:24] still, fixing it in proxy is a good start [21:50:31] doing a redirect in haproxy might also be a solution [21:51:08] the only problem I see is that tube.o.o currently points to widehat [21:51:29] I seriously hope not too many people abuse it as an alias for widehat or download.o.o, but you never know ;-) [21:51:35] I can change it to point to the proxy [21:53:25] right, a DNS change is easy, but see the possible problem "alias for widehat" which might in worst (hopefully unlikely) case break someone's repos [21:53:42] now it points to the proxy [21:53:42] are we fine with risking such breakage? [21:53:53] yes [21:54:04] if someone is using tube.o.o for his repos, good for him [21:54:13] enjoy downloading youtube videos instead of rpms [21:54:21] lol [21:54:32] I'll add the redirect in the haproxy config [21:56:18] cboltz: https://progress.opensuse.org/issues/18372 what about this? [21:57:36] good question :-/ [21:58:09] I'd say send him mail to put his opensuse category/tag rss only [21:58:28] the original request is valid, planet opensuse is supposed to be about opensuse or at least linux / community related [21:59:26] https://progress.opensuse.org/issues/17702 this one I'll close it, it's a salt design issue [21:59:30] nothing we can fix [21:59:40] upstream is aware btw and not really willing to provide the functionality [22:00:03] PROBLEM: HAProxy on elsa.infra.opensuse.org - HAPROXY CRITICAL - Active service dale is DOWN on dale proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=elsa.infra.opensuse.org&service=HAProxy [22:00:04] PROBLEM: HAProxy on anna.infra.opensuse.org - HAPROXY CRITICAL - Active service dale is DOWN on dale proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=anna.infra.opensuse.org&service=HAProxy [22:00:06] PROBLEM: HAProxy on mufasa.infra.opensuse.org - HAPROXY CRITICAL - Active service riesling is DOWN on riesling proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mufasa.infra.opensuse.org&service=HAProxy [22:00:47] I wonder if there's a way to add a dependency "required for pkg.* states"... [22:01:36] that would also be useful to ensure the zypper config (especially "no recommends") is done before installing packages [22:01:55] that's exactly what the ticket is about [22:02:54] I have to admit that I don't remember all details from this ticket ;-) [22:03:07] so the idea comes from puppet [22:03:22] in puppet you can define stages [22:03:39] there is a base stage, but you can define a stage eg pre or post [22:03:54] and say that everything that belongs to pre stage runs before base [22:04:12] so for example, you can add all the firewall enable rules on pre, and all pkgrepo.* on pre [22:04:16] then it will run the base [22:04:26] and on post you put eg all the firewall block rules [22:04:42] so that way all the pkgrepo.* instances will run before pkg.* [22:04:56] and all firewall block rules will run after the firewall enable rules [22:05:05] so that you don't kick yourself out for example :) [22:05:37] the oldest ticket is https://progress.opensuse.org/issues/1154 [22:05:41] what about https://docs.saltstack.com/en/latest/ref/states/ordering.html [22:05:44] and I think we can close [22:06:11] the ordering stuff you can see there are between defined states [22:06:26] while puppet supports all of pkgrepo.* isntances to run before anything else [22:06:29] see the diff? [22:06:54] yes, I see the diff [22:07:37] and while I don't say it's perfect, order: 1 (or even order: 0 or -1) to ensure a few things run first might be an acceptable workaround [22:08:23] we would need to do it on all pkgrepo.* states though [22:08:32] even the ones that come from eg a formula [22:09:30] if a formula adds a repo (which is a bad idea IMHO), I'd expect that it is only used for packages related to this formula, and that the formula handles the correct order internally [22:09:56] so I'd only care about the zypper config (no-recommends) and our own repos [22:10:22] actually even our own repos aren't important enough - in worst case, we need to run highstate twice [22:10:42] (or do we have newer versions of some packages than Leap has?) [22:11:50] agree [22:12:20] so let's improve the zypper formula [22:12:50] I left a comment on https://progress.opensuse.org/issues/1154 if I get no reply in a week I'll close it [22:13:17] sounds like a good idea (both the ticket comment and improving zypper formula) [22:13:28] cboltz: https://progress.opensuse.org/issues/33685 give him a reply please, I forgot the name again [22:13:32] and close the ticket [22:13:55] ok, will do [22:17:11] https://progress.opensuse.org/issues/9754 this I'll close it [22:17:18] it's really pointless to have such tickets [22:17:39] we might as well open a ticket for every service we want to saltify for no gain :) [22:17:53] agree? [22:18:30] yes, but with a different reason ;-) [22:18:45] we set the IP address in initial_setup.sh to avoid a chicken and egg problem [22:18:53] so salting it is probably pointless anyway [22:19:52] managing resolv.conf might be a good idea _if_ we ever change the nameserver IPs [22:20:05] but since this hopefully won't happen ;-) ignore that for now [22:20:13] yep [22:20:23] I closed also https://progress.opensuse.org/issues/11242 for the same reason [22:21:29] having a working postfix (and if it's only to send out root mails) sounds more useful than salting the IP address ;-) [22:21:43] yep but we don't need the ticket for that [22:21:59] (actually Lars did that in the JeOS image, but that doesn't help for existing VMs) [22:22:22] I think enough for today [22:22:43] for planet - I checked, and it already *is* the opensuse feed... [22:23:20] then a polite warning message to not post irrelevant stuff? [22:24:14] I'll check with google translate if it's really irrelevant ;-) [22:30:20] tube.o.o is now a redirect - ticket closed :-)