2017-05-08 #opensuse-admin Heroes Meeting [20:02:37] Or let's start. [20:02:58] yup [20:03:10] At first: Questions from the community [20:03:40] Have we got new people with questions? [20:03:53] is this a general item or just in case ? [20:04:03] just in case [20:04:19] and "questions" in general item [20:04:54] okay. Then perhaps we ought to start with a listing of whos is attending. [20:05:32] * cboltz hides once more [20:05:37] just raise your hand and say "Im here" [20:05:49] * tampakrap [20:06:45] Then to the second item: status reports about everything (see ticket #17272 / March meeting for a list) [20:06:47] I'm here [20:06:55] Lars :) [20:07:11] Hi! [20:07:14] you really miss me? Wonderful ;-) [20:07:31] looks like I broke not too much ;-) [20:07:38] it was getting a bit lonely here [20:07:40] I was surprised, that you were inactive. [20:08:26] sometimes even I fell inactive [20:08:34] status report? [20:08:41] Yes. [20:08:48] connect.o.o - anyone with news about that beast ? [20:09:08] It lives like it is...? [20:09:32] DNS: up and running (which allowed us to add IPv6 very quickly :-) [20:09:42] Monitoring and status page [20:09:44] good stuff! [20:10:02] I added an additional item for Icinga. [20:10:15] pjessen: I'm looking forward to the openVPN machine, so next time I can leave the solving completely up to you [20:10:37] Ada_Lovelace: where? [20:10:38] We spoke about using Salt Stack for checks. [20:10:55] On our list for today. [20:11:05] haha, what have i committed to? [20:11:15] https://progress.opensuse.org/issues/18456 [20:12:11] Additional to that Lars and I created a wiki page: https://progress.opensuse.org/projects/opensuse-admin-wiki/wiki/Monitoring [20:12:30] Ada_Lovelace: you know that this needs still developement? [20:12:54] What do other admins say about using zeromq in combination with SaltStack, API and Icinga? [20:13:04] Ada_Lovelace: so I would leave the "Using salt for monitoring" out for now - until you implemented the missing features in Salt and Icinga [20:13:06] :-) [20:13:16] Yes, but why not the meaning? [20:13:37] or opinion ^^ [20:13:38] Ada_Lovelace: fine with me - go ahead :-) [20:14:32] Nothing? Thhen we would try it. ^^ [20:15:18] kl_eisbaer1: Anything about the status page additional to that? [20:16:11] if you need help with salt let me know as always [20:16:18] I am currently working on an update to the next version - but need to figure out how to package all the additional vendor php packages comming in via the composer requires [20:16:24] Yes. Thanks! [20:17:28] more= [20:17:30] ? [20:17:31] kl_eisbaer1: I already had some fun with composer when packaging the MediaWiki extensions [20:17:41] cboltz: so where are your packages? [20:17:49] home:cboltz:infra [20:18:05] feel free to steal my script ;-) [20:18:07] cboltz: ok, thanks. I will have a look [20:18:42] (actually, with more users, maybe we should split it into generic script + config file instead of carrying 10 slightly modified copies) [20:19:09] cboltz: which package contains "your script" ? [20:19:57] is there any documentation/script for openSUSE that helps packaging https://packagist.org/ ? [20:19:58] for example mediawiki_1_27-Elastica and mediawiki_1_27-GitHub [20:20:20] * kl_eisbaer1 might need to ask on #opensuse-packaging ;-) [20:21:21] more? [20:21:42] a small bugreport: when I click on "Monitoring page" instead of the "Monitor openSUSE" button on http://monitor.opensuse.org/ a layer with broken content pops up [20:22:56] I doubt we need that layer for additional information, so making everything a link to /icinga/ is probably the easiest solution [20:24:05] cboltz: that "welcome" page was (as always) just a quick hack from me to show what we have [20:24:35] cboltz: IMHO we might discuss about a "project"(?) below github.com/opensuse/ to store simple pages like that [20:25:22] at the moment, that page is just in a local git repo [20:26:11] I'd vote for one repo per page/domain, in this case the ideal name would be monitor.opensuse.org (or monitor.o.o if you want a shorter name) [20:26:29] cboltz: ok, fine with me [20:26:35] simple pages change rarely, and that name makes it easier to find it again ;-) [20:28:31] next item? [20:29:20] ticket triage ? [20:30:34] Max wanted to do a lot for the conference and not tickets. [20:30:41] One note about the monitor.o.o - I just noticed that we have https://github.com/orgs/openSUSE/teams/opensuse-admin-tools already [20:31:19] What is to do there after the last year with the conference in the same location and other events with great internet access? [20:31:36] Ada_Lovelace: ask Max [20:31:59] Ok [20:32:07] I was surprised... [20:32:23] Ada_Lovelace: you are not alone ... [20:32:56] mmaher_: are you near your keyboard, or is only your computer online? ;-) [20:32:58] I'll write him. [20:33:18] cboltz: should we create a "openSUSE heroes" team there or is it ok if we use the openSUSE-admin team ? [20:34:17] I don't care about the name on github too much, so use whatever is easier ;-) [20:35:09] cboltz: ok - I'll take this as action item to prepare something for the next meeting [20:35:29] Next status report? [20:35:33] => new wiki [20:35:56] well, I'm still waiting for the auth proxy [20:36:14] kl_eisbaer1: did you have some time to work on it already? [20:36:29] nope - as I said: I would prefer to have openVPN first [20:36:57] I *love* dependency chains... [20:37:00] yes, openvpn is high priority task now that we also have freeipa [20:37:03] cboltz: if you want to use ldap-auth, this might be a bit easier, but even that is nothing I would do in a minute (firewalling, etc. needs to be done there) [20:37:22] just checking - have I promised something? I've been busy with openVPN in the business anyway. [20:37:47] wrt openVPN [20:38:05] kl_eisbaer1: sounds like I should look at openID again - I'll need to modify the extension to enforce the username, but maybe it isn't too hard [20:38:39] cboltz: so you will become a mediawiki extension author - what a surprise ;-) [20:39:23] pjessen: I guess tampakrap will take you on his list of openVPN contacts ;-) [20:39:25] I hope that I'll only become a one-time patch contributor ;-) [20:39:41] btw: Salt / Gitlab - any news here ? [20:39:45] BTW: did I already mention http://en.test.opensuse.org/Map_test ? [20:40:12] pjessen: I'll contact you via mail on this topic on monday or tuesday to set up a separate hands-on meeting [20:40:23] gitlab and salt are two different topics [20:40:25] cboltz: you might want to start to document your work on progress.o.o ? [20:40:38] gitlab news: it is fully integrated now with freeipa auth yey! [20:40:39] okay, sounds good. [20:41:19] kl_eisbaer1: that's on my TODO list, but in some cases I prefer to wait until I know which way we go before documenting it [20:41:22] non-suse people can not access the freeipa instance yet to reset their passwords, so either I send them a password via encrypted mail or you wait until we have the openvpn in place [20:41:45] cboltz: that does not really work regarding "release often, release early", right? [20:42:04] cboltz: because in that case nobody can jump in and help if you have no documentation [20:42:15] regarding salt: we have new NTP servers set up by Rudi and me managed fully in salt [20:42:22] We have got different extensions which need old packages... [20:42:28] cboltz: you might get also some additional testers/contributors by making some "marketing" [20:42:49] they are running sle and are managed by the suse salt master, I will "upgrade" them to leap and move them to the opensuse salt [20:42:57] now we are back at the dependency chain [20:43:09] I need the authentification running because testing makes sense [20:43:29] second news regarding salt: powerdns for slave nameserver is fully in salt, chip.o.o is our slave DNS server set up by darix and me [20:43:34] (the current way of "find and use the 'lost password' link is slightly insane ;-) [20:43:39] cboltz: ...and if you would document the dependency chains in our admin-wiki, we might know better how to help you ;-) [20:43:47] and right now I am working on rsyslog clients, then will work on rsyslog server [20:44:12] ;-) [20:45:03] rsyslog and powerdns formulas already got full opensuse support as well [20:45:07] about salt - I wrote a role for AppArmor: https://gitlab.opensuse.org/infra/salt/merge_requests/10 [20:45:48] it might become a formula later [20:46:21] yes, we need to work on this as well [20:46:31] I want also to extract profile/zypper to a formula [20:46:48] as soon as it gets merged, all salt-managed machines will get the default profiles (from apparmor-profiles package) [20:46:55] and you can easily add your own profiles [20:47:30] +1 from me [20:47:34] there are already apparmor profiles running on some machines, should we import them also to salt before we merge that one? [20:47:58] tampakrap: release early, release often, I would say... [20:48:10] we can import them whenever we have time - my salt code won't destroy anything ;-) [20:48:23] cool [20:48:27] I do not see where the commit can break something that was not broken from the beginning [20:48:29] also https://gitlab.opensuse.org/infra/salt/merge_requests/11 this got merged a few mins ago [20:49:04] no, I mean if the commit will cause salt to overwrite apparmor profiles that are in the machines set up manually [20:49:44] tampakrap: sorry, but how? Can you point me to the command that is doing this ? [20:49:53] the current version will install the apparmor-profiles package which might *in theory* overwrite something [20:50:25] but the profiles in that packages are installed on nearly every openSUSE system in the default install, so I'm sure they won't break anything [20:51:11] IMHO the files are already marked as %config(noreplace) - so I do not see anything that might break because of a package that should be installed on productive machines anyway [20:51:45] indeed [20:52:09] guys, you're digressing. The agenda says "naming pattern for VMs". [20:52:22] tampakrap: what about accepting it tomorrow - and we will have a close look at the nodes during the day ? [20:52:41] pjessen: correct [20:52:53] I guess that topic comes from tampakrap, right? [20:53:29] tampakrap: ? [20:53:32] I thought we had this topic some months ago on the mailing list... [20:53:50] kl_eisbaer1: regarding the overwrite - the scenario is: I have written a file by hand, then I use file.managed in salt and overwrite the content [20:54:14] regarding accepting it tomorrow, I was hoping that we can finish it with cboltz today [20:54:20] there are a few comments I need to make there [20:54:36] so cboltz if you got time after the meeting we can do it [20:54:41] tampakrap: right now, there isn't any custom AppArmor profile in salt yet, "just" the framework to handle them [20:54:52] okay [20:55:01] tampakrap: regarding "accepting it today" - in that case you need to take care of the machines on your own, as I need to go to sleep after the meeting ;-) [20:55:07] I understood it, wanted to make sure only that this scenario happens [20:55:27] tampakrap: VM naming scheme ? [20:55:29] good point, so we can finish the code today and deploy only in a couple of machines [20:55:38] and finish the deployment tomorrow [20:55:58] so naming scheme [20:56:44] first the domain: it will be either $location.infra.o.o or just infra.o.o - I personally believe we don't really need the location on the FQDN of the machine [20:56:53] it is marked as a grain [20:57:16] makes sense? [20:57:17] agree, location is superlfuous. [20:57:27] indeed, good point [20:57:50] It depends on whether we want to have geo clustering in the future. [20:58:12] If no, then yes (own experience). ;) [20:58:19] Ada_Lovelace: I don't think that we need it for the infra.o.o domain [20:58:34] geo clustering happens behind tho scenes. [20:58:36] we don't [20:58:49] so $machine.infra.o.o ? [20:58:55] the infra.o.o will be obvious behind the VPN, and on each machine's `hostname -f` [20:58:55] yes [20:59:02] and also an A record [20:59:08] also behind the openvpn [20:59:31] right [20:59:50] so since the location will not be on the domain, this means that the hostname has to be unique on each machine [21:00:05] regarding the naming pattern of the hostnames, flame on :) [21:00:20] next topic [21:00:38] hostnames: two syllable names, in english, french or german. [21:01:01] french? [21:01:10] pjessen: fine with me [21:01:14] I'd vote for boring, numbered hostnames like "wiki1" [21:01:29] +1 [21:01:38] cboltz: please no. [21:01:49] so you can know for what something is... [21:01:50] cboltz: in that case I vote for plain chinese hostnames ;-) [21:02:20] let me warn you that I like crazy ideas ;-)) [21:02:22] known names that people generally know - I had trouble with baloo .... [21:02:56] 维客.infra.opensuse.org [21:03:06] the problem with the cartoon names comes because: 1) we don't define the roles in salt on ALL machines 2) we don't create automatically a cname based on the role [21:03:17] looks at least interesting - and will increase the need for better monitor resolutions for me ;-) [21:03:24] hostname != service names - "wiki1" is good as CNAME to [21:03:43] so if baloo had a role 'lists' on salt, and it would create automatically your CNAME, and maybe also add it in the motd and/or in the bash shell like baloo (lists) $ [21:03:53] then whatever name we pick will be problematic :) [21:04:06] problem with cartoon names - they're not unviersally known. Gaston Lagaffe anyone? [21:04:24] there is a specific list on where to pick those names [21:05:06] anyway, two syllables is good and memorable. London, Dresden, Airbus - plenty to be found. [21:05:43] IMHO the problem with cartoon names or two syllables is that they don't give you an idea what the host does [21:06:19] true, but add a service name for that. [21:06:20] so I'd really prefer the boring "wiki1" way - even if we have "wiki 125" one day ;-) [21:06:24] exactly my point, so unless we don't set up roles and take advantage of them, pick whatever you want. the result will be confusing [21:06:45] cboltz: correct. But you should be able to use the alias to log in to the machine (ssh wiki1 will log you in to boing for example) [21:07:10] cboltz: The funniest story about hostnames in my work experience is the word "Wurstbrot" in the work documentation for HR. :) [21:07:19] my main concern about those hostnames is more the PS1 output and the serial console naming [21:07:26] And Wurstbrot was a hostname. [21:07:49] sounds like a bad company for vegetarians ;-) [21:08:05] Ada_Lovelace: can easily be topped: ismirwurscht.suse.de still exists ;-) [21:08:20] hahaha [21:08:58] but I guess tampakrap wants a solution today for the hostnames, right ? [21:09:02] bahnhof.suse.de ? [21:09:04] sounds like the perfect solution for the *.infra.opensuse.org hostnames - ismirwurscht, mirdochegal etc. ;-) [21:09:17] Host bahnhof.suse.de not found: 3(NXDOMAIN) [21:09:38] see, good one for a hostname - two syllables [21:09:49] but: "senil.suse.de has address 192.168.1.129" [21:10:04] not really :) [21:10:13] ...and debil exists, too ;-) [21:10:24] Enough with funny hostnames... [21:10:31] Next topic [21:10:32] I would prefer that someone works on populating the roles to motd and/or PS1 [21:10:38] where is the emoticon for ROFLAO [21:11:19] tampakrap: but what happens if a host has more than one role ? [21:11:57] two CNAMES pointing to the same host? [21:12:02] it will have the roles comma separated in the PS1 [21:12:12] :D [21:12:25] also, this host is candidate for splitting the service to another host [21:12:38] root@mailinglists,ircbot,webserver ~# [21:13:21] ...so we need to think about which role is worth to become mentioned in "PS1" [21:13:32] root@funnyname (lists, ircbot, webserver) ~# [21:13:59] ...in that case, we can use whatever name the admin of the machine prefers, right? [21:14:13] seems ;-) [21:14:15] sure [21:14:21] that's also a possibility [21:14:22] we just need to watch that the role line does not become too long for the serial console [21:14:37] actually, that sounds good. [21:14:57] kl_eisbaer1: whatever role(s) is/are listed in pillar/id/* (which also means roles included in one of that roles won't get listed) [21:15:40] I guess, we should start with that and evaluate next year if it paid out [21:16:25] agreed [21:16:36] pjessen, Ada_Lovelace, cboltz ? [21:16:48] agreed [21:16:48] agreed [21:16:52] agreed [21:17:13] next topic. "Support other Open Source communities" [21:17:26] yeah, I was wondering about that one [21:17:28] I wrote something about this in the ticket already [21:17:42] while SUSE lists https://www.suse.com/company/open-source/ [21:17:56] openSUSE just has: http://suse.github.io/ [21:18:00] Why not, if we have got capacities available. [21:18:18] Ada: that is exactly the question. [21:18:29] so I was wondering if we could not check if we want to support - for example - the NTP pool with a machine or other projects [21:18:38] I would say, yes. [21:19:04] I would say: technically, we have capacity - but not admin / aka time wise [21:19:07] I agree that we can and should do it, but a) someone has to do it ;-) and b) it should not stop us from getting our own stuff done [21:19:38] I would say yes, no question about it, but it has to be up to the admin. [21:19:39] but we should mention it somewhere, if we can do it [21:20:09] keyserver.o.o is one example: I am maintaining it anyway, so getting it added to the public pool was a low brainer [21:20:11] How about, technical yes, but communities need own volunteers? [21:20:41] Ada_Lovelace: agreed [21:20:56] I just wanted to mentoin two points: [21:21:14] a) openSUSE likes to support, if community needs it [21:21:27] if we, as the openUSE community have resources to spare, we should offer them, as we can [21:21:30] b) openSUSE has an own interest not to waste resources ;-) [21:21:45] (b) very important [21:21:54] totally support the idea, let's do it [21:22:24] ...and do not forget to talk about it ;-) [21:22:35] ;) [21:22:35] but I guess that's all about this topic at the moment [21:22:48] Theo can talk about it at the oSC. [21:22:59] you are in my mind [21:23:14] any immedciate candidates ? [21:23:46] our NTP servers could be [21:23:54] they need public IP and public domain [21:24:09] and protection [21:24:28] ...a lot of protection, as they would open direct access to the infra.o.o network ... [21:25:00] true [21:25:18] do NOT use our internal instances for the pool [21:25:23] ntp is another favourite of mine, been running a stratus 1 mirror for ten years. (from dcf77) [21:26:04] pjessen: /me is wondering what interesting stuff you are NOT doing ;-) [21:26:23] but I guess we leave the topic for now and switch to the next one? [21:26:30] Ideas / plans for the Heroes workshop at oSC17 [21:27:15] beer lots of beer? [21:27:17] who added this ? [21:27:31] I added it [21:27:37] You as the organisator? [21:28:01] darix: "malt beer"? [21:28:12] Ada_Lovelace: no, that's why I'm asking... [21:28:19] mate beer [21:28:21] cboltz: so please start :-) [21:28:30] the basic question is: do we want to have a pure workshop with discussions and hands-on, or do we want to have some lightning talks? [21:28:31] osc17 is in Nürnberg / home ? [21:28:35] darix: .oO :D [21:28:50] pjessen: yes. same location as last year [21:28:55] I would bring some fellow students with me. ;) [21:29:09] Ada_Lovelace: to the workshop ? [21:29:23] Why not? They should learn. ^^ [21:29:30] I'll have to think about a little road-trip [21:29:40] pjessen: definitively !!!!!!!!!! [21:29:50] I have to admit that the reason for asking is a bit selfish - Sarah and I had proposed a talk which didn't fit into the overly crowded schedule, so we might have some topics for lightning talks [21:29:53] pjessen: you know about the "travel support programm" ? [21:29:58] We have to restructure our Linux laboratory in the next semester. [21:30:41] yeah, maybe I can file my petrol-receipts :-) [21:30:46] Ada_Lovelace: ...and what has "restructuring the Linux laboratory" to do with a workshop that was planned to sit together and work (for example) on wiki / openID / freeIPA and other stuff ? [21:31:00] pjessen: ...and your Hotel... [21:31:30] We will do something like that,too. And they can see how we are doing that in the community. [21:31:49] Ada_Lovelace: my original plan was to have a small room with people who want to work together on different topics (4 eyes coding) [21:32:15] Ok. I thought about bringing them to us... [21:32:56] I you bring a huge amount of people, I need to reconsider the original plan [21:33:15] Not such a large number. [21:33:17] ...and that would mean that we probably do not end up in "getting things done" - but more in "talking about things" [21:33:40] that's not a big deal - but this would, from my pow, the consequence [21:33:53] Then I let them in other talks... [21:34:19] while I'm happy to share knowledge and also to give some presentations (if needed), I would vote to use the time together to get things done instead [21:34:49] Agreed [21:35:03] I -as example - want to learn from tampakrap how I could test my Salt changes on my nodes before I submit them to git [21:35:32] ...and we might want to get the new wiki stuff online together [21:35:50] saltifying more machines is another "topic" [21:35:58] That would be great! :) [21:35:58] so - to make it short: [21:36:02] services, not machines [21:36:15] there is not really a full, public schedule for this "workshop" [21:36:40] as I wanted to do something similar we did during Christmas [21:36:52] but this time the other way around: less talking, more coding ;-) [21:37:32] talk is cheap, show me the code [21:37:33] there will definitively some learning - and some guys will also be asked to put their console on a screen [21:37:57] but I want to use the workshop to "work" (and not to talk about shopping ;-) [21:38:43] Ada_Lovelace, cboltz: I hope that makes my original idea a bit more clear ? [21:38:51] Yes. [21:38:51] yes, it does [21:39:02] sorry if the description in the event page was not so clear [21:39:27] does it make sense to everyone ? [21:39:30] But that sounds like more than the planned time. ;) [21:39:53] I asked for 3 days and got 3 hours ... ;-) [21:39:57] next item - "mirror status - 31 open tickets " [21:40:53] I added this - mostly a little egotistical. One of my guys set up to mirror for Switzerland, but it's been waiting [21:41:02] kl_eisbaer1: talking about the workshop description: can you please add wine as another option? I don't like beer ;-) [21:41:17] cboltz: only if you bring something with you [21:41:19] schnapps? [21:41:35] pjessen: I guess switzerland has more to give ;-) [21:41:55] pjessen: but schnapps might be a good "starter" :-) [21:42:29] kl_eisbaer1: should be possible, but I can't promise that it will be cooled as it should be (I doubt Z-Bau will let me use their fridge) [21:42:31] pjessen: I asume that most tickets are currently stalled because Max is busy with preparing the oSC17. Is this correct, Ada_Lovelace? [21:42:58] It seems. [21:43:08] it looks like *everybody* who could handle mirror tickets is busy [21:43:20] I'll ask what he is doing. [21:43:39] Yes, I think so too. I don't want to push anyone, but mirroing has become very public lately [21:43:54] I'll say it. [21:44:41] Can I help out with mirrroing? [21:45:00] not yet unfortunately [21:45:05] insert 'r' [21:45:17] the important service is still behind the suse network [21:45:38] okay, just offering. Let me know if I can help. [21:46:00] pjessen: IMHO it would be more helpful if someone can finish the work on mirrorpinky, so admins would be able to administrate their mirror servers on their own [21:46:14] that as well yes [21:46:30] https://github.com/openSUSE/mirrorpinky [21:46:38] tampakrap: that one you can move tonight. just needs ram [21:46:49] that sounds really cool, but it's news ro me. mirrorpinky? [21:47:18] pjessen: "A self management frontend for the mirrorbrain.org database" [21:47:42] pjessen: mirrorpinky and mirrorbrain [21:48:12] I'll have a look - the more we can offload to the mirror the better [21:49:07] pjessen: right. If you find someone with some Rails-skills, that would be perfect [21:49:59] rails ... umm, not sure. not my cup of tea. [21:50:21] ...at the end, when everything is stored in the database, we "just" need some commandline scripts to adapt the settings (like writting the "host allow" line in the rsyncd.conf for example) [21:51:14] that way adding/deleting/editing a mirror could be done by any admin himself (once he has registered for an openSUSE account) [21:52:30] pjessen: anything more about this topic? [21:52:56] I guess we're just short on resources, so wait and see. [21:53:23] ok [21:53:48] so the only topic left is "Salt Stack for Icinga Checks with ZeroMQ", added by Ada_Lovelace [21:54:00] but I guess we skip this as she left already [21:55:44] ending meeting ? [21:55:57] yes! [21:56:15] yes [21:56:23] last thing I wanted to say: long live openSUSE Leap 15! [21:57:08] see you guys later [21:57:15] I expect a lifetime of at least 3 years - and then we'll have Leap 16 ;-) [21:57:17] bye! [21:57:24] ciao [21:57:34] cboltz: voting for 21!