tickets #161411
openDedicated networks for openSUSE GitHub Runners
40%
Description
The SUSE Labs department will sponsor an unused old four-node chassis for usage as GitHub Runners. The maintenance will be done by me (Enno Gotthold/SchoolGuy) during my work hours. One of the nodes will be used for the Cobbler org but the other three can be freely integrated into the openSUSE GitHub Org.
As GitHub Runners are essentially executing untrusted code by design they should be isolated as much as possible. I am proposing a VLAN for each GitHub Org (one for Cobbler and one for openSUSE).
The idea is to use https://github.com/actions/actions-runner-controller on top of a k3s cluster to manage the runners. Furthermore, I would desire to use MicroOS as the base OS.
The host is not yet configured with a static network configuration. The four nodes each have a dedicated BMC that only has a Java Web Start based UI for machine access.
Updated by crameleon 7 months ago
Hi Enno,
I will try to configure the network soon. From reading your SUSE ticket, I should probably be able to find the physical connections in SUSE RackTables. Is there some networking configured I can use to connect to the BMC to then configure the correct addresses for our management network? If not, I could spawn a temporary DHCP server.
I understand why MicroOS would be a good candidate for this application. However, I had terrible experience integrating it with our infrastructure in the past. A lot of the Salt states either do not support transactional operation at all, or require dirty hacks. Also a lot of packages are not included in the base distribution, and required maintaining a separate project with various links: https://build.opensuse.org/project/show/openSUSE:infrastructure:Micro. It lead me to eventually move the two servers I tried it with to Leap again and to give up with pursuing the effort to make it work.
Hence I suggest to make your servers Leap based as well but to confine the relevant services with systemd hardening and AppArmor.
I have an AutoYaST profile we can use for deployment of the base OS (there's currently no network boot server in our infrastructure since we rarely ever have new hardware, hence I'd just load it with an image through the BMC, if possible).
The names are fine with me.
Updated by SchoolGuy 7 months ago
crameleon wrote in #note-3:
Hi Enno,
I will try to configure the network soon. From reading your SUSE ticket, I should probably be able to find the physical connections in SUSE RackTables. Is there some networking configured I can use to connect to the BMC to then configure the correct addresses for our management network? If not, I could spawn a temporary DHCP server.
I understand why MicroOS would be a good candidate for this application. However, I had terrible experience integrating it with our infrastructure in the past. A lot of the Salt states either do not support transactional operation at all, or require dirty hacks. Also a lot of packages are not included in the base distribution, and required maintaining a separate project with various links: https://build.opensuse.org/project/show/openSUSE:infrastructure:Micro. It lead me to eventually move the two servers I tried it with to Leap again and to give up with pursuing the effort to make it work.
Hence I suggest to make your servers Leap based as well but to confine the relevant services with systemd hardening and AppArmor.I have an AutoYaST profile we can use for deployment of the base OS (there's currently no network boot server in our infrastructure since we rarely ever have new hardware, hence I'd just load it with an image through the BMC, if possible).
The names are fine with me.
Feel free to go ahead with Leap. I just wanted to save myself a bit of maintenance. The BMC should have DHCP, so spawning a temporary DHCP server should make them accessible. Username/Password I will give you via the work messenger.
Updated by SchoolGuy 7 months ago
crameleon wrote in #note-4:
On a second thought, I wonder if the names shouldn't be something more generic.
I know we will only use these machines as GitHub runners now, but I have this fear of finding a new purpose at some point in the future making the names no longer make sense. ;-)
I have no hard feelings about other names. It was an idea from me. I don't know if we have a naming schema in the openSUSE Infra but if yes then feel free to apply it.
Updated by crameleon 7 months ago
Thanks, found the credentials. Will try them soon and let you know.
Naming scheme is sometimes service related and sometimes just creativity. For physical machines usually the latter (as I feel those are more involved to relabel down the line). What about apollo-chassis + apollo0{1,2,3,4}?
Updated by crameleon 7 months ago
- Precedes tickets #161963: Prepare GitHub runner servers added
Updated by crameleon 7 months ago · Edited
- % Done changed from 10 to 20
Created network allocations:
2a07:de40:b27e:1207::/64
- Machine network for Cobbler runners
https://netbox.infra.opensuse.org/ipam/prefixes/35
with
VLAN 1207 openSUSE-GHR-Cobbler
https://netbox.infra.opensuse.org/ipam/vlans/33
2a07:de40:b27e:1208::/64
- Machine network for openSUSE runners
https://netbox.infra.opensuse.org/ipam/prefixes/36
with
VLAN 1208 openSUSE-GHR-openSUSE
https://netbox.infra.opensuse.org/ipam/vlans/34
2a07:de40:b27e:4003::/64
- K3S Cluster network for Cobbler runners
https://netbox.infra.opensuse.org/ipam/prefixes/37
2a07:de40:b27e:4004::/64
- K3S Service network for Cobbler runners
https://netbox.infra.opensuse.org/ipam/prefixes/38
2a07:de40:b27e:4005::/64
- K3S Cluster network for openSUSE runners
https://netbox.infra.opensuse.org/ipam/prefixes/39
2a07:de40:b27e:4006::/64
- K3S Service network for openSUSE runners
https://netbox.infra.opensuse.org/ipam/prefixes/40
For configuring K3S networking, https://docs.k3s.io/networking/basic-network-options#single-stack-ipv6-networking should be followed (we don't use router advertisements so the warning is not relevant).
Updated by crameleon 7 months ago
- % Done changed from 20 to 30
Patch for routing configuration and firewall baseline submitted as https://gitlab.infra.opensuse.org/infra/salt/-/merge_requests/1917.