action #137198
closedPrepare cgroup-dependent test runs for container engines
0%
Description
In 15-SP6 we will have cgroups v2, previously we only had cgroups v1. Once released we need to test the container engines on both.
We should introduce a new setting, e.g. CGROUPS_VERSION that allows us to control which version a single test run is using. We don't need to support multiple cgroup versions per test run, one is enough.
This setting should make sure the appropriate cgroups settings are set in the kernel parameters (GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub). To enable v2, the systemd.unified_cgroup_hierarchy=1 setting should be set there.
Alternatively we can also use mount to check if cgroup2 is present in /sys/fs/cgroup.
# mount -l | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate)
We need to do this at least for podman and docker, and if possible for all other container engines as well.
Acceptance criteria¶
- Introduce a new setting e.g.
CGROUPS_VERSIONthat allows us to define the cgroups version to container engine test runs - Check if this can be applied on Tumbleweed (if there are still cgroups v1 possible) as testing ground because we cannot use cgroupsv2 on SLES15-SP5 and older
- Schedule container engine tests using both cgroups versions, if possible on Tumbleweed
Related links¶
Updated by rpalethorpe 9 months ago
Hi, I just came across this while looking for a kernel related cgroup ticket.
FYI, In cases where both V1 and V2 are enabled (which unfortunately was the default in systemd for a while) you need to either check 'cgroup.controllers' to ensure the V2 controller versions are enabled or check 'mount' for the cgroup V1 controllers. It's possible to have both the cgroup2 mount present (there is only one for V2) and the V1 controller mounts.
On Tumbleweed cgroups V2 is the default AFAIK so we shouldn't need to do anything.
Updated by rbranco 7 months ago
From PED-5849:
2022-06: SLE15 SP4 adds support for unified hierarchy (defaults to hybrid)
Confirming with:
https://documentation.suse.com/sles/15-SP4/html/SLES-all/cha-tuning-cgroups.html
If I'm reading this right, we should also test 15-SP4+
Updated by ph03nix 7 months ago
rbranco wrote in #note-8:
From PED-5849:
2022-06: SLE15 SP4 adds support for unified hierarchy (defaults to hybrid)
Confirming with:
https://documentation.suse.com/sles/15-SP4/html/SLES-all/cha-tuning-cgroups.htmlIf I'm reading this right, we should also test 15-SP4+
Yes, I read it the same way. We should also test 15-SP4+.
Updated by rbranco 7 months ago
- Status changed from Workable to In Progress
Introduce switch_cgroup() function
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18184
Updated by rbranco 7 months ago
Introduce CONTAINERS_CGROUPS_VERSION variable
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/18201
TODO:
- Add scheduling for TW on opensuse-jobgroups
- Add scheduling for SLE 15-SP3+ on qac-openqa-yaml
Updated by rbranco 7 months ago
Opened bug:
podman fails to run containers when using hybrid cgroup hierarchy
https://bugzilla.opensuse.org/show_bug.cgi?id=1217590
Updated by rbranco 7 months ago
Add job for testing cgroup v1 on docker
https://github.com/os-autoinst/opensuse-jobgroups/pull/392
Updated by rbranco 7 months ago
Add docker_cgroupv2_tests to SLES 15-SP3+
https://gitlab.suse.de/qac/qac-openqa-yaml/-/merge_requests/1408
Updated by rbranco 7 months ago
Add cgroups v2 test to 15-SP3+ and cgroups v1 to 15-SP6+
https://gitlab.suse.de/qac/qac-openqa-yaml/-/merge_requests/1409
Updated by rbranco 7 months ago
- Status changed from In Progress to Blocked
Waiting for resolution of https://bugzilla.opensuse.org/show_bug.cgi?id=1217590
Updated by rbranco 7 months ago
Add cgroupv1 tests for podman
https://github.com/os-autoinst/opensuse-jobgroups/pull/395
Updated by rbranco 7 months ago
Kubernetes 1.25 adds official support for cgroup v2:
https://kubernetes.io/blog/2022/08/31/cgroupv2-ga-1-25/
Information for nerdctl:
https://github.com/containerd/nerdctl/blob/main/docs/faq.md#how-to-change-the-cgroup-driver
Rancher (k3s):
https://docs.k3s.io/advanced
Cgroup v1 and Hybrid v1/v2 are not supported; only pure Cgroup v2 is supported. If K3s fails to start due to missing cgroups when running rootless, it is likely that your node is in Hybrid mode, and the "missing" cgroups are still bound to a v1 controller.
Updated by rbranco 7 months ago
systemd version 256 will drop support for v1:
https://github.com/os-autoinst/opensuse-jobgroups/pull/399