Actions
action #180710
closedopenQA Project (public) - coordination #157969: [epic] Upgrade all our infrastructure, e.g. o3+osd workers+webui and production workloads, to openSUSE Leap 15.6
Upgrade Power8 o3 workers to openSUSE Leap 15.6 - qa-power8-3 size:S
Status:
Resolved
Priority:
Normal
Assignee:
Category:
Organisational
Target version:
Start date:
2024-11-14
Due date:
% Done:
0%
Estimated time:
Description
Motivation¶
qa-power8-3 is still on Leap 15.5 after #169939 which is problematic because 15.5 is outdated, potentially insecure and also does not feature a sufficiently up-to-date os-autoinst. We need to find a solution to ensure that only current Leap is used on o3 ppc64le machines
Acceptance criteria¶
- AC1: all power8 o3 worker machines run a clean upgraded openSUSE Leap 15.6 (no failed systemd services, no left over .rpm-new files, etc.)
Suggestions¶
- Read comments in #169939 to understand the context and differences for machines
- read https://progress.opensuse.org/projects/openqav3/wiki#Distribution-upgrades
- Reserve some time when the workers are only executing a few or no openQA test jobs
- Keep IPMI interface ready and test that Serial-over-LAN works for potential recovery
- Try using old package version of kernel and utils-linux, refer: #119008-14 (consider using package locks then)
- After upgrade reboot and check everything working as expected, additionally use https://github.com/os-autoinst/scripts/blob/master/reboot-stability-check, if not rollback, e.g. with
transactional-update rollback
Further details¶
- Don't worry, everything can be repaired :) If by any chance the worker gets misconfigured there are btrfs snapshots to recover, the IPMI Serial-over-LAN, a reinstall is possible and not hard, there is no important data on the host (it's only an openQA worker) and there are also other machines that can jobs while one host might be down for a little bit longer.
Updated by okurz about 2 months ago
- Copied from action #169939: Upgrade Power8 o3 workers to openSUSE Leap 15.6 size:M added
Updated by livdywan about 1 month ago
- Subject changed from Upgrade Power8 o3 workers to openSUSE Leap 15.6 - qa-power8-3 to Upgrade Power8 o3 workers to openSUSE Leap 15.6 - qa-power8-3 size:S
- Description updated (diff)
Updated by livdywan about 1 month ago
- Related to action #180698: "Rollback of updates" wiki section outdated size:S added
Updated by okurz 15 days ago
- Copied to action #182585: Ensure all osd PowerNV ppc machines run a clean upgraded openSUSE Leap 15.6 w/o kernel package locks added
Actions