action #163469
closed
openQA Project (public) - coordination #157969: [epic] Upgrade all our infrastructure, e.g. o3+osd workers+webui and production workloads, to openSUSE Leap 15.6
Upgrade a single o3 worker to openSUSE Leap 15.6
Added by okurz 5 months ago.
Updated about 2 months ago.
Category:
Feature requests
Description
Motivation¶
- Need to upgrade workers before EOL of Leap 15.5 and have a consistent environment
Acceptance criteria¶
- AC1: a single o3 worker machine runs a clean upgraded openSUSE Leap 15.6 (no failed systemd services, no left over .rpm-new files, etc.)
Suggestions¶
- read https://progress.opensuse.org/projects/openqav3/wiki#Distribution-upgrades
- Keep IPMI interface ready and test that Serial-over-LAN works for potential recovery
- Apply the workaround for #162296, i.e.
zypper al -m "boo#1227616" *firewall*
- Use the instructions from above
- After upgrade reboot and check everything working as expected, if not rollback, e.g. with
transactional-update rollback
- Monitor for effect on special test scenarios, e.g. iscsi which showed problems in the past
- Record important details into the "upgrade all other" ticket #157972
Further details¶
- Don't worry, everything can be repaired :) If by any chance the worker gets misconfigured there are btrfs snapshots to recover, the IPMI Serial-over-LAN, a reinstall is possible and not hard, there is no important data on the host (it's only an openQA worker) and there are also other machines that can jobs while one host might be down for a little bit longer. And okurz can hold your hand :)
- Copied from action #157972: Upgrade o3 workers to openSUSE Leap 15.6 size:S added
- Related to action #162683: s390x libvirt started kvm machines on Leap 15.6 fail with "unsupported configuration: machine type 's390-ccw-virtio-8.2' does not support ACPI" size:M added
- Related to action #162296: openQA workers crash with Linux 6.4 after upgrade openSUSE Leap 15.6 size:S added
- Project changed from openQA Project (public) to openQA Infrastructure (public)
- Description updated (diff)
- Category changed from Organisational to Feature requests
- Status changed from Blocked to New
- Assignee deleted (
okurz)
- Target version changed from Tools - Next to Ready
Seems like this is still blocked on #162296
No, we shouldn't block on this longer. That's why I updated this ticket in #163469-6 and explained that we need to apply a workaround with a package lock
- Status changed from New to In Progress
The worker is upgraded, a reboot is pending. As I don't have access to IPMI via jumpy, I need some help with this.
- Status changed from In Progress to Blocked
gpathak wrote in #note-16:
The worker is upgraded, a reboot is pending. As I don't have access to IPMI via jumpy, I need some help with this.
Let's block on SD-170670
After connecting to IMPI of openqaworker21, found out that the worker lost network connection.
None of the physical interface eth0, eth1 has an IP address assigned.
Do we use static IP addresses or dynamic ones assigned via an internal dhcp server?
dynamic with o3 aka ariel running dnsmasq for DHCP. Are you sure you applied the workaround mentioned in the description? The network might also recover with a reboot. If not consider a rollback with snapper
- Status changed from Blocked to In Progress
@okurz I missed the workaround zypper al -m "boo#1227616" *firewall*
Performed a rollback to 15.5, executed the command to lock the firewall package upgrade and now doing a dist upgrade again.
openqaworker21.openqanet.opensuse.org is Upgraded to
openqaworker21:~ # cat /etc/os-release
NAME="openSUSE Leap"
VERSION="15.6"
ID="opensuse-leap"
ID_LIKE="suse opensuse"
VERSION_ID="15.6"
PRETTY_NAME="openSUSE Leap 15.6"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:15.6"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org/"
DOCUMENTATION_URL="https://en.opensuse.org/Portal:Leap"
LOGO="distributor-logo-Leap"
openqaworker21:~ #
- Status changed from In Progress to Feedback
Need to run tests on o3 worker21
- Related to action #168454: `openqaworker21` fails with no `Qemu/KVM found` added
- Status changed from Feedback to Resolved
Also available in: Atom
PDF