QA » openQA Project » openQA Infrastructure

be aware of the problems encountered in #72079 though. So upgrading other machines can help to crosscheck if problems reproduce there but this can also increase the impact for more users.

okurz wrote:

be aware of the problems encountered in #72079 though. So upgrading other machines can help to crosscheck if problems reproduce there but this can also increase the impact for more users.

Aye, I went through the docs and the steps of the upgrade but didn't activate the snapshot for now.

Two error messages stood out, and I was wondering if you saw this with openqa-aarch64:

• /usr/sbin/grub2-install: warning: Couldn't find physical volume (null)
• dracut module 'biosdevname' will not be installed, because command 'biosdevname' could not be found

I couldn't find anything in the grub configuration. Of course the missing module could simply be installed - I'm mostly wondering if this is pointing to an existing misconfiguration.

Haven't seen these messages but I wouldn't worry. You can just make sure you have access over IPMI for a quick recovery if needed and try the reboot.

@cdywan I assume you did not yet upgrade any further workers. Do you still plan?

All the problems in #72079 I consider likely to be aarch64 specifics so you can go ahead with the upgrade.

Little hint: To try out upgrading in a safe environment you can run a openSUSE Leap 15.1 container, e.g. podman run --rm -it registry.opensuse.org/opensuse/leap:15.1 and run upgrade with https://github.com/okurz/auto-upgrade/blob/master/auto-upgrade

You need to run zypper ref outside the transactional-update shell.

I did with host "rebel" now:

sed -i -e 's@15\.1@\$releasever@g' -e 's@https:@http:@g' /etc/zypp/repos.d/*
zypper --releasever=15.2 -n ref
transactional-update --no-selfupdate shell
zypper --releasever=15.2 --no-refresh --no-gpg-check -n dup --auto-agree-with-licenses --replacefiles --download-in-advance
rpmconfigcheck
for i in $(cat /var/adm/rpmconfigcheck) ; do vimdiff ${i%.rpm*} $i ; done
rm $(cat /var/adm/rpmconfigcheck)
exit
reboot

what previously failed is either repository refresh or download of packages due to certificate mismatches.
Worked then with changing all https repos to http and calling zypper ref outside the transactional shell

EDIT: 2020-10-23 16:51 Fixed sed expression with second '-e'

Revised command(s) taking boo#1149131 into account:

export new_version=15.2; zypper --releasever=$new_version ref
transactional-update shell
. /etc/os-release && sed -i -e "s/${VERSION_ID}/\$releasever/g" /etc/zypp/repos.d/* && zypper --releasever=$new_version ref && zypper --releasever=$new_version dup --auto-agree-with-licenses --replacefiles --download-in-advance && rpmconfigcheck

for i in $(cat /var/adm/rpmconfigcheck); do vimdiff ${i%.rpm*} $i ; done && rm $(cat /var/adm/rpmconfigcheck)

Note: I uninstalled samba-libs-python-4.9.5 to resolve a conflict.

The upgrade of openqaworker1 went through now.

imagetester worked with these commands based on what @okurz used (but note the second -e in the sed) - the double ref did not work like it did on openqaworker1:

sed -i -e 's@15\.1@\$releasever@g' -e 's@https:@http:@g' /etc/zypp/repos.d/* && zypper --releasever=15.2 -n ref && transactional-update --no-selfupdate shell
zypper --releasever=15.2 --no-refresh --no-gpg-check -n dup --auto-agree-with-licenses --replacefiles --download-in-advance && rpmconfigcheck
for i in $(cat /var/adm/rpmconfigcheck) ; do vimdiff ${i%.rpm*} $i ; done && rm $(cat /var/adm/rpmconfigcheck)

I fail to access power8, it's stuck in petitboot it seems.
Following comments in https://progress.opensuse.org/issues/55607#note-6 I looked up in the shell of petitboot what mount points are available and found /var/petitboot/mnt/dev/sda3/boot/ with all the necessary kernel images installed.

Based on these files and what I could find in /var/petitboot/mnt/dev/sda3/boot/grub2/grub.cfg I did

kexec -l /var/petitboot/mnt/dev/sda3/boot/vmlinux-5.3.18-lp152.47-default --initrd=/var/petitboot/mnt/dev/sda3/boot/initrd-5.3.18-lp152.47-default "root=UUID=e0110d2d-84f3-4590-8d12-a472f82cac8b  root=/dev/disk/by-id/scsi-1IBM_IPR-0_5DB3D30000000020-part3 disk=/dev/disk/by-id/scsi-1IBM_IPR-0_5DB3D30000000020 resume=/dev/disk/by-id/scsi-1IBM_IPR-0_5DB3D30000000020-part2 nomodeset console=hvc console=tty"
kexec -e

so now the machine is up but this is not reboot safe.

I'm trying to confirm the state of the bootloader before marking it as Resolved, will report once I know if I have a work-around

So I tried out the work-around that @nicksinger used in #68053 alas no dice, it´s still getting stuck in petitboot:

power8:~ # nvram --print-config
"common" Partition
---------------------
petitboot,bootdev=uuid:e0110d2d-84f3-4590-8d12-a472f82cac8b
petitboot,bootdevs=uuid:e0110d2d-84f3-4590-8d12-a472f82cac8b

power8:~ # nvram -p common --update-config "petitboot,snapshots?=false"
power8:~ # reboot

ssh root@power8
ssh: connect to host power8 port 22: No route to host

Next step, creating a new boot entry with sda3, /boot/vmlinux and /boot/initrd, kernels args from the kexec command above, boot works, entry is gone after reboot... 🙄 Tried a couple variations but couldn´t make my changes stick.

To add insult to injury, the temporary manual boot entry stops at entering emergency mode now, just like booting from the detected entries does.

cdywan wrote:

The file being parsed should be /var/petitboot/mnt/dev/sda3/boot/grub2/grub.cfg although this version of petitbot won't seem to reveal that, line 239 would be the hiddenentry block at the very end. So I uncommented that block and hurrah, items show up in petiboot's menu!

Tried re-enabling the contents of the block (enabling textmode) and finally undoing my fix, and it still doesn't boot. I guess I'll try and pick some more of @nicksinger's brain in the morning.