QA » openQA Project

for i in openqaworker1 openqaworker4 openqaworker7 imagetester rebel power8; do ssh root@$i "zypper ps | tail -n 2; rebootmgrctl status; uptime"; done shows that a reboot is pending but was never planned. on openqaworker1 journalctl --since=today -u transactional-update shows:

-- Logs begin at Thu 2022-05-12 03:17:54 CEST, end at Fri 2022-06-17 17:47:58 CEST. --
Jun 17 00:22:02 openqaworker1 systemd[1]: Starting Update the system...
Jun 17 00:22:02 openqaworker1 openqa-check-devel-repo[4675]: devel:openQA looks good for Leap 15.3 (x86_64)
Jun 17 00:22:02 openqaworker1 transactional-update[4686]: Checking for newer version.
Jun 17 00:22:10 openqaworker1 transactional-update[4686]: transactional-update 3.6.2 started
Jun 17 00:22:10 openqaworker1 transactional-update[4686]: Options: cleanup dup reboot
Jun 17 00:22:10 openqaworker1 transactional-update[4686]: Separate /var detected.
Jun 17 00:22:14 openqaworker1 transactional-update[4686]: 2022-06-17 00:22:10 tukit 3.6.2 started
Jun 17 00:22:14 openqaworker1 transactional-update[4686]: 2022-06-17 00:22:10 Options: -c1336 open
Jun 17 00:22:14 openqaworker1 transactional-update[4686]: 2022-06-17 00:22:13 Using snapshot 1336 as base for new snapshot 1803.
Jun 17 00:22:14 openqaworker1 transactional-update[4686]: 2022-06-17 00:22:13 Parent snapshot 1330 does not exist any more - skipping rsync
Jun 17 00:22:14 openqaworker1 transactional-update[4686]: ID: 1803
Jun 17 00:22:14 openqaworker1 transactional-update[4686]: 2022-06-17 00:22:14 Transaction completed.
Jun 17 00:22:14 openqaworker1 transactional-update[4686]: Calling zypper --no-cd dup
Jun 17 00:22:19 openqaworker1 transactional-update[4686]: zypper: nothing to update
Jun 17 00:22:19 openqaworker1 transactional-update[4686]: Removing snapshot #1803...
Jun 17 00:22:19 openqaworker1 transactional-update[5535]: 2022-06-17 00:22:19 tukit 3.6.2 started
Jun 17 00:22:19 openqaworker1 transactional-update[5535]: 2022-06-17 00:22:19 Options: abort 1803
Jun 17 00:22:19 openqaworker1 transactional-update[5535]: 2022-06-17 00:22:19 Discarding snapshot 1803.
Jun 17 00:22:20 openqaworker1 transactional-update[5535]: 2022-06-17 00:22:20 Transaction completed.
Jun 17 00:22:20 openqaworker1 transactional-update[4686]: transactional-update finished
Jun 17 00:22:20 openqaworker1 systemd[1]: transactional-update.service: Succeeded.
Jun 17 00:22:20 openqaworker1 systemd[1]: Finished Update the system.

so "nothing to update" and nothing is done, no reboot is triggered.

$ ssh o3
Last login: Wed Jul  6 09:55:54 2022 from 192.168.47.252
okurz@ariel:~> for i in openqaworker1 openqaworker4 openqaworker7 imagetester rebel power8; do ssh root@$i "zypper ps | tail -n 2; rebootmgrctl status; uptime"; done
Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.
Status: Reboot not requested
 13:40:56  up 37 days  2:50,  0 users,  load average: 1.57, 1.59, 3.06
No core libraries or services have been updated since the last system boot.
Reboot is probably not necessary.
Status: Reboot not requested
 13:40:57  up 1 day 10:06,  0 users,  load average: 1.30, 1.69, 2.43
No core libraries or services have been updated since the last system boot.
Reboot is probably not necessary.
Status: Reboot not requested
 13:40:58  up  10:04,  0 users,  load average: 0.38, 0.62, 1.04
Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.
Status: Reboot not requested
 11:40:59  up 37 days  2:52,  0 users,  load average: 0.00, 0.02, 0.09
Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.
Status: Reboot not requested
 13:40:59  up 37 days  2:51,  0 users,  load average: 0.00, 0.00, 0.00
Check failed:
Please install package 'lsof' first.
Error: The name org.opensuse.RebootMgr was not provided by any .service files
 11:41:00  up 37 days  1:13,  0 users,  load average: 0.00, 0.02, 0.21
okurz@ariel:~> 

and

for i in openqaworker1 openqaworker4 openqaworker7 imagetester rebel power8; do ssh root@$i "zypper ps | tail -n 2"; done
Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.
No core libraries or services have been updated since the last system boot.
Reboot is probably not necessary.
No core libraries or services have been updated since the last system boot.
Reboot is probably not necessary.
Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.
Since the last system boot core libraries or services have been updated.
Reboot is suggested to ensure that your system benefits from these updates.
Check failed:
Please install package 'lsof' first.

so some machines are up since 37 days and not rebooted. We discussed in weekly SUSE QE Tools unblock 2022-07-06 and agreed that likely the best approach is to just use the openqa-auto-update service which also, same as the transactional update, will most likely never need to install anything as the continuous update already does but it will check if a reboot is needed and request a reboot over rebootmgr. And with the maintenance window still setup correctly to be nightly there will be a nightly reboot but only when necessary.

I did

for i in openqaworker1 openqaworker4 openqaworker7 imagetester rebel power8; do ssh root@$i "systemctl enable --now openqa-auto-update.timer"; done

and now back to monitoring over the next days.

EDIT: Wait, where is aarch64? … Ah, it's missing! Apparently we had openQA-auto-update installed on all machines but not on aarch64. Fixed that with zypper -n in openQA-auto-update && systemctl enable --now openqa-auto-update.timer