Actions
action #70978
closedopenQA Project - coordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
automatic reboots on o3 to activate new kernel versions
Added by okurz over 3 years ago. Updated almost 3 years ago.
Start date:
2020-09-04
Due date:
% Done:
0%
Estimated time:
Description
Observation¶
[04/09/2020 14:45:11] <Martchus_> I'm currently having a look at o3's almost full root partition. Can I clean up old kernel versions (e.g. only leaving the 2 most recent versions)? I would also like to get rid of the zypper cache and old log files. These are the main culprits according to ncdu.
…
[04/09/2020 14:49:52] <okurz> Martchus: regarding kernel versions the problem is that o3 is not having recurring automatic reboots. We have that on osd by now for the same reason: To boot the most recent kernel version and allow older versions to be pruned.
Problem¶
- Kernel updates are not used
- Disk space is taken up by accumulating kernel( package)s
Suggestion¶
- Enable automatic pruning of kernels on o3
- Investigate if we can automatically reboot o3
- Look at the setup used on osd
Workaround¶
- Delete old kernels and reboot manually
Updated by okurz over 3 years ago
- Related to action #70939: [alert] ** PROBLEM Service Alert: ariel-opensuse.suse.de/root partition is WARNING ** added
Updated by livdywan over 3 years ago
- Description updated (diff)
- Status changed from New to Workable
Updated by okurz over 3 years ago
- Status changed from Workable to In Progress
- Assignee set to okurz
To apply the same approach as on osd following https://gitlab.suse.de/openqa/salt-states-openqa/-/tree/master/auto-update I did on o3:
zypper -n --no-refresh in rebootmgr zypper-needs-restarting
systemctl enable --now rebootmgr
sed -i 's/^\(window-start=\)\(.*\)$/window-start=Sun, 03:30/' /etc/rebootmgr.conf
cat <<EOF > /etc/systemd/system/auto-update.timer
[Unit]
Description=Nightly trigger of auto-update.
[Timer]
OnCalendar=3:00
Persistent=true
[Install]
WantedBy=timers.target
EOF
cat <<EOF > /etc/systemd/system/auto-update.service
[Unit]
Description=Automatically patch system packages. See https://progress.opensuse.org/issues/18006 and https://progress.opensuse.org/issues/70978
After=network.target
StartLimitInterval=3600
StartLimitBurst=12
[Service]
Type=simple
ExecStart=/bin/sh -c '. /etc/os-release && result="\$(curl -s "https://api.opensuse.org/public/build/devel:openQA/_result?repository=openSUSE_Leap_\${VERSION}&arch=\$(uname -i)")" && echo -e "\$result" | grep -q "project.*state=.published." && echo "\$result" | grep "status package" | (! grep -vE "(disabled|succeeded|excluded)") && zypper -n dup --replacefiles --auto-agree-with-licenses --force-resolution --download-in-advance && needs-restarting --reboothint >/dev/null || (command -v rebootmgrctl >/dev/null && rebootmgrctl reboot ||:)'
Restart=on-failure
RestartSec=300
EOF
systemctl daemon-reload
rm -f /etc/cron.d/auto-update
with an adapted auto-update.service to look for the repo status first.
Former content of deleted /etc/cron.d/auto-update:
# https://progress.opensuse.org/issues/18006
0 3 * * * root . /etc/os-release && result="$(curl -s "https://api.opensuse.org/public/build/devel:openQA/_result?repository=openSUSE_Leap_${VERSION}&arch=$(uname -i)")" && echo -e "$result" | grep -q 'project.*state="published"' && echo "$result" | grep 'status package' | (! grep -v '\(disabled\|succeeded\|excluded\)') && zypper -n dup --replacefiles --auto-agree-with-licenses --force-resolution --download-in-advance
Updated by okurz over 3 years ago
- Status changed from In Progress to Resolved
Triggered service once with systemctl start auto-update and confirmed with rebootmgrctl status that a reboot was requested:
Status: Reboot requested, waiting for maintenance window
Triggered that explicit reboot now (as it's not a busy time) and the machine came back up in I would say less than a minute. Checking again with rebootmgrctl status confirms that Status: Reboot not requested.
Updated by okurz over 3 years ago
- Status changed from Resolved to In Progress
something is fishy. I checked if #70792 is live and found that still o3 did not really do updates for some days. And /etc/systemd/system/auto-update.service does not look like what https://progress.opensuse.org/issues/70978#note-3 says and if I just update auto-update.service accordingly then it fails with Oct 29 21:10:52 ariel systemd[1]: /etc/systemd/system/auto-update.service:11: Ignoring unknown escape sequences: in the log and more syntax errors.
I put the content now in a script /usr/local/bin/auto-update :
#!/bin/bash
set -e
set -o pipefail
. /etc/os-release && result="$(curl -s "https://api.opensuse.org/public/build/devel:openQA/_result?repository=openSUSE_Leap_${VERSION}&arch=$(uname -i)")" && echo -e "$result" | grep -q "project.*state=.published." && echo "$result" | grep "status package" | (! grep -vE "(disabled|succeeded|excluded)") && zypper -n dup --replacefiles --auto-agree-with-licenses --force-resolution --download-in-advance && needs-restarting --reboothint >/dev/null || (command -v rebootmgrctl >/dev/null && rebootmgrctl reboot ||:)
and call that from the systemd service:
[Unit]
Description=Automatically patch system packages. See https://progress.opensuse.org/issues/18006 and https://progress.opensuse.org/issues/70978
After=network.target
StartLimitInterval=3600
StartLimitBurst=12
[Service]
Type=simple
ExecStart=/usr/local/bin/auto-update
Restart=on-failure
RestartSec=300
Triggered the service now. But it seems to be stuck sometimes on /usr/bin/systemd-tty-ask-password-agent --watch, need to check again.
Updated by okurz over 3 years ago
o3 was correctly automatically rebooted last night:
$ journalctl --since=today -u auto-update -u rebootmgr
-- Logs begin at Tue 2020-02-25 12:35:55 UTC, end at Sun 2020-11-08 08:10:35 UTC. --
Nov 08 03:00:01 ariel systemd[1]: Started Automatically patch system packages. See https://progress.opensuse.org/issues/18006 and https://progress.opensuse.>
Nov 08 03:00:06 ariel auto-update[26416]: Loading repository data...
Nov 08 03:00:07 ariel auto-update[26416]: Reading installed packages...
Nov 08 03:00:09 ariel auto-update[26416]: Warning: You are about to do a distribution upgrade with all enabled repositories. Make sure these repositories ar>
Nov 08 03:00:09 ariel auto-update[26416]: Computing distribution upgrade...
Nov 08 03:00:09 ariel auto-update[26416]: The following 4 packages are going to be upgraded:
Nov 08 03:00:09 ariel auto-update[26416]: openQA openQA-client openQA-common perl-DBI
Nov 08 03:00:09 ariel auto-update[26416]: 4 packages to upgrade.
Nov 08 03:00:09 ariel auto-update[26416]: Overall download size: 3.5 MiB. Already cached: 0 B. After the operation, additional 2.1 KiB will be used.
Nov 08 03:00:09 ariel auto-update[26416]: Continue? [y/n/v/...? shows all options] (y): y
Nov 08 03:00:38 ariel auto-update[26416]: Retrieving package perl-DBI-1.643-lp151.3.1.x86_64 (1/4), 748.4 KiB ( 2.1 MiB unpacked)
Nov 08 03:00:38 ariel auto-update[26416]: Retrieving: perl-DBI-1.643-lp151.3.1.x86_64.rpm [done]
Nov 08 03:00:38 ariel auto-update[26416]: Retrieving package openQA-common-4.6.1604770960.7bfaab847-lp151.3375.1.noarch (2/4), 335.5 KiB (356.6 KiB unpacked)
Nov 08 03:00:39 ariel auto-update[26416]: Retrieving: openQA-common-4.6.1604770960.7bfaab847-lp151.3375.1.noarch.rpm [.done (185.8 KiB/s)]
Nov 08 03:00:39 ariel auto-update[26416]: Retrieving package openQA-client-4.6.1604770960.7bfaab847-lp151.3375.1.noarch (3/4), 266.2 KiB ( 56.5 KiB unpacked)
Nov 08 03:00:39 ariel auto-update[26416]: Retrieving: openQA-client-4.6.1604770960.7bfaab847-lp151.3375.1.noarch.rpm [done]
Nov 08 03:00:39 ariel auto-update[26416]: Retrieving package openQA-4.6.1604770960.7bfaab847-lp151.3375.1.noarch (4/4), 2.2 MiB ( 12.0 MiB unpacked)
Nov 08 03:00:39 ariel auto-update[26416]: Retrieving: openQA-4.6.1604770960.7bfaab847-lp151.3375.1.noarch.rpm [done]
Nov 08 03:00:40 ariel auto-update[26416]: Checking for file conflicts: [.......done]
Nov 08 03:00:40 ariel [RPM][26765]: Transaction ID 5fa75f58 started
Nov 08 03:00:40 ariel [RPM][26765]: erase perl-DBI-1.643-lp151.2.2.x86_64: success
Nov 08 03:00:40 ariel [RPM][26765]: install perl-DBI-1.643-lp151.3.1.x86_64: success
Nov 08 03:00:40 ariel [RPM][26765]: erase perl-DBI-1.643-lp151.2.2.x86_64: success
Nov 08 03:00:40 ariel [RPM][26765]: install perl-DBI-1.643-lp151.3.1.x86_64: success
Nov 08 03:00:40 ariel [RPM][26765]: Transaction ID 5fa75f58 finished: 0
Nov 08 03:00:40 ariel auto-update[26416]: (1/4) Installing: perl-DBI-1.643-lp151.3.1.x86_64 [...................done]
Nov 08 03:00:40 ariel [RPM][26766]: Transaction ID 5fa75f58 started
Nov 08 03:00:40 ariel [RPM][26766]: erase openQA-common-4.6.1604520546.c5601c4c9-lp151.3338.1.noarch: success
Nov 08 03:00:40 ariel [RPM][26766]: install openQA-common-4.6.1604770960.7bfaab847-lp151.3375.1.noarch: success
Nov 08 03:00:40 ariel [RPM][26766]: erase openQA-common-4.6.1604520546.c5601c4c9-lp151.3338.1.noarch: success
Nov 08 03:00:40 ariel [RPM][26766]: install openQA-common-4.6.1604770960.7bfaab847-lp151.3375.1.noarch: success
Nov 08 03:00:40 ariel [RPM][26766]: Transaction ID 5fa75f58 finished: 0
Nov 08 03:00:40 ariel auto-update[26416]: (2/4) Installing: openQA-common-4.6.1604770960.7bfaab847-lp151.3375.1.noarch [..................done]
Nov 08 03:00:40 ariel [RPM][26774]: Transaction ID 5fa75f58 started
Nov 08 03:00:40 ariel [RPM][26774]: erase openQA-client-4.6.1604520546.c5601c4c9-lp151.3338.1.noarch: success
Nov 08 03:00:40 ariel [RPM][26774]: install openQA-client-4.6.1604770960.7bfaab847-lp151.3375.1.noarch: success
Nov 08 03:00:40 ariel [RPM][26774]: erase openQA-client-4.6.1604520546.c5601c4c9-lp151.3338.1.noarch: success
Nov 08 03:00:40 ariel [RPM][26774]: install openQA-client-4.6.1604770960.7bfaab847-lp151.3375.1.noarch: success
Nov 08 03:00:40 ariel [RPM][26774]: Transaction ID 5fa75f58 finished: 0
Nov 08 03:00:40 ariel auto-update[26416]: (3/4) Installing: openQA-client-4.6.1604770960.7bfaab847-lp151.3375.1.noarch [.................done]
Nov 08 03:00:41 ariel [RPM][26775]: Transaction ID 5fa75f59 started
Nov 08 03:00:41 ariel [RPM][26775]: erase openQA-4.6.1604520546.c5601c4c9-lp151.3338.1.noarch: success
Nov 08 03:00:43 ariel [RPM][26775]: install openQA-4.6.1604770960.7bfaab847-lp151.3375.1.noarch: success
Nov 08 03:00:49 ariel [RPM][26775]: erase openQA-4.6.1604520546.c5601c4c9-lp151.3338.1.noarch: success
Nov 08 03:00:49 ariel [RPM][26775]: install openQA-4.6.1604770960.7bfaab847-lp151.3375.1.noarch: success
Nov 08 03:00:49 ariel [RPM][26775]: Transaction ID 5fa75f59 finished: 0
Nov 08 03:00:49 ariel auto-update[26416]: (4/4) Installing: openQA-4.6.1604770960.7bfaab847-lp151.3375.1.noarch [...................done]
Nov 08 03:00:50 ariel auto-update[26416]: There are running programs which still use files and libraries deleted or updated by recent upgrades. They should >
Nov 08 03:00:50 ariel auto-update[26416]:
Nov 08 03:00:50 ariel auto-update[26416]: Core libraries or services have been updated.
Nov 08 03:00:50 ariel auto-update[26416]: Reboot is required to ensure that your system benefits from these updates.
Nov 08 03:00:50 ariel rebootmgrd[1517]: Reboot already in progress, ignored
…
Nov 08 03:30:00 ariel rebootmgrd[1517]: rebootmgr: reboot triggered now!
-- Reboot --
Nov 08 03:30:34 ariel systemd[1]: Starting Reboot Manager...
Nov 08 03:30:34 ariel systemd[1]: Started Reboot Manager.
so everything works as expected. But I think it's worthwhile to ensure that the corresponding script and service is also stored in a persistent way. So I created
https://github.com/os-autoinst/openQA/pull/3524
to add the auto-upgrade script as well as the according systemd service as part of a specific package that one can install.
This brings me to the idea that we can try to consolidate the infrastructure of o3 by ensuring all necessary parts are installed from packages. By that we can also benefit upstream from configuration and setup we have on o3 which can be helpful for other openQA users as well.
Updated by okurz over 3 years ago
PR merged. Should check state of packages later and tomorrow check if it can be deployed to o3 to replace the former manual solution.
Updated by okurz over 3 years ago
introduced a mistake in the service ExecStart statement :,( -> https://github.com/os-autoinst/openQA/pull/3556
Also I should use a consistent name "auto-update".
Updated by okurz over 3 years ago
- Due date set to 2020-11-18
PR merged and installed on o3.
Did:
systemctl enable --now openqa-auto-update.timer
preparing to remove the old services with systemctl cat auto-update.timer; systemctl cat auto-update.service; cat /usr/local/bin/auto-update:
# /etc/systemd/system/auto-update.timer
[Unit]
Description=Nightly trigger of auto-update.
[Timer]
OnCalendar=3:00
Persistent=true
[Install]
WantedBy=timers.target
# /etc/systemd/system/auto-update.service
[Unit]
Description=Automatically patch system packages. See https://progress.opensuse.org/issues/18006 and https://progress.opensuse.org/issues/70978
After=network.target
StartLimitInterval=3600
StartLimitBurst=12
[Service]
Type=simple
ExecStart=/usr/local/bin/auto-update
Restart=on-failure
RestartSec=300
#!/bin/bash
set -e
set -o pipefail
. /etc/os-release && result="$(curl -s "https://api.opensuse.org/public/build/devel:openQA/_result?repository=openSUSE_Leap_${VERSION}&arch=$(uname -i)")" && echo -e "$result" | grep -q "project.*state=.published." && echo "$result" | grep "status package" | (! grep -vE "(disabled|succeeded|excluded)") && zypper -n dup --replacefiles --auto-agree-with-licenses --force-resolution --download-in-advance && needs-restarting --reboothint >/dev/null || (command -v rebootmgrctl >/dev/null && rebootmgrctl reboot ||:)
and removed with
rm /etc/systemd/system/auto-update.* /usr/local/bin/auto-update && systemctl daemon-reload && rebootmgrctl cancel
Then tomorrow I can check if the auto-update.service ran and if it has requested a reboot.
Updated by okurz over 3 years ago
- Due date changed from 2020-11-18 to 2020-11-20
the service was running and triggering a request for reboot but I have the suspicion that it does not like to "update itself":
Nov 17 07:45:27 ariel [RPM][30148]: Transaction ID 5fb37f97 started
Nov 17 07:45:27 ariel [RPM][30148]: erase openQA-auto-update-4.6.1605530208.6b910f3fc-lp151.3447.1.noarch: success
Nov 17 07:45:27 ariel systemd[1]: Stopping Automatically patch system packages. See https://progress.opensuse.org/issues/18006 and https://progress.opensuse.org/issues/70978...
Nov 17 07:45:27 ariel openqa-auto-update[29869]: (1/4) Removing openQA-auto-update-4.6.1605530208.6b910f3fc-lp151.3447.1.noarch [...error]
Nov 17 07:46:57 ariel systemd[1]: openqa-auto-update.service: State 'stop-final-sigterm' timed out. Killing.
Nov 17 07:46:57 ariel systemd[1]: openqa-auto-update.service: Killing process 30148 (rpm) with signal SIGKILL.
Nov 17 07:46:57 ariel systemd[1]: openqa-auto-update.service: Killing process 30150 (sh) with signal SIGKILL.
Nov 17 07:46:57 ariel systemd[1]: openqa-auto-update.service: Killing process 30169 (sh) with signal SIGKILL.
Nov 17 07:46:57 ariel systemd[1]: openqa-auto-update.service: Killing process 30170 (systemctl) with signal SIGKILL.
Nov 17 07:46:57 ariel systemd[1]: Stopped Automatically patch system packages. See https://progress.opensuse.org/issues/18006 and https://progress.opensuse.org/issues/70978.
Nov 17 07:46:57 ariel systemd[1]: openqa-auto-update.service: Unit entered failed state.
Nov 17 07:46:57 ariel systemd[1]: openqa-auto-update.service: Failed with result 'timeout'.
Nov 17 07:46:57 ariel systemd[1]: Started Automatically patch system packages. See https://progress.opensuse.org/issues/18006 and https://progress.opensuse.org/issues/70978.
Trying something:
# /etc/systemd/system/openqa-auto-update.service.d/override.conf
[Service]
RestartSec=1m
Restart=on-abort
EDIT: 2020-11-20: This did not help to prevent that the service is killing itself during update causing the package to be reinstalled multiple times every minute. However, everything else works, i.e. all packages are updated and also a reboot is triggered if necessary. As the openqa-auto-update package replaces the systemd service that is running, this is causing the problem. I wonder if there is any better idea how to handle that or if it is just a stupid idea to replace the service in an update of itself.
Finally I found something: https://github.com/os-autoinst/openQA/pull/3575
Updated by okurz over 3 years ago
- Due date changed from 2020-11-20 to 2020-12-04
found a new approach: https://github.com/os-autoinst/openQA/pull/3575
waiting for review and results on o3.
Updated by okurz over 3 years ago
- Priority changed from Normal to Low
lowering prio as the original problem is solved since some time. The current work is the "upstreaming" and some beautification :)
Updated by okurz over 3 years ago
- Status changed from Feedback to Resolved
latest results on o3 look ok, e.g.:
Nov 29 03:00:08 ariel openqa-auto-update[4141]: Loading repository data...
Nov 29 03:00:09 ariel openqa-auto-update[4141]: Reading installed packages...
Nov 29 03:00:10 ariel openqa-auto-update[4141]: Warning: You are about to do a distribution upgrade with all enabled repositories. Make sure these repositories are compatible before you c>
Nov 29 03:00:10 ariel openqa-auto-update[4141]: Computing distribution upgrade...
Nov 29 03:00:10 ariel openqa-auto-update[4141]: The following 4 packages are going to be upgraded:
Nov 29 03:00:10 ariel openqa-auto-update[4141]: openQA openQA-auto-update openQA-client openQA-common
Nov 29 03:00:10 ariel openqa-auto-update[4141]: 4 packages to upgrade.
Nov 29 03:00:10 ariel openqa-auto-update[4141]: Overall download size: 3.0 MiB. Already cached: 0 B. After the operation, additional 162.0 B will be used.
Nov 29 03:00:10 ariel openqa-auto-update[4141]: Continue? [y/n/v/...? shows all options] (y): y
Nov 29 03:00:15 ariel openqa-auto-update[4141]: Retrieving package openQA-auto-update-4.6.1606497673.f3348bd43-lp152.3503.1.noarch (1/4), 259.9 KiB ( 1.4 KiB unpacked)
Nov 29 03:00:15 ariel openqa-auto-update[4141]: Retrieving: openQA-auto-update-4.6.1606497673.f3348bd43-lp152.3503.1.noarch.rpm [done]
Nov 29 03:00:15 ariel openqa-auto-update[4141]: Retrieving package openQA-common-4.6.1606497673.f3348bd43-lp152.3503.1.noarch (2/4), 341.6 KiB (359.3 KiB unpacked)
Nov 29 03:00:16 ariel openqa-auto-update[4141]: Retrieving: openQA-common-4.6.1606497673.f3348bd43-lp152.3503.1.noarch.rpm [.done]
Nov 29 03:00:16 ariel openqa-auto-update[4141]: Retrieving package openQA-client-4.6.1606497673.f3348bd43-lp152.3503.1.noarch (3/4), 271.6 KiB ( 56.5 KiB unpacked)
Nov 29 03:00:16 ariel openqa-auto-update[4141]: Retrieving: openQA-client-4.6.1606497673.f3348bd43-lp152.3503.1.noarch.rpm [done]
Nov 29 03:00:16 ariel openqa-auto-update[4141]: Retrieving package openQA-4.6.1606497673.f3348bd43-lp152.3503.1.noarch (4/4), 2.2 MiB ( 12.0 MiB unpacked)
Nov 29 03:00:16 ariel openqa-auto-update[4141]: Retrieving: openQA-4.6.1606497673.f3348bd43-lp152.3503.1.noarch.rpm [done]
Nov 29 03:00:17 ariel openqa-auto-update[4141]: Checking for file conflicts: [.....done]
Nov 29 03:00:17 ariel [RPM][4477]: Transaction ID 5fc30ec1 started
Nov 29 03:00:17 ariel [RPM][4477]: erase openQA-auto-update-4.6.1606497673.f3348bd43-lp152.3501.1.noarch: success
Nov 29 03:00:17 ariel systemd[1]: Reloading.
Nov 29 03:00:17 ariel [RPM][4477]: install openQA-auto-update-4.6.1606497673.f3348bd43-lp152.3503.1.noarch: success
Nov 29 03:00:18 ariel systemd[1]: Reloading.
Nov 29 03:00:18 ariel systemd[1]: Stopped Nightly trigger of auto-update..
Nov 29 03:00:18 ariel systemd[1]: Stopping Nightly trigger of auto-update..
Nov 29 03:00:18 ariel systemd[1]: Started Nightly trigger of auto-update..
Nov 29 03:00:18 ariel [RPM][4477]: erase openQA-auto-update-4.6.1606497673.f3348bd43-lp152.3501.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4477]: install openQA-auto-update-4.6.1606497673.f3348bd43-lp152.3503.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4477]: Transaction ID 5fc30ec1 finished: 0
Nov 29 03:00:18 ariel openqa-auto-update[4141]: (1/4) Installing: openQA-auto-update-4.6.1606497673.f3348bd43-lp152.3503.1.noarch [..........done]
Nov 29 03:00:18 ariel [RPM][4522]: Transaction ID 5fc30ec2 started
Nov 29 03:00:18 ariel [RPM][4522]: erase openQA-common-4.6.1606497673.f3348bd43-lp152.3501.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4522]: install openQA-common-4.6.1606497673.f3348bd43-lp152.3503.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4522]: erase openQA-common-4.6.1606497673.f3348bd43-lp152.3501.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4522]: install openQA-common-4.6.1606497673.f3348bd43-lp152.3503.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4522]: Transaction ID 5fc30ec2 finished: 0
Nov 29 03:00:18 ariel openqa-auto-update[4141]: (2/4) Installing: openQA-common-4.6.1606497673.f3348bd43-lp152.3503.1.noarch [..................done]
Nov 29 03:00:18 ariel [RPM][4525]: Transaction ID 5fc30ec2 started
Nov 29 03:00:18 ariel [RPM][4525]: erase openQA-client-4.6.1606497673.f3348bd43-lp152.3501.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4525]: install openQA-client-4.6.1606497673.f3348bd43-lp152.3503.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4525]: erase openQA-client-4.6.1606497673.f3348bd43-lp152.3501.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4525]: install openQA-client-4.6.1606497673.f3348bd43-lp152.3503.1.noarch: success
Nov 29 03:00:18 ariel [RPM][4525]: Transaction ID 5fc30ec2 finished: 0
Nov 29 03:00:18 ariel openqa-auto-update[4141]: (3/4) Installing: openQA-client-4.6.1606497673.f3348bd43-lp152.3503.1.noarch [.................done]
Nov 29 03:00:18 ariel [RPM][4531]: Transaction ID 5fc30ec2 started
Nov 29 03:00:19 ariel [RPM][4531]: erase openQA-4.6.1606497673.f3348bd43-lp152.3501.1.noarch: success
Nov 29 03:00:19 ariel systemd[1]: Reloading.
Nov 29 03:00:20 ariel systemd[1]: Reloading.
Nov 29 03:00:20 ariel [RPM][4531]: install openQA-4.6.1606497673.f3348bd43-lp152.3503.1.noarch: success
Nov 29 03:00:20 ariel systemd[1]: Reloading.
Nov 29 03:00:21 ariel systemd[1]: Stopping Handler for live view in openQA's web UI...
Nov 29 03:00:21 ariel openqa-livehandler-daemon[2089]: Use of uninitialized value $ret in exit at /usr/share/openqa/script/openqa-livehandler line 31.
Nov 29 03:00:21 ariel openqa-livehandler-daemon[2089]: Web application available at http://127.0.0.1:9528
Nov 29 03:00:21 ariel openqa-livehandler-daemon[2089]: Web application available at http://[::1]:9528
Nov 29 03:00:21 ariel systemd[1]: Stopping The openQA web UI...
Nov 29 03:00:21 ariel systemd[1]: Stopping The openQA daemon for various background tasks like cleanup and saving needles...
Nov 29 03:00:21 ariel systemd[1]: Stopped Enqueues an audit event cleanup task for the openQA database every day..
Nov 29 03:00:21 ariel systemd[1]: Stopping Enqueues an audit event cleanup task for the openQA database every day..
Nov 29 03:00:21 ariel systemd[1]: Started Enqueues an audit event cleanup task for the openQA database every day..
Nov 29 03:00:21 ariel systemd[1]: Stopped Handler for live view in openQA's web UI.
Nov 29 03:00:21 ariel systemd[1]: Started Handler for live view in openQA's web UI.
Nov 29 03:00:21 ariel systemd[1]: Stopped The openQA web UI.
Nov 29 03:00:21 ariel systemd[1]: Stopped Enqueues an asset cleanup task for openQA every hour..
Nov 29 03:00:21 ariel systemd[1]: Stopping Enqueues an asset cleanup task for openQA every hour..
Nov 29 03:00:21 ariel systemd[1]: Started Enqueues an asset cleanup task for openQA every hour..
Nov 29 03:00:21 ariel systemd[1]: Stopping The openQA WebSockets server...
Nov 29 03:00:21 ariel systemd[1]: Stopped Enqueues a result/screenshots/logs cleanup task for openQA every day..
Nov 29 03:00:21 ariel systemd[1]: Stopping Enqueues a result/screenshots/logs cleanup task for openQA every day..
Nov 29 03:00:21 ariel systemd[1]: Started Enqueues a result/screenshots/logs cleanup task for openQA every day..
Nov 29 03:00:21 ariel systemd[1]: Stopped Enqueues a bug cleanup task for the openQA database every week..
Nov 29 03:00:21 ariel systemd[1]: Stopping Enqueues a bug cleanup task for the openQA database every week..
Nov 29 03:00:21 ariel systemd[1]: Started Enqueues a bug cleanup task for the openQA database every week..
Nov 29 03:00:21 ariel openqa-websockets-daemon[2092]: Use of uninitialized value $ret in exit at /usr/share/openqa/script/openqa-websockets line 31.
Nov 29 03:00:21 ariel openqa-websockets-daemon[2092]: Web application available at http://127.0.0.1:9527
Nov 29 03:00:21 ariel openqa-websockets-daemon[2092]: Web application available at http://[::1]:9527
Nov 29 03:00:21 ariel systemd[1]: Stopped The openQA WebSockets server.
Nov 29 03:00:21 ariel systemd[1]: Stopping The openQA Scheduler...
Nov 29 03:00:21 ariel openqa-scheduler-daemon[2091]: Web application available at http://127.0.0.1:9529
Nov 29 03:00:21 ariel openqa-scheduler-daemon[2091]: Web application available at http://[::1]:9529
Nov 29 03:00:21 ariel systemd[1]: Stopped The openQA Scheduler.
Nov 29 03:00:21 ariel systemd[1]: Started The openQA Scheduler.
Nov 29 03:00:21 ariel systemd[1]: Started The openQA WebSockets server.
Nov 29 03:00:21 ariel systemd[1]: Started The openQA web UI.
Nov 29 03:00:24 ariel openqa-webui-daemon[4615]: Web application available at http://127.0.0.1:9526
Nov 29 03:00:24 ariel openqa-webui-daemon[4615]: Web application available at http://[::1]:9526
Nov 29 03:00:24 ariel systemd[1]: Stopped The openQA daemon for various background tasks like cleanup and saving needles.
Nov 29 03:00:24 ariel systemd[1]: Started The openQA daemon for various background tasks like cleanup and saving needles.
Nov 29 03:00:24 ariel systemd[1]: Reloading.
Nov 29 03:00:26 ariel [RPM][4531]: erase openQA-4.6.1606497673.f3348bd43-lp152.3501.1.noarch: success
Nov 29 03:00:26 ariel [RPM][4531]: install openQA-4.6.1606497673.f3348bd43-lp152.3503.1.noarch: success
Nov 29 03:00:26 ariel [RPM][4531]: Transaction ID 5fc30ec2 finished: 0
Nov 29 03:00:26 ariel openqa-auto-update[4141]: (4/4) Installing: openQA-4.6.1606497673.f3348bd43-lp152.3503.1.noarch [...................done]
Nov 29 03:00:27 ariel openqa-auto-update[4141]: There are running programs which still use files and libraries deleted or updated by recent upgrades. They should be restarted to benefit f>
Nov 29 03:00:27 ariel openqa-auto-update[4141]:
Nov 29 03:00:27 ariel openqa-auto-update[4141]: Core libraries or services have been updated.
Nov 29 03:00:27 ariel openqa-auto-update[4141]: Reboot is required to ensure that your system benefits from these updates.
Nov 29 03:00:27 ariel rebootmgrd[1709]: Reboot already in progress, ignored
Actions