action #158814
closedo3 logreport: Active version 24 is greater than the latest version 23 at .../Mojo/Pg.pm size:M
Description
Observation¶
From /var/log/openqa
:
[2024-04-10T15:20:04.190828Z] [error] [6XDd4QtKL9Pc] Active version 24 is greater than the latest version 23 at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Pg.pm line 100.
[2024-04-10T15:25:06.279066Z] [error] [UYVJthN6XmUF] Active version 24 is greater than the latest version 23 at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Pg.pm line 100.
[2024-04-10T15:45:04.107327Z] [error] [T4fj_bgqmapR] Active version 24 is greater than the latest version 23 at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Pg.pm line 100.
[2024-04-10T15:50:02.488807Z] [error] [xYSu0-F1l8A6] Active version 24 is greater than the latest version 23 at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/Pg.pm line 100.
I can't find that in the osd openqa journal.
Looking at the time, it looks related to #158805
Acceptance criteria¶
- AC1: All our services using Minion are restarted automatically as necessary on an update of the perl-Minion package
Suggestions¶
- Catch the above error in active openQA components and trigger a self-restart (without causing an endless loop of restarts)
- Use a systemd path unit to trigger an auto-restart
Updated by mkittler 6 months ago
- Subject changed from o3 logreport: Active version 24 is greater than the latest version 23 at .../Mojo/Pg.pm to o3 logreport: Active version 24 is greater than the latest version 23 at .../Mojo/Pg.pm size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by jbaier_cz 6 months ago
Maybe this could be just https://github.com/os-autoinst/openQA/pull/5573 ?
Updated by jbaier_cz 6 months ago
- Status changed from Workable to In Progress
I installed the changed rpms manually, enable the auto-restart unit via systemctl enable --now openqa-minion-restart.path
and after faking Minion update via zypper install --force perl-Minion
, I spotted the following:
Apr 11 15:49:33 dagr.qam.suse.cz [RPM][26338]: Transaction ID 6617ea6d started
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: Starting Restarts services which are using Minion...
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: Stopping Handler for live view in openQA's web UI...
Apr 11 15:49:34 dagr.qam.suse.cz openqa-webui-daemon[25830]: [warn] Stopping worker 25984 immediately
...
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: Stopping The openQA web UI...
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: Stopping The openQA daemon for various background tasks like cleanup and saving needles...
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: openqa-livehandler.service: Deactivated successfully.
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: Stopped Handler for live view in openQA's web UI.
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: Started Handler for live view in openQA's web UI.
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: openqa-webui.service: Deactivated successfully.
Apr 11 15:49:34 dagr.qam.suse.cz systemd[1]: Stopped The openQA web UI.
Apr 11 15:49:34 dagr.qam.suse.cz [RPM][26338]: install perl-Minion-10.290.0-lp155.2.1.noarch: success
Apr 11 15:49:34 dagr.qam.suse.cz [RPM][26338]: install perl-Minion-10.290.0-lp155.2.1.noarch: success
Apr 11 15:49:34 dagr.qam.suse.cz [RPM][26338]: Transaction ID 6617ea6d finished: 0
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: openqa-gru.service: Deactivated successfully.
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: Stopped The openQA daemon for various background tasks like cleanup and saving needles.
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: Starting Setup local PostgreSQL database for openQA...
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: openqa-setup-db.service: Deactivated successfully.
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: Finished Setup local PostgreSQL database for openQA.
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: Started The openQA web UI.
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: Started The openQA daemon for various background tasks like cleanup and saving needles.
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: openqa-minion-restart.service: Deactivated successfully.
Apr 11 15:49:37 dagr.qam.suse.cz systemd[1]: Finished Restarts services which are using Minion.
I believe that's what we want.
Updated by openqa_review 6 months ago
- Due date set to 2024-04-26
Setting due date based on mean cycle time of SUSE QE Tools
Updated by jbaier_cz 6 months ago
The code is merged and deployed. I did systemctl enable --now openqa-minion-restart.path
on o3 to enable the service. Do we want that also for osd? If yes: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1155
Updated by livdywan 6 months ago
jbaier_cz wrote in #note-8:
The code is merged and deployed. I did
systemctl enable --now openqa-minion-restart.path
on o3 to enable the service. Do we want that also for osd? If yes: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1155
I guess this isn't quite working yet... or maybe deployed too early?
ID: openqa-minion-restart.path
Function: service.running
Result: False
Comment: The named service openqa-minion-restart.path is not available
Started: 00:57:40.673701
Duration: 43.035 ms
Changes:
https://gitlab.suse.de/openqa/salt-states-openqa/-/jobs/2500285
Updated by jbaier_cz 6 months ago
livdywan wrote in #note-11:
jbaier_cz wrote in #note-8:
The code is merged and deployed. I did
systemctl enable --now openqa-minion-restart.path
on o3 to enable the service. Do we want that also for osd? If yes: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1155I guess this isn't quite working yet... or maybe deployed too early?
According to
# ls -l /usr/lib/systemd/system/openqa-minion-restart.path
-rw-r--r-- 1 root root 134 Apr 15 17:46 /usr/lib/systemd/system/openqa-minion-restart.path
that file should be deployed, so it might be a salt thing not understanding I do not want a .service
file but a .path
unit file
Updated by livdywan 6 months ago ยท Edited
# ls -l /usr/lib/systemd/system/openqa-minion-restart.path -rw-r--r-- 1 root root 134 Apr 15 17:46 /usr/lib/systemd/system/openqa-minion-restart.path
that file should be deployed, so it might be a salt thing not understanding I do not want a
.service
file but a.path
unit file
Checking the docs, neither salt.states.service nor salt.modules.systemd_service says anything about path units. The implementation seems to recognize it, though?
Updated by jbaier_cz 6 months ago
Upstream handled in https://github.com/os-autoinst/openQA/pull/5584