action #163571
closedopenQA Project (public) - coordination #154768: [saga][epic][ux] State-of-art user experience for openQA
openQA Project (public) - coordination #168127: [epic] Up-to-date Perl stack
cpanspec: Move cronjob for auto updating devel:languages:perl:CPAN-{A..Z} off the obsolete engcloud size:M
0%
Description
Motivation¶
engcloud will be switched off, so we need to move the process to a different cloud server.
See also https://confluence.suse.com/pages/viewpage.action?pageId=450594448
https://build.opensuse.org/project/show/devel:languages:perl:CPAN-A etc. are regularly updated with the newest CPAN uploads.
Acceptance Criteria¶
AC1: Everything is moved from engcloud to a different provider
Suggestions¶
- The daily cronjob needs to be moved.
- The cache files under
~/obs-mirror/obs-cache/
need to be copied. They contain the information about all modules and versions. Without the cache it can take several days to query the API to gather the information again - Consider github actions and using git LFS for storing the state for each lettered project and the list of packages where cpanspec fails
- when working in a github repository with GHA the state could be commited with git LFS into a separate branch with even just amending the last commit instead of adding new commits
- Consider OpenPlatform (Harvester) as a fallback solution if GitHub does not work out: https://op-prg2.openplatform.suse.com/dashboard/home
- We also have munin running on that server with useful information about the build status, e.g. http://10.86.1.204/munin/obs-cpan-day.html . Ideally this should also be moved, but it doesn't have to run on the same server. The code for the munin data is also in https://github.com/openSUSE/autoupdate-perl
Updated by tinita 6 months ago
- Copied from action #160688: cpanspec: Run autoupdate for devel:languages:perl in github workflow size:M added
Updated by livdywan 5 months ago
- Subject changed from cpanspec: Move cronjob for auto updating devel:languages:perl:CPAN-{A..Z} off the obsolete engcloud to cpanspec: Move cronjob for auto updating devel:languages:perl:CPAN-{A..Z} off the obsolete engcloud size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by livdywan 3 months ago
- Consider OpenPlatform (Harvester) as a fallback solution if GitHub does not work out: https://op-prg2.openplatform.suse.com/dashboard/home
We looked into that last week and created virtual machines via Virtualization Management in the qe namespace to test it out. The machine could also be setup by someone else to help @tinita. This includes checking what image and resources to use and how to make the machine available to the outside and ensuring multiple people in the team have access to it.
Updated by okurz 3 months ago
- Priority changed from High to Normal
As we discussed the cost of delay is that due to the absence of the cache the update procedure becomes lengthy and eventually packages or newer versions will be missing from devel:languages:perl which can become annoying for manual submitters.
Updated by livdywan about 2 months ago
- Status changed from Workable to Blocked
livdywan wrote in #note-15:
- Consider OpenPlatform (Harvester) as a fallback solution if GitHub does not work out: https://op-prg2.openplatform.suse.com/dashboard/home
We looked into that last week and created virtual machines via Virtualization Management in the qe namespace to test it out. The machine could also be setup by someone else to help @tinita. This includes checking what image and resources to use and how to make the machine available to the outside and ensuring multiple people in the team have access to it.
I think at this point we need a new ticket to block on since I have the feeling people forget we need a new server here before we can continue. And moving this back to High likely won't help us.
Updated by livdywan about 2 months ago
- Copied to action #169633: Setup an environment to run cpan cronjobs size:S added
Updated by openqa_review about 1 month ago
- Due date set to 2024-11-29
Setting due date based on mean cycle time of SUSE QE Tools
Updated by tinita 24 days ago
- Status changed from Workable to In Progress
Actually, I found the IP in my shell history and just tried to connect. The VM is still there, up and running, just the cluster in openplatform is gone 🤷
I now ran several parts of the updating jobs and need to fix a few things, but it looks good.
Since the cluster is under maintenance, I won't schedule any cronjob for now.
Updated by openqa_review 24 days ago
- Due date set to 2024-12-17
Setting due date based on mean cycle time of SUSE QE Tools
Updated by tinita 22 days ago · Edited
Installed munin for some basic monitoring: http://10.144.110.163/munin/
edit: new ip
Updated by tinita 22 days ago
Configured logrotate.
https://github.com/openSUSE/autoupdate-perl/pull/10
Updated a bunch of packages. Will continue tomorrow.
Taking regular backups is not yet supported in the currently used harvester version (1.2).
One can take manual VM Snapshots, VM Backups and Volume Snapshots. I tried out all three. The VM Backup has a note "The current backup target does not match the existing one." Not sure how to influence that as I couldn't select anything when creating it.
Still missing: I would like to have a basic alerting.
Updated by tinita 22 days ago
Updated https://confluence.suse.com/pages/viewpage.action?pageId=450594448 with the new OpenPlatform info
Updated by tinita 21 days ago
All packages are uptodate.
I enabled the cronjobs:
IMAGE=registry.opensuse.org/home/tinita/cpanspec/container/containers/opensuse/autoupdate-perl
CPANSPEC_WRAP_BUILD_COMMAND="chgrp -R cpan ..; chmod -R g+rx ..; chmod -R g+rwx .; su cpanbuild -c '\$COMMAND'"
5 3 * * * podman run -i --rm -e HOME=/home/$USER -v$HOME:/home/$USER $IMAGE /home/$USER/autoupdate-perl/bin/fetch-cpan.sh >> $HOME/obs-mirror/logs/cpan.log 2>&1
5 4 * * * podman run -i --rm -e HOME=/home/$USER -v$HOME:/home/$USER -e CPANSPEC_WRAP_BUILD_COMMAND $IMAGE /home/$USER/autoupdate-perl/bin/update-dlp-cpan.sh >> $HOME/obs-mirror/logs/cpanupdate.log 2>&1
The vlan had to be changed, and the IP address is fixed now: 10.144.110.163
Updated by okurz 21 days ago
https://gitlab.suse.de/suse/wiki/-/merge_requests/6 is merged. You might want to consider https://github.com/openSUSE/os-update/ and maybe also systemd-mail-status, also mentioned on the os-update wiki page. Both are in openSUSE so just zypper in os-update systemd-mail-status
and configure
Updated by tinita 17 days ago
- Status changed from In Progress to Feedback
I added hopefully everything necessary to setup the server here: https://confluence.suse.com/pages/viewpage.action?pageId=450594448
Updated by okurz 3 days ago
- Copied to action #174709: Select better network for obs-cpan-mirror VM on op-prg2 harvester added