action #163571
closed
openQA Project (public) - coordination #127031: [saga][epic] openQA for SUSE customers
openQA Project (public) - coordination #168127: [epic] Up-to-date Perl stack
cpanspec: Move cronjob for auto updating devel:languages:perl:CPAN-{A..Z} off the obsolete engcloud size:M
Added by tinita 10 months ago.
Updated about 1 month ago.
Description
Motivation¶
engcloud will be switched off, so we need to move the process to a different cloud server.
See also https://confluence.suse.com/pages/viewpage.action?pageId=450594448
https://build.opensuse.org/project/show/devel:languages:perl:CPAN-A etc. are regularly updated with the newest CPAN uploads.
Acceptance Criteria¶
AC1: Everything is moved from engcloud to a different provider
Suggestions¶
- The daily cronjob needs to be moved.
- The cache files under
~/obs-mirror/obs-cache/ need to be copied. They contain the information about all modules and versions. Without the cache it can take several days to query the API to gather the information again
- Consider github actions and using git LFS for storing the state for each lettered project and the list of packages where cpanspec fails
- when working in a github repository with GHA the state could be commited with git LFS into a separate branch with even just amending the last commit instead of adding new commits
- Consider OpenPlatform (Harvester) as a fallback solution if GitHub does not work out: https://op-prg2.openplatform.suse.com/dashboard/home
- We also have munin running on that server with useful information about the build status, e.g. http://10.86.1.204/munin/obs-cpan-day.html . Ideally this should also be moved, but it doesn't have to run on the same server. The code for the munin data is also in https://github.com/openSUSE/autoupdate-perl
- Copied from action #160688: cpanspec: Run autoupdate for devel:languages:perl in github workflow size:M added
- Target version set to Ready
@tinita I would like to reconsider this with you. As this is about updating packages on OBS we should try really hard to use a platform that is available to the community, i.e. not SUSE internal openplatform. I am sure we can find a good solution.
- Subject changed from cpanspec: Move cronjob for auto updating devel:languages:perl:CPAN-{A..Z} to openplatform to cpanspec: Move cronjob for auto updating devel:languages:perl:CPAN-{A..Z} off the obsolete engcloud
- Description updated (diff)
- Description updated (diff)
- Subject changed from cpanspec: Move cronjob for auto updating devel:languages:perl:CPAN-{A..Z} off the obsolete engcloud to cpanspec: Move cronjob for auto updating devel:languages:perl:CPAN-{A..Z} off the obsolete engcloud size:M
- Description updated (diff)
- Status changed from New to Workable
I asked in Slack about when engcloud will be shutdown.
- Priority changed from Normal to High
The plan is to shutdown "in September", so I guess this should be high prio now.
We don't have a backup for the obs mirror host, so I started to rsync files, in case the machine gets turned off.
tinita wrote in #note-9:
We don't have a backup for the obs mirror host, so I started to rsync files, in case the machine gets turned off.
Where do the files end up? Somewhere on osd?
Where do the files end up? Somewhere on osd?
on my own laptop for now
We should discuss this again. As this is a High ticket, but effectively currently treated as a Low ticket. So either we figure out how to help out here to get it done asap or really re-consider the priority of it.
Will try and bring it up at the next opportunity this week
- Description updated (diff)
We looked into that last week and created virtual machines via Virtualization Management in the qe namespace to test it out. The machine could also be setup by someone else to help @tinita. This includes checking what image and resources to use and how to make the machine available to the outside and ensuring multiple people in the team have access to it.
The cloud server was shutdown in the meantime. This should be done soon.
- Priority changed from High to Normal
As we discussed the cost of delay is that due to the absence of the cache the update procedure becomes lengthy and eventually packages or newer versions will be missing from devel:languages:perl which can become annoying for manual submitters.
- Parent task set to #168127
- Status changed from Workable to Blocked
livdywan wrote in #note-15:
We looked into that last week and created virtual machines via Virtualization Management in the qe namespace to test it out. The machine could also be setup by someone else to help @tinita. This includes checking what image and resources to use and how to make the machine available to the outside and ensuring multiple people in the team have access to it.
I think at this point we need a new ticket to block on since I have the feeling people forget we need a new server here before we can continue. And moving this back to High likely won't help us.
- Copied to action #169633: Setup an environment to run cpan cronjobs size:S added
- Status changed from Blocked to In Progress
- Due date set to 2024-11-29
Setting due date based on mean cycle time of SUSE QE Tools
- Status changed from In Progress to Workable
- Due date deleted (
2024-11-29)
The cluster I created the VM on is in maintenance. It is supposed to take at least until the end of tuesday (tomorrow), so I can't work on it.
- Status changed from Workable to In Progress
Actually, I found the IP in my shell history and just tried to connect. The VM is still there, up and running, just the cluster in openplatform is gone 🤷
I now ran several parts of the updating jobs and need to fix a few things, but it looks good.
Since the cluster is under maintenance, I won't schedule any cronjob for now.
- Due date set to 2024-12-17
Setting due date based on mean cycle time of SUSE QE Tools
Configured logrotate.
https://github.com/openSUSE/autoupdate-perl/pull/10
Updated a bunch of packages. Will continue tomorrow.
Taking regular backups is not yet supported in the currently used harvester version (1.2).
One can take manual VM Snapshots, VM Backups and Volume Snapshots. I tried out all three. The VM Backup has a note "The current backup target does not match the existing one." Not sure how to influence that as I couldn't select anything when creating it.
Still missing: I would like to have a basic alerting.
All packages are uptodate.
I enabled the cronjobs:
IMAGE=registry.opensuse.org/home/tinita/cpanspec/container/containers/opensuse/autoupdate-perl
CPANSPEC_WRAP_BUILD_COMMAND="chgrp -R cpan ..; chmod -R g+rx ..; chmod -R g+rwx .; su cpanbuild -c '\$COMMAND'"
5 3 * * * podman run -i --rm -e HOME=/home/$USER -v$HOME:/home/$USER $IMAGE /home/$USER/autoupdate-perl/bin/fetch-cpan.sh >> $HOME/obs-mirror/logs/cpan.log 2>&1
5 4 * * * podman run -i --rm -e HOME=/home/$USER -v$HOME:/home/$USER -e CPANSPEC_WRAP_BUILD_COMMAND $IMAGE /home/$USER/autoupdate-perl/bin/update-dlp-cpan.sh >> $HOME/obs-mirror/logs/cpanupdate.log 2>&1
The vlan had to be changed, and the IP address is fixed now: 10.144.110.163
- Status changed from In Progress to Feedback
- Status changed from Feedback to Resolved
- Copied to action #174709: Select proper non-CC network for obs-cpan-mirror VM on op-prg2 harvester size:S added
- Due date deleted (
2024-12-17)
Also available in: Atom
PDF