action #180707
Updated by gpuliti 6 days ago
## Observation
Trying to do a rollback for #180695 I found that openqaworker20 there are no cached versions of os-autoinst and mostly only one package which is outdated: useless:
```
okurz@ariel:~> hosts="openqaworker20 openqaworker21 openqaworker22 openqaworker23 openqaworker24 openqaworker25 openqaworker26 openqaworker27 openqaworker28 openqaworker-arm21 openqaworker-arm22 qa-power8-3"; for i in $hosts; do echo "### $i" && ssh root@$i "rpm -q os-autoinst" ; done
### openqaworker20
os-autoinst-4.6.1739536098.bc54195-lp156.2047.1.x86_64
### openqaworker21
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.x86_64
### openqaworker22
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.x86_64
… ### openqaworker23
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.x86_64
### openqaworker24
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.x86_64
### openqaworker25
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.x86_64
### openqaworker26
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.x86_64
### openqaworker27
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.x86_64
### openqaworker28
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.x86_64
### openqaworker-arm21
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.aarch64
### openqaworker-arm22
os-autoinst-5.1744220409.84f2b44-lp156.2074.1.aarch64
…
```
From `journalctl -u openqa-auto-update` since 2025-02-18 (!)
```
Feb 18 03:00:19 openqaworker20 systemd[1]: Started Automatically patch system packages. See https://progress.opensuse.org/issues/18006 and https://progress.opensuse.org/issues/70978.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45450]: devel:openQA looks good for Leap 15.6 (x86_64)
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: Repository 'Update repository of openSUSE Backports' is up to date.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: Repository 'Update repository with updates from SUSE Linux Enterprise 15' is up to date.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: Repository 'devel_openQA' is up to date.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: Retrieving repository 'devel_openQA_Leap' metadata [.error]
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: Repository 'devel_openQA_Leap' is invalid.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: [devel_openQA_Leap|http://download.opensuse.org/repositories/devel:openQA:Leap:15.4/15.4] Failed to retrieve new repository metadata.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: History:
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: - [devel_openQA_Leap|http://download.opensuse.org/repositories/devel:openQA:Leap:15.4/15.4] Repository type can't be determined.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: Please check if the URIs defined for this repository are pointing to a valid repository.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: Skipping repository 'devel_openQA_Leap' because of the above error.
Feb 18 03:00:19 openqaworker20 openqa-auto-update[45464]: Repository 'repo-oss' is up to date.
Feb 18 03:00:20 openqaworker20 openqa-auto-update[45464]: Some of the repositories have not been refreshed because of an error.
Feb 18 03:00:20 openqaworker20 systemd[1]: openqa-auto-update.service: Main process exited, code=exited, status=4/NOPERMISSION
Feb 18 03:00:20 openqaworker20 systemd[1]: openqa-auto-update.service: Failed with result 'exit-code'.
```
## Acceptance criteria
* **AC1:** All o3 workers have a consistent os-autoinst version
* **AC2:** It is understood why w20 is outdated
## Suggestions
* Understand what brought us Look into the situation https://github.com/os-autoinst/openQA/commit/d4e1252241d33da29f8228758b95969e7c66cf66
* The `openqa-auto-update.service` is failing (at the Introduce proper checks to ensure that we don't clean out complete caches again
* Monitor over a reasonable time we are estimating this ticket).
* Ensure that *all* o3 workers have at least the previously installed version of os-autoinst and all relevant openQA packages
Back