action #125141
Salt pillars deployment pipeline failed on "tumblesle" due to invalid security sensor repo size:M
Start date:
2023-02-28
Due date:
% Done:
0%
Estimated time:
Description
Observation¶
ID: security-sensor.repo Function: pkgrepo.managed Result: False Comment: Failed to configure repo 'security-sensor.repo': Zypper command failure: Repository 'security-sensor.repo' is invalid. [security-sensor.repo|https://download.opensuse.org/repositories/security:/sensor/15.4] Valid metadata not found at specified URL History: - Signature verification failed for repomd.xml - Can't provide /repodata/repomd.xml Please check if the URIs defined for this repository are pointing to a valid repository. Skipping repository 'security-sensor.repo' because of the above error. Could not refresh the repositories because of errors.Forcing raw metadata refresh Retrieving repository 'security-sensor.repo' metadata [.......... Warning: File 'repomd.xml' from repository 'security-sensor.repo' is unsigned. Note: Signing data enables the recipient to verify that no modifications occurred after the data were signed. Accepting data with no, wrong or unknown signature can lead to a corrupted system and in extreme cases even to a system compromise. Note: File 'repomd.xml' is the repositories master index file. It ensures the integrity of the whole repo. Warning: We can't verify that no one meddled with this file, so it might not be trustworthy anymore! You should not continue unless you know it's safe. File 'repomd.xml' from repository 'security-sensor.repo' is unsigned, continue? [yes/no] (no): no error] Started: 09:39:50.917365 Duration: 9775.41 ms Changes: ---------- ID: security-sensor.repo Function: pkg.latest Name: velociraptor-client Result: False Comment: One or more requisite failed: security_sensor.security-sensor.repo Started: 09:40:00.699471 Duration: 0.011 ms Changes: … Summary for tumblesle -------------- Succeeded: 231 (changed=1) Failed: 2 -------------- Total states run: 233
(https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1427053/raw)
Suggestions¶
- Find out what the host "tumblesle" is -> a VM on qamaster.qa.suse.de (according to https://racktables.suse.de/index.php?page=object&tab=default&object_id=1300), the full domain is tumblesle.qa.suse.de
- Check whether the problem persists -> no the repo can be refreshed (on tumblesle)
- Check whether the error handling (retries) is in accordance with how other repos are configured -> we use
pkgrepo.managed: - retry: attempts: 5
for our own devel repos, maybe the same would make sense forsecurity:sensor
as well; we don't have a retry for all repos configured viapkgrepo.managed
so far, though
Remarks¶
- Likely not specific to "tumblesle".
- Looks like a temporary signing problem of security-sensor.repo (and not like a network issue). DONE So maybe a one-time issue and we don't need to introduce a retry. -> It is reproducible on tumblesle.qa.suse.de with
for i in {001..100}; do echo "## $i" && zypper ref --force -r security-sensor.repo; done
after 23 runs. Directly afterwards it was working to retrieve the file.
- Optional Try to reproduce the above problem in a clean container environment, at best for crosschecking both Leap and Tumbleweed
- Based on the above report an issue to zypper on https://github.com/openSUSE/zypper/ as zypper claims "File is unsigned" which is apparently not true. It's likely a temporary connection issue. Better retry
- Optional: Additionally report an issue with the openSUSE infrastructure with a cross-reference
History
#1
Updated by mkittler 3 months ago
- Subject changed from Salt pillars deployment pipeline failed due to invalid security sensor repo to Salt pillars deployment pipeline failed on "tumblesle" due to invalid security sensor repo
- Description updated (diff)
- Status changed from New to Feedback
- Target version set to Ready
I'll keep this in feedback after gathering more info and adding it to the ticket. We can discuss this ticket in the next estimation meeting.
#2
Updated by okurz 3 months ago
- Tags set to infra, salt, alert, pillars, tumblesle, reactive work
The problem looks familiar for problems I observed in other scopes regarding signing on download.opensuse.org. I suggest we look for a more generic "upstream solution", e.g. ask around in broader scope chat rooms, mailing lists, etc.
#3
Updated by okurz 3 months ago
- Subject changed from Salt pillars deployment pipeline failed on "tumblesle" due to invalid security sensor repo to Salt pillars deployment pipeline failed on "tumblesle" due to invalid security sensor repo size:M
- Description updated (diff)
- Status changed from Feedback to Workable
- Assignee deleted (
mkittler)
Estimated with mkittler and we could also reproduce easily:
for i in {001..100}; do echo "## $i" && zypper ref --force -r security-sensor.repo; done
#5
Updated by cdywan 3 months ago
Another incidence this morning: https://gitlab.suse.de/openqa/salt-states-openqa/-/jobs/1457431/raw
ID: security-sensor.repo Function: pkgrepo.managed Result: False Comment: Failed to configure repo 'security-sensor.repo': Zypper command failure: Repository 'security-sensor.repo' is invalid. [security-sensor.repo|https://download.opensuse.org/repositories/security:/sensor/15.4] Valid metadata not found at specified URL History: - [|] Error trying to read from 'https://download.opensuse.org/repositories/security:/sensor/15.4' - Download (curl) error for 'https://download.opensuse.org/repositories/security:/sensor/15.4/content': Error code: Connection failed Error message: Could not resolve host: download.opensuse.org Please check if the URIs defined for this repository are pointing to a valid repository. Skipping repository 'security-sensor.repo' because of the above error. Could not refresh the repositories because of errors.Forcing raw metadata refresh Retrieving repository 'security-sensor.repo' metadata [.error] Started: 08:54:20.143484 Duration: 794.301 ms Changes: ---------- ID: security-sensor.repo Function: pkg.latest Name: velociraptor-client Result: False Comment: One or more requisite failed: security_sensor.security-sensor.repo Started: 08:54:20.947657 Duration: 0.047 ms Changes: