action #111446
closedopenQA-in-openQA tests fail due to corrupted downloaded rpm auto_review:"Test died: command '.*zypper -n in os-autoinst-distri-opensuse-deps' failed at openqa//tests/install/test_distribution.pm line 1.*":retry
0%
Description
Observation¶
https://openqa.opensuse.org/tests/2364769#step/test_distribution/8 shows in the screenshot
so a digest failed for an RPM package. I wonder what users are expected to do. There is a question to "retry retrieval" which of course is negated due to the non-interactive mode. Of course we try to get around that with another "retry" on the outer level but this becomes annoying quickly so we should aim to get a proper solution into the OS itself, e.g. into zypper.
Acceptance criteria¶
- AC1: Common openSUSE related openQA test distributions (e.g. os-autoinst-distri-opensuse+os-autoinst-distri-openQA) uses retry on related errors automatically
Suggestions¶
There are already upstream feature requests like already resolved https://github.com/openSUSE/zypper/issues/177 and https://github.com/openSUSE/zypper/issues/312 but also there is at least one open feature request https://github.com/openSUSE/zypper/issues/420 which we can support, propose an implementation to, ask nicely to be prioritized, etc.
Files
Updated by andriinikitin over 2 years ago
To me these "digest failed" and "file not found" in the next job look like concurrency issues when rpm is replaced on server during download or after zypper refresh.
There is no protection from that in infrastructure and it is likely to happen if source gets released / published often.
The best idea I have at the moment is just retry zypper ref -f && zypper in ... again
Updated by okurz over 2 years ago
andriinikitin wrote:
To me these "digest failed" and "file not found" in the next job look like concurrency issues when rpm is replaced on server during download or after zypper refresh.
There is no protection from that in infrastructure and it is likely to happen if source gets released / published often.
The best idea I have at the moment is just retryzypper ref -f && zypper in ...again
Yes, but IMHO that is something that zypper should allow to do in the non-interactive mode. It already offers to retry but I see no point in needing to do that in another way in non-interactive scripts when zypper already offers it in the interactive way.
Updated by okurz over 2 years ago
- Status changed from New to Blocked
- Assignee set to okurz
Updated by okurz over 2 years ago
- Blocked by action #112232: [tools] Multiple recurring failures due to zypper failing to download packages temporarily added
Updated by okurz over 2 years ago
- Subject changed from openQA-in-openQA tests fail due to corrupted downloaded rpm auto_review:"Test died: command 'zypper -n in os-autoinst-distri-opensuse-deps' failed at openqa//tests/install/test_distribution.pm line 13":retry to openQA-in-openQA tests fail due to corrupted downloaded rpm auto_review:"Test died: command '.*zypper -n in os-autoinst-distri-opensuse-deps' failed at openqa//tests/install/test_distribution.pm line 1.*":retry
In the meantime we have updated the test code to use retries but that's not enough, see #112232, the blocker. Updated auto-review expression accordingly.
Updated by okurz over 2 years ago
- Status changed from Blocked to Resolved
We track the upstream issue. We use https://software.opensuse.org/package/retry and are ok with that because we don't expect the upstream issue to be ever fixed (but never say never). Welcome to the real world