Project

General

Profile

Actions

action #111446

closed

openQA-in-openQA tests fail due to corrupted downloaded rpm auto_review:"Test died: command '.*zypper -n in os-autoinst-distri-opensuse-deps' failed at openqa//tests/install/test_distribution.pm line 1.*":retry

Added by okurz almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Low
Assignee:
Target version:
Start date:
2022-05-23
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://openqa.opensuse.org/tests/2364769#step/test_distribution/8 shows in the screenshot

canvas.png

so a digest failed for an RPM package. I wonder what users are expected to do. There is a question to "retry retrieval" which of course is negated due to the non-interactive mode. Of course we try to get around that with another "retry" on the outer level but this becomes annoying quickly so we should aim to get a proper solution into the OS itself, e.g. into zypper.

Acceptance criteria

  • AC1: Common openSUSE related openQA test distributions (e.g. os-autoinst-distri-opensuse+os-autoinst-distri-openQA) uses retry on related errors automatically

Suggestions

There are already upstream feature requests like already resolved https://github.com/openSUSE/zypper/issues/177 and https://github.com/openSUSE/zypper/issues/312 but also there is at least one open feature request https://github.com/openSUSE/zypper/issues/420 which we can support, propose an implementation to, ask nicely to be prioritized, etc.


Files

canvas.png (44.2 KB) canvas.png okurz, 2022-05-23 09:40

Related issues 1 (0 open1 closed)

Blocked by QA - action #112232: [tools] Multiple recurring failures due to zypper failing to download packages temporarilyResolvedokurz2022-06-09

Actions
Actions #1

Updated by okurz almost 2 years ago

  • Tags set to reactive work
Actions #2

Updated by andriinikitin almost 2 years ago

To me these "digest failed" and "file not found" in the next job look like concurrency issues when rpm is replaced on server during download or after zypper refresh.
There is no protection from that in infrastructure and it is likely to happen if source gets released / published often.
The best idea I have at the moment is just retry zypper ref -f && zypper in ... again

Actions #3

Updated by okurz almost 2 years ago

andriinikitin wrote:

To me these "digest failed" and "file not found" in the next job look like concurrency issues when rpm is replaced on server during download or after zypper refresh.
There is no protection from that in infrastructure and it is likely to happen if source gets released / published often.
The best idea I have at the moment is just retry zypper ref -f && zypper in ... again

Yes, but IMHO that is something that zypper should allow to do in the non-interactive mode. It already offers to retry but I see no point in needing to do that in another way in non-interactive scripts when zypper already offers it in the interactive way.

Actions #4

Updated by okurz almost 2 years ago

  • Priority changed from Normal to Low
Actions #5

Updated by okurz almost 2 years ago

  • Status changed from New to Blocked
  • Assignee set to okurz
Actions #6

Updated by okurz over 1 year ago

  • Blocked by action #112232: [tools] Multiple recurring failures due to zypper failing to download packages temporarily added
Actions #7

Updated by okurz over 1 year ago

  • Subject changed from openQA-in-openQA tests fail due to corrupted downloaded rpm auto_review:"Test died: command 'zypper -n in os-autoinst-distri-opensuse-deps' failed at openqa//tests/install/test_distribution.pm line 13":retry to openQA-in-openQA tests fail due to corrupted downloaded rpm auto_review:"Test died: command '.*zypper -n in os-autoinst-distri-opensuse-deps' failed at openqa//tests/install/test_distribution.pm line 1.*":retry

In the meantime we have updated the test code to use retries but that's not enough, see #112232, the blocker. Updated auto-review expression accordingly.

Actions #8

Updated by okurz over 1 year ago

  • Status changed from Blocked to Resolved

We track the upstream issue. We use https://software.opensuse.org/package/retry and are ok with that because we don't expect the upstream issue to be ever fixed (but never say never). Welcome to the real world

Actions

Also available in: Atom PDF