Project

General

Profile

action #110803

[tools] openqa_from_containers: test fails in setup_env size:S

Added by jbaier_cz 3 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
Start date:
2022-05-09
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario openqa-Tumbleweed-dev-x86_64-openqa_from_containers@64bit-2G fails in
setup_env

The error message from docker suggests, there could be a problem with the docker registry itself (maybe a limit has been exceeded?).

[  870.989997] dockerd[2355]: time="2022-05-09T07:39:08.406609692-04:00" level=error msg="Handler for POST /v1.41/images/create returned error: Head \"https://registry-1.docker.io/v2/library/postgres/manifests/latest\": Get \"https://auth.docker.io/token?scope=repository%3Alibrary%2Fpostgres%3Apull&service=registry.docker.io\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"

Retry did help in this case. The actual container image we try to retrieve from the docker hub is "postgres" here.

Test suite description

Maintainer: okurz@suse.de Test for running openQA itself from containers. To be used with "openqa" distri. Introduced retry on the job level due to https://progress.opensuse.org/issues/108665 as there can still be sporadic network issues sometimes.

Reproducible

Fails since (at least) Build :TW.11254 (current job)

Expected result

Last good: :TW.11253 (or more recent)

Suggestions

  • Use podman instead of docker (might actually be relevant to fixing in case of timeouts)
  • Consider using a postgres container from the openSUSE registry
  • Consider adding retrying around the docker call within the test (we have RETRY on the test)

Further details

Always latest result in this scenario: latest


Related issues

Copied to openQA Project - action #110911: Do not trigger investigation jobs and notifications on jobs where openQA automatically retries size:MResolved2022-05-09

History

#1 Updated by maritawerner 3 months ago

okurz do you want to take that? As a community member?

#2 Updated by okurz 3 months ago

  • Tags set to reactive work
  • Subject changed from openqa_from_containers: test fails in setup_env to [tools] openqa_from_containers: test fails in setup_env
  • Priority changed from Normal to High
  • Target version set to Ready

Oops, that's actually a ticket by the team, for the team. Thank you for noticing

#3 Updated by cdywan 3 months ago

  • Subject changed from [tools] openqa_from_containers: test fails in setup_env to [tools] openqa_from_containers: test fails in setup_env size:S
  • Description updated (diff)
  • Status changed from New to Workable

We discussed keeping this as High despite the retry since it's part of the openQA self tests

#4 Updated by okurz 3 months ago

  • Copied to action #110911: Do not trigger investigation jobs and notifications on jobs where openQA automatically retries size:M added

#5 Updated by cdywan 3 months ago

  • Status changed from Workable to In Progress
  • Assignee set to cdywan

Let's see if I can address this with a trivial PR

#6 Updated by cdywan 3 months ago

  • Status changed from In Progress to Feedback

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84

openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84 https://openqa.opensuse.org/tests/2335254
Created job #2339617: openqa-Tumbleweed-dev-x86_64-Build:TW.11254-openqa_from_containers@64bit-2G -> https://openqa.opensuse.org/t2339617

There seems to be no postgres on the openSUSE registry so I'm not following that suggestion.

#7 Updated by cdywan 3 months ago

cdywan wrote:

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84

openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84 https://openqa.opensuse.org/tests/2335254
Created job #2339617: openqa-Tumbleweed-dev-x86_64-Build:TW.11254-openqa_from_containers@64bit-2G -> https://openqa.opensuse.org/t2339617

There seems to be no postgres on the openSUSE registry so I'm not following that suggestion.

Successful run w/o the podman changes: https://openqa.opensuse.org/tests/2339705

#8 Updated by cdywan 3 months ago

cdywan wrote:

cdywan wrote:

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84

openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84 https://openqa.opensuse.org/tests/2335254
Created job #2339617: openqa-Tumbleweed-dev-x86_64-Build:TW.11254-openqa_from_containers@64bit-2G -> https://openqa.opensuse.org/t2339617

There seems to be no postgres on the openSUSE registry so I'm not following that suggestion.

Successful run w/o the podman changes: https://openqa.opensuse.org/tests/2339705

Potential alternative: https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/86

#9 Updated by cdywan 3 months ago

  • Status changed from Feedback to Resolved

Seems to work fine

Also available in: Atom PDF