Project

General

Profile

Actions

action #110803

closed

[tools] openqa_from_containers: test fails in setup_env size:S

Added by jbaier_cz over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
Start date:
2022-05-09
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario openqa-Tumbleweed-dev-x86_64-openqa_from_containers@64bit-2G fails in
setup_env

The error message from docker suggests, there could be a problem with the docker registry itself (maybe a limit has been exceeded?).

[  870.989997] dockerd[2355]: time="2022-05-09T07:39:08.406609692-04:00" level=error msg="Handler for POST /v1.41/images/create returned error: Head \"https://registry-1.docker.io/v2/library/postgres/manifests/latest\": Get \"https://auth.docker.io/token?scope=repository%3Alibrary%2Fpostgres%3Apull&service=registry.docker.io\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"

Retry did help in this case. The actual container image we try to retrieve from the docker hub is "postgres" here.

Test suite description

Maintainer: okurz@suse.de Test for running openQA itself from containers. To be used with "openqa" distri. Introduced retry on the job level due to https://progress.opensuse.org/issues/108665 as there can still be sporadic network issues sometimes.

Reproducible

Fails since (at least) Build :TW.11254 (current job)

Expected result

Last good: :TW.11253 (or more recent)

Suggestions

  • Use podman instead of docker (might actually be relevant to fixing in case of timeouts)
  • Consider using a postgres container from the openSUSE registry
  • Consider adding retrying around the docker call within the test (we have RETRY on the test)

Further details

Always latest result in this scenario: latest


Related issues 1 (0 open1 closed)

Copied to openQA Project - action #110911: Do not trigger investigation jobs and notifications on jobs where openQA automatically retries size:MResolvedokurz2022-05-09

Actions
Actions #1

Updated by maritawerner over 2 years ago

@okurz do you want to take that? As a community member?

Actions #2

Updated by okurz over 2 years ago

  • Tags set to reactive work
  • Subject changed from openqa_from_containers: test fails in setup_env to [tools] openqa_from_containers: test fails in setup_env
  • Priority changed from Normal to High
  • Target version set to Ready

Oops, that's actually a ticket by the team, for the team. Thank you for noticing

Actions #3

Updated by livdywan over 2 years ago

  • Subject changed from [tools] openqa_from_containers: test fails in setup_env to [tools] openqa_from_containers: test fails in setup_env size:S
  • Description updated (diff)
  • Status changed from New to Workable

We discussed keeping this as High despite the retry since it's part of the openQA self tests

Actions #4

Updated by okurz over 2 years ago

  • Copied to action #110911: Do not trigger investigation jobs and notifications on jobs where openQA automatically retries size:M added
Actions #5

Updated by livdywan over 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to livdywan

Let's see if I can address this with a trivial PR

Actions #6

Updated by livdywan over 2 years ago

  • Status changed from In Progress to Feedback

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84

openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84 https://openqa.opensuse.org/tests/2335254
Created job #2339617: openqa-Tumbleweed-dev-x86_64-Build:TW.11254-openqa_from_containers@64bit-2G -> https://openqa.opensuse.org/t2339617

There seems to be no postgres on the openSUSE registry so I'm not following that suggestion.

Actions #7

Updated by livdywan over 2 years ago

cdywan wrote:

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84

openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84 https://openqa.opensuse.org/tests/2335254
Created job #2339617: openqa-Tumbleweed-dev-x86_64-Build:TW.11254-openqa_from_containers@64bit-2G -> https://openqa.opensuse.org/t2339617

There seems to be no postgres on the openSUSE registry so I'm not following that suggestion.

Successful run w/o the podman changes: https://openqa.opensuse.org/tests/2339705

Actions #8

Updated by livdywan over 2 years ago

cdywan wrote:

cdywan wrote:

https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84

openqa-clone-custom-git-refspec https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/84 https://openqa.opensuse.org/tests/2335254
Created job #2339617: openqa-Tumbleweed-dev-x86_64-Build:TW.11254-openqa_from_containers@64bit-2G -> https://openqa.opensuse.org/t2339617

There seems to be no postgres on the openSUSE registry so I'm not following that suggestion.

Successful run w/o the podman changes: https://openqa.opensuse.org/tests/2339705

Potential alternative: https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/86

Actions #9

Updated by livdywan over 2 years ago

  • Status changed from Feedback to Resolved

Seems to work fine

Actions

Also available in: Atom PDF