Project

General

Profile

Actions

action #72118

closed

[qe-core][qem][rmt] test fails in rmt_feature

Added by dzedro over 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2020-09-30
Due date:
% Done:

100%

Estimated time:
Difficulty:

Description

Observation

Test is failing every day, this looks like SCC issue.
Please open bug and workaround it or fix the test.

openQA test in scenario sle-15-Server-DVD-Updates-x86_64-qam_rmt@64bit fails in
rmt_feature

Test suite description

Meant for scheduling rmt tests on QAM Updates https://progress.opensuse.org/issues/55784

Reproducible

Fails since (at least) Build 20200930-1 (current job)

Expected result

Last good: 20200929-1 (or more recent)

Further details

Always latest result in this scenario: latest

Actions #1

Updated by mgrifalconi over 3 years ago

Thanks for the finding Jozef!

I see some of 504 on SCC even on successful runs: https://openqa.suse.de/tests/4713655#step/rmt_feature/36

Seems like the 3 times retry with 30s delay is not enough https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/5cf18bfa356f00b4a42baac72107480382579b7d/lib/repo_tools.pm#L277

Actions #2

Updated by dzedro over 3 years ago

The problem is that looks like this is happening always on night run and at that time there is something happening or broken with SCC.
I assume 504 is gateway timeout.

Actions #4

Updated by dzedro over 3 years ago

Still failed https://openqa.suse.de/tests/4758701#step/rmt_feature/47
The thing is that there are 9 modules. This test is running approx. 1h30m
Wouldn't be 1-3 modules enough ?
Is there any log or verbose arg to know what repo update is exactly failing ?

Actions #6

Updated by tjyrinki_suse over 3 years ago

  • Status changed from New to Workable
Actions #7

Updated by mgrifalconi over 3 years ago

  • Status changed from Workable to Blocked

Waiting for feedback on bsc

Actions #8

Updated by dzedro over 3 years ago

It is always failing on product 1797 https://openqa.suse.de/tests/4882338#step/rmt_feature/36
Deactivate syncing of this product as workaround.

Actions #9

Updated by mgrifalconi over 3 years ago

Doesn't 1797 mean only the timeout in seconds?

timeout 1797 rmt-cli sync;

No idea why it is 3 seconds short of the configured timeout though: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/11116/files#diff-df032fc1142ccdd3287c7dbc03ae0e11238efd02ddb8ed1ec7bed6e7ccc6cd2fR277

script_retry 'rmt-cli sync', delay => 60, retry => 6, timeout => 1800;

Actions #10

Updated by dzedro over 3 years ago

mgrifalconi wrote:

Doesn't 1797 mean only the timeout in seconds?

timeout 1797 rmt-cli sync;

No idea why it is 3 seconds short of the configured timeout though: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/11116/files#diff-df032fc1142ccdd3287c7dbc03ae0e11238efd02ddb8ed1ec7bed6e7ccc6cd2fR277

script_retry 'rmt-cli sync', delay => 60, retry => 6, timeout => 1800;

hahah rofl I'm dumb, I have been looking on the code and have seen product numbers enabled and disabled and automatically thought this number is also product number .... ignore

Actions #11

Updated by dzedro over 3 years ago

Unscheduled from SLE 15 & 15-SP1 as this versions don't have any other SLE failures now.
Scheduled on 15-SP2 where are another failures blocking automatic approval.

Actions #12

Updated by tjyrinki_suse over 3 years ago

  • Subject changed from [qam] [rmt] test fails in rmt_feature to [qe-core][qam] [rmt] test fails in rmt_feature
Actions #13

Updated by dzedro over 3 years ago

I enabled rmt test on SLE 15 & 15-SP1 as it was passing on 15-SP2, no idea what changed ...

Actions #14

Updated by tjyrinki_suse over 3 years ago

  • Subject changed from [qe-core][qam] [rmt] test fails in rmt_feature to [qe-core][qem][rmt] test fails in rmt_feature
Actions #15

Updated by mgrifalconi about 3 years ago

  • Status changed from Blocked to Resolved
  • % Done changed from 0 to 100

Hello, considering that:

  • tests are enabled and green right now
  • there was no response on the bug

I would close both this issue and the bug.

Actions #16

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: qam_rmt
https://openqa.suse.de/tests/6859780

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The label in the openQA scenario is removed
Actions

Also available in: Atom PDF