Project

General

Profile

action #115340

[qem] test fails in add_update_test_repo[and zypper_lifecycle], seems performance issue when accessing maintenance repositories

Added by rfan1 about 1 month ago. Updated about 1 month ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
-
Start date:
2022-08-16
Due date:
% Done:

100%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP4-Server-DVD-Updates-x86_64-mau-webserver+15@64bit fails in
add_update_test_repo

Test suite description

Testsuite maintained at https://gitlab.suse.de/qa-maintenance/qam-openqa-yml. SLE15+ version of mau-webserver

Reproducible

Fails since (at least) Build 20220815-1

Expected result

Last good: 20220814-1 (or more recent)

Further details

Always latest result in this scenario: latest


Related issues

Related to openQA Tests - action #115406: [qe-core] needle fails on add_update_test_repoRejected2022-08-17

History

#1 Updated by rfan1 about 1 month ago

Lots of Core Maintenance Updates tests failed at add_update_test_repo and zypper_lifecycle. seems we are hitting some performance issues when accessing maintenance repositories.

#2 Updated by apappas about 1 month ago

  • Tags set to bugbusters

rfan1 when you make tickets like that please add the bugbusters tag.

#3 Updated by rfan1 about 1 month ago

As Michael Andres's kindly reply, there might be some problem with download.suse.de

Even small requests resolved by download.suse.de itself take time:
+++++++++++++++++++++++++++
+++ 04:42:49 http://download.suse.de/ibs/SUSE:/Maintenance:/25413/SUSE_Updates_SLE-Module-Containers_15-SP4_s390x/repodata/repomd.xml.key
++ 04:42:54 HTTP response: 200
The slow connection to download.suse.de is probably nothing zypper can influence

#4 Updated by rfan1 about 1 month ago

Add one more commet:

I checked one of the failed case, and I can see lots of error messages in serial console,
https://openqa.suse.de/tests/9345669/logfile?filename=serial0.txt

2022-08-16 18:59:20 <5> install(17374) [zypp] Exception.cc(log):166 MediaCurl.cc(evaluateCurlCode):1126 THROW:    File '/repo/repoindex.xml' not found on medium 'http://download.suse.de/ibs/SUSE:/Maintenance:/24571/SUSE_Updates_SLE-SERVER_12-SP5_s390x/'
2022-08-16 18:59:20 <3> install(17374) [zypp] MediaCurl.cc(doGetFileCopyFile):1575 curl error: 22: The requested URL returned error: 404 Not Found, temp file size 0 bytes.
2022-08-16 18:59:20 <5> install(17374) [zypp] Exception.cc(log):166 MediaCurl.cc(evaluateCurlCode):1126 THROW:    File './media.1/products' not found on medium 'http://download.suse.de/ibs/SUSE:/Maintenance:/24571/SUSE_Updates_SLE-SERVER_12-SP5_s390x/'
2022-08-16 18:59:20 <5> install(17374) [zypp] Exception.cc(log):166 MediaCurl.cc(doGetFileCopyFile):1591 RETHROW:  File './media.1/products' not found on medium 'http://download.suse.de/ibs/SUSE:/Maintenance:/24571/SUSE_Updates_SLE-SERVER_12-SP5_s390x/'
2022-08-16 18:59:20 <5> install(17374) [zypp] Exception.cc(log):166 MediaMultiCurl.cc(doGetFileCopy):1363 RETHROW:  File './media.1/products' not found on medium 'http://download.suse.de/ibs/SUSE:/Maintenance:/24571/SUSE_Updates_SLE-SERVER_12-SP5_s390x/'
2022-08-16 18:59:20 <5> install(17374) [zypp] Exception.cc(log):166 MediaCurl.cc(getFileCopy):1023 RETHROW:  File './media.1/products' not found on medium 'http://download.suse.de/ibs/SUSE:/Maintenance:/24571/SUSE_Updates_SLE-SERVER_12-SP5_s390x/'
2022-08-16 18:59:20 <5> install(17374) [zypp] Exception.cc(log):166 MediaProducts.h(productsInMedia):103 CAUGHT:   File './media.1/products' not found on medium 'http://download.suse.de/ibs/SUSE:/Maintenance:/24571/SUSE_Updates_SLE-SERVER_12-SP5_s390x/'
2022-08-16 18:59:21 <3> install(17374) [zypp] MediaCurl.cc(doGetFileCopyFile):1575 curl error: 22: The requested URL returned error: 404 Not Found, temp file size 0 bytes.

Who can help with this issue?

#5 Updated by tjyrinki_suse about 1 month ago

Buildops can now reproduce the apache timeouts that cause this.

#6 Updated by rfan1 about 1 month ago

  • Related to action #115406: [qe-core] needle fails on add_update_test_repo added

#7 Updated by andriinikitin about 1 month ago

  • Status changed from New to Feedback

tjyrinki_suse wrote:

Buildops can now reproduce the apache timeouts that cause this.

Apparently the incoming load on download.suse.de increased at least twice comparing to recent weeks (maybe find why?), so the first explanation was that machine's resources must be scaled as well.
After adding more CPU power and reboot Apache is back to ultra fast mode.

I will keep monitoring, but let's consider that the issue is resolved and let me know if anything.

#8 Updated by rfan1 about 1 month ago

The issue is gone, thanks much andriinikitin, can you please assign this ticket to yourself and close it?
BR//Richard.

#9 Updated by rfan1 about 1 month ago

  • Status changed from Feedback to Resolved
  • Assignee set to andriinikitin
  • % Done changed from 0 to 100

Also available in: Atom PDF