Project

General

Profile

Actions

action #99591

closed

[qe-core] many test called with qa_run.pm fail

Added by mgrifalconi over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Immediate
Assignee:
Category:
Bugs in existing tests
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:

Description

Observation

This is one example, almost all tests on SLE-12 are failing for the same issue after a 3h timeout.

openQA test in scenario sle-12-SP3-Server-DVD-Updates-x86_64-mau-qa_kernel_lynis@64bit fails in
kernel_lynis

Test suite description

Testsuite maintained at https://gitlab.suse.de/qa-maintenance/qam-openqa-yml.

Reproducible

Fails since (at least) Build 20211001-1 (current job)

Expected result

Last good: 20210930-1 (or more recent)

Further details

Always latest result in this scenario: latest


Files


Related issues 1 (0 open1 closed)

Related to openQA Tests - action #99714: [qem] test fails in kernel_multipathClosed2021-10-04

Actions
Actions #1

Updated by tjyrinki_suse over 2 years ago

  • Status changed from New to Workable
  • Target version set to QE-Core: Ready
  • Start date deleted (2021-10-01)
Actions #2

Updated by tjyrinki_suse over 2 years ago

From Michael to lessen the impact on openQA: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/13376/

If qa_run is still to be used in the future, it's useful to file a new ticket before closing this one to change it back possibly, but so far that's only a beginning and we do not have a fix for the test failures.

Actions #3

Updated by mgrifalconi over 2 years ago

Cloning a test that worked yesterday still fails https://openqa.suse.de/tests/7290761#

More info on the situation https://suse.slack.com/archives/C02D16TCP99/p1633075991136300

Actions #4

Updated by dzedro over 2 years ago

I created ticket https://sd.suse.com/servicedesk/customer/portal/1/SD-62274
Looks like dashboard.qa2.suse.asia is unreachable.

# nc -zv dashboard.qa2.suse.asia 8888
nc: connect to dashboard.qa2.suse.asia port 8888 (tcp) failed: Connection timed out

Actions #5

Updated by kgw_suse over 2 years ago

  • Related to action #99714: [qem] test fails in kernel_multipath added
Actions #6

Updated by maritawerner over 2 years ago

@dzedro China is on Public Holiday and they had to switch of the servers due to a power switch off in the building. The team will be back on Thursday.

Actions #7

Updated by vpelcak over 2 years ago

Do we have then some action plan for the future?

Like having tests geo-distributed or maybe even removed?

Actions #8

Updated by dzedro over 2 years ago

  • Status changed from Workable to Blocked
  • Assignee set to dzedro
Actions #9

Updated by pcervinka over 2 years ago

Maybe it is right time to drop (at least part of them) these old ctcs2 based tests. Their outcome was questionable all the time and some of of those tests were not touched for couple of years.

Actions #10

Updated by mgrifalconi over 2 years ago

@dzedro That is great! Thanks Jozef!
@vpelcak @pcervinka Yes it is planned to get rid of it https://progress.opensuse.org/issues/99657

Actions #11

Updated by dzedro over 2 years ago

  • Status changed from Blocked to Resolved
  • % Done changed from 0 to 100

The issue is resolved, there is another ticket to handle the old tests.

Actions #12

Updated by cachen over 2 years ago

Hello Guys, like Marita mentioned dashboard.qa2.suse.asia was shutdown due to power down of the office building. There is a mistake in code probably, dashboard.qa2.suse.asia is using by qe-performance squad and message_queue to fetching its status is internal service shouldn't affect your testing. @Jerry Tang will take a look and see how to fix the influence.

Actions #13

Updated by jerrytang over 2 years ago

dzedro wrote:

I created ticket https://sd.suse.com/servicedesk/customer/portal/1/SD-62274
Looks like dashboard.qa2.suse.asia is unreachable.

# nc -zv dashboard.qa2.suse.asia 8888
nc: connect to dashboard.qa2.suse.asia port 8888 (tcp) failed: Connection timed out


Yes, root cause is dashboard.qa2.suse.asia power off.
Automation test will update progress of test by message queue API.
When server is down,it will increase running time because of connection hung.

I'll submit a PR to set it off by default.

Actions #14

Updated by jerrytang over 2 years ago

Actions #15

Updated by jerrytang over 2 years ago

qa-testsuite commit id : eb83983dbd6c5734540f822424c977c61ec0e813

"set message queue api disable by default"

Actions

Also available in: Atom PDF