Project

General

Profile

action #50765

[sle][functional][u] test fails in bootloader - svirt-xen vnc connection refused

Added by jorauch over 2 years ago. Updated over 1 year ago.

Status:
Rejected
Priority:
Normal
Assignee:
Category:
Bugs in existing tests
Target version:
SUSE QA - Milestone 30
Start date:
2019-04-25
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP1-Installer-DVD-x86_64-allpatterns@svirt-xen-hvm fails in
bootloader

According to the logs the connection to the VNC server is refused
This happened on openqaworker2:X with svirt-xen tests

Tasks

  • investigate why this happens
  • try to fix this behaviour if it's an openqa or test issue
  • file a bug if it's a product issue

Test suite description

Maintainers: okurz Installation with all patterns selected for installation to check for potential package conflicts, how the system handles big space usage, etc.

allpatterns installations can take longer, especially on non-x86_64 architectures.

Reproducible

Fails since (at least) Build 216.4 (current job)

Expected result

Last good: 216.1 (or more recent)

Further details

Always latest result in this scenario: latest

Entry in the logs:


[2019-04-25T12:59:16.896 CEST] [debug] considering VNC stalled, no update for 4.01 seconds
[2019-04-25T12:59:18.902 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5902>: IO::Socket::INET: connect: Connection refused
[2019-04-25T12:59:19.907 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5902>: IO::Socket::INET: connect: Connection refused
[2019-04-25T12:59:20.909 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5902>: IO::Socket::INET: connect: Connection refused
[2019-04-25T12:59:21.912 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5902>: IO::Socket::INET: connect: Connection refused
[2019-04-25T12:59:22.914 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5902>: IO::Socket::INET: connect: Connection refused
[2019-04-25T12:59:23.916 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5902>: IO::Socket::INET: connect: Connection refused
[2019-04-25T12:59:24.925 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5902>: IO::Socket::INET: connect: Connection refused
[2019-04-25T12:59:25.927 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5902>: IO::Socket::INET: connect: Connection refused

10.162.0.72_gi51-day.png (2.47 KB) 10.162.0.72_gi51-day.png mgriessmeier, 2019-04-26 08:17
7895

Related issues

Related to openQA Tests - action #46919: [functional][u][svirt][sporadic] auto_review:"IO::Socket::INET: connect: Connection timed out"Rejected2018-12-01

Is duplicate of openQA Tests - action #49964: [functional][u] test fails in timezone - Error connecting to VNC serverRejected2019-04-03

History

#1 Updated by szarate over 2 years ago

I'm having the feeling that this is a network related issue... Somehow when there's a new build this will happen pretty often...

#2 Updated by szarate over 2 years ago

  • Related to action #46919: [functional][u][svirt][sporadic] auto_review:"IO::Socket::INET: connect: Connection timed out" added

#3 Updated by mgriessmeier over 2 years ago

szarate wrote:

I'm having the feeling that this is a network related issue... Somehow when there's a new build this will happen pretty often...

but why in the middle of the installation process?
like here: https://openqa.suse.de/tests/2829674#
we were already connected

#4 Updated by mgriessmeier over 2 years ago

  • Assignee set to mgriessmeier

triggered 50 jobs with custom scheduling, aborting the test after addon_products_sle to gather statistics:
https://openqa.suse.de/tests/overview?distri=sle&version=15-SP1&build=mgriessmeier_poo50765

#5 Updated by mgriessmeier over 2 years ago

7895

so the 50 jobs doesnt show anything.
so most likely this is a network issue, because it

  • a) happens only when a new build get triggered
  • b) restarting solves it
  • c) http://mrtg.suse.de/qanet12nue/10.162.0.72_gi51.html shows clear spikes of the uplink in our QA LAB which are exactly in the timeframe where the new build/incompletes happen (see attached picture, since the link is not static)

we need to come up with a mitigation for this, suggestions so far:

  • a) moving the machine out of our lab
  • b) be less strict with timeouts
  • c) limit syncing speed somehow

unassigning

#6 Updated by mgriessmeier over 2 years ago

  • Is duplicate of action #49964: [functional][u] test fails in timezone - Error connecting to VNC server added

#7 Updated by zluo over 2 years ago

  • Assignee set to zluo

take over and checking current status for this issue.

#8 Updated by zluo over 2 years ago

  • Status changed from Workable to In Progress

#9 Updated by zluo over 2 years ago

  • Status changed from In Progress to Workable
  • Target version changed from Milestone 23 to Milestone 26

since this becomes epic ticket, set it to target milestone 26.

#10 Updated by zluo over 2 years ago

  • Subject changed from [functional][u] test fails in bootloader - svirt-xen vnc connection refused to [sle][functional][u] test fails in bootloader - svirt-xen vnc connection refused
  • Status changed from Workable to In Progress

#11 Updated by zluo over 2 years ago

https://openqa.suse.de/tests/2874059#step/bootloader/3 (openqaworker2:12) shows still the problem for 5 days ago.

However the latest test runs are fine.

#14 Updated by zluo over 2 years ago

The results of 200 test runs on osd shows clearly that the issue is related to network, see problem with firefox for loading html page, or test module welcome for connection to SCC.
Actually all tests got successfully executed for bootloader.

Reject this ticket for now.

#15 Updated by zluo over 2 years ago

  • Status changed from In Progress to Rejected

#16 Updated by szarate almost 2 years ago

  • Status changed from Rejected to Workable
  • Assignee deleted (zluo)
  • Priority changed from High to Normal
  • Target version changed from Milestone 26 to Milestone 28

This is still valid: https://openqa.suse.de/tests/3501451

[2019-10-21T17:19:00.026 CEST] [debug] no change: 27.9s
[2019-10-21T17:19:01.027 CEST] [debug] no change: 26.9s
[2019-10-21T17:19:01.688 CEST] [debug] considering VNC stalled, no update for 4.01 seconds
[2019-10-21T17:19:03.691 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:04.692 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:05.692 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:06.693 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:07.694 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:08.695 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:09.696 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:10.697 CEST] [debug] Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:11.701 CEST] [debug] Backend process died, backend errors are reported below in the following lines:
Error connecting to VNC server <openqaw5-xen.qa.suse.de:5901>: IO::Socket::INET: connect: Connection refused
[2019-10-21T17:19:11.702 CEST] [debug] Destroying openQA-SUT-1 virtual machine

#17 Updated by szarate almost 2 years ago

  • Target version changed from Milestone 28 to Milestone 29

#18 Updated by mgriessmeier over 1 year ago

  • Target version changed from Milestone 29 to Milestone 30

needs to be discussed offline

#19 Updated by zluo over 1 year ago

  • Status changed from Workable to Rejected
  • Assignee set to zluo

I checked for sles 15 sp2 on osd, atm we don't have this issue of bootloader for svirt-xen. So reject this again.

Also available in: Atom PDF