Project

General

Profile

Actions

action #71227

open

coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA

coordination #102909: [epic] Prevent more incompletes already within os-autoinst or openQA

[ux] job incompletes with auto_review:"backend died: 'current_console' is not set at /usr/lib/os-autoinst/backend/baseclass.pm line 932."

Added by Xiaojing_liu over 4 years ago. Updated over 2 years ago.

Status:
Workable
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
QA (public, currently private due to #173521) - future
Start date:
2020-09-11
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://openqa.suse.de/tests/4673747 in incomplete, reason is: "Backend process died: 'current_console' is not set at /usr/lib/os-autoinst/backend/baseclass.pm line 932."
https://openqa.suse.de/tests/4673747/file/autoinst-log.txt shows:

[2020-09-11T02:59:50.763 CEST] [debug] Backend process died, backend errors are reported below in the following lines:
'current_console' is not set at /usr/lib/os-autoinst/backend/baseclass.pm line 932.
    backend::baseclass::wait_serial(backend::svirt=HASH(0x55a087239aa8), HASH(0x55a08804a420)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 89
    backend::baseclass::handle_command(backend::svirt=HASH(0x55a087239aa8), HASH(0x55a088c2c420)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 625
    backend::baseclass::check_socket(backend::svirt=HASH(0x55a087239aa8), IO::Handle=GLOB(0x55a087bb56f8), 0) called at /usr/lib/os-autoinst/backend/svirt.pm line 337
    backend::svirt::check_socket(backend::svirt=HASH(0x55a087239aa8), IO::Handle=GLOB(0x55a087bb56f8), 0) called at /usr/lib/os-autoinst/backend/baseclass.pm line 276
    eval {...} called at /usr/lib/os-autoinst/backend/baseclass.pm line 191
    backend::baseclass::run_capture_loop(backend::svirt=HASH(0x55a087239aa8)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 146
    backend::baseclass::run(backend::svirt=HASH(0x55a087239aa8), 13, 16) called at /usr/lib/os-autoinst/backend/driver.pm line 86
    backend::driver::__ANON__(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
    eval {...} called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
    Mojo::IOLoop::ReadWriteProcess::_fork(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48), CODE(0x55a088ef7228)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 477
    Mojo::IOLoop::ReadWriteProcess::start(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48)) called at /usr/lib/os-autoinst/backend/driver.pm line 88
    backend::driver::start(backend::driver=HASH(0x55a0884b1268)) called at /usr/lib/os-autoinst/backend/driver.pm line 52
    backend::driver::new("backend::driver", "svirt") called at /usr/bin/isotovideo line 222
    main::init_backend() called at /usr/bin/isotovideo line 273

Checking the git log of test distri between "first bad" and "last good" (when tests were green again) shows:

$ git log1 --no-merges de9fd2b5dcf5d9bc31807e29cd720ac80a1c7f9f..bb6dc4672fc0177c4fb20d7dfd08ddb824e0d0a4
f9f7351ab Remove /tmp as subvolume in profile validation
ee4098d8f Improve pre- and post-feature test
74cd182e8 Rename sap.slice to SAP.slice
f2d6865d7 Wait snapper to settle down to assert root-console

with some changes to "lib/virt_feature_test_base.pm" which might be related.

Problem

This can happen if the console is not properly initialized by test code.
From the error message it is unclear if test code is the culprit or some temporary, flaky error in the infrastructure. This could be cleaner.

Acceptance criteria

  • AC1: It is obvious to the test reviewer if this is a (temporary) infrastructure issue or a problem in test code that needs to be fixed

Suggestions

  • Crosscheck how "current_console" is used within os-autoinst
  • Improve error handling with explicit error messages on misuse
  • Optional: Apply something similar for "reset_consoles". For this also see #60437 and https://github.com/os-autoinst/os-autoinst/pull/1288

Related issues 1 (0 open1 closed)

Related to openQA Tests (public) - action #71230: [SLE] Test 'default_install_svirt' is incomplete in OSD with error: 'Backend process died'ResolvedXiaojing_liu2020-09-11

Actions
Actions

Also available in: Atom PDF