action #71227
opencoordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA
coordination #102909: [epic] Prevent more incompletes already within os-autoinst or openQA
[ux] job incompletes with auto_review:"backend died: 'current_console' is not set at /usr/lib/os-autoinst/backend/baseclass.pm line 932."
0%
Description
Observation¶
https://openqa.suse.de/tests/4673747 in incomplete, reason is: "Backend process died: 'current_console' is not set at /usr/lib/os-autoinst/backend/baseclass.pm line 932."
https://openqa.suse.de/tests/4673747/file/autoinst-log.txt shows:
[37m[2020-09-11T02:59:50.763 CEST] [debug] Backend process died, backend errors are reported below in the following lines:
'current_console' is not set at /usr/lib/os-autoinst/backend/baseclass.pm line 932.
backend::baseclass::wait_serial(backend::svirt=HASH(0x55a087239aa8), HASH(0x55a08804a420)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 89
backend::baseclass::handle_command(backend::svirt=HASH(0x55a087239aa8), HASH(0x55a088c2c420)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 625
backend::baseclass::check_socket(backend::svirt=HASH(0x55a087239aa8), IO::Handle=GLOB(0x55a087bb56f8), 0) called at /usr/lib/os-autoinst/backend/svirt.pm line 337
backend::svirt::check_socket(backend::svirt=HASH(0x55a087239aa8), IO::Handle=GLOB(0x55a087bb56f8), 0) called at /usr/lib/os-autoinst/backend/baseclass.pm line 276
eval {...} called at /usr/lib/os-autoinst/backend/baseclass.pm line 191
backend::baseclass::run_capture_loop(backend::svirt=HASH(0x55a087239aa8)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 146
backend::baseclass::run(backend::svirt=HASH(0x55a087239aa8), 13, 16) called at /usr/lib/os-autoinst/backend/driver.pm line 86
backend::driver::__ANON__(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
eval {...} called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
Mojo::IOLoop::ReadWriteProcess::_fork(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48), CODE(0x55a088ef7228)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 477
Mojo::IOLoop::ReadWriteProcess::start(Mojo::IOLoop::ReadWriteProcess=HASH(0x55a087733d48)) called at /usr/lib/os-autoinst/backend/driver.pm line 88
backend::driver::start(backend::driver=HASH(0x55a0884b1268)) called at /usr/lib/os-autoinst/backend/driver.pm line 52
backend::driver::new("backend::driver", "svirt") called at /usr/bin/isotovideo line 222
main::init_backend() called at /usr/bin/isotovideo line 273
Checking the git log of test distri between "first bad" and "last good" (when tests were green again) shows:
$ git log1 --no-merges de9fd2b5dcf5d9bc31807e29cd720ac80a1c7f9f..bb6dc4672fc0177c4fb20d7dfd08ddb824e0d0a4
f9f7351ab Remove /tmp as subvolume in profile validation
ee4098d8f Improve pre- and post-feature test
74cd182e8 Rename sap.slice to SAP.slice
f2d6865d7 Wait snapper to settle down to assert root-console
with some changes to "lib/virt_feature_test_base.pm" which might be related.
Problem¶
This can happen if the console is not properly initialized by test code.
From the error message it is unclear if test code is the culprit or some temporary, flaky error in the infrastructure. This could be cleaner.
Acceptance criteria¶
- AC1: It is obvious to the test reviewer if this is a (temporary) infrastructure issue or a problem in test code that needs to be fixed
Suggestions¶
- Crosscheck how "current_console" is used within os-autoinst
- Improve error handling with explicit error messages on misuse
- Optional: Apply something similar for "reset_consoles". For this also see #60437 and https://github.com/os-autoinst/os-autoinst/pull/1288