action #41330
closed
[functional][y][s390x][investigation][timebox:4h] test fails in welcome - half-open socket in post_fail_hook causing incomplete job
Added by okurz about 6 years ago.
Updated almost 6 years ago.
Category:
Bugs in existing tests
Target version:
SUSE QA - Milestone 22
Description
Observation¶
openQA test in scenario sle-15-SP1-Installer-DVD-s390x-btrfs+low_ram@s390x-zVM-vswitch-l3 fails in
welcome
Reproducible¶
Fails since (at least) Build 19.3 reproducibly
Expected result¶
Job should not incomplete
Further details¶
Always latest result in this scenario: latest
Same issue with sle-15-SP1-Installer-DVD-s390x-Build58.1-create_hdd_textmode@s390x-kvm-sle12
[2018-10-02T23:15:32.0330 CEST] [debug] <<< backend::console_proxy::__ANON__(wrapped_call={
'function' => 'stop_serial_grab',
'console' => 'svirt',
'args' => []
})
DIE The console isn't responding correctly. Maybe half-open socket? at /usr/lib/os-autoinst/backend/baseclass.pm line 241.
at /usr/lib/os-autoinst/backend/baseclass.pm line 80.
backend::baseclass::die_handler('The console isn\'t responding correctly. Maybe half-open sock...') called at /usr/lib/os-autoinst/backend/baseclass.pm line 241
eval {...} called at /usr/lib/os-autoinst/backend/baseclass.pm line 156
backend::baseclass::run_capture_loop('backend::svirt=HASH(0x72aad18)') called at /usr/lib/os-autoinst/backend/baseclass.pm line 129
backend::baseclass::run('backend::svirt=HASH(0x72aad18)', 13, 16) called at /usr/lib/os-autoinst/backend/driver.pm line 77
backend::driver::__ANON__('Mojo::IOLoop::ReadWriteProcess=HASH(0x74a5d70)') called at /usr/lib/perl5/vendor_perl/5.18.2/Mojo/IOLoop/ReadWriteProcess.pm line 325
eval {...} called at /usr/lib/perl5/vendor_perl/5.18.2/Mojo/IOLoop/ReadWriteProcess.pm line 325
Mojo::IOLoop::ReadWriteProcess::_fork('Mojo::IOLoop::ReadWriteProcess=HASH(0x74a5d70)', 'CODE(0x2e2b7f0)') called at /usr/lib/perl5/vendor_perl/5.18.2/Mojo/IOLoop/ReadWriteProcess.pm line 476
Mojo::IOLoop::ReadWriteProcess::start('Mojo::IOLoop::ReadWriteProcess=HASH(0x74a5d70)') called at /usr/lib/os-autoinst/backend/driver.pm line 79
backend::driver::start('backend::driver=HASH(0x71722b8)') called at /usr/lib/os-autoinst/backend/driver.pm line 50
backend::driver::new('backend::driver', 'svirt') called at /usr/bin/isotovideo line 184
main::init_backend() called at /usr/bin/isotovideo line 248
[2018-10-02T23:19:30.0705 CEST] [debug] Destroying openQA-SUT-3 virtual machine
[2018-10-02T23:19:30.0861 CEST] [debug] Connection to root@s390p8.suse.de established
- Due date set to 2018-12-18
- Target version changed from Milestone 20 to Milestone 21
- Related to action #32746: [sle][tools][remote-backends][hard] Incomplete job because console isn't responding correctly. Half-open socket on IPMI added
- Subject changed from [functional][y][s390x][zvm] test fails in welcome - half-open socket in post_fail_hook causing incomplete job to [functional][y][s390x] test fails in welcome - half-open socket in post_fail_hook causing incomplete job
most recent example: sle-15-SP1-Installer-DVD-s390x-Build107.5-textmode@s390x-kvm-sle12, autoinst-log.txt tells:
[2018-12-01T18:02:44.637 CET] [debug] /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/reboot_after_installation.pm:39 called power_action_utils::power_action
[2018-12-01T18:02:44.637 CET] [debug] <<< backend::console_proxy::__ANON__(wrapped_call={
'args' => [],
'console' => 'svirt',
'function' => 'stop_serial_grab'
})
[2018-12-01T18:05:34.561 CET] [debug] Backend process died, backend errors are reported below in the following lines The console isn't responding correctly. Maybe half-open socket? at /usr/lib/os-autoinst/backend/baseclass.pm line 241.
- Status changed from New to Workable
- Subject changed from [functional][y][s390x] test fails in welcome - half-open socket in post_fail_hook causing incomplete job to [functional][y][s390x][investigation][timebox:4h] test fails in welcome - half-open socket in post_fail_hook causing incomplete job
- Due date changed from 2018-12-18 to 2019-01-29
- Target version changed from Milestone 21 to Milestone 22
- Status changed from Workable to Resolved
There was a patch provided by mgriessmeier which should resolve such issues in general and in case it reoccurs it should incomplete as is backend issue.
There were only 2 runs where we got incomplete during the postfailhook, which both somehow appeared during attempts of collecting strace and gdb output. I was not able to reproduce the issue manually though.
Also available in: Atom
PDF