action #54275
closed[kernel]kotd][ltp][s390x] test fails in install_ltp - failure after install_kotd, console issue
0%
Description
Observation¶
openQA test in scenario sle-12-SP5-Server-DVD-s390x-install_ltp+sle+Server-DVD+KOTD@s390x-kvm-sle12 fails in
install_ltp
Test suite description¶
maintainer: pcervinka, pvorel
Reproducible¶
Fails since (at least) Build 0222
Expected result¶
Last good: (unknown) (or more recent)
Further details¶
Always latest result in this scenario: latest
Test probably needs to schedule again bootloader_zkvm after reboot in install_kotd to properly setup test scenario.
Updated by pcervinka over 5 years ago
- Subject changed from [kernel]kotd][ltp][s390x] test fails in install_ltp - failure after install_kotd, bootloader_zkvm needed to [kernel]kotd][ltp][s390x] test fails in install_ltp - failure after install_kotd, console issue
Experimented with reconnect management console, it helps standard console, but not much to serial terminal.
When is SUT rebooted, it is not possible reconnect by virsh console as session already exist.
[2019-07-15T12:02:53.080 CEST] [debug] Command executed: '[ -f '/tmp/serial_terminal.txt.HSwG7zmFqv2QmkFK' ] && rm -v /tmp/serial_terminal.txt.HSwG7zmFqv2QmkFK', ret=0
[2019-07-15T12:02:53.080 CEST] [debug] Backend process died, backend errors are reported below in the following lines:
problem with virsh: cmd: 'virsh console openQA-SUT-4 console1', output of script wrapper: 'Script started on Mon Jul 15 12:02:51 2019
Connected to domain openQA-SUT-4
Escape character is ^]
error: operation failed: Active console session exists for this domain
Updated by pcervinka over 5 years ago
I will try --force to virsh console but not sure about side effects.
Updated by pvorel over 5 years ago
I guess you're talking about https://openqa.suse.de/tests/3062903
Can't we add a hook into power_action() to disable consoles::sshVirtshSUT on reboot?
I'd try avoid --force. It should be safe, but it can lead to really cryptic errors.
Updated by pcervinka over 5 years ago
Yes, I'm afraid --force could lead something, which we can't predict at the moment. Will try to look on your hint about disabling console.
Updated by pcervinka over 5 years ago
The problem is more complex than it looks. It fails on attaching console1 in command virsh console openQA-SUT-4 console1. This console should be connected all the time and it doesn't count with system reboot at all, because should be initialized only once per test.
Console init is done in function open_serial_console_via_ssh in `svirt.pm'. Description of the function says:
Opens SSH connection to grab serial terminal log
(using consoles::serial_screen, saved into serial_terminal.txt).
This method is not supposed to be called twice for test run due logging
into file.
If I understand, we don't have any option to cleanly stop and reconnect console1 in the same job. Adding --force to $cmd = "virsh console $name $devname$port"; didn't work.
Updated by pcervinka over 5 years ago
Workaround for KOTD installation could be in use of root-console for svirt backend. @pvorel what do you think? I think this is the quickest solution we can have now.
Updated by pcervinka over 5 years ago
PR created https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/7974, verification on OSD on all backends in progress.
Updated by pcervinka over 5 years ago
I realized that normal LTP installation in install_ltp+sle+Server-DVD is done with SERIAL_CONSOLE=0, so probably ma workaround is not needed.
Restarted KOTD install job on osd https://openqa.suse.de/tests/3109961 and added SERIAl_CONSOLE=0.
Updated by pvorel over 5 years ago
SERIAl_CONSOLE=0 is just a temporary workaround, we plan to remove it once we fix svirt #54260.
Updated by pcervinka over 5 years ago
- Status changed from In Progress to Feedback
Added SERIAL_CONSOLE=0 to install_ltp+sle+Server-DVD+KOTD to have LTP with KOTD working. As we agreed, I will create new poo for better fix, not just related to KOTD.
Updated by pcervinka over 5 years ago
- Related to action #54260: [kernel][s390x] Occasional failures due timeout added
Updated by pcervinka over 5 years ago
KOTD job on svirt is OK now. Let's wait for next build.
Updated by pcervinka over 5 years ago
- Status changed from Feedback to Resolved
Build 242 OK: https://openqa.suse.de/tests/3120594
Updated by jlausuch over 5 years ago
- Status changed from Resolved to Closed
- Target version changed from 445 to 457
Updated by pcervinka about 4 years ago
- Target version changed from 457 to QE Kernel Done