action #13914
open[qe-core][functional][ipmi] wait_serial does not get expected output because ipmi console connection is closed
0%
Description
Test failed due to wait_serial does not get output. From serial0.txt, the ipmi session was already closed due to "excess errors received"
Failure step:
https://openqa.suse.de/tests/587781#step/install_package/4
Build link:
https://openqa.suse.de/tests/overview?distri=sle&version=12-SP2&build=2141&groupid=46
Serial output link:
https://openqa.suse.de/tests/587781/file/serial0.txt
Key serial output errors:
[�[0;32m OK �[0m] Started Serial Getty on ttyS1.
[�[0;32m OK �[0m] Started Serial Getty on hvc0.
Starting X Display Manager...
[�[0;32m OK �[0m] Started Getty on tty1.
[�[0;32m OK �[0m] Reached target Login Prompts.
[�[0;32m OK �[0m] Started /etc/init.d/after.local Compatibility.
[�[0;32m OK �[0m] Started Load dom0 backend drivers.
Starting The Xen xenstore...
[SOL established]
[error received]: excess errors received
[closing the connection]
Updated by okurz about 8 years ago
hm, what do you think could be the potential error sources? Is it the IPMI itself or network connection or related to test or backend changes or maybe the product itself?
Updated by xlai about 8 years ago
I do not have any proof yet. But I strongly suspect the ipmi performance under high test pressure.
Updated by okurz almost 8 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: gi-guest_sles11sp4-on-host_sles12sp3-xen
http://openqa.suse.de/tests/781640
Updated by okurz almost 8 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: gi-guest_sles11sp4-on-host_sles12sp3-xen
http://openqa.suse.de/tests/781640
Updated by okurz over 7 years ago
- Related to action #18144: [tools] restart ipmi management controller before every ipmi job added
Updated by okurz over 7 years ago
https://openqa.suse.de/tests/964225#step/install_and_reboot/10 looks related, serial0.txt says
IKG0B-162-
DIE Can't close(GLOB(0x610ab00)) filehandle: 'No child processes' at /usr/lib/os-autoinst/backend/baseclass.pm line 267
at /usr/lib/os-autoinst/backend/baseclass.pm line 73.
backend::baseclass::die_handler('autodie::exception=HASH(0x6200770)') called at (eval 1258) line 75
so same?
https://openqa.suse.de/tests/966840#step/zypper_lr/2 looks similar or related. serial0.txt shows the output but wait_serial does not show the expected output. As it looks like in the same job wait_serial failed in before in an intermediate step, e.g. https://openqa.suse.de/tests/966840#step/logpackages/4
@coolo is this related to your latest work on ipmi+ssh?
Updated by okurz over 7 years ago
Retriggered job https://openqa.suse.de/tests/967718 shows the same symptoms, no serial output recorded in the wait_serial calls
Updated by okurz over 7 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: gnome@64bit-ipmi
https://openqa.suse.de/tests/1002264
Updated by okurz over 7 years ago
- Related to action #23404: Serial output gets lost added
Updated by xlai about 7 years ago
This issue is absolutely ipmi itself unstability issue, has nothing to do with test itself. If relevant, then we can only say that high work pressure on ipmi may lead to this much more.
Updated by okurz about 7 years ago
- Description updated (diff)
- Category changed from Bugs in existing tests to Infrastructure
- Priority changed from Normal to High
updated description format. IMHO it's an "infrastructure" issue. I don't see anything done wrong in the test code itself. Of course, if we can not find a more stable solution in any IMPI worker that means the ticket is about workarounds in the test code or the backend itself.
Updated by okurz over 6 years ago
- Related to action #34699: [qe-core][functional][ipmi] access to serial log during installation added
Updated by okurz almost 6 years ago
- Related to coordination #44843: [qe-core][functional][epic] Cleanup the use of serial-/virtio-/ssh-consoles in our tests (was: use $self->select_serial_terminal instead of checking IPMI in every module) added
Updated by okurz almost 6 years ago
- Subject changed from [ipmi] wait_serial does not get expected output because ipmi console connection is closed to [functional][u][ipmi] wait_serial does not get expected output because ipmi console connection is closed
- Status changed from New to Blocked
- Assignee set to okurz
- Target version set to future
waiting for #34699 first
Updated by okurz over 5 years ago
- Assignee changed from okurz to mgriessmeier
Move to new QSF-u PO after I moved to the "tools"-team. I mainly checked the subject line so in individual instances you might not agree to take it over completely into QSF-u. Feel free to discuss with me or reassign to me or someone else in this case. Thanks.
Updated by SLindoMansilla over 4 years ago
- Related to deleted (action #34699: [qe-core][functional][ipmi] access to serial log during installation)
Updated by SLindoMansilla over 4 years ago
- Blocked by action #34699: [qe-core][functional][ipmi] access to serial log during installation added
Updated by SLindoMansilla over 4 years ago
- Assignee changed from mgriessmeier to SLindoMansilla
Updated by tjyrinki_suse about 4 years ago
- Subject changed from [functional][u][ipmi] wait_serial does not get expected output because ipmi console connection is closed to [qe-core][functional][ipmi] wait_serial does not get expected output because ipmi console connection is closed
Updated by SLindoMansilla over 3 years ago
- Status changed from Blocked to New
- Assignee deleted (
SLindoMansilla)
No time to work on this :(
Updated by okurz about 3 years ago
- Priority changed from High to Normal
This ticket was set to "High" priority but was not updated within 120 days which is 4 times the period of the SLO for "High" tickets (30 days) as described on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#SLOs-service-level-objectives . The ticket will be set to the next lower priority of "Normal".
Updated by slo-gin about 2 years ago
This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.
Updated by slo-gin 8 months ago
This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.