action #10208


Tests sometimes fail with 'timeout reading hmp socket'

Added by dimstar about 8 years ago. Updated almost 8 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


from autoinst-log.txt (

DIE ERROR: timeout reading hmp socket
at /usr/lib/os-autoinst/backend/ line 58 thread 1.
backend::baseclass::die_handler("ERROR: timeout reading hmp socket\x{a}") called at /usr/lib/os-autoinst/backend/ line 723 thread 1
backend::qemu::_read_hmp(backend::qemu=HASH(0x57e1d98)) called at /usr/lib/os-autoinst/backend/ line 790 thread 1
backend::qemu::_send_hmp(backend::qemu=HASH(0x57e1d98), "savevm lastgood") called at /usr/lib/os-autoinst/backend/ line 138 thread 1
backend::qemu::do_savevm(backend::qemu=HASH(0x57e1d98), HASH(0x7f7b781f65d0)) called at /usr/lib/os-autoinst/backend/ line 53 thread 1
backend::baseclass::handle_command(backend::qemu=HASH(0x57e1d98), HASH(0x7f7b781f28a0)) called at /usr/lib/os-autoinst/backend/ line 374 thread 1
backend::baseclass::check_socket(backend::qemu=HASH(0x57e1d98), IO::Handle=GLOB(0x45d9820)) called at /usr/lib/os-autoinst/backend/ line 815 thread 1
backend::qemu::check_socket(backend::qemu=HASH(0x57e1d98), IO::Handle=GLOB(0x45d9820)) called at /usr/lib/os-autoinst/backend/ line 176 thread 1
eval {...} called at /usr/lib/os-autoinst/backend/ line 143 thread 1
backend::baseclass::run_capture_loop(backend::qemu=HASH(0x57e1d98), IO::Select=ARRAY(0x5433ed0)) called at /usr/lib/os-autoinst/backend/ line 98 thread 1
backend::baseclass::run(backend::qemu=HASH(0x57e1d98), 14, 17) called at /usr/lib/os-autoinst/backend/ line 66 thread 1
backend::driver::_run(backend::qemu=HASH(0x57e1d98), 14, 17) called at /usr/lib/os-autoinst/backend/ line 53 thread 1
eval {...} called at /usr/lib/os-autoinst/backend/ line 53 thread 1
waitpid for 28972 returned 28972
QEMU: qemu-system-x86_64: /home/abuild/rpmbuild/BUILD/qemu-2.1.3/hw/i386/kvm/clock.c:69: kvmclock_current_nsec: Assertion `time.tsc_timestamp <= migration_tsc' failed.
sending magic and exit
received magic close

openQA is nice and auto-reschedules those tests, but it would be even better to find why those fail

Actions #1

Updated by coolo about 8 years ago

It saves the vm and doesn't get an answer from qemu. This is also part of your log:

QEMU: qemu-system-x86_64: /home/abuild/rpmbuild/BUILD/qemu-2.1.3/hw/i386/kvm/clock.c:69: kvmclock_current_nsec: Assertion `time.tsc_timestamp <= migration_tsc' failed.

And I think that's the problem. Now I'm not a qemu hacker, so I can't tell you what the meaning of this is.

Actions #2

Updated by dimstar about 8 years ago

Some pointers I found so far seem to hint at a kernel issue, that should be fixed with:

commit 7f187922ddf6b67f2999a76dcb71663097b75497
KVM: x86: update masterclock values on TSC writes

of course I'm not sure if our 3.16.7 from openSUSE 13.2 has that commit - I might just open a boo# entry to check with the kernel team

Actions #3

Updated by dimstar about 8 years ago


My quick glance at the kernel comes to the conclusion that we do not have the patch in our kernel

Actions #4

Updated by okurz about 8 years ago

I did not know openQA already has a "re-scheduling" feature. Is the "reschedule" somehow reported in a way easily visible in the UI?

Actions #5

Updated by dimstar about 8 years ago

It might also be an external script that restarted the 'incomplete' tests
(check, search for tests containing "update_leap_" (including obsolete tests)

The audit trail lists those restarted ones as:
about 2 hours ago coolo efc296cb75b729269943cb097320711e job_duplicate { auto => 1, id => 113600, result => 1...

Actions #6

Updated by coolo about 8 years ago

that's just the worker running under my credentionals. The restart is done by the worker if incomplete, i.e. crashes os-autoinst

Actions #7

Updated by RBrownSUSE about 8 years ago

  • Target version set to Milestone 1
Actions #8

Updated by dimstar about 8 years ago

As per : maintenance update is being prepared. Once out, the kernel update is hopefully eliminating that issue for us.

Actions #9

Updated by RBrownSUSE about 8 years ago

  • Assignee set to dimstar
Actions #10

Updated by dimstar about 8 years ago

openSUSE-SU-2016:0301-1: An update that solves 57 vulnerabilities and has 21
fixes is now available.

Category: security (important)
Bug References:
CVE References:
Sources used:

Actions #11

Updated by dimstar almost 8 years ago

  • Status changed from New to Resolved

The kernel has been released and I have not seen this issue surface in a while now - seems it was the right fix in the end

Actions #12

Updated by dimstar almost 8 years ago

  • % Done changed from 0 to 100

Also available in: Atom PDF