action #69691


coordination #102906: [saga][epic] Increased stability of tests with less "known failures", known incompletes handled automatically within openQA

coordination #102909: [epic] Prevent more incompletes already within os-autoinst or openQA

Improve incomplete output for qemu related problems, e.g. auto_review:"Failed to allocate KVM HPT.*Cannot allocate memory":retry instead of "can't open qmp"

Added by okurz almost 4 years ago. Updated over 2 years ago.

Feature requests
Target version:
Start date:
Due date:
% Done:


Estimated time:


Observation shows reason "backend died: can't open qmp at /usr/lib/os-autoinst/OpenQA/Qemu/ line 448." so one needs to take a look into the logfile which shows:

[2020-08-06T12:11:40.169 UTC] [debug] starting: /usr/bin/qemu-system-ppc64 -g 1024x768 -vga std -only-migratable -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -soundhw hda -global isa-fdc.driveA= -m 4096 -machine usb=off -cpu host -netdev user,id=qanet0 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -boot order=c -device nec-usb-xhci -device usb-tablet -device usb-kbd -smp 4 -enable-kvm -no-shutdown -vnc :93,share=force-shared -device virtio-serial -chardev pipe,id=virtio_console,path=virtio_console,logfile=virtio_console.log,logappend=on -device virtconsole,chardev=virtio_console,name=org.openqa.console.virtio_console -chardev socket,path=qmp_socket,server,nowait,id=qmp_socket,logfile=qmp_socket.log,logappend=on -qmp chardev:qmp_socket -S -device virtio-scsi-pci,id=scsi0 -blockdev driver=file,node-name=hd0-overlay2-file,filename=/var/lib/openqa/pool/3/raid/hd0-overlay2, -blockdev driver=qcow2,node-name=hd0-overlay2,file=hd0-overlay2-file, -device virtio-blk,id=hd0-device,drive=hd0-overlay2,bootindex=0,serial=hd0 -blockdev driver=file,node-name=cd0-overlay2-file,filename=/var/lib/openqa/pool/3/raid/cd0-overlay2, -blockdev driver=qcow2,node-name=cd0-overlay2,file=cd0-overlay2-file, -device scsi-cd,id=cd0-device,drive=cd0-overlay2,serial=cd0 -incoming defer
[2020-08-06T12:11:40.174 UTC] [debug] Waiting for 0 attempts
[2020-08-06T12:11:58.443 UTC] [debug] Waiting for 19 attempts
[2020-08-06T12:11:59.444 UTC] [debug] Backend process died, backend errors are reported below in the following lines:
can't open qmp at /usr/lib/os-autoinst/OpenQA/Qemu/ line 448.

[2020-08-06T12:11:59.444 UTC] [info] ::: OpenQA::Qemu::Proc::save_state: Saving QEMU state to qemu_state.json
[2020-08-06T12:11:59.445 UTC] [debug] flushing frames
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: QEMU emulator version (openSUSE Leap 15.1)
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: Copyright (c) 2003-2018 Fabrice Bellard and the QEMU Project developers
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: Unknown host!
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: Unknown host!
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: Unknown host!
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: Unknown host!
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: Unknown host!
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: Unknown host!
[2020-08-06T12:11:59.447 UTC] [debug] QEMU: qemu-system-ppc64: Failed to allocate KVM HPT of order 25 (try smaller maxmem?): Cannot allocate memory
[2020-08-06T12:11:59.447 UTC] [debug] sending magic and exit
[2020-08-06T12:11:59.448 UTC] [debug] received magic close
[2020-08-06T12:11:59.449 UTC] [debug] THERE IS NOTHING TO READ 15 4 3

where the latter is only visible in the log file.

Acceptance criteria

  • AC1: The reason includes the content from the last qemu output


  • For qemu related problems try to parse the (last) line that starts with "QEMU: " and put that into the reason instead of "can't open qmp"
Actions #1

Updated by okurz over 3 years ago

  • Parent task set to #62420
Actions #2

Updated by okurz over 2 years ago

  • Parent task changed from #62420 to #102909

Also available in: Atom PDF