action #111602
closed18-qemu-options.t makes apparently unsafe assumptions about qemu behaviour with multiple params size:M
0%
Description
Observation¶
okurz recently changed 18-qemu-options.t so the second run in the qemu_append_option
subtest appends -M ? -version
. Previously it appended -M ?
. The test assumes that qemu will always print only the version information (not supported machines) when run this way, but that doesn't seem safe.
In Fedora's build environment, we get both the version information and the supported machines:
2: # [2022-05-25T19:04:44.856378Z] [debug] starting: /usr/bin/qemu-system-i386 -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -audiodev none,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 -global isa-fdc.fdtypeA=none -m 1024 -netdev user,id=qanet0 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -boot once=d -device qemu-xhci -device usb-tablet -smp 1 -no-shutdown -vnc :90,share=force-shared -device virtio-serial -chardev pipe,id=virtio_console,path=virtio_console,logfile=virtio_console.log,logappend=on -device virtconsole,chardev=virtio_console,name=org.openqa.console.virtio_console -chardev pipe,id=virtio_console1,path=virtio_console1,logfile=virtio_console1.log,logappend=on -device virtconsole,chardev=virtio_console1,name=org.openqa.console.virtio_console1 -chardev socket,path=qmp_socket,server=on,wait=off,id=qmp_socket,logfile=qmp_socket.log,logappend=on -qmp chardev:qmp_socket -S -M ? -version -device virtio-scsi-pci,id=scsi0 -blockdev driver=file,node-name=hd0-file,filename=/tmp/18-qemu-options.t-dNVG/pool/raid/hd0,cache.no-flush=on -blockdev driver=qcow2,node-name=hd0,file=hd0-file,cache.no-flush=on -device virtio-blk,id=hd0-device,drive=hd0,serial=hd0
2: # [2022-05-25T19:04:44.862180Z] [debug] Waiting for 0 attempts
2: # perl: warning: Setting locale failed.
2: # perl: warning: Please check that your locale settings:
2: # LANGUAGE = (unset),
2: # LC_ALL = "en_US.UTF-8",
2: # LANG = "en_US.UTF-8"
2: # are supported and installed on your system.
2: # perl: warning: Falling back to the standard locale ("C").
2: # [2022-05-25T19:04:45.032982Z] [debug] Waiting for 1 attempts
2: # [2022-05-25T19:04:45.033289Z] [info] ::: backend::baseclass::die_handler: Backend process died, backend errors are reported below in the following lines:
2: # QEMU terminated before QMP connection could be established. Check for errors below
2: # [2022-05-25T19:04:45.033642Z] [info] ::: OpenQA::Qemu::Proc::save_state: Saving QEMU state to qemu_state.json
2: # [2022-05-25T19:04:45.035028Z] [debug] Passing remaining frames to the video encoder
2: # [2022-05-25T19:04:45.087372Z] [debug] Waiting for video encoder to finalize the video
2: # [2022-05-25T19:04:45.087556Z] [debug] The built-in video encoder (pid 960824) terminated
2: # [2022-05-25T19:04:45.088978Z] [debug] QEMU: QEMU emulator version 6.2.0 (qemu-6.2.0-10.fc36)
2: # [2022-05-25T19:04:45.089061Z] [debug] QEMU: Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
2: # [2022-05-25T19:04:45.089115Z] [debug] QEMU: Supported machines are:
2: # [2022-05-25T19:04:45.089170Z] [debug] QEMU: microvm microvm (i386)
…
2: # [2022-05-25T19:04:45.090022Z] [debug] QEMU: pc-i440fx-3.0
2: # [2022-05-25T19:04:45.091199Z] [debug] sending magic and exit
2: # [2022-05-25T19:04:45.091579Z] [debug] received magic close
2: # [2022-05-25T19:04:45.097114Z] [debug] backend process exited: 0
2: # failed to start VM at /builddir/build/BUILD/os-autoinst-548335fc1544beeb51e86fe83e025a0f0aa4901e/t/../backend/driver.pm line 107.
2: # [2022-05-25T19:04:45.197976Z] [debug] stopping command server 960811 because test execution ended through exception
2: # [2022-05-25T19:04:45.398914Z] [debug] done with command server
2: # [2022-05-25T19:04:45.399034Z] [debug] stopping autotest process 960814
2: # [2022-05-25T19:04:45.599884Z] [debug] done with autotest process
2: # 960807: EXIT 1
2: # '
2: # matches '(?^u:Supported machines are\:)'
2: # Looks like you failed 1 test of 14.
On my local system, I get just the supported machines, no version information:
[adamw@xps13k os-autoinst (rawhide *%)]$ qemu-system-i386 -M ? -version
Supported machines are:
microvm microvm (i386)
xenfv-4.2 Xen Fully-virtualized PC
xenfv Xen Fully-virtualized PC (alias of xenfv-3.1)
xenfv-3.1 Xen Fully-virtualized PC
pc Standard PC (i440FX + PIIX, 1996) (alias of pc-i440fx-7.0)
pc-i440fx-7.0 Standard PC (i440FX + PIIX, 1996) (default)
…
pc-q35-2.10 Standard PC (Q35 + ICH9, 2009)
isapc ISA-only PC
none empty machine
x-remote Experimental remote machine
xenpv Xen Para-virtualized PC
[adamw@xps13k os-autoinst (rawhide *%)]$
I'm not really sure what the point of this test is, anyway. Why are we testing qemu behaviour here? Shouldn't we only care whether the parameters got appended properly?
Acceptance criteria¶
- AC1: A conscious decision was made if we want to do such test at all or not
Suggestions¶
- Read the question from AdamWilliamson why we test this at all, understand why the according test was included, make sure that we only do test what makes sense feature and coverage wise
Updated by livdywan over 2 years ago
What I currently see is this:
$ cmake --build build -t test-perl-testsuite TESTS="t/18-qemu-options.t"
Failed test 'Supported machines not listed'
at t/18-qemu-options.t line 78.
[...]
matches '(?^u:Supported machines are\:)'
Looks like you failed 1 test of 14.
This seems to match the "Fedora" case described?
Updated by okurz over 2 years ago
- Related to action #112403: 18-qemu-options.t fails on Leap 15.4 and Tumbleweed size:M added
Updated by okurz over 2 years ago
- Status changed from New to Blocked
- Assignee set to okurz
Updated by okurz over 2 years ago
- Status changed from Blocked to New
- Assignee deleted (
okurz)
Updated by okurz over 2 years ago
- Subject changed from 18-qemu-options.t makes apparently unsafe assumptions about qemu behaviour with multiple params to 18-qemu-options.t makes apparently unsafe assumptions about qemu behaviour with multiple params size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by okurz over 2 years ago
- Due date set to 2022-07-22
- Status changed from Workable to Feedback
- Assignee set to okurz
@AdamWill is there still an issue after #112403 was resolved?
The test using -M ?
was initially added in e6cbcfd4 in 2018 to test the behaviour of "QEMU_APPEND" so it's main purpose is to check that parameters are correctly passed to qemu, not trying to test qemu internals. As we are only checking that parameters are correctly added and that having the according effect without checking for the specific content within the list of supported machines I assume we are good?
Updated by okurz over 2 years ago
- Due date deleted (
2022-07-22) - Status changed from Feedback to Resolved
with that I assume we are good