Project

General

Profile

Actions

action #111602

closed

18-qemu-options.t makes apparently unsafe assumptions about qemu behaviour with multiple params size:M

Added by AdamWill almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2022-05-25
Due date:
% Done:

0%

Estimated time:

Description

Observation

okurz recently changed 18-qemu-options.t so the second run in the qemu_append_option subtest appends -M ? -version. Previously it appended -M ?. The test assumes that qemu will always print only the version information (not supported machines) when run this way, but that doesn't seem safe.

In Fedora's build environment, we get both the version information and the supported machines:

    2:     # [2022-05-25T19:04:44.856378Z] [debug] starting: /usr/bin/qemu-system-i386 -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -audiodev none,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 -global isa-fdc.fdtypeA=none -m 1024 -netdev user,id=qanet0 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -boot once=d -device qemu-xhci -device usb-tablet -smp 1 -no-shutdown -vnc :90,share=force-shared -device virtio-serial -chardev pipe,id=virtio_console,path=virtio_console,logfile=virtio_console.log,logappend=on -device virtconsole,chardev=virtio_console,name=org.openqa.console.virtio_console -chardev pipe,id=virtio_console1,path=virtio_console1,logfile=virtio_console1.log,logappend=on -device virtconsole,chardev=virtio_console1,name=org.openqa.console.virtio_console1 -chardev socket,path=qmp_socket,server=on,wait=off,id=qmp_socket,logfile=qmp_socket.log,logappend=on -qmp chardev:qmp_socket -S -M ? -version -device virtio-scsi-pci,id=scsi0 -blockdev driver=file,node-name=hd0-file,filename=/tmp/18-qemu-options.t-dNVG/pool/raid/hd0,cache.no-flush=on -blockdev driver=qcow2,node-name=hd0,file=hd0-file,cache.no-flush=on -device virtio-blk,id=hd0-device,drive=hd0,serial=hd0
    2:     # [2022-05-25T19:04:44.862180Z] [debug] Waiting for 0 attempts
    2:     # perl: warning: Setting locale failed.
    2:     # perl: warning: Please check that your locale settings:
    2:     #     LANGUAGE = (unset),
    2:     #     LC_ALL = "en_US.UTF-8",
    2:     #     LANG = "en_US.UTF-8"
    2:     #     are supported and installed on your system.
    2:     # perl: warning: Falling back to the standard locale ("C").
    2:     # [2022-05-25T19:04:45.032982Z] [debug] Waiting for 1 attempts
    2:     # [2022-05-25T19:04:45.033289Z] [info] ::: backend::baseclass::die_handler: Backend process died, backend errors are reported below in the following lines:
    2:     #   QEMU terminated before QMP connection could be established. Check for errors below
    2:     # [2022-05-25T19:04:45.033642Z] [info] ::: OpenQA::Qemu::Proc::save_state: Saving QEMU state to qemu_state.json
    2:     # [2022-05-25T19:04:45.035028Z] [debug] Passing remaining frames to the video encoder
    2:     # [2022-05-25T19:04:45.087372Z] [debug] Waiting for video encoder to finalize the video
    2:     # [2022-05-25T19:04:45.087556Z] [debug] The built-in video encoder (pid 960824) terminated
    2:     # [2022-05-25T19:04:45.088978Z] [debug] QEMU: QEMU emulator version 6.2.0 (qemu-6.2.0-10.fc36)
    2:     # [2022-05-25T19:04:45.089061Z] [debug] QEMU: Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
    2:     # [2022-05-25T19:04:45.089115Z] [debug] QEMU: Supported machines are:
    2:     # [2022-05-25T19:04:45.089170Z] [debug] QEMU: microvm              microvm (i386)
…
    2:     # [2022-05-25T19:04:45.090022Z] [debug] QEMU: pc-i440fx-3.0       
    2:     # [2022-05-25T19:04:45.091199Z] [debug] sending magic and exit
    2:     # [2022-05-25T19:04:45.091579Z] [debug] received magic close
    2:     # [2022-05-25T19:04:45.097114Z] [debug] backend process exited: 0
    2:     # failed to start VM at /builddir/build/BUILD/os-autoinst-548335fc1544beeb51e86fe83e025a0f0aa4901e/t/../backend/driver.pm line 107.
    2:     # [2022-05-25T19:04:45.197976Z] [debug] stopping command server 960811 because test execution ended through exception
    2:     # [2022-05-25T19:04:45.398914Z] [debug] done with command server
    2:     # [2022-05-25T19:04:45.399034Z] [debug] stopping autotest process 960814
    2:     # [2022-05-25T19:04:45.599884Z] [debug] done with autotest process
    2:     # 960807: EXIT 1
    2:     # '
    2:     #           matches '(?^u:Supported machines are\:)'
    2:     # Looks like you failed 1 test of 14.

On my local system, I get just the supported machines, no version information:

    [adamw@xps13k os-autoinst (rawhide *%)]$ qemu-system-i386 -M ? -version
    Supported machines are:
    microvm              microvm (i386)
    xenfv-4.2            Xen Fully-virtualized PC
    xenfv                Xen Fully-virtualized PC (alias of xenfv-3.1)
    xenfv-3.1            Xen Fully-virtualized PC
    pc                   Standard PC (i440FX + PIIX, 1996) (alias of pc-i440fx-7.0)
    pc-i440fx-7.0        Standard PC (i440FX + PIIX, 1996) (default)
 …
    pc-q35-2.10          Standard PC (Q35 + ICH9, 2009)
    isapc                ISA-only PC
    none                 empty machine
    x-remote             Experimental remote machine
    xenpv                Xen Para-virtualized PC
    [adamw@xps13k os-autoinst (rawhide *%)]$ 

I'm not really sure what the point of this test is, anyway. Why are we testing qemu behaviour here? Shouldn't we only care whether the parameters got appended properly?

Acceptance criteria

  • AC1: A conscious decision was made if we want to do such test at all or not

Suggestions

  • Read the question from AdamWilliamson why we test this at all, understand why the according test was included, make sure that we only do test what makes sense feature and coverage wise

Related issues 1 (0 open1 closed)

Related to openQA Project - action #112403: 18-qemu-options.t fails on Leap 15.4 and Tumbleweed size:MResolvedmkittler2022-06-14

Actions
Actions #1

Updated by okurz almost 2 years ago

  • Target version set to Ready
Actions #2

Updated by okurz almost 2 years ago

  • Priority changed from Normal to Low
Actions #3

Updated by livdywan almost 2 years ago

What I currently see is this:

$ cmake --build build -t test-perl-testsuite TESTS="t/18-qemu-options.t"
   Failed test 'Supported machines not listed'
   at t/18-qemu-options.t line 78.
[...]
          matches '(?^u:Supported machines are\:)'
Looks like you failed 1 test of 14.

This seems to match the "Fedora" case described?

Actions #4

Updated by okurz almost 2 years ago

  • Related to action #112403: 18-qemu-options.t fails on Leap 15.4 and Tumbleweed size:M added
Actions #5

Updated by okurz almost 2 years ago

  • Status changed from New to Blocked
  • Assignee set to okurz
Actions #6

Updated by okurz over 1 year ago

  • Status changed from Blocked to New
  • Assignee deleted (okurz)
Actions #7

Updated by okurz over 1 year ago

  • Subject changed from 18-qemu-options.t makes apparently unsafe assumptions about qemu behaviour with multiple params to 18-qemu-options.t makes apparently unsafe assumptions about qemu behaviour with multiple params size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #8

Updated by okurz over 1 year ago

  • Description updated (diff)
Actions #9

Updated by okurz over 1 year ago

  • Due date set to 2022-07-22
  • Status changed from Workable to Feedback
  • Assignee set to okurz

@AdamWill is there still an issue after #112403 was resolved?

The test using -M ? was initially added in e6cbcfd4 in 2018 to test the behaviour of "QEMU_APPEND" so it's main purpose is to check that parameters are correctly passed to qemu, not trying to test qemu internals. As we are only checking that parameters are correctly added and that having the according effect without checking for the specific content within the list of supported machines I assume we are good?

Actions #10

Updated by okurz over 1 year ago

  • Tags set to reactive work
Actions #11

Updated by okurz over 1 year ago

  • Due date deleted (2022-07-22)
  • Status changed from Feedback to Resolved

with that I assume we are good

Actions

Also available in: Atom PDF