QA » openQA Project

Observation¶

qemu 6.1.0 requires that, if you specify a backing file for qemu-img create, you must also specify the backing format. Up to 6.0.0 it was warning that not specifying the backing format was deprecated, but it was a warning, not fatal - it would auto-detect the format and proceed. See e.g. this autoinst log:

    [2021-09-02T12:43:17.932 UTC] [debug] Initializing block device images
    [2021-09-02T12:43:17.933 UTC] [debug] running /usr/bin/qemu-img create -f qcow2 /var/lib/openqa/pool/26/raid/hd0 10G
    [2021-09-02T12:43:17.961 UTC] [debug] Formatting '/var/lib/openqa/pool/26/raid/hd0', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10737418240 lazy_refcounts=off refcount_bits=16
    [2021-09-02T12:43:17.961 UTC] [debug] running /usr/bin/qemu-img create -f qcow2 -b /var/lib/openqa/pool/26/Fedora-Server-dvd-x86_64-35-20210902.n.0.iso /var/lib/openqa/pool/26/raid/cd0-overlay0 2236612608
    [2021-09-02T12:43:17.986 UTC] [debug] qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of raw)
      Formatting '/var/lib/openqa/pool/26/raid/cd0-overlay0', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2236612608 backing_file=/var/lib/openqa/pool/26/Fedora-Server-dvd-x86_64-35-20210902.n.0.iso lazy_refcounts=off refcount_bits=16
    [2021-09-02T12:43:17.986 UTC] [debug] running /usr/bin/qemu-img create -f qcow2 -b /usr/share/edk2/ovmf/OVMF_CODE.fd /var/lib/openqa/pool/26/raid/pflash-code-overlay0 1966080
    [2021-09-02T12:43:18.011 UTC] [debug] qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of raw)
      Formatting '/var/lib/openqa/pool/26/raid/pflash-code-overlay0', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=1966080 backing_file=/usr/share/edk2/ovmf/OVMF_CODE.fd lazy_refcounts=off refcount_bits=16
    [2021-09-02T12:43:18.011 UTC] [debug] running /usr/bin/qemu-img create -f qcow2 -b /usr/share/edk2/ovmf/OVMF_VARS.fd /var/lib/openqa/pool/26/raid/pflash-vars-overlay0 131072
    [2021-09-02T12:43:18.036 UTC] [debug] qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of raw)
      Formatting '/var/lib/openqa/pool/26/raid/pflash-vars-overlay0', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=131072 backing_file=/usr/share/edk2/ovmf/OVMF_VARS.fd lazy_refcounts=off refcount_bits=16
    [2021-09-02T12:43:18.036 UTC] [debug] init_blockdev_images: Finished creating block devices

Steps to reproduce¶

This is what you get if you just run the tests from current HEAD with qemu 6.1.0:

 # [2021-08-31T14:48:35.199 UTC] [debug] Initializing block device images
 # [2021-08-31T14:48:35.199 UTC] [debug] running /usr/bin/qemu-img create -f qcow2 /tmp/18-qemu-options.t-8oXw/pool/raid/hd0 10G
 # [2021-08-31T14:48:35.217 UTC] [debug] Formatting '/tmp/18-qemu-options.t-8oXw/pool/raid/hd0', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10737418240 lazy_refcounts=off refcount_bits=16
 # [2021-08-31T14:48:35.217 UTC] [debug] running /usr/bin/qemu-img create -f qcow2 -b /builddir/build/BUILD/os-autoinst-ad28b4b59369ada5731ebd7f0916bc2e52324988/t/data/Core-7.2.iso /tmp/18-qemu-options.t-8oXw/pool/raid/cd0-overlay0 11116544
 # [2021-08-31T14:48:35.231 UTC] [debug] qemu-img: /tmp/18-qemu-options.t-8oXw/pool/raid/cd0-overlay0: Backing file specified without backing format
 #   Detected format of raw.
 # [2021-08-31T14:48:35.231 UTC] [info] ::: backend::baseclass::die_handler: Backend process died, backend errors are reported below in the following lines:
 #   runcmd '/usr/bin/qemu-img create -f qcow2 -b /builddir/build/BUILD/os-autoinst-ad28b4b59369ada5731ebd7f0916bc2e52324988/t/data/Core-7.2.iso /tmp/18-qemu-options.t-8oXw/pool/raid/cd0-overlay0 11116544' failed with exit code 1: 'qemu-img: /tmp/18-qemu-options.t-8oXw/pool/raid/cd0-overlay0: Backing file specified without backing format
 #   Detected format of raw.' at /builddir/build/BUILD/os-autoinst-ad28b4b59369ada5731ebd7f0916bc2e52324988/t/../osutils.pm line 121.

Suggestion¶

• Prevent failure of qemu-img create with qemu 6.1.0

Further details¶

I was trying to just send a PR for this, but the relevant code is heavily abstracted and I haven't been able to figure it out completely. So far I have this, which ought to be correct, but actually it seems we are not currently specifying the correct "driver" for the objects that represent the backing files. If you apply this patch and run the tests, the ones that expect '-F raw' will fail because they instead get '-F qcow2'. We should be specifying 'raw' as the "driver" for firmware and ISO backing files, but we apparently are not doing so in this context, and I don't know where to fix that.

    From 282806def4c5b1021f431e58e8258d1dea4d6882 Mon Sep 17 00:00:00 2001
    From: Adam Williamson <awilliam@redhat.com>
    Date: Fri, 3 Sep 2021 11:02:42 -0700
    Subject: [PATCH] Specify backing file format for qemu-img commands (qemu
     6.1.0)

    qemu 6.1.0 requires the format to be specified for backing files
    in `qemu-img create`. In 6.0.0 and before, not specifying it was
    deprecated, and you can see the warnings in autoinst-log.txt,
    e.g.:

    [2021-09-02T12:42:37.401 UTC] [debug] qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of qcow2)

    Signed-off-by: Adam Williamson <awilliam@redhat.com>
    ---
     OpenQA/Qemu/BlockDev.pm |  6 ++++--
     t/18-qemu.t             | 10 +++++-----
     2 files changed, 9 insertions(+), 7 deletions(-)

    diff --git a/OpenQA/Qemu/BlockDev.pm b/OpenQA/Qemu/BlockDev.pm
    index 19f70a98..c8d7cbd8 100644
    --- a/OpenQA/Qemu/BlockDev.pm
    +++ b/OpenQA/Qemu/BlockDev.pm
    @@ -163,8 +163,10 @@ sub gen_qemu_img_cmdlines {
         return @cmdlns unless $self->needs_creating;

         my @params = ('create', '-f', $self->driver);
    -    push(@params, ('-b', $self->backing_file->file))
    -      if defined $self->backing_file;
    +    if (defined $self->backing_file) {
    +        push(@params, ('-b', $self->backing_file->file));
    +        push(@params, ('-F', $self->backing_file->driver));
    +    }
         push(@params, $self->file);
         push(@params, $self->size);

    diff --git a/t/18-qemu.t b/t/18-qemu.t
    index 147ed8a2..c8b887fd 100755
    --- a/t/18-qemu.t
    +++ b/t/18-qemu.t
    @@ -84,7 +84,7 @@ $bdc->add_existing_drive('hd1', '/abs/path/sle15-minimal.qcow2', 'virtio-blk', 2
     @gcmdl = $bdc->gen_cmdline();
     is_deeply(\@gcmdl, \@cmdl, 'Generate qemu command line for single existing drive');

    -@cmdl  = ([qw(create -f qcow2 -b /abs/path/sle15-minimal.qcow2 raid/hd1-overlay0 22548578304)]);
    +@cmdl  = ([qw(create -f qcow2 -b /abs/path/sle15-minimal.qcow2 -F qcow2 raid/hd1-overlay0 22548578304)]);
     @gcmdl = $bdc->gen_qemu_img_cmdlines();
     is_deeply(\@gcmdl, \@cmdl, 'Generate qemu-img command line for single existing drive');

    @@ -114,7 +114,7 @@ $proc  = qemu_proc('-foo', \%vars);
     @gcmdl = $proc->gen_cmdline();
     is_deeply(\@gcmdl, \@cmdl, 'Generate qemu command line for single existing UEFI disk using vars');

    -@cmdl  = ([qw(create -f qcow2 -b), "$Bin/data/Core-7.2.iso", qw(raid/hd0-overlay0 11116544)]);
    +@cmdl  = ([qw(create -f qcow2 -b), "$Bin/data/Core-7.2.iso", qw(-F raw raid/hd0-overlay0 11116544)]);
     @gcmdl = $proc->blockdev_conf->gen_qemu_img_cmdlines();
     is_deeply(\@gcmdl, \@cmdl, 'Generate qemu-img command line for single existing UEFI disk');

    @@ -229,8 +229,8 @@ $proc->deserialise_state(path($path)->slurp());
     is_deeply(\@gcmdl, \@cmdl, 'Command line after snapshot and serialisation')
       || diag(explain(\@gcmdl));

    -@cmdl = ([qw(create -f qcow2 -b raid/hd0 raid/hd0-overlay1 10G)],
    -    [qw(create -f qcow2 -b raid/cd0-overlay0 raid/cd0-overlay1 11116544)]);
    +@cmdl = ([qw(create -f qcow2 -b raid/hd0 -F qcow2 raid/hd0-overlay1 10G)],
    +    [qw(create -f qcow2 -b raid/cd0-overlay0 -F qcow2 raid/cd0-overlay1 11116544)]);
     @gcmdl = $bdc->gen_qemu_img_cmdlines();
     is_deeply(\@gcmdl, \@cmdl, 'Generate reverted snapshot images');

    @@ -384,7 +384,7 @@ subtest 'relative assets' => sub {
         symlink("$Bin/data/Core-7.2.iso", "./Core-7.2.iso");
         $proc = qemu_proc('-foo', \%vars);
         my @gcmdl = $proc->blockdev_conf->gen_qemu_img_cmdlines();
    -    @cmdl = map { [qw(create -f qcow2 -b), "$dir/Core-7.2.iso", "raid/$_-overlay0", 11116544] } qw(hd0 cd0 cd1);
    +    @cmdl = map { [qw(create -f qcow2 -b), "$dir/Core-7.2.iso", "-F", "raw", "raid/$_-overlay0", 11116544] } qw(hd0 cd0 cd1);
         push @cmdl, ["create", "-f", "qcow2", "-b", "$Bin/data/uefi-code.bin", "raid/pflash-code-overlay0", 1966080];
         push @cmdl, ["create", "-f", "qcow2", "-b", "$dir/Core-7.2.iso",       "raid/pflash-vars-overlay0", 11116544];
         is_deeply(\@gcmdl, \@cmdl, 'find the asset real path');
    -- 
    2.32.0