QA » openQA Project

can you please update the ticket description instead of adding the new description as comment?

Also, https://openqa.opensuse.org/minion/jobs?offset=0&task=download_asset might provide some insight

okurz wrote:

can you please update the ticket description instead of adding the new description as comment?

Sorry, I thought that the Edit button edits the description…

Also, https://openqa.opensuse.org/minion/jobs?offset=0&task=download_asset might provide some insight

Not really. This one here downloaded a zero bytes file: https://openqa.opensuse.org/minion/jobs?id=656141 and this one https://openqa.opensuse.org/minion/jobs?id=656140 failed due to "too many download errors" but I really don't see any additional information on that page.

From o3 grep 'Extracting.*failed' /var/log/openqa_gru

[2021-07-01T09:10:37.0930 UTC] [error] [#655446] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-custom-partitions.x86_64-1.15.1-Build9.25.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:10:39.0374 UTC] [error] [#655448] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk.x86_64-1.42.1-Build9.25.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:10:41.0366 UTC] [error] [#655452] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk-simple.x86_64-1.42.1-Build9.25.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:10:47.0370 UTC] [error] [#655450] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk-legacy.x86_64-1.42.1-Build11.3.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:10:53.0960 UTC] [error] [#655456] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-lvm.x86_64-1.42.1-Build9.25.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:10:56.0834 UTC] [error] [#655460] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-overlayroot.x86_64-1.42.1-Build9.25.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:10:56.0901 UTC] [error] [#655462] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-suse-on-dnf.x86_64-1.42.1-Build9.25.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:11:03.0550 UTC] [error] [#655464] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk-ramdisk.x86_64-1.42.1-Build9.25.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:11:06.0027 UTC] [error] [#655458] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-orthos.x86_64-1.13.1-Build10.2.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T09:11:31.0623 UTC] [error] [#655454] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-luks.x86_64-1.15.1-Build9.25.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T12:22:08.0957 UTC] [error] [#655995] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk-simple.x86_64-1.42.1-Build9.26.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T12:22:09.0747 UTC] [error] [#655991] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-custom-partitions.x86_64-1.15.1-Build9.26.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T12:22:10.0917 UTC] [error] [#655993] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk.x86_64-1.42.1-Build9.26.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T12:22:13.0050 UTC] [error] [#656011] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk-ramdisk.x86_64-1.42.1-Build9.26.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T12:22:14.0522 UTC] [error] [#656008] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk-legacy.x86_64-1.42.1-Build11.4.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T13:04:45.0582 UTC] [error] [#656142] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-overlayroot.x86_64-1.42.1-Build9.26.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T13:04:47.0314 UTC] [error] [#656138] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-lvm.x86_64-1.42.1-Build9.26.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T13:04:49.0611 UTC] [error] [#656144] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-suse-on-dnf.x86_64-1.42.1-Build9.26.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T13:04:53.0115 UTC] [error] [#656140] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-orthos.x86_64-1.13.1-Build10.3.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-01T13:04:56.0766 UTC] [error] [#656136] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-luks.x86_64-1.15.1-Build9.26.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-02T11:36:16.0944 UTC] [error] [#660689] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-custom-partitions.x86_64-1.15.3-Build9.7.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-02T11:36:19.0054 UTC] [error] [#660691] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk.x86_64-1.15.3-Build9.7.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-02T11:36:20.0007 UTC] [error] [#660693] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk-simple.x86_64-1.15.3-Build9.7.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-02T11:36:23.0652 UTC] [error] [#660700] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-lvm.x86_64-1.15.3-Build9.7.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-02T11:36:28.0885 UTC] [error] [#660705] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-overlayroot.x86_64-1.15.3-Build9.7.vmdk.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-02T11:36:29.0208 UTC] [error] [#660707] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-disk-ramdisk.x86_64-1.15.3-Build9.7.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.
[2021-07-02T11:36:35.0801 UTC] [error] [#660697] Extracting "/var/lib/openqa/share/factory/tmp/kiwi-test-image-luks.x86_64-1.15.3-Build9.7.raw.xz" failed: You do not have 'IO::Uncompress::UnXz' installed - Please install it as soon as possible.

so

1. Why is the same error detail not accessible over the webUI?
2. Can we look into monitoring logs for errors again?
3. Should IO::Uncompress::UnXz be a general dependency of the openQA package? According to https://metacpan.org/dist/IO-Compress IO::Uncompress is included in there but https://build.opensuse.org/package/show/devel:languages:perl/perl-IO-Compress is not currently part of openSUSE Factory . zypper in -C 'perl(IO::Uncompress::UnXz)' installs perl-IO-Compress-Lzma, but not on Leap 15.2 it seems

1. Why is the same error detail not accessible over the webUI?

The error is not visible in the webUI, because the error message is generated in lib/OpenQA/Task/Asset/Download.pm:104:

    unless ($downloader->download($url, $assetpath, $options)) {
        $ctx->error(my $msg = qq{Downloading "$url" failed because of too many download errors});
        return $job->fail($msg);
    }

and the download function swallows the actual cause of the error (from lib/OpenQA/Downloader.pm:106):

            try {
                die "Could not determine archive type\n"
                  unless my $ae = Archive::Extract->new(archive => $tempfile);
                die $ae->error unless $ae->extract(to => $target);
            }
            catch {
                $log->error(qq{Extracting "$tempfile" failed: $_});
                $ret = $code;
            };

I would suggest that we refactor this function and make the error message visible in the webUI as well. Especially as atm every error, be it download, decompression or /tmp/ is full will result in the error message Downloading "$url" failed because of too many download errors (which is not overly helpful anyway).

1. Can we look into monitoring logs for errors again?

2. Should IO::Uncompress::UnXz be a general dependency of the openQA package?
   According to https://metacpan.org/dist/IO-Compress IO::Uncompress is included in there but https://build.opensuse.org/package/show/devel:languages:perl/perl-IO-Compress is not currently part of openSUSE Factory . zypper in -C 'perl(IO::Uncompress::UnXz)' installs perl-IO-Compress-Lzma, but not on Leap 15.2 it seems

It should become a dependency imho.

We discussed this in the team.
The problem is that the dual life module IO::Compress is not in Factory, but IO::Compress::Lzma needs the newer version from CPAN, the one in the perl core is too old.
There are comments trying to explain why we can't put it into Factory:
https://build.opensuse.org/package/show/devel:languages:perl/perl-IO-Compress

We now decided to link the necessary modules from devel:languages:perl in https://build.opensuse.org/project/show/devel:openQA:Leap:15.2 et al

I did 15.2 and the OBS build for the PR https://github.com/os-autoinst/openQA/pull/4011 doesn't say "unresolvable" any more, so this worked.
I linked the following modules:

• perl-IO-Compress
• perl-Compress-Raw-Lzma
• perl-Compress-Raw-Bzip2
• perl-Compress-Raw-Zlib
• perl-IO-Compress-Lzma

e.g.

osc linkpac openSUSE:Factory perl-IO-Compress-Lzma  devel:openQA:Leap:15.3
osc linkpac devel:languages:perl perl-Compress-Raw-Zlib  devel:openQA:Leap:15.2
...

Will do that for 15.1 and 15.3 too

Edit: done for 15.1 and 15.3 too