Project

General

Profile

action #75259

Updated by okurz 11 months ago

## Observation
Currently, all powerpc tests end with result incomplete

E.g https://openqa.opensuse.org/tests/1448500

From the log:

[2020-10-25T08:55:26.0807 UTC] [info] +++ setup notes +++
[2020-10-25T08:55:26.0807 UTC] [info] Running on power8:6 (Linux 5.3.18-lp152.47-default #1 SMP Thu Oct 15 16:05:25 UTC 2020 (41f7396) ppc64le)
[2020-10-25T08:55:26.0810 UTC] [debug] Found ASSET_256, caching openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso.sha256
[2020-10-25T08:55:26.0812 UTC] [info] Downloading openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso.sha256, request #37952 sent to Cache Service
[2020-10-25T08:55:31.0853 UTC] [info] Download of openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso.sha256 processed:
[info] [#37952] Cache size of "/var/lib/openqa/cache" is 600GiB, with limit 600GiB
[info] [#37952] Downloading "openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso.sha256" from "http://openqa1-opensuse/tests/1448500/asset/other/openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso.sha256"
[info] [#37952] Content of "/var/lib/openqa/cache/openqa1-opensuse/openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso.sha256" has not changed, updating last use

[2020-10-25T08:55:31.0856 UTC] [debug] Found ISO, caching openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso
[2020-10-25T08:55:31.0858 UTC] [info] Downloading openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso, request #37957 sent to Cache Service
[2020-10-25T08:55:36.0899 UTC] [info] Download of openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso processed:
[info] [#37957] Cache size of "/var/lib/openqa/cache" is 600GiB, with limit 600GiB
[info] [#37957] Downloading "openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso" from "http://openqa1-opensuse/tests/1448500/asset/iso/openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso"
[info] [#37957] Content of "/var/lib/openqa/cache/openqa1-opensuse/openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso" has not changed, updating last use

[2020-10-25T08:55:36.0902 UTC] [info] Rsync from 'rsync://openqa1-opensuse/tests' to '/var/lib/openqa/cache/openqa1-opensuse', request #37961 sent to Cache Service
[2020-10-25T08:55:41.0954 UTC] [info] Output of rsync:
[info] [#37961] Calling: rsync -avHP rsync://openqa1-opensuse/tests/ --delete /var/lib/openqa/cache/openqa1-opensuse/tests/
receiving incremental file list

sent 1,713 bytes received 1,099,636 bytes 734,232.67 bytes/sec
total size is 4,532,146,856 speedup is 4,115.09

[2020-10-25T08:55:41.0954 UTC] [info] Finished to rsync tests
[2020-10-25T08:55:41.0958 UTC] [debug] +++ worker notes +++
[2020-10-25T08:55:42.377 UTC] [debug] Current version is 4.6.1603469837.f54bdeab [interface v20]
[2020-10-25T08:55:42.384 UTC] [debug] git hash in /var/lib/openqa/cache/openqa1-opensuse/tests/opensuse: c12d1da2e511d1cb87aeb302b752bc035a2c8c60
[2020-10-25T08:55:42.560 UTC] [debug] usingenv DESKTOP=kde
[2020-10-25T08:55:42.560 UTC] [debug] usingenv DISTRI=opensuse
[2020-10-25T08:55:42.560 UTC] [debug] usingenv QEMUCPU=host
[2020-10-25T08:55:42.560 UTC] [debug] usingenv QEMUCPUS=4
[2020-10-25T08:55:42.560 UTC] [debug] usingenv INSTLANG=en_US
[2020-10-25T08:55:42.560 UTC] [debug] usingenv DVD=1
[2020-10-25T08:55:42.560 UTC] [debug] usingenv KDE=1
[2020-10-25T08:55:42.560 UTC] [debug] usingenv ISO=/var/lib/openqa/pool/6/openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso
[2020-10-25T08:55:42.561 UTC] [debug] usingenv ISO_MAXSIZE=4700372992
[2020-10-25T08:55:42.561 UTC] [debug] usingenv NOIMAGES=1
[2020-10-25T08:55:42.561 UTC] [debug] scheduling mediacheck tests/installation/mediacheck.pm
[2020-10-25T08:55:42.567 UTC] [info] cmdsrv: daemon reachable under http://*:20063/qL0cvoaoKR5zOTFr/
[2020-10-25T08:55:42.568 UTC] [debug] git hash in /var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles: 5482b32dfe22ba8e0376db50c26e07cc01ee56fb
[2020-10-25T08:55:42.568 UTC] [debug] init needles from /var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles
[2020-10-25T08:55:42.569 UTC] [info] Listening at "http://[::]:20063"
Web application available at http://[::]:20063
[2020-10-25T08:55:42.884 UTC] [debug] inst-console-20200224 contains inst-console twice
[2020-10-25T08:55:43.715 UTC] [debug] loaded 8201 needles
[2020-10-25T08:55:43.862 UTC] [debug] Blocking SIGTERM
[2020-10-25T08:55:43.884 UTC] [debug] Unblocking SIGTERM
[2020-10-25T08:55:43.981 UTC] [debug] Blocking SIGTERM
19934: channel_out 15, channel_in 14
[2020-10-25T08:55:44.000 UTC] [debug] Unblocking SIGTERM
19982: cmdpipe 13, rsppipe 16
[2020-10-25T08:55:44.001 UTC] [debug] started mgmt loop with pid 19982
remove_tree qemuscreenshot
[2020-10-25T08:55:44.025 UTC] [debug] qemu version detected: 4.2.1
[2020-10-25T08:55:44.026 UTC] [debug] running /usr/bin/chattr -f +C /var/lib/openqa/pool/6/raid
[2020-10-25T08:55:44.032 UTC] [debug] running /usr/bin/qemu-img info --output=json /var/lib/openqa/pool/6/openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso
[2020-10-25T08:55:44.039 UTC] [debug] running /usr/bin/qemu-img create -f qcow2 /var/lib/openqa/pool/6/raid/hd0 20G
[2020-10-25T08:55:44.076 UTC] [debug] Formatting '/var/lib/openqa/pool/6/raid/hd0', fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off refcount_bits=16
[2020-10-25T08:55:44.076 UTC] [debug] running /usr/bin/qemu-img create -f qcow2 -b /var/lib/openqa/pool/6/openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso /var/lib/openqa/pool/6/raid/cd0-overlay0 3745804288
[2020-10-25T08:55:44.119 UTC] [debug] Formatting '/var/lib/openqa/pool/6/raid/cd0-overlay0', fmt=qcow2 size=3745804288 backing_file=/var/lib/openqa/pool/6/openSUSE-Tumbleweed-DVD-ppc64le-Snapshot20201024-Media.iso cluster_size=65536 lazy_refcounts=off refcount_bits=16
[2020-10-25T08:55:44.120 UTC] [debug] starting: /usr/bin/qemu-system-ppc64 -g 1024x768 -vga std -only-migratable -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -audiodev none,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 -global isa-fdc.driveA= -m 4096 -machine usb=off -cpu host -netdev user,id=qanet0 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -boot once=d -device nec-usb-xhci -device usb-tablet -device usb-kbd -smp 4 -enable-kvm -no-shutdown -vnc :96,share=force-shared -device virtio-serial -chardev pipe,id=virtio_console,path=virtio_console,logfile=virtio_console.log,logappend=on -device virtconsole,chardev=virtio_console,name=org.openqa.console.virtio_console -chardev socket,path=qmp_socket,server,nowait,id=qmp_socket,logfile=qmp_socket.log,logappend=on -qmp chardev:qmp_socket -S -device virtio-scsi-pci,id=scsi0 -blockdev driver=file,node-name=hd0-file,filename=/var/lib/openqa/pool/6/raid/hd0,cache.no-flush=on -blockdev driver=qcow2,node-name=hd0,file=hd0-file,cache.no-flush=on -device virtio-blk,id=hd0-device,drive=hd0,serial=hd0 -blockdev driver=file,node-name=cd0-overlay0-file,filename=/var/lib/openqa/pool/6/raid/cd0-overlay0,cache.no-flush=on -blockdev driver=qcow2,node-name=cd0-overlay0,file=cd0-overlay0-file,cache.no-flush=on -device scsi-cd,id=cd0-device,drive=cd0-overlay0,serial=cd0
[2020-10-25T08:55:44.123 UTC] [debug] Waiting for 0 attempts
[2020-10-25T08:55:44.185 UTC] [debug] Waiting for 1 attempts
[2020-10-25T08:55:44.185 UTC] [debug] Backend process died, backend errors are reported below in the following lines:
QEMU terminated before QMP connection could be established at /usr/lib/os-autoinst/OpenQA/Qemu/Proc.pm line 444.

[2020-10-25T08:55:44.185 UTC] [info] ::: OpenQA::Qemu::Proc::save_state: Saving QEMU state to qemu_state.json
[2020-10-25T08:55:44.186 UTC] [debug] Passing remaining frames to the video encoder
[2020-10-25T08:55:44.187 UTC] [debug] Waiting for video encoder to finalize the video
[2020-10-25T08:55:44.187 UTC] [debug] The built-in video encoder (pid 20002) terminated
[2020-10-25T08:55:44.188 UTC] [debug] QEMU: QEMU emulator version 4.2.1 (openSUSE Leap 15.2)
[2020-10-25T08:55:44.188 UTC] [debug] QEMU: Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
[2020-10-25T08:55:44.188 UTC] [debug] QEMU: Unknown host!
[2020-10-25T08:55:44.188 UTC] [debug] QEMU: Unknown host!
[2020-10-25T08:55:44.188 UTC] [debug] QEMU: Unknown host!
[2020-10-25T08:55:44.188 UTC] [debug] QEMU: qemu-system-ppc64: Requested safe cache capability level not supported by kvm, try appending -machine cap-cfpc=broken

## Problem

This very likely happened due to the upgrade of the worker host to openSUSE Leap 15.2 in #73189

## Suggestions
* First consider rolling back the installation or maybe just the qemu version
* Try the suggestion in the log file: "try appending -machine cap-cfpc=broken", to the qemu command line in jobs, then add that maybe in the openQA "machine" configuration?
* Consider including the necessary setting in os-autoinst itself

Back