Project

General

Profile

Actions

action #107941

closed

coordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes

coordination #98952: [epic] t/full-stack.t sporadically fails "clickElement: element not interactable" and other errors

[sporadic] openQA Fullstack test t/full-stack.t can still fail with "udevadm" log message size:M

Added by mkittler about 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2022-03-07
Due date:
% Done:

0%

Estimated time:

Description

Observation

This problem showed up in https://app.circleci.com/pipelines/github/os-autoinst/openQA/9123/workflows/42eb45bf-126b-43be-97f4-b64af16c27ed/jobs/86387/steps (PR: https://github.com/os-autoinst/openQA/pull/4496) and is again a problem of the os-autoinst test itself - or rather unexpected behavior of the SUT.

Checkout the attached log file and especially to see what's going on. Summary:

  • sudo poweroff is not typed completely (only sudo pow)
  • After the prompt, the following message is logged:
udevadm settle - timeout of 5 seconds reached, the event queue contains
  /sys/devices/pci0000:00/0000:00:07/virtio3/block/vda (1174)
  • Then nothing happens so we see the usual log messages in autoinst-log.txt complaining that the system doesn't shutdown.

Further details

  • mkittler had to conduct approximediately 60 runs of openQA's fullstack test in circleCI to show the problem once. (And one run of openQA's fullstack test runs the problematic openQA test at least two times.)
  • QEMU commmand-line: /usr/bin/qemu-system-i386 -only-migratable -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -audiodev none,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 -m 1024 -netdev user,id=qanet0 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -boot once=d -smp 1 -no-shutdown -vnc :91,share=force-shared -device virtio-serial -chardev pipe,id=virtio_console,path=virtio_console,logfile=virtio_console.log,logappend=on -device virtconsole,chardev=virtio_console,name=org.openqa.console.virtio_console -chardev pipe,id=virtio_console1,path=virtio_console1,logfile=virtio_console1.log,logappend=on -device virtconsole,chardev=virtio_console1,name=org.openqa.console.virtio_console1 -chardev socket,path=qmp_socket,server=on,wait=off,id=qmp_socket,logfile=qmp_socket.log,logappend=on -qmp chardev:qmp_socket -S -device virtio-scsi-pci,id=scsi0 -blockdev driver=file,node-name=hd0-file,filename=/home/squamata/project/test-results/fullstack/full-stack.d/openqa/pool/1/raid/hd0,cache.no-flush=on -blockdev driver=qcow2,node-name=hd0,file=hd0-file,cache.no-flush=on -device virtio-blk,id=hd0-device,drive=hd0,serial=hd0 -blockdev driver=file,node-name=cd0-overlay0-file,filename=/home/squamata/project/test-results/fullstack/full-stack.d/openqa/pool/1/raid/cd0-overlay0,cache.no-flush=on -blockdev driver=qcow2,node-name=cd0-overlay0,file=cd0-overlay0-file,cache.no-flush=on -device scsi-cd,id=cd0-device,drive=cd0-overlay0,serial=cd0

Suggestions

  • Only if we actually see it at least a second time we should look into this again
  • Try to send the command again e.g. 2-3 times

Files

autoinst-log.txt (22.7 KB) autoinst-log.txt mkittler, 2022-03-07 14:46
video.ogg (171 KB) video.ogg mkittler, 2022-03-07 14:46
Actions

Also available in: Atom PDF