Project

General

Profile

Actions

action #104077

closed

backend died: Can't syswrite(IO::Socket::UNIX=GLOB(0x558d9dd5cb68), <BUFFER>): Broken pipe at /usr/lib/os-autoinst/backend/qemu.pm line 985 size:M

Added by GraceWang about 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2021-12-16
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://openqa.suse.de/tests/7857824/logfile?filename=autoinst-log.txt shows

[2021-12-15T16:37:40.022513+01:00] [info] ::: backend::baseclass::die_handler: Backend process died, backend errors are reported below in the following lines:
  Can't syswrite(IO::Socket::UNIX=GLOB(0x558d9dd5cb68), <BUFFER>): Broken pipe at /usr/lib/os-autoinst/backend/qemu.pm line 985
      backend::qemu::syswrite(IO::Socket::UNIX=GLOB(0x558d9dd5cb68), "{\"execute\":\"query-status\"}\x{a}") called at /usr/lib/os-autoinst/backend/qemu.pm line 985
      backend::qemu::handle_qmp_command(backend::qemu=HASH(0x558d9f2f1220), HASH(0x558d9ed9f280), "fatal", 1) called at /usr/lib/os-autoinst/backend/qemu.pm line 204
      backend::qemu::_wait_while_status_is(backend::qemu=HASH(0x558d9f2f1220), qr(migrate)u, 300, "Timed out while loading snapshot") called at /usr/lib/os-autoinst/backend/qemu.pm line 459
      backend::qemu::load_snapshot(backend::qemu=HASH(0x558d9f2f1220), HASH(0x558d9dd8ba68)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 74
      backend::baseclass::handle_command(backend::qemu=HASH(0x558d9f2f1220), HASH(0x558d9dd86b58)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 545
      backend::baseclass::check_socket(backend::qemu=HASH(0x558d9f2f1220), IO::Handle=GLOB(0x558d9f25b780)) called at /usr/lib/os-autoinst/backend/qemu.pm line 1060
      backend::qemu::check_socket(backend::qemu=HASH(0x558d9f2f1220), IO::Handle=GLOB(0x558d9f25b780), 0) called at /usr/lib/os-autoinst/backend/baseclass.pm line 236
      backend::baseclass::do_capture(backend::qemu=HASH(0x558d9f2f1220), undef, 1639581624.84715) called at /usr/lib/os-autoinst/backend/baseclass.pm line 265
      eval {...} called at /usr/lib/os-autoinst/backend/baseclass.pm line 265
      backend::baseclass::run_capture_loop(backend::qemu=HASH(0x558d9f2f1220)) called at /usr/lib/os-autoinst/backend/baseclass.pm line 128
      backend::baseclass::run(backend::qemu=HASH(0x558d9f2f1220), 13, 16) called at /usr/lib/os-autoinst/backend/driver.pm line 70
      backend::driver::__ANON__(Mojo::IOLoop::ReadWriteProcess=HASH(0x558d9f8f8040)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
      eval {...} called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 326
      Mojo::IOLoop::ReadWriteProcess::_fork(Mojo::IOLoop::ReadWriteProcess=HASH(0x558d9f8f8040), CODE(0x558d9eb1e900)) called at /usr/lib/perl5/vendor_perl/5.26.1/Mojo/IOLoop/ReadWriteProcess.pm line 488
      Mojo::IOLoop::ReadWriteProcess::start(Mojo::IOLoop::ReadWriteProcess=HASH(0x558d9f8f8040)) called at /usr/lib/os-autoinst/backend/driver.pm line 71
      backend::driver::start(backend::driver=HASH(0x558d9f2b1ef8)) called at /usr/lib/os-autoinst/backend/driver.pm line 38
      backend::driver::new("backend::driver", "qemu") called at /usr/bin/isotovideo line 211
      main::init_backend() called at /usr/bin/isotovideo line 262
[2021-12-15T16:37:40.022856+01:00] [info] ::: OpenQA::Qemu::Proc::save_state: Saving QEMU state to qemu_state.json

A cloned job https://openqa.suse.de/tests/7863925 does not immediately reproduce this problem

Reproducible

So far only a single occurence was found

Acceptance criteria

  • AC1: We at least improved the code a little bit

Problem

We see a not so helpful stack trace.

Suggestions

  • Look at the exact code section and take a look what could go wrong there, improve test code coverage, logging and error handling.
  • Don't try to fix a problem that we can't reproduce :)
  • Switch over to transactional upgrades

Related issues 1 (0 open1 closed)

Related to openQA Infrastructure (public) - action #99192: Upgrade osd workers and openqa-monitor to openSUSE Leap 15.3 size:MResolvedlivdywan

Actions
Actions

Also available in: Atom PDF