Project

General

Profile

Actions
Copy link

action #127046

open

t: os-autoinst - qemu keeps running when test runs into timelimit size:M

Added by tinita over 1 year ago. Updated over 1 year ago.

Status:
Workable
Priority:
Low
Assignee:
-
Category:
Regressions/Crashes
Target version:
QA (public) - future
Start date:
2023-04-01
Due date:
% Done:

0%

Estimated time:

Description

Observation

We use OpenQA::Test::TimeLimit to limit test runtimes.
When the limit is exceeded for t/14-isotovideo.t for example, then /usr/bin/qemu-system-i386 keeps running, and the next test will die.

Acceptance Criteria

  • AC1: Terminating a test will cleanup everything including running child processes

Suggestions

  • Send SIGTERM instead, or send SIGTERM first and SIGKILL only after another timeout?
  • Use a process group for all processes created so the group can be killed atomically similar to the openqa worker which uses this approach already
  • Maybe isotovideo is what's killed and can't propagate the signal

Related issues 1 (0 open — 1 closed)

Related to openQA Project (public) - action #157339: os-autoinst t/14-isotovideo.t is again taking too long (>20s on my setup) size:MResolvedybonatakis2024-03-15

Actions
Actions
Copy link
 #1

Updated by tinita over 1 year ago

  • Subject changed from t: qemu keeps running when test runs into timelimit to t: os-autoinst - qemu keeps running when test runs into timelimit
Actions
Copy link
 #2

Updated by okurz over 1 year ago

  • Priority changed from Normal to Low
  • Target version set to Ready
Actions
Copy link
 #3

Updated by livdywan over 1 year ago

  • Subject changed from t: os-autoinst - qemu keeps running when test runs into timelimit to t: os-autoinst - qemu keeps running when test runs into timelimit size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #4

Updated by tinita over 1 year ago

I had a quick look, and was able to see that in case of an exceeded timelimit the BAIL_OUT happens, and while the test is stopped, isotovideo and qemu are still running.
isotovideo will finish at some point, it seems to just finish its regular job, but qemu will run forever.
None of the signal handlers in isotovideo is triggered.

I printed the isotovideo log for this case, and I can see this error (which happens before qemu is even started

[2023-06-22T14:17:40.369410+02:00] [debug] [pid:8418] running `/usr/bin/chattr +C /tmp/14-isotovideo.t-rQQQ/pool/raid`                             
[2023-06-22T14:17:40.377104+02:00] [debug] [pid:8418] Fatal error in command `/usr/bin/chattr +C /tmp/14-isotovideo.t-rQQQ/pool/raid`: open3: exec of /usr/bin/chattr +C /tmp/14-isotovideo.t-rQQQ/pool/raid failed: No such file or directory at /usr/lib/perl5/vendor_perl/5.2
6.1/Mojo/IOLoop/ReadWriteProcess.pm line 127.
Actions
Copy link
 #5

Updated by okurz over 1 year ago

  • Target version changed from Ready to future

As suggested by the team removing from backlog as we are exceeding our backlog length

Actions
Copy link
 #6

Updated by okurz 10 months ago

  • Related to action #157339: os-autoinst t/14-isotovideo.t is again taking too long (>20s on my setup) size:M added
Actions
Copy link

Also available in: Atom PDF