action #127046
open
t: os-autoinst - qemu keeps running when test runs into timelimit size:M
Added by tinita about 1 year ago.
Updated 10 months ago.
Category:
Regressions/Crashes
Description
Observation¶
We use OpenQA::Test::TimeLimit to limit test runtimes.
When the limit is exceeded for t/14-isotovideo.t for example, then /usr/bin/qemu-system-i386 keeps running, and the next test will die.
Acceptance Criteria¶
- AC1: Terminating a test will cleanup everything including running child processes
Suggestions¶
- Send SIGTERM instead, or send SIGTERM first and SIGKILL only after another timeout?
- Use a process group for all processes created so the group can be killed atomically similar to the openqa worker which uses this approach already
- Maybe isotovideo is what's killed and can't propagate the signal
- Subject changed from t: qemu keeps running when test runs into timelimit to t: os-autoinst - qemu keeps running when test runs into timelimit
- Priority changed from Normal to Low
- Target version set to Ready
- Subject changed from t: os-autoinst - qemu keeps running when test runs into timelimit to t: os-autoinst - qemu keeps running when test runs into timelimit size:M
- Description updated (diff)
- Status changed from New to Workable
I had a quick look, and was able to see that in case of an exceeded timelimit the BAIL_OUT happens, and while the test is stopped, isotovideo and qemu are still running.
isotovideo will finish at some point, it seems to just finish its regular job, but qemu will run forever.
None of the signal handlers in isotovideo is triggered.
I printed the isotovideo log for this case, and I can see this error (which happens before qemu is even started
[2023-06-22T14:17:40.369410+02:00] [debug] [pid:8418] running `/usr/bin/chattr +C /tmp/14-isotovideo.t-rQQQ/pool/raid`
[2023-06-22T14:17:40.377104+02:00] [debug] [pid:8418] Fatal error in command `/usr/bin/chattr +C /tmp/14-isotovideo.t-rQQQ/pool/raid`: open3: exec of /usr/bin/chattr +C /tmp/14-isotovideo.t-rQQQ/pool/raid failed: No such file or directory at /usr/lib/perl5/vendor_perl/5.2
6.1/Mojo/IOLoop/ReadWriteProcess.pm line 127.
- Target version changed from Ready to future
As suggested by the team removing from backlog as we are exceeding our backlog length
- Related to action #157339: os-autoinst t/14-isotovideo.t is again taking too long (>20s on my setup) size:M added
Also available in: Atom
PDF