Project

General

Profile

Actions

action #57662

closed

monitoring considered harmful

Added by coolo over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2019-10-03
Due date:
% Done:

0%

Estimated time:

Description

Since osd runs a lot of ltp tests for kernel live patches, we noticed a dramatic slowdown in uploading results. Stracing the mojo workers, they were doing a log of systemv ipc and it was unclear why.
Attaching gdb to it, it was found that this is all triggered by IPC::ShareLite used by Mojolicious::Plugin::Status. I removed both and disabled the monitoring option in openqa.ini

Before:

Oct 03 07:16:14 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-40.txt
Oct 03 07:16:17 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-41.txt
Oct 03 07:16:19 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-42.txt
Oct 03 07:16:28 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-43.txt
Oct 03 07:16:34 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-44.txt
Oct 03 07:16:37 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-45.txt
Oct 03 07:16:41 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-46.txt
Oct 03 07:16:41 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-47.txt
Oct 03 07:16:43 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-48.txt
Oct 03 07:16:50 openqaworker8 worker[18535]: [debug] [pid:23592] Uploading artefact boot_ltp-49.txt

After:

Oct 03 07:21:17 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-41.txt
Oct 03 07:21:18 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-42.txt
Oct 03 07:21:18 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-43.txt
Oct 03 07:21:18 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-44.txt
Oct 03 07:21:19 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-45.txt
Oct 03 07:21:19 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-46.txt
Oct 03 07:21:19 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-47.txt
Oct 03 07:21:20 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-48.txt
Oct 03 07:21:20 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-49.txt
Oct 03 07:21:20 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-50.txt
Oct 03 07:21:21 openqaworker8 worker[18535]: [debug] [pid:25212] Uploading artefact shutdown_ltp-51.txt

I guess either this monitor plugin can be implemented without locking or we need to remove the option.

Actions

Also available in: Atom PDF