Project

General

Profile

action #110266

[alert] Disk I/O time for sr0 size:S

Added by tinita 2 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Observation

  [Alerting] storage: Disk I/O time alert
      Metric name
      Value
      sr0
      14351.852

View your Alert rule [stats.openqa-monitor.qa.suse.de]
Go to the Alerts page [stats.openqa-monitor.qa.suse.de]

https://stats.openqa-monitor.qa.suse.de/d/GDstorage/dashboard-for-storage?tab=alert&viewPanel=56720&orgId=1&refresh=1m

Acceptance criteria

  • AC1: No more similar alerts

Suggestions

  • Research how currently the grafana query is constructed, is it a jinja loop iterating over all block devices? If yes, then exclude sr0 or virtual CD from there
  • Switch off alerts for virtual CD's

Related issues

Copied from openQA Infrastructure - action #96242: [alert] Disk I/O time for /dev/vde (/space-slow) alert 2021-07-28 size:MResolved2021-07-29

History

#1 Updated by tinita 2 months ago

  • Copied from action #96242: [alert] Disk I/O time for /dev/vde (/space-slow) alert 2021-07-28 size:M added

#2 Updated by tinita 2 months ago

  • Start date deleted (2021-07-29)

#3 Updated by jbaier_cz 2 months ago

The /dev/sr0 reminds me #107158

#4 Updated by cdywan 2 months ago

  • Subject changed from [alert] Disk I/O time for sr0 to [alert] Disk I/O time for sr0 size:S
  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from Urgent to High

#6 Updated by okurz 2 months ago

  • Status changed from Workable to Feedback
  • Assignee set to jbaier_cz

#7 Updated by jbaier_cz 2 months ago

  • Status changed from Feedback to In Progress

For some reason, we still have IO data for sr0 on storage causing alerts. Needs further investigation.

#8 Updated by openqa_review 2 months ago

  • Due date set to 2022-05-16

Setting due date based on mean cycle time of SUSE QE Tools

#9 Updated by okurz about 2 months ago

  • Tags set to reactive work

#10 Updated by jbaier_cz about 2 months ago

  • Status changed from In Progress to Resolved

Did not see the alert for a while, so I assume the changes worked.

#11 Updated by jbaier_cz about 2 months ago

  • Due date deleted (2022-05-16)

Also available in: Atom PDF