Project

General

Profile

Actions
Copy link

action #110269

closed

[alert] QA-Power8-4-kvm + QA-Power8-5-kvm: Disk I/O time alert size:M

Added by tinita about 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
kraih
Category:
-
Target version:
openQA Project - Ready
Start date:
Due date:
% Done:

0%

Estimated time:
Tags:
reactive work

Description

Observation

  [Alerting] QA-Power8-4-kvm: Disk I/O time alert

      Metric name
      Value
      sdf
      11500.000
      sdi
      11500.000

View your Alert rule [stats.openqa-monitor.qa.suse.de]
Go to the Alerts page [stats.openqa-monitor.qa.suse.de]

https://stats.openqa-monitor.qa.suse.de/d/WDQA-Power8-4-kvm/worker-dashboard-qa-power8-4-kvm?tab=alert&viewPanel=56720&orgId=1&refresh=1m

I paused the alert for now.

Acceptance criteria

  • AC1: No more similar alerts
  • AC2: Relevancy of I/O alerts is understood

Suggestions

  • No apparent problems due to this (bump values?)
  • Research and consider what makes sense
  • Check the disk health
  • Consider performance
  • Unpause the alert but change the notification target for evaluation

Rollback steps

  • Unpause alert on QA-Power8-4-kvm: Disk I/O time alert
  • Unpause alert on QA-Power8-5-kvm: Disk I/O time alert

Related issues 4 (1 open — 3 closed)

Related to openQA Infrastructure - action #96242: [alert] Disk I/O time for /dev/vde (/space-slow) alert 2021-07-28 size:MResolvedmkittler2021-07-29

Actions
Related to openQA Infrastructure - action #70834: [alert] Refine I/O time alerts for OSDResolvedokurz2020-09-02

Actions
Related to openQA Infrastructure - action #59621: osd: Sporadically high CPU and IO load (vdd), grafana alerts "Disk I/O time for /dev/vdd" and "CPU usage", also other disksNew2019-11-14

Actions
Related to openQA Infrastructure - action #112196: [alert][sporadic] QA-Power8-4-kvm: Disk I/O time alert size:MResolvedokurz2022-06-08

Actions
Actions
Copy link

Also available in: Atom PDF