Project

General

Profile

Actions

action #127274

closed

[alert] Usage of partition mmcblk0p3 on openqa-piworker exceeds threshold size:M

Added by mkittler over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Start date:
2023-04-05
Due date:
% Done:

0%

Estimated time:
Tags:

Description

Observation

This triggers the alert https://stats.openqa-monitor.qa.suse.de/alerting/grafana/partitions_usage_alert_openqa-piworker/view I've been adding a silence for.

Strangely I haven't received a notification mail about it.

Acceptance Criteria

  • AC1: The alert is no longer firing
  • AC2: We will still be notified if the disk is really almost full

Suggestions

  • Tweak the alert for that worker specifically (so we can utilize more of it without triggering the alert)
  • Check with e.g. ncdu what could be cleaned up
  • Possibly make the journal non-volatile again; however, maybe we should just utilize the disk space we have as the journal is valuable (so basically suggestion one seems preferable)
Actions

Also available in: Atom PDF