action #163778
Updated by tinita 5 months ago
## Observation We had several alerts regarding s390zl12 today, firing and resolving shortly after each other: http://stats.openqa-monitor.qa.suse.de/alerting/grafana/5ddd66bd99b31f7597fd68af2cc96304f8d9e480/view?orgId=1 http://stats.openqa-monitor.qa.suse.de/alerting/grafana/openqa_ping_time_alert_s390zl12/view?orgId=1 ``` Date: Thu, 11 Jul 2024 14:24:34 +0200 From: Grafana <osd-admins@suse.de> To: osd-admins@suse.de Subject: [FIRING:2] s390zl12 2 firing alert instances [IMAGE] 📁 GROUPED BY hostname=s390zl12 🔥 2 firing instances Firing [stats.openqa-monitor.qa.suse.de] s390zl12: Ping time alert View alert [stats.openqa-monitor.qa.suse.de] Values B0=372.96250000000003 Labels alertname s390zl12: Ping time alert grafana_folder Generic hostname s390zl12 rule_uid ping_time_alert_s390zl12 type generic Silence [stats.openqa-monitor.qa.suse.de] View dashboard [stats.openqa-monitor.qa.suse.de] View panel [stats.openqa-monitor.qa.suse.de] Observed 34s before this notification was delivered, at 2024-07-11 14:24:00 +0200 CEST Firing [stats.openqa-monitor.qa.suse.de] s390zl12: OpenQA Ping time alert View alert [stats.openqa-monitor.qa.suse.de] Values B0=372.96250000000003 Labels alertname s390zl12: OpenQA Ping time alert grafana_folder openQA hostname s390zl12 rule_uid openqa_ping_time_alert_s390zl12 type worker ## Rollback steps * Remove the two silences