action #155716
Updated by okurz 3 months ago
## Observation
From https://monitor.qa.suse.de/d/KToPYLEWz/failed-systemd-services?orgId=1
`ssh worker29.oqa.prg2.suse.org "journalctl -u openqa-worker-cacheservice"` says
```
Feb 21 09:25:43 worker29 openqa-workercache-daemon[86009]: [86009] [e] Database has been corrupted: DBD::SQLite::db commit failed: disk I/O error at /u>
Feb 21 09:25:43 worker29 openqa-workercache-daemon[86009]: [86009] [e] Killing processes accessing the database file handles and removing database
```
## Suggestions
* *DONE* Add silence(s) silence
## Rollback actions
* Remove silence `alertname=Failed systemd services alert` from https://monitor.qa.suse.de/alerting/silences
* Remove silence `alertname=Broken workers alert` from https://monitor.qa.suse.de/alerting/silences
Back