action #132383
Updated by okurz 11 months ago
## Observation
Multiple FC Basement OSD hosts are not reachable since 2023-07-06 01:50 CEST as visible on https://monitor.qa.suse.de/alerting/list?search=state:firing mentioning
* [openqa-piworker](https://monitor.qa.suse.de/alerting/grafana/host_up_alert_openqa-piworker/view?returnTo=%2Falerting%2Flist%3Fsearch%3Dstate%3Afiring)
* [openqaworker1](https://monitor.qa.suse.de/alerting/grafana/host_up_alert_openqaworker1/view?returnTo=%2Falerting%2Flist%3Fsearch%3Dstate%3Afiring)
* [sapworker2](https://monitor.qa.suse.de/alerting/grafana/host_up_alert_sapworker2/view?returnTo=%2Falerting%2Flist%3Fsearch%3Dstate%3Afiring)
* [sapworker3](https://monitor.qa.suse.de/alerting/grafana/host_up_alert_sapworker3/view?returnTo=%2Falerting%2Flist%3Fsearch%3Dstate%3Afiring)
plus the ["Packet loss" alert](https://monitor.qa.suse.de/alerting/grafana/2Z025iB4km/view?returnTo=%2Falerting%2Flist%3Fsearch%3Dstate%3Afiring)
## Suggestions
* Wait for https://sd.suse.com/servicedesk/customer/portal/1/SD-126191 resolution
* Ensure all relevant machines in FC Basement are reachable again
* Retry https://gitlab.suse.de/openqa/osd-deployment/-/pipelines and other gitlab CI pipelines where failed
* Ensure alerts are ok again