Actions
action #153346
closedUpdate Grafana API call to get firing alerts to new API size:M
Start date:
2024-01-10
Due date:
% Done:
0%
Estimated time:
Tags:
Description
Observation¶
We are requesting https://stats.openqa-monitor.qa.suse.de/api/alerts?state=alerting in our post-deploy monitor step in osd-deployment.
However, that returns nothing, and even without a filter https://stats.openqa-monitor.qa.suse.de/api/alerts it returns an empty list.
After the migration to the unified alerting we probably need to use new API calls.
It's not obvious which call it would be though. See https://grafana.com/docs/grafana/latest/developers/
Our goal is to detect broken deployments, i.e. new alerts firing after deployment.
Suggestions¶
- Read how https://gitlab.suse.de/openqa/osd-deployment/-/blob/master/.gitlab-ci.yml?ref_type=heads#L19
- I found this openapi editor url which lists some API calls that I can't find in the documentation. e.g. https://stats.openqa-monitor.qa.suse.de/api/alertmanager/grafana/api/v2/alerts does return something, but not much. But it includes the current silenced one https://monitor.qa.suse.de/alerting/grafana/2Z025iB4km/view
- Check https://gitlab.suse.de/openqa/salt-states-openqa#updating-alert-rules-with-the-help-of-the-grafana-web-ui
- Update our check, maybe extend the jq hacking we have
- Crosscheck that what we did in the past actually worked
Updated by nicksinger 8 months ago
- Status changed from Workable to In Progress
- Assignee set to nicksinger
Updated by nicksinger 8 months ago
- Status changed from In Progress to Feedback
Updated by okurz 8 months ago
- Status changed from Feedback to Resolved
https://gitlab.suse.de/openqa/osd-deployment/-/merge_requests/55 merged. We can reopen if we see problems.
Actions