action #112733
closedopenQA Project - coordination #109846: [epic] Ensure all our database tables accomodate enough data, e.g. bigint for ids
Webui Summary dashboard in Grafana is missing I/O panels size:M
0%
Description
Summary¶
https://stats.openqa-monitor.qa.suse.de/d/WebuiDb/webui-summary shows no I/O panels any longer. I could imagine that this might be related to our generation of the dashboard but has to be checked.
Acceptance criteria¶
- AC1: We have IO panels and data back for openQA webUI and workers
Suggestions¶
- Maybe related to grafana version update so that the maybe still in the background existing panels are visible (vague guess by nicksinger).
Investigate if the generated JSON shows invisible panels, try to play around in the graphical editor how a newly created panel would look like in the generated JSON: okurz thinks that the stuff is still there, from https://stats.openqa-monitor.qa.suse.de/d/WebuiDb/webui-summary?orgId=1&editview=dashboard_json
{ "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 54 }, "id": 60, "panels": [], "title": "IO of vda (/ , fast storage)", "type": "row" }, { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 55 }, "id": 92, "panels": [], "title": "IO of vdb (/srv , fast storage)", "type": "row" }, { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 56 }, "id": 64, "panels": [], "title": "IO of vdc (/assets , cheap+slow storage)", "type": "row" },
Read about https://grafana.com/blog/2020/06/09/learn-grafana-how-to-automatically-repeat-rows-and-panels-in-dynamic-dashboards/ and if we can make use of it here
After fixing it take a look if there is a better way to generate which is less error-prone
Updated by okurz over 2 years ago
- Related to coordination #112718: [alert][osd] openqa.suse.de is not reachable anymore, response times > 30s, multiple alerts over the weekend added
Updated by livdywan over 2 years ago
- Subject changed from Webui Summary dashboard is missing IO panels to Webui Summary dashboard in Grafana is missing I/O panels size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by nicksinger over 2 years ago
- Status changed from Workable to Feedback
- Assignee set to nicksinger
Seems like we lost them with https://gitlab.suse.de/openqa/salt-states-openqa/-/commit/038d155bfc02f3562fceca7f6255a3b2458959ea#83363be13fc620ae79e6efa4c1e0cca5f9e930cb_3441_3387 or at least critical data (e.g. the query) to render them properly.
I created a first draft with https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/708/diffs?commit_id=7a12180cc4842122447a7a04363916c4b9fb352b which introduces a variable based on which we could repeat the panels/rows for each disk we have in a system. This would make the dashboard highly dynamic (e.g. new telegraf disks would show up automatically) but has the drawback of loosing some descriptive text (e.g. that vda is / and stored on fast disks). Have to think and ask people about it.
Updated by nicksinger over 2 years ago
- Status changed from Feedback to In Progress
I found a pretty nifty solution which might be overkill but works: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/708/diffs?commit_id=8b9e103a02211a0117bebe322f82b4a498a228f6
These descriptions can be updated per host by using our pillars: https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/419/diffs
Updated by openqa_review over 2 years ago
- Due date set to 2022-07-27
Setting due date based on mean cycle time of SUSE QE Tools
Updated by nicksinger over 2 years ago
- Copied to action #113671: [timeboxed][10h] Configure write of I/O panels to be on the negative Y-axis again once we're on grafana 8.4 size:S added
Updated by nicksinger over 2 years ago
- Copied to deleted (action #113671: [timeboxed][10h] Configure write of I/O panels to be on the negative Y-axis again once we're on grafana 8.4 size:S)
Updated by nicksinger over 2 years ago
- Related to action #113671: [timeboxed][10h] Configure write of I/O panels to be on the negative Y-axis again once we're on grafana 8.4 size:S added
Updated by nicksinger over 2 years ago
- Related to coordination #113674: [epic] Configure I/O alerts again for the webui after migrating to the "unified alerting" in grafana size:M added
Updated by nicksinger over 2 years ago
- Status changed from In Progress to Feedback
https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/708 is ready for review and merge
Updated by okurz over 2 years ago
merged and looks nice on https://monitor.qa.suse.de/d/WebuiDb/webui-summary
Updated by nicksinger over 2 years ago
- Status changed from Feedback to Resolved