Project

General

Profile

action #112733

openQA Project - coordination #109846: [epic] Ensure all our database tables accomodate enough data, e.g. bigint for ids

Webui Summary dashboard in Grafana is missing I/O panels size:M

Added by nicksinger about 2 months ago. Updated 14 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Start date:
2022-06-20
Due date:
% Done:

0%

Estimated time:

Description

Summary

https://stats.openqa-monitor.qa.suse.de/d/WebuiDb/webui-summary shows no I/O panels any longer. I could imagine that this might be related to our generation of the dashboard but has to be checked.

Acceptance criteria

  • AC1: We have IO panels and data back for openQA webUI and workers

Suggestions

  • Maybe related to grafana version update so that the maybe still in the background existing panels are visible (vague guess by nicksinger).
  • Investigate if the generated JSON shows invisible panels, try to play around in the graphical editor how a newly created panel would look like in the generated JSON: okurz thinks that the stuff is still there, from https://stats.openqa-monitor.qa.suse.de/d/WebuiDb/webui-summary?orgId=1&editview=dashboard_json

    {
      "collapsed": false,
      "gridPos": {
        "h": 1,
        "w": 24,
        "x": 0,
        "y": 54
      },
      "id": 60,
      "panels": [],
      "title": "IO of vda (/ , fast storage)",
      "type": "row"
    },
    {
      "collapsed": false,
      "gridPos": {
        "h": 1,
        "w": 24,
        "x": 0,
        "y": 55
      },
      "id": 92,
      "panels": [],
      "title": "IO of vdb (/srv , fast storage)",
      "type": "row"
    },
    {
      "collapsed": false,
      "gridPos": {
        "h": 1,
        "w": 24,
        "x": 0,
        "y": 56
      },
      "id": 64,
      "panels": [],
      "title": "IO of vdc (/assets , cheap+slow storage)",
      "type": "row"
    },
    
  • Read about https://grafana.com/blog/2020/06/09/learn-grafana-how-to-automatically-repeat-rows-and-panels-in-dynamic-dashboards/ and if we can make use of it here

  • After fixing it take a look if there is a better way to generate which is less error-prone


Related issues

Related to openQA Infrastructure - coordination #112718: [alert][osd] openqa.suse.de is not reachable anymore, response times > 30s, multiple alerts over the weekendResolved2022-06-22

Related to openQA Infrastructure - action #113671: [timeboxed][10h] Configure write of I/O panels to be on the negative Y-axis again once we're on grafana 8.4 size:SFeedback2022-06-20

Related to openQA Infrastructure - action #113674: Configure I/O alerts again for the webui after migrating to the "unified alerting" in grafana size:MWorkable2022-06-20

History

#1 Updated by okurz about 2 months ago

  • Related to coordination #112718: [alert][osd] openqa.suse.de is not reachable anymore, response times > 30s, multiple alerts over the weekend added

#2 Updated by okurz about 2 months ago

  • Target version set to Ready

#3 Updated by okurz about 2 months ago

  • Parent task set to #109846

#4 Updated by cdywan about 1 month ago

  • Subject changed from Webui Summary dashboard is missing IO panels to Webui Summary dashboard in Grafana is missing I/O panels size:M
  • Description updated (diff)
  • Status changed from New to Workable

#5 Updated by nicksinger about 1 month ago

  • Status changed from Workable to Feedback
  • Assignee set to nicksinger

Seems like we lost them with https://gitlab.suse.de/openqa/salt-states-openqa/-/commit/038d155bfc02f3562fceca7f6255a3b2458959ea#83363be13fc620ae79e6efa4c1e0cca5f9e930cb_3441_3387 or at least critical data (e.g. the query) to render them properly.
I created a first draft with https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/708/diffs?commit_id=7a12180cc4842122447a7a04363916c4b9fb352b which introduces a variable based on which we could repeat the panels/rows for each disk we have in a system. This would make the dashboard highly dynamic (e.g. new telegraf disks would show up automatically) but has the drawback of loosing some descriptive text (e.g. that vda is / and stored on fast disks). Have to think and ask people about it.

#6 Updated by nicksinger about 1 month ago

  • Status changed from Feedback to In Progress

#7 Updated by openqa_review about 1 month ago

  • Due date set to 2022-07-27

Setting due date based on mean cycle time of SUSE QE Tools

#8 Updated by nicksinger about 1 month ago

  • Copied to action #113671: [timeboxed][10h] Configure write of I/O panels to be on the negative Y-axis again once we're on grafana 8.4 size:S added

#9 Updated by nicksinger about 1 month ago

  • Copied to deleted (action #113671: [timeboxed][10h] Configure write of I/O panels to be on the negative Y-axis again once we're on grafana 8.4 size:S)

#10 Updated by nicksinger about 1 month ago

  • Related to action #113671: [timeboxed][10h] Configure write of I/O panels to be on the negative Y-axis again once we're on grafana 8.4 size:S added

#11 Updated by nicksinger about 1 month ago

  • Related to action #113674: Configure I/O alerts again for the webui after migrating to the "unified alerting" in grafana size:M added

#12 Updated by nicksinger about 1 month ago

  • Status changed from In Progress to Feedback

#14 Updated by nicksinger 29 days ago

  • Status changed from Feedback to Resolved

#15 Updated by okurz 14 days ago

  • Due date deleted (2022-07-27)

Also available in: Atom PDF