Project

General

Profile

Actions
Copy link

action #134816

closed

[tools] grafana dashboard for `OpenQA Jobs test` partially without any data from OSD migration size:M

Added by osukup over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Urgent
Assignee:
okurz
Category:
-
Target version:
openQA Project (public) - Ready
Start date:
2023-08-30
Due date:
% Done:

0%

Estimated time:
Tags:
alert, osd, monitor, infra, reactive work, prg2

Description

Observation

Dashboard https://stats.openqa-monitor.qa.suse.de/d/nRDab3Jiz/openqa-jobs-test?orgId=1

missing data in graphs showing running tests from yesterday migration

Acceptance criteria

  • AC1: No missing data for osd on Grafana
  • AC2: Alerts related to affected panels are functioning

Suggestions

  • In salt states in monitoring/telegraf/telegraf-webui.conf instead of grains['fqdn'] use something like grains.get('primary_webui_domain', grains.get('fqdn'))`. Alternatively we could use the "id" in place of the FQDN
  • If the above does not work then use an OR expression since we already have data with different domains in the db (or implement that to cover the data from 2023-08-29 to today)
  • Also check whether alerts need to be covered
  • As alternative can we change the FQDN of osd to again point to openqa.suse.de
    • Apparently a bad idea according to mcaj (not sure why)
  • See existing MR: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/953
Actions
Copy link
 #1

Updated by okurz over 1 year ago

  • Tags set to infra, osd, prg2, monitor, alert, reactive work
  • Priority changed from Normal to Urgent
  • Target version set to Ready
Actions
Copy link
 #4

Updated by osukup over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to osukup
Actions
Copy link
 #5

Updated by openqa_review over 1 year ago

  • Due date set to 2023-09-14

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #6

Updated by livdywan over 1 year ago

  • Subject changed from [tools] graphana dashboard for `OpenQA Jobs test` partially without any data from OSD migration to [tools] graphana dashboard for `OpenQA Jobs test` partially without any data from OSD migration size:M
  • Description updated (diff)
Actions
Copy link
 #7

Updated by livdywan over 1 year ago

  • Assignee changed from osukup to okurz

Oli is going to come up with a branch/MR based on the id suggestion above

Actions
Copy link
 #9

Updated by tinita over 1 year ago

  • Subject changed from [tools] graphana dashboard for `OpenQA Jobs test` partially without any data from OSD migration size:M to [tools] grafana dashboard for `OpenQA Jobs test` partially without any data from OSD migration size:M
Actions
Copy link
 #10

Updated by okurz over 1 year ago

  • Due date deleted (2023-09-14)
  • Status changed from In Progress to Resolved

MR merged. https://monitor.qa.suse.de/d/nRDab3Jiz/openqa-jobs-test?orgId=1&from=1693218539722&to=1693544149511 now again shows data. There is a gap of 2 days in some graphs but I think we can live with that. The alerts are configured for the same panels and we did not change anything in grafana.

Actions
Copy link

Also available in: Atom PDF