Project

General

Profile

Actions

action #134816

closed

[tools] grafana dashboard for `OpenQA Jobs test` partially without any data from OSD migration size:M

Added by osukup 8 months ago. Updated 8 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
Start date:
2023-08-30
Due date:
% Done:

0%

Estimated time:

Description

Observation

Dashboard https://stats.openqa-monitor.qa.suse.de/d/nRDab3Jiz/openqa-jobs-test?orgId=1

missing data in graphs showing running tests from yesterday migration

Acceptance criteria

  • AC1: No missing data for osd on Grafana
  • AC2: Alerts related to affected panels are functioning

Suggestions

  • In salt states in monitoring/telegraf/telegraf-webui.conf instead of grains['fqdn'] use something like grains.get('primary_webui_domain', grains.get('fqdn'))`. Alternatively we could use the "id" in place of the FQDN
  • If the above does not work then use an OR expression since we already have data with different domains in the db (or implement that to cover the data from 2023-08-29 to today)
  • Also check whether alerts need to be covered
  • As alternative can we change the FQDN of osd to again point to openqa.suse.de
    • Apparently a bad idea according to mcaj (not sure why)
  • See existing MR: https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/953

Related issues 1 (0 open1 closed)

Related to QA - action #132146: Support migration of osd VM to PRG2 - 2023-08-29 size:MResolvedmkittler2023-06-29

Actions
Actions #1

Updated by okurz 8 months ago

  • Tags set to infra, osd, prg2, monitor, alert, reactive work
  • Priority changed from Normal to Urgent
  • Target version set to Ready
Actions #2

Updated by okurz 8 months ago

  • Related to action #132146: Support migration of osd VM to PRG2 - 2023-08-29 size:M added
Actions #4

Updated by osukup 8 months ago

  • Status changed from New to In Progress
  • Assignee set to osukup
Actions #5

Updated by openqa_review 8 months ago

  • Due date set to 2023-09-14

Setting due date based on mean cycle time of SUSE QE Tools

Actions #6

Updated by livdywan 8 months ago

  • Subject changed from [tools] graphana dashboard for `OpenQA Jobs test` partially without any data from OSD migration to [tools] graphana dashboard for `OpenQA Jobs test` partially without any data from OSD migration size:M
  • Description updated (diff)
Actions #7

Updated by livdywan 8 months ago

  • Assignee changed from osukup to okurz

Oli is going to come up with a branch/MR based on the id suggestion above

Actions #9

Updated by tinita 8 months ago

  • Subject changed from [tools] graphana dashboard for `OpenQA Jobs test` partially without any data from OSD migration size:M to [tools] grafana dashboard for `OpenQA Jobs test` partially without any data from OSD migration size:M
Actions #10

Updated by okurz 8 months ago

  • Due date deleted (2023-09-14)
  • Status changed from In Progress to Resolved

MR merged. https://monitor.qa.suse.de/d/nRDab3Jiz/openqa-jobs-test?orgId=1&from=1693218539722&to=1693544149511 now again shows data. There is a gap of 2 days in some graphs but I think we can live with that. The alerts are configured for the same panels and we did not change anything in grafana.

Actions

Also available in: Atom PDF