Project

General

Custom queries

Profile

Actions

action #153925

closed

Support YAM squad to get backlogger running in our salt states (and fix our pipelines again)

Added by okurz 11 months ago. Updated 11 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Start date:
2024-01-19
Due date:
% Done:

0%

Estimated time:


Files


Related issues 1 (0 open1 closed)

Related to qe-yam - action #153205: Connect backlog assistant to GrafanaResolvedrainerkoenig2024-01-08

Actions
Actions #1

Updated by livdywan 11 months ago

  • Related to action #153205: Connect backlog assistant to Grafana added
Actions #2

Updated by nicksinger 11 months ago

  • Status changed from New to In Progress
  • Assignee set to nicksinger
Actions #3

Updated by nicksinger 11 months ago

  • Status changed from In Progress to Feedback
  • Priority changed from Urgent to Normal

After https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1093 being merged the pipeline itself was able to test the pillars properly again which enabled us to rebase https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1092 to pass the pipeline there. Unfortunately with my revert (https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1089) I did not delete the according, generated telegraf-config in /etc/telegraf/telegraf.d which left us with the old and wrong API key and therefore telegraf was still failing. With MR!1092 merged this also got fixed and we now stumble over a new issue: https://gitlab.suse.de/openqa/salt-states-openqa/-/jobs/2186381#L129

This seems to be caused by https://github.com/rakoenig/qe-yam-backlog-assistant/blob/master/queries.yaml#L31 which triggers a missing escaping implementation in out backlogger tool. I looked up the rules and implemented https://github.com/openSUSE/backlogger/pull/28 as permanent fix. As urgency mitigation I adjusted /etc/telegraf/scripts/tools-yam-backlog/queries.yaml manually and added a backslash before every comma. I'd assume salt overwrites this repo but maybe due to an unclean state it doesn't and apparently seems to work for now.

Actions #4

Updated by jbaier_cz 11 months ago

Just for the record, the implemented escaping seems to be similar to influx_line_protocol, maybe as a follow-up we can use influxdb-client-python to communicate with influxdb directly (or to at least create the output).

Actions #5

Updated by okurz 11 months ago

  • Status changed from Feedback to New

good. https://github.com/openSUSE/backlogger/pull/28 is now merged as well. What do you plan as next step?

Actions #6

Updated by nicksinger 11 months ago

I checked and my changes are already deployed on the monitor host. I reverted the config and ran the script again - this time successful. Metrics are available in grafana as well:

Actions

Also available in: Atom PDF