action #88450: Flaky NTP offset alert - openQA Infrastructure (public) - openSUSE Project Management Tool

Actions

Copy link

action #88450

closed

Flaky NTP offset alert

Added by livdywan almost 4 years ago. Updated almost 4 years ago.

Status:

Resolved

Priority:

High

Assignee:

mkittler

Category:

Target version:

openQA Project (public) - Ready

Start date:

2021-02-04

Due date:

% Done:

Estimated time:

Description

Observation¶

Metric name
Stratum 3

Value
3073.250

Alert seen 8.44, OK seen 10.28

Acceptance criteria¶

AC1: Alert not flaky

Suggestions¶

To remove urgency just bump up the threshold time to prevent flaky alerts
Check how ntp is kept in sync, adjust timeouts, servers used
Possibly the same as #71224 but totally guessing here. That was supposed to have been fixed according to the comments.

Actions

Copy link

Updated by okurz almost 4 years ago

Description updated (diff)
Status changed from New to Workable
Priority changed from Normal to Urgent
Target version set to Ready

The urgency could be resolved by simply bumping the threshold time.

Actions

Copy link

Updated by mkittler almost 4 years ago

Priority changed from Urgent to High

Considering the frequency of these spikes it doesn't seem to be too urgent: https://stats.openqa-monitor.qa.suse.de/d/WebuiDb/webui-summary?tab=query&editPanel=86&viewPanel=86&orgId=1&from=now-90d&to=now

We need to allow at least a 5 second offset (at least for a certain time period) to clear out the warnings (judging by the data from the last 90 days). That sounds quite high.

Note that the current warning only checks for positive offsets so far and not negative ones. This could also be fixed (if we're touching the warning conditions anyways).

Actions

Copy link