Project

General

Profile

Actions
Copy link

action #135230

closed

salt pillars pipelines failing due to Temporary failure in name resolution

Added by livdywan 8 months ago. Updated 8 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
nicksinger
Category:
-
Target version:
openQA Project - Ready
Start date:
2023-09-06
Due date:
% Done:

0%

Estimated time:
Tags:
gitlab, infra, reactive work

Description

Observation

https://gitlab.suse.de/openqa/salt-pillars-openqa/-/jobs/1808645

openqaworker-arm-2.suse.de:
    telegraf is fine
openqa.suse.de:
    2023-09-06T08:13:41Z E! [inputs.http] Error in plugin: [url=https://openqa.suse.de/admin/influxdb/minion]: Get "https://openqa.suse.de/admin/influxdb/minion": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
[...]
atus 2 - /usr/bin/ping: qa-jump.qe.nue2.suse.org: Name or service not known
    2023-09-06T08:13:38Z E! [telegraf] Error running agent: input plugins recorded 2 errors
    2023-09-06T08:13:41Z E! [inputs.http] Error in plugin: [url=https://openqa.suse.de/admin/influxdb/minion]: Get "https://openqa.suse.de/admin/influxdb/minion": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
    2023-09-06T08:13:41Z E! [inputs.http] Error in plugin: [url=https://openqa.suse.de/admin/influxdb/jobs]: Get "https://openqa.suse.de/admin/influxdb/jobs": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
    2023-09-06T08:13:41Z E! [telegraf] Error running agent: input plugins recorded 2 errors

Acceptance criteria

  • AC1: salt pillars pipelines succeed

Suggestions

  • Report an eng-infra ticket

Related issues 2 (0 open — 2 closed)

Related to openQA Infrastructure - action #134879: reverse DNS resolution PTR for openqa.oqa.prg2.suse.org. yields "3(NXDOMAIN)" for PRG1 workers (NUE1+PRG2 are fine) size:MResolvedokurz2023-08-31

Actions
Has duplicate openQA Infrastructure - action #135206: [tools] GitlabCI telegraf step on salt-states-openqa failedRejectednicksinger2023-09-05

Actions
Actions
Copy link
 #2

Updated by livdywan 8 months ago

  • Status changed from New to Blocked
  • Assignee set to nicksinger
Actions
Copy link
 #3

Updated by okurz 8 months ago

There had also been user reports which could be related to DNS resolution problems in https://suse.slack.com/archives/C02CANHLANP/p1693986939350679 and https://suse.slack.com/archives/C02CANHLANP/p1693988984111069

Actions
Copy link
 #4

Updated by osukup 8 months ago

Actions
Copy link
 #5

Updated by nicksinger 8 months ago

  • Has duplicate action #135206: [tools] GitlabCI telegraf step on salt-states-openqa failed added
Actions
Copy link
 #6

Updated by nicksinger 8 months ago

  • Project changed from QA to openQA Infrastructure
Actions
Copy link
 #7

Updated by nicksinger 8 months ago

  • Related to action #134879: reverse DNS resolution PTR for openqa.oqa.prg2.suse.org. yields "3(NXDOMAIN)" for PRG1 workers (NUE1+PRG2 are fine) size:M added
Actions
Copy link
 #8

Updated by okurz 8 months ago

  • Tags changed from infra, arm, FC Basement to infra, gitlab, reactive
  • Parent task deleted (#130955)
Actions
Copy link
 #9

Updated by okurz 8 months ago

  • Tags changed from infra, gitlab, reactive to infra, gitlab, reactive work
Actions
Copy link
 #10

Updated by nicksinger 8 months ago

  • Status changed from Blocked to Resolved

so the issue was a wrong configuration on the DNS servers for qa.suse.cz hosted e.g. on qanet.suse.cz. Martin was able to resolve the issue in https://progress.opensuse.org/issues/134879#note-12 and we have a working pipeline again: https://gitlab.suse.de/openqa/salt-states-openqa/-/jobs/1810909

Actions
Copy link

Also available in: Atom PDF