Project

General

Profile

Actions
Copy link

action #170026

closed

[QA][tools][monitor] monitor.qa.suse.de is down

Added by waynechen55 about 1 month ago. Updated 27 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
okurz
Category:
Regressions/Crashes
Target version:
openQA Project (public) - Ready
Start date:
2024-11-18
Due date:
% Done:

0%

Estimated time:
Tags:
infra, reactive work, nue2

Description

Observation

Can not reach monitor.qa.suse.de at the moment.

Can not open monitor.qa.suse.de webpage

localhost:/home/waynechen # ping -c5 -4 monitor.qa.suse.de
PING (10.168.192.191) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable

--- ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4062ms
pipe 4
localhost:/home/waynechen # ping -c5 -6 monitor.qa.suse.de
PING monitor.qa.suse.de(2a07:de40:a102:5:5054:ff:fe00:894e (2a07:de40:a102:5:5054:ff:fe00:894e)) 56 data bytes

--- monitor.qa.suse.de ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4084ms

Steps to reproduce

Impact

  • Can not open dashboard
  • Can not store metrics data

Problem

  • monitor.qa.suse.de down unexpectedly ?

Suggestions

  • Check host on which monitor.qa.suse.de runs
  • Check related services
  • Check firewall rules

Workaround

n/a

Related issues 1 (1 open — 0 closed)

Related to openQA Infrastructure (public) - action #170077: Put more storage into qamaster "to make our lives easier in general" size:MBlockedokurz2024-11-19

Actions
Actions
Copy link
 #1

Updated by okurz about 1 month ago

  • Category set to Regressions/Crashes
  • Status changed from New to Resolved
  • Assignee set to okurz
  • Target version set to Ready

There was a short power outage in NUE2 on 2024-11-17 causing problems on the TOR network switches which needed manual intervention by IT. monitor.qa.suse.de is back up and fully reachable again. Details in #170038

Actions
Copy link
 #2

Updated by okurz about 1 month ago

  • Tags set to infra, reactive work, nue2
Actions
Copy link
 #3

Updated by waynechen55 27 days ago

  • Status changed from Resolved to Workable

monitor.qa.suse.de is not accessible again. Power outage again ? @okurz

Actions
Copy link
 #4

Updated by okurz 27 days ago

No, now it's #170077

Actions
Copy link
 #5

Updated by okurz 27 days ago

  • Related to action #170077: Put more storage into qamaster "to make our lives easier in general" size:M added
Actions
Copy link
 #6

Updated by okurz 27 days ago

  • Status changed from Workable to Blocked
Actions
Copy link
 #7

Updated by waynechen55 27 days ago

okurz wrote in #note-4:

No, now it's #170077

localhost:/home/waynechen/Projects/QE_Metrics/deployment_backup # ping -c5 -4 qamaster.qe.nue2.suse.org
PING  (10.168.192.188) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable

---  ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4071ms
pipe 4
localhost:/home/waynechen/Projects/QE_Metrics/deployment_backup # ping -c5 -6 qamaster.qe.nue2.suse.org
PING qamaster.qe.nue2.suse.org(2a07:de40:a102:5:225:90ff:fef1:d8e4 (2a07:de40:a102:5:225:90ff:fef1:d8e4)) 56 data bytes

--- qamaster.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4103ms
Actions
Copy link
 #8

Updated by okurz 27 days ago

  • Status changed from Blocked to Resolved

it's back

Actions
Copy link

Also available in: Atom PDF