Actions
action #157834
closed[openQA][ipmi] IPMI backend machines in NUE2 can not be reached auto_review:"Reason: backend died: ipmitool.*Address lookup for.*qe.nue2.suse.org":retry
Status:
Resolved
Priority:
High
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2024-03-25
Due date:
% Done:
0%
Estimated time:
Description
Observation¶
All ipmi backend SUT machines in NUE can not be reached:
localhost:~ # ping -c5 openqaipmi5-sp.qe.nue2.suse.org
PING openqaipmi5-sp.qe.nue2.suse.org (10.168.192.202) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable
--- openqaipmi5-sp.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4060ms
pipe 4
localhost:~ # ping -c5 scooter-sp.qe.nue2.suse.org
PING scooter-sp.qe.nue2.suse.org (10.168.192.86) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable
--- scooter-sp.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4054ms
pipe 4
localhost:~ # ping -c5 kermit-sp.qe.nue2.suse.org
PING kermit-sp.qe.nue2.suse.org (10.168.192.88) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable
--- kermit-sp.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4058ms
pipe 4
localhost:~ # ping -c5 ix64ph1075-sp.qe.nue2.suse.org
PING ix64ph1075-sp.qe.nue2.suse.org (10.168.192.204) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable
--- ix64ph1075-sp.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4061ms
pipe 4
localhost:~ # ping -c5 amd-zen3-gpu-sut1-sp.qe.nue2.suse.org
PING amd-zen3-gpu-sut1-sp.qe.nue2.suse.org (10.168.192.83) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable
--- amd-zen3-gpu-sut1-sp.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4078ms
pipe 4
localhost:~ # ping -c5 gonzo-sp.qe.nue2.suse.org
PING gonzo-sp.qe.nue2.suse.org (10.168.192.90) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable
--- gonzo-sp.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4079ms
pipe 4
localhost:~ # ping -c5 unreal2-sp.qe.nue2.suse.org
PING unreal2-sp.qe.nue2.suse.org (10.168.192.160) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable
--- unreal2-sp.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4066ms
pipe 4
localhost:~ # ping -c5 unreal3-sp.qe.nue2.suse.org
PING unreal3-sp.qe.nue2.suse.org (10.168.192.162) 56(84) bytes of data.
From 81.95.8.245 (81.95.8.245) icmp_seq=1 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=2 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=3 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=4 Destination Host Unreachable
From 81.95.8.245 (81.95.8.245) icmp_seq=5 Destination Host Unreachable
--- unreal3-sp.qe.nue2.suse.org ping statistics ---
5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4062ms
pipe 4
Test run also failed, for example, failure 1 and failure 2.
Steps to reproduce¶
- Ping SUT BMC
- Schedule ipmi backend test run
Impact¶
Only two machines in PRG2 are available, no enough resources and not convenient for debugging work or test run.
Problem¶
- Network glitch ???
- Power outage ???
Suggestions¶
- Look into power supply
- Look into network status
Workaround¶
n/a
Actions