Project

General

Profile

Actions
Copy link

action #117580

closed

web interface of qanet02nue.qa.suse.de and qanet03nue.qa.suse.de can not be reached over neither IPv4 nor IPv6 size:M

Added by okurz about 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
nicksinger
Category:
-
Target version:
openQA Project (public) - Ready
Start date:
2022-10-05
Due date:
% Done:

0%

Estimated time:
Tags:
reactive work

Description

qanet02nue.qa.suse.de and qanet03nue.qa.suse.de can not be reached over neither IPv4 nor IPv6

Observation

$ ping qanet03nue.qa.suse.de
PING qanet03nue.qa.suse.de(2620:113:80c0:80a0:10:162:0:63 (2620:113:80c0:80a0:10:162:0:63)) 56 data bytes
From 2620:113:80c0:8120::5 (2620:113:80c0:8120::5) icmp_seq=1 Destination unreachable: Address unreachable
From 2620:113:80c0:8120::5 (2620:113:80c0:8120::5) icmp_seq=2 Destination unreachable: Address unreachable
^C
--- qanet03nue.qa.suse.de ping statistics ---
2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1001ms

okurz@linux-28d7:~ 1 (master) $ ping -4 qanet03nue.qa.suse.de
PING  (10.162.0.63) 56(84) bytes of data.
64 bytes from qanet03nue.qa.suse.de (10.162.0.63): icmp_seq=1 ttl=62 time=2625 ms
64 bytes from qanet03nue.qa.suse.de (10.162.0.63): icmp_seq=2 ttl=62 time=1613 ms

ssh access works, http not.

So two problems:

  1. IPv6 DNS resolution broken
  2. web interface not accessible

Acceptance criteria

Suggestions

  • Check for firmware upgrades for these switches
  • Optional: Apply the same solution to other QA switches to prevent further problems there
  • Check dhcpv6 setup on qanet to understand which v6 address these switches should have
  • If IPv6 is not feasible to use at all remove AAAA records

Workaround

If the web interface is not available a manual power cycle can help

Actions
Copy link
 #1

Updated by okurz about 2 years ago

  • Tags set to next-office-day
  • Subject changed from web interface of qanet03nue.qa.suse.de can not be reached over neither IPv4 nor IPv6 to web interface of qanet02nue.qa.suse.de and qanet03nue.qa.suse.de can not be reached over neither IPv4 nor IPv6
  • Status changed from New to In Progress
  • Assignee set to okurz

Same problem seems to appear on qanet02nue. Also tried "reload" but it seems the switch is stuck in "shutting down" for long. nicksinger stated that according to his research a process might block the shutdown.

Actions
Copy link
 #2

Updated by okurz about 2 years ago

After disconnecting and reconnecting the power cord both switches came back and their web configuration interface as well so this part works.

Actions
Copy link
 #3

Updated by openqa_review about 2 years ago

  • Due date set to 2022-10-21

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #4

Updated by okurz about 2 years ago

regarding IPv6 the configuration on https://qanet03nue.qa.suse.de/ looks like IPv6 is enabled but not having the right address. From the qanet config I see that the address should be 2620:0113:80c0:80a0:10:162:0:63. We likely need to add that in config. We did with prefix 64, same as the address of qanet, but we couldn't ping it back or forth.

Actions
Copy link
 #5

Updated by okurz about 2 years ago

  • Due date deleted (2022-10-21)
  • Status changed from In Progress to New
  • Assignee deleted (okurz)
Actions
Copy link
 #6

Updated by livdywan about 2 years ago

  • Subject changed from web interface of qanet02nue.qa.suse.de and qanet03nue.qa.suse.de can not be reached over neither IPv4 nor IPv6 to web interface of qanet02nue.qa.suse.de and qanet03nue.qa.suse.de can not be reached over neither IPv4 nor IPv6 size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions
Copy link
 #7

Updated by okurz about 2 years ago

I just found a new firmware image on https://software.cisco.com/download/home/283019669/type/282463181/release/1.4.11.5 from 18-Jun-2020. Maybe we can try out the firmware upgrade first on switches which are currently in "QA cold storage".

Actions
Copy link
 #9

Updated by okurz about 2 years ago

  • Tags changed from next-office-day to reactive work
  • Parent task set to #109743
Actions
Copy link
 #10

Updated by nicksinger about 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to nicksinger

I will give it a try with qanet03 first

Actions
Copy link
 #11

Updated by nicksinger about 2 years ago

update of qanet03nue worked flawlessly. Will continue with qanet02nue after my (late) lunch break

Actions
Copy link
 #12

Updated by openqa_review about 2 years ago

  • Due date set to 2022-11-01

Setting due date based on mean cycle time of SUSE QE Tools

Actions
Copy link
 #13

Updated by nicksinger about 2 years ago

Firmware update was successful on both switches and they are now running on the most recent 1.4.11.5 version.

Actions
Copy link
 #14

Updated by nicksinger about 2 years ago

  • Status changed from In Progress to Resolved

I changed the "IPv6 Interfaces" settings for the "Management Interface" to be available in VLAN12 (like with v4) instead of VLAN1 (our current configuration). Afterwards added the according static entries from our dhcpd6.conf - both switches are now reachable over v4 and v6 and with the update in place the webui hopefully doesn't crash any longer.

Actions
Copy link
 #15

Updated by okurz about 2 years ago

Awesome, thanks :)

Actions
Copy link
 #16

Updated by okurz almost 2 years ago

  • Due date deleted (2022-11-01)
Actions
Copy link

Also available in: Atom PDF