Project

General

Profile

Actions

action #119443

closed

coordination #121720: [saga][epic] Migration to QE setup in PRG2+NUE3 while ensuring availability

coordination #116623: [epic] Migration of SUSE Nbg based openQA+QA+QAM systems to new security zones

Conduct the migration of SUSE openQA systems from Nbg SRV1 to new security zones size:M

Added by okurz almost 2 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
Start date:
2022-11-17
Due date:
% Done:

0%

Estimated time:
Tags:

Description

Motivation

See parent #116623

Acceptance criteria

  • AC1: All openQA machines in Nbg SRV1 are in new security zones
  • AC2: All openQA machines in Nbg SRV1 are fully usable in production

Suggestions

Open points

  • DONE Failed to connect to gitlab.suse.de port 443 from both worker11.oqa.suse.de and worker12.oqa.suse.de
  • DONE https://openqa.suse.de/tests/9870589#step/suseconnect_scc/23 failed trying to access scc.suse.com . I thought there would be no restrictions contacting services outside the network zones. What are the actual rules applied? --> Specific rules are managed within the firewall by lhaleplidis and will be documented later on wiki but unfortunately can not currently be dynamically visible to users
  • DONE https://openqa.suse.de/tests/9870976#step/sys_param_check/19 fails to curl -f -v "qa-css-hq.qa.suse.de/robot.tar.gz" --> see https://progress.opensuse.org/issues/119443?issue_count=97&issue_position=19&next_issue_id=118660&prev_issue_id=81192#note-17
  • DONE I try to access VNC services on the hosts. That seems to be blocked as well.
  • DONE Where can we see which services are blocked ourselves? --> Specific rules are managed within the firewall by lhaleplidis and will be documented later on wiki but unfortunately can not currently be dynamically visible to users. Not really done though. Extracted into a new ticket #120145
  • DONE hosts within the new domain .oqa.suse.de. should search for matches within that domain so that nslookup $(hostname) works, e.g. nslookup worker13 should work. I assume that salt is relying on that to return a proper match for grains.fqdn
  • DONE worker13 back in production
  • DONE worker10 back in production
  • DONE worker3 back in production
  • DONE worker5 back in production
  • DONE worker6 back in production
  • DONE worker8 back in production
  • DONE worker9 back in production
  • DONE Unpause "Packet loss between worker hosts and other hosts alert"
  • DONE worker2 back in production
  • Unpause "job age (scheduled) (max)" and "job age (scheduled) (median)"

Out-of-scope

  • This is not including o3 (openqa.opensuse.org) machines as they are in a dedicated network already
  • Not including non-openQA systems, see #120264 about that

Related issues 13 (1 open12 closed)

Related to openQA Infrastructure - action #109241: Prefer to use domain names rather than IPv4 in salt pillars size:MResolvedokurz

Actions
Related to openQA Infrastructure - action #120025: [openQA][ipmi][worker] Worker host hostname changed and broken networking connectionResolvedokurz2022-11-07

Actions
Related to openQA Infrastructure - action #120112: worker worker2.oqa.suse.de auto_review:"Error connecting to <root@win2k19.qa.suse.cz>: Connection timed out":retry size:MResolvedokurz2022-11-08

Actions
Related to openQA Infrastructure - action #120261: tests should try to access worker by WORKER_HOSTNAME FQDN but sometimes get 'worker2' or something auto_review:".*curl.*worker\d+:.*failed at.*":retry size:meowResolvedmkittler2022-11-10

Actions
Related to openQA Infrastructure - action #113701: [qe-core] Move workers back to grenacheNew

Actions
Related to openQA Infrastructure - action #120339: QEMU DNS fails to resolve openqa.suse.de via IP addressResolvedokurz2022-11-11

Actions
Copied from QA - action #116629: Preparation planning for migration of SUSE openQA+QA systems to new security zones size:MResolvedokurz2022-09-15

Actions
Copied to QA - action #119446: Conduct the migration of SUSE openQA+QA systems from Nbg SRV2 to new security zonesResolvedokurz2022-09-15

Actions
Copied to QA - action #119638: Ensure every physical machine within .qam.suse.de has an IPMI+eth L2 address entry in racktables size:MResolvedokurz

Actions
Copied to openQA Infrastructure - action #120163: Use salt grains instead of manually specifying IPs in "bridge_ip" size:MResolvedmkittler

Actions
Copied to QA - action #120264: Conduct the migration of SUSE QA systems (non-tools-team maintained) from Nbg SRV1 to new security zones size:MResolvedokurz2022-09-15

Actions
Copied to openQA Infrastructure - action #120270: Conduct the migration of SUSE openQA systems IPMI from Nbg SRV1 to new security zones size:MResolvedmkittler

Actions
Copied to openQA Infrastructure - action #120807: [alert] openqa.suse.de - worker12.oqa.suse.de 100% packet loss due to outdated AAAA recordResolvedokurz2022-11-17

Actions
Actions

Also available in: Atom PDF