Project

General

Profile

Actions

coordination #116623

closed

coordination #121720: [saga][epic] Migration to QE setup in PRG2+NUE3 while ensuring availability

[epic] Migration of SUSE Nbg based openQA+QA+QAM systems to new security zones

Added by okurz over 1 year ago. Updated 4 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Start date:
2022-09-14
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Tags:

Description

Motivation

The SUSE Cybersecurity team plans to provide better network segmentation to improve security. Proposals exist e.g. on https://confluence.suse.com/pages/viewpage.action?pageId=952173193 and https://confluence.suse.com/pages/viewpage.action?pageId=1006108843 .
mgriessmeier and nsinger and okurz had a meeting with Lazaros Haleplidis on 2022-09-15. We should now plan which systems and services need which security rules, e.g. which ports accessible, etc.

Acceptance criteria

  • AC1: All Nbg based QA/QAM machines are within the new security zones (including OSD machines, excluding O3 machines)
  • AC2: All QA provided services continue to be operational

Suggestions

  • Read existing materials and proposals, e.g. above mentioned confluence pages
  • okurz suggests to make sure racktables Nuremberg&QA&QAM is the complete list for all the machines we need to care about
  • Come up with a proposal for what network security zones we need and what security rules should apply for thos
  • Provide a list of all machines with FQDN, MAC, VLAN, IPv4, IPv6 for machines as well as BMCs as required by Lazaros Haleplidis, at best readable directly from Racktables

Out of scope

Currently the dedicated openqa.opensuse.org network is not covered by this change. According to Lazaros Haleplidis no public facing machines which is including https://openqa.opensuse.org are touched by this.

Further details

  1. What are your requirements that need to be fulfilled?
    All inbound traffic needs to be well defined.

  2. Do we have any benefits from this change?
    Better separation within SUSE networks

  3. How can the security rules be controlled?
    Creating a ticket. Automation, e.g. using terraform, etc., is evaluated

  4. Do we need two networks, one for openQA and QA?
    Right now we use machines within the Eng-Infra network. We can specify requirements

  5. We need HTTP communication to various hosts within the .suse.de domain. download.suse.de, gitlab.suse.de, etc.
    All of these need to be specifically specified

BMCs are planned to be accessible over jump hosts. It is planned to migrate IP access to machines first and keep IPMI till the end. Jump hosts is planned to be a Linux VM accessible over SSH from where we can access BMCs of the systems.

It is possible to have dedicated "test networks" so equivalent to our QA network where we have machines+BMCs within the same network. It might not be the suggested setup but is possible.

We meet again on 2022-09-22, 1500 CEST. Lazaros Haleplidis will invite us for 2022-09-22.


Subtasks 33 (0 open33 closed)

action #116626: Migration of SUSE QA systems to new security zones - QAM systemsResolvedokurz2022-09-15

Actions
action #116629: Preparation planning for migration of SUSE openQA+QA systems to new security zones size:MResolvedokurz2022-09-15

Actions
openQA Infrastructure - action #116689: Do not rely on statically configured IPv4 addresses for the salt master in /etc/hosts size:SResolvedokurz2022-09-14

Actions
action #117043: Request DHCP+DNS services for new QE network zones, same as already provided for .qam.suse.de and .qa.suse.czResolvedokurz

Actions
action #119443: Conduct the migration of SUSE openQA systems from Nbg SRV1 to new security zones size:MResolvedokurz2022-11-17

Actions
action #119446: Conduct the migration of SUSE openQA+QA systems from Nbg SRV2 to new security zonesResolvedokurz2022-09-15

Actions
action #119449: Conduct the migration of SUSE openQA+QA systems from Nbg QA labs to new security zonesResolvedokurz2022-09-15

Actions
action #119638: Ensure every physical machine within .qam.suse.de has an IPMI+eth L2 address entry in racktables size:MResolvedokurz

Actions
openQA Infrastructure - action #120025: [openQA][ipmi][worker] Worker host hostname changed and broken networking connectionResolvedokurz2022-11-07

Actions
openQA Infrastructure - action #120163: Use salt grains instead of manually specifying IPs in "bridge_ip" size:MResolvedmkittler

Actions
action #120264: Conduct the migration of SUSE QA systems (non-tools-team maintained) from Nbg SRV1 to new security zones size:MResolvedokurz2022-09-15

Actions
action #120267: Conduct the migration of openqa-ses aka. "storage.qa.suse.de" size:MResolvedmkittler2022-09-15

Actions
openQA Infrastructure - action #120270: Conduct the migration of SUSE openQA systems IPMI from Nbg SRV1 to new security zones size:MResolvedmkittler

Actions
openQA Tests - action #120288: [tools] cloud based tests fail due to traffic to cloud blocked auto_review:"2022-11-0.*Test died: (Waiting for Godot.*ssh|Cannot find image after upload)":retryResolvedokurz2022-11-10

Actions
openQA Project - action #120333: [os-autoinst][ipmi] Add support for ssh jump host in IPMI backendRejectedokurz2022-11-11

Actions
openQA Infrastructure - action #120339: QEMU DNS fails to resolve openqa.suse.de via IP addressResolvedokurz2022-11-11

Actions
openQA Infrastructure - action #120441: OSD parallel jobs failed with "get_job_autoinst_url: No worker info for job xxx available" size:meowResolvedokurz2022-11-15

Actions
openQA Tests - action #120789: [virtualization] tests fail to upload to qadb on dbproxy.suse.de with "Access denied, this account is locked"Resolved

Actions
openQA Infrastructure - action #120807: [alert] openqa.suse.de - worker12.oqa.suse.de 100% packet loss due to outdated AAAA recordResolvedokurz2022-11-17

Actions
openQA Project - coordination #122650: [epic] Fix firewall block and improve error reporting when test fails in curl log uploadResolvedokurz2022-12-29

Actions
openQA Tests - action #122539: test fails in curl log from openqa and connect with FQDN worker2.oqa.suse.de always fails by time out size:MClosed2022-12-29

Actions
openQA Project - action #122608: exit code of shell command not received by script_runResolvedokurz2023-01-02

Actions
openQA Infrastructure - action #122653: Ask SUSE-IT network admins to REJECT packets instead of DROP so that we get more clear results size:SRejectedokurz2023-01-03

Actions
openQA Infrastructure - action #122656: Ask SUSE-IT network admins to *not* block this traffic which we need for tests regarding s390x within SUSE network size:MResolvedokurz2023-01-03

Actions
openQA Project - action #122659: Improved error reporting in openQA tests when curl times out on connection attemptsRejectedokurz2023-01-03

Actions
action #123697: Conduct the migration of SUSE QA systems s390x zVM instances to new security zones size:MResolvedokurz2022-09-15

Actions
openQA Infrastructure - action #124119: Conduct the migration of remaining SUSE openQA systems IPMI to new security zonesResolvedokurz2023-02-08

Actions
openQA Infrastructure - action #124715: Failing pipelines because of unreachable machine openqaworker-arm-1Rejected2023-02-08

Actions
coordination #124721: [epic] Ensure proper QE maintainership of Nbg QAM machinesResolvedokurz2023-02-17

Actions
action #124724: Ensure Nbg QAM machines have a current maintainer as "contact person" size:SResolvedokurz2023-02-17

Actions
action #125144: Give members of SUSE QE Tools team a chance to get familiar with Nbg QAM machines size:MResolvedokurz2023-02-17

Actions
action #125234: Decommission obsolete machines in qam.suse.de size:MResolvedokurz2023-03-01

Actions
openQA Infrastructure - action #124877: Failing pipelines because of unreachable machine openqaworker-arm-1Resolvedmkittler2023-02-08

Actions

Related issues 1 (0 open1 closed)

Related to QA - action #125450: Improve collaboration with Eng-Infra - Firewall management access, potentially also DHCP+DNS size:MResolvedokurz2023-03-06

Actions
Actions

Also available in: Atom PDF