QA

Discussed with mgriessmeier. We see that currently existing QE infrastructure can benefit from structured infrastructure management, e.g. automatic upgrades, reboots, ssh key handling with salt. We prefer to have a separate git repository for the QE infrastructure.

Double-checked all machines by git grepping salt-pillars, ssh and sudo salt-key -L respectively:

• ada: Blocked by #115562
• arm4.qe.suse.de is not reachable via SSH
  • check with UV squad what the state of it is
• backup.qam.suse.de #131528
• borg.qam.suse.de ssh: connect to host borg.qam.suse.de port 22: Connection refused
  • file ticket
• conan.qam.suse.de: Blocked by #115562
• enterprise-nx02.qam.suse.de
  • file ticket to add the machine to salt
• fibonacci.qam.suse.de
  • file ticket to add the machine to salt
  • "Shutdown on 20.7.22" comment in racktables!?
• galileo.qam.suse.de
  • machcine needs to be salted
  • running SLES 11 SP4 so probably needs to be updated first
• grenache.qa.suse.de
  • salted
  • grenache is the same machine / the chassis
• ix64ph1075.qa.suse.de
  • salted
• Linux ONE III
  • rack, can't salt this
• openqa-service.qe.suse.de
  • needs salt
• openqaw5-xen.qa.suse.de
  • salted, all good
• powerqaworker-qam-1.qa.suse.de
  • salted, all good
• QA-Power8-4.qa.suse.de
  • salted, all good
• QA-Power8-4.qa.suse.de
  • salted, all good
• qamaster.qa.suse.de
  • salted, all good
• qanet.qa.suse.de
  • old machine that we still need in NUE1, see #81192 . Expected to be obsoleted with #130955
• styx.qam.suse.de
  • not currently salted
  • formerly maintained by dabatianni+apappas
  • check with virt squad and UV squad. We can login using the old QAM root password. It's an ESXi server, not saltable
  • remove Production tag?
  • copy #131528 and make a new ticket
• walter1.qe.nue2.suse.org
  • worker, salted
• walter2.qe.nue2.suse.org
  • DHCP, maintained by eng infra
• whale.qam.suse.de
  • see styx
• worker10/11
  • salted

General notes:

• Can we merge QA/QAM tags or replace both with something sensible?
• Racks should be visible as "production but not saltable" grenache, Linux ONE III

As discussed my suggestions:

1. racktables and netbox are out of sync. Wait for #132293 before trying to come up with a fancy solution regarding tags. However I think we can still add tags in racktables if we come up with a reasonable suggestion
2. For any specific machine needing clarification clone #131528 with the same parent for the specific machine
3. Go over the list of all machines in racktables without the production tag as well to find out if there is any QE/QA machine that should have the production tag

okurz wrote:

1. Go over the list of all machines in racktables without the production tag as well to find out if there is any QE/QA machine that should have the production tag

Looking through machines w/o the Production tag:

• andromeda.openqanet.opensuse.org has no Production tag, presumably owned by QAC, probably Production but not tagged as such - maybe we can assume "Team" means not ours, and it's o3 so we shouldn't care in this context. Just mentioning it here since I wasn't sure at first.
• blackcurrant has the "Team" tag and is ours
  • should it be in production? -> No, it has tag "Testing" and it's PowerPC used actually for testing so it's fine. I added a comment in https://racktables.nue.suse.com/index.php?page=object&tab=default&object_id=11350
  • not responding to SSH -> yes, that's expected because only the LPARs would be reachable directly, e.g. "blackcurrant-1" or however they would be called
• davinci.qam.suse.de
  • dabatianni+apappas
  • should it be in production? -> yes, because the machine is used as DHCP/DNS server in the qam.suse.de domain . I added a comment in https://racktables.nue.suse.com/index.php?page=object&tab=default&object_id=10308 and links to both https://confluence.suse.com/display/maintenanceqa/DNS+Server+NUE and https://gitlab.suse.de/OPS-Service/salt/-/blob/production/pillar/id/davinci_qam_suse_de.sls
  • "Managed by SaltStack"
• frisch.qam.suse.de -> TODO ticket
  • dabatianni+apappas
  • not responding to ping nor SSH
  • should it be in production?
• haldir.qa.suse.de
  • 2023-05-23: According to discussion https://suse.slack.com/archives/C02CANHLANP/p1684777943823339 [^] the machine is currently not used
  • The machine is in the process to be moved to PRG2. Coordinated in #132140
• ix64ph1080
  • #116716
• kadmeia.qe.nue2.suse.org
  • 2023-04-19: Partially connected, can be repurposed
• kynane.qe.nue2.suse.org
  • 2023-04-19: Partially connected, can be repurposed
• loge.qam.suse.de -> qam ref host is not "Production"
  • Q A - M K E R N E L U P D A T E R E F E R E N C E H O S T
  • Shutdown on 20.7.22
  • not responsive to SSH
  • is this machine still there?
• mime.qam.suse.de -> qam ref host is not "Production"
  • Kernel ref host
  • Shutdown on 20.7.22
  • not responsive to SSH
  • is this machine still there?
• nofx.arch.suse.de -> everything in arch can be considered "Testing", not "Production"
  • no owner or purpose specified
  • seems to be online but couldn't successfully connect
• quake.qe.nue2.suse.org -> not to be salted as intended for "workstation replacements" hence not "Production"
  • #130796
• unreal.qe.nue2.suse.org
  • #131552
• sol.qam.suse.de -> to be decommissioned, not "Production". updated https://racktables.nue.suse.com/index.php?page=object&tab=default&object_id=9314
  • no owner or purpose specified
• serial.qam.suse.de -> "Move to Frankencampus" for decommissioning
  • no owner or purpose specified
• thunderx21.qe.nue2.suse.org -> Marked as "Development" in https://racktables.nue.suse.com/index.php?page=object&tab=default&object_id=9570, not "Production"
  • no owner or purpose specified
  • seems to be offline -> to be handled in #132383
• {seth,osiris}.qe.nue2.suse.org -> to be salted, see #132452
  • no owner or purpose specified -> I added a description and wiki link

Next up: #132359 #131552

#132359 and all related actually in-use machines had been put into production or marked accordingly as unused in racktables. This leaves only #151390, blocking on that