Project

General

Profile

Actions

action #125204

closed

Move QA labs NUE-2.2.14-B to Frankencampus labs - non-bare-metal machines size:M

Added by okurz over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Start date:
2022-10-28
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Rent at Maxtorhof will end. The new location at Nbg Frankencampus will have lab rooms in the same building where the office rooms are. We must prepare and execute the move of QA equipment from the old location, mostly NUE-2.2.13 QA cold storage and NUE-2.2.14 (TAM) to an according Frankencampus room

Acceptance criteria

  • AC1: All servers from NUE-2.2.14 (TAM) are operational from the new location
  • AC2: racktables is up-to-date

Suggestions

  • Wait for initial setup in #119548
  • Organize transport of hot equipment
  • Setup hot equipment at new location
  • Ensure racktables is up-to-date

Files


Related issues 4 (0 open4 closed)

Related to QA - action #124221: Repurpose quake.qe.nue2.suse.org (formerly known as cloud4) as employee-workstation replacement size:MResolvedokurz2023-02-09

Actions
Copied from QA - action #119551: Move QA labs NUE-2.2.14-B to Frankencampus labs - bare-metal openQA workers size:MResolvednicksinger2023-03-10

Actions
Copied to QA - action #128390: Move QA labs NUE-2.2.14-B to Frankencampus labs - infrastructure management improvementsResolvedokurz

Actions
Copied to QA - action #128393: Move QA labs NUE-2.2.14-B to Frankencampus labs - recover openQA staging test setup size:MResolvedokurz2022-10-28

Actions
Actions #1

Updated by okurz over 1 year ago

  • Copied from action #119551: Move QA labs NUE-2.2.14-B to Frankencampus labs - bare-metal openQA workers size:M added
Actions #2

Updated by okurz over 1 year ago

  • Due date deleted (2023-03-02)
Actions #3

Updated by okurz over 1 year ago

  • Tags set to infra
Actions #4

Updated by okurz over 1 year ago

  • Project changed from 46 to QA
  • Category deleted (Infrastructure)
Actions #5

Updated by okurz over 1 year ago

  • Tags changed from infra to infra, next-office-day, frankencampus
  • Status changed from New to In Progress
  • Assignee set to okurz
Actions #6

Updated by okurz over 1 year ago

  • Related to action #124221: Repurpose quake.qe.nue2.suse.org (formerly known as cloud4) as employee-workstation replacement size:M added
Actions #7

Updated by openqa_review over 1 year ago

  • Due date set to 2023-03-17

Setting due date based on mean cycle time of SUSE QE Tools

Actions #10

Updated by okurz over 1 year ago

After a bit of waiting since above I could now reach

I powered on osiris+seth over the https BMC interfaces and after some minutes of bootup I could login over root-ssh. corosync reports as failed on both, rest not failed.

I updated https://gitlab.suse.de/qa-sle/qanet-configs/-/merge_requests/54 big-time and IHMO we should merge that first before we continue, e.g. to update a CNAME entry for qsf-cluster and such. Anyway, I already found one issue that /etc/libvirt/storage/dist.suse.de.xml included an old IPv4 entry for dist.suse.de. I changed that manually to use the FQDN dist.suse.de and then systemctl start libvirtd was starting up fine. I could connect using virt-manager to osiris-1.qe.nue2.suse.org but this showed only "first-test-vm". I think I saw this already some time ago when the cluster thingy was not working.

With grep -R 10.162.2 /etc I could find

/etc/drbd.d/qsf-cluster.res:        address   10.162.2.95:7789;
/etc/drbd.d/qsf-cluster.res:        address   10.162.2.96:7789;
/etc/drbd.d/qsf-cluster.res.salttest:        address   10.162.2.95:7789;
/etc/drbd.d/qsf-cluster.res.salttest:        address   10.162.2.96:7789;
/etc/corosync/corosync.conf:                ring0_addr: 10.162.2.95
/etc/corosync/corosync.conf:                ring0_addr: 10.162.2.96

so I updated that using the FQDNs and restarted corosync. But libvirtd still shows only first-test-vm. Maybe something in #78206, #35506, #117697 helps

Actions #11

Updated by okurz over 1 year ago

  • Tags changed from infra, frankencampus to infra, frankencampus, next-office-day
  • Due date deleted (2023-03-17)

I did fibre connections to both openqaworker-arm-4+5, BMC still pending.

  • TODO next-office-day connect BMC for both openqaworker-arm-4+5

EDIT: 2023-04-19: BMC connections for openqaworker-arm-4+5 done

Actions #13

Updated by okurz over 1 year ago

  • Tags changed from infra, frankencampus, next-office-day to infra, frankencampus
Actions #15

Updated by okurz over 1 year ago

  • Copied to action #128390: Move QA labs NUE-2.2.14-B to Frankencampus labs - infrastructure management improvements added
Actions #16

Updated by okurz over 1 year ago

  • Copied to action #128393: Move QA labs NUE-2.2.14-B to Frankencampus labs - recover openQA staging test setup size:M added
Actions #17

Updated by okurz over 1 year ago

  • Status changed from Blocked to In Progress

https://sd.suse.com/servicedesk/customer/portal/1/SD-113959 resolved, we have root access to the DHCP servers walter1/2 in FC Basement LSG QE now.

#124643 for later.

Still blocked on

moved to #128390

thincsus to be handled in #128393.

I think all other equipment is usable from the new location and racktables is having the information. We should walk over NUE-FC-B LSG QE to consolidate, e.g. use FQDNs for all entries.

Actions #18

Updated by okurz over 1 year ago

  • Subject changed from Move QA labs NUE-2.2.14-B to Frankencampus labs - non-bare-metal machines to Move QA labs NUE-2.2.14-B to Frankencampus labs - non-bare-metal machines size:M
Actions #19

Updated by okurz over 1 year ago

  • Status changed from In Progress to Feedback

With mkittler and nsinger we walked over complete NUE-FC-B LSG QE and used proper pseudo-FQDNs for all entries. "pseudo" because in some cases machines are not connected so they are not reachable by the FQDN as such.
https://gitlab.suse.de/qa-sle/qanet-configs/-/merge_requests/61 (merged) "Cleanup more entries of machines moved to FC Basement".
I also kept old CNAME entries as aliases in FQDNs after the A record FQDN entry.

https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/529 to use new qe.nue2.suse.org FQDNs where applicable.

Then crosschecked properties and port entries in racktables again.

For IPv4 entries I reviewed https://racktables.nue.suse.com/index.php?id=1386&page=ipv4net&tab=default . I went over every entry corresponding to IPv4 entries in https://gitlab.suse.de/OPS-Service/salt/-/blob/production/pillar/domain/qe_nue2_suse_org/hosts.yaml by replacing the IPv4 address in the URL like
https://racktables.nue.suse.com/index.php?page=ipaddress&tab=assignment&ip=10.168.192.77
and add according entries for each address-machine-pair.

EDIT: Waiting for https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/529

Actions #20

Updated by okurz over 1 year ago

Actions #21

Updated by okurz over 1 year ago

  • Status changed from Feedback to Resolved

https://openqa.suse.de/tests/11025832 shows use of the new FQDNs. In the meantime no other complaints received.

Actions

Also available in: Atom PDF