Project

General

Profile

Actions

action #133490

closed

QA - coordination #121720: [saga][epic] Migration to QE setup in PRG2+NUE3 while ensuring availability

QA - coordination #123800: [epic] Provide SUSE QE Tools services running in PRG2 aka. Prg CoLo

Migration of o3 VM to PRG2 - Fix o3 bare metal hosts iPXE booting size:M

Added by okurz 11 months ago. Updated 9 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Motivation

The openQA webUI VM for o3 has moved to PRG2. For now we don't have bare metal tests working, see https://openqa.opensuse.org/tests/3446336#step/ipxe_install/2 and it will take some time until hardware is available in PRG2. Let's fix it with o3 VM in PRG2 and hardware in NUE1 for the time being

Expected result

Acceptance criteria

  • AC1: bare metal tests can work on o3 after move as long as hardware is in NUE1

Suggestions

  • DONE Wait for the move
  • DONE After move connect NUE1 based workers over https to PRG2 based o3, see #132134 for details
  • Fix o3 bare metal hosts iPXE booting, see https://openqa.opensuse.org/tests/3446336#step/ipxe_install/2, i.e.
    • try to either point to new-ariel as PXE server in /etc/dnsmasq.d/openqa.conf from old-ariel
    • As alternative keep the /srv/tftpboot in sync with new-ariel, e.g. mount that directory remotely over the SSH tunnel

Related issues 1 (0 open1 closed)

Copied from openQA Infrastructure - action #132647: Migration of o3 VM to PRG2 - bare-metal tests size:MResolvedokurz

Actions
Actions #1

Updated by okurz 11 months ago

  • Copied from action #132647: Migration of o3 VM to PRG2 - bare-metal tests size:M added
Actions #3

Updated by okurz 11 months ago

  • Status changed from New to In Progress
  • Assignee set to nicksinger
Actions #4

Updated by okurz 11 months ago

  • Subject changed from Migration of o3 VM to PRG2 - Fix o3 bare metal hosts iPXE booting to Migration of o3 VM to PRG2 - Fix o3 bare metal hosts iPXE booting size:M
  • Description updated (diff)
Actions #5

Updated by openqa_review 11 months ago

  • Due date set to 2023-08-12

Setting due date based on mean cycle time of SUSE QE Tools

Actions #6

Updated by nicksinger 11 months ago

  • Assignee deleted (nicksinger)

We tried to apply some forwarding from the old network to new ariel but for some reason tftp doesn't like this. It needs further debugging if e.g. some special ports are required to make it work (we assumed no because it should be "trivial"). I haven't worked on this and put it back into our queue if somebody else wants to give it a shot

Actions #7

Updated by nicksinger 11 months ago

  • Status changed from In Progress to Workable
Actions #8

Updated by nicksinger 11 months ago

  • Assignee set to nicksinger
Actions #9

Updated by okurz 11 months ago

  • Project changed from QA to openQA Infrastructure
Actions #10

Updated by livdywan 11 months ago

  • Due date deleted (2023-08-12)
  • Assignee deleted (nicksinger)
  • Start date deleted (2023-07-28)

I haven't worked on this and put it back into our queue if somebody else wants to give it a shot

Unassigning for clarity and because someone really should pick this up if possible.

Actions #11

Updated by dheidler 11 months ago

  • Status changed from Workable to In Progress
  • Assignee set to dheidler

The issue appears because openqa.opensuse.org used to be not terminating SSL.
Now we get a redirect to ssl which iPXE is not compiled with.

Actions #12

Updated by dheidler 11 months ago

Reconfigured o3 to serve this asset paths (/assets and /images) directly without redirect even on unencrypted http.
This is needed because iPXE HTTPS support is not very good.

2do:

  • Update ipxe menu on old-ariel to load config from new-ariel.
  • Update ipxe module used in referenced o3 test to work with updated ipxe menu.
Actions #13

Updated by vojha 11 months ago

Hello Team,
For https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=20230809&groupid=38
virt-guest-installation-kvm@64bit-ipmi

virt-guest-installation-xen@64bit-ipmi
above mentioned tests are failing at installation.

I've identified the root cause of the test failures and the associated error. The issue seems to stem from an outdated file at the following location: http://openqa.opensuse.org/assets/other/autoyast_opensuse_kvm_sshd.xml. This file wasn't updated during the O3 migration triggered initially and I was working on this autoyast file at that time, even though you may have made updates to the "ipxe menu on old-ariel to load config from new-ariel."

However, please note that the file at this location on old Ariel is not being updated. O3: http://openqa.opensuse.org/assets/other/autoyast_opensuse_kvm_sshd.xml.
To access and update the file on old Ariel, you can use the following path: vojha@old-ariel:> sudo vi /var/lib/openqa/share/factory/other/autoyast_opensuse_kvm_sshd.xml

It's important that both locations have the same file content to ensure consistency. This will allow us to make necessary changes to the autoyast profile in the future, in line with our requirements.

Thank you for your attention to this matter. I hope this helps

For the time being I would request to please overwrite the file completely from https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/virt_autotest/host_unattended_installation_files/autoyast_opensuse_kvm_sshd.xml

OR FROM old-ariel:> sudo vi /var/lib/openqa/share/factory/other/autoyast_opensuse_kvm_sshd.xml

and copy to http://openqa.opensuse.org/assets/other/autoyast_opensuse_kvm_sshd.xml

Actions #14

Updated by openqa_review 11 months ago

  • Due date set to 2023-08-25

Setting due date based on mean cycle time of SUSE QE Tools

Actions #16

Updated by dheidler 11 months ago

  • Status changed from In Progress to Feedback

Changes applied to old-ariel and menu on new-ariel.
Opened PR for openqa test and ipxe-scripts.

Actions #17

Updated by dheidler 10 months ago

  • Status changed from Feedback to Resolved
Actions #18

Updated by okurz 9 months ago

  • Due date deleted (2023-08-25)
Actions

Also available in: Atom PDF