action #133490: Migration of o3 VM to PRG2 - Fix o3 bare metal hosts iPXE booting size:M - openQA Infrastructure - openSUSE Project Management Tool

Custom queries

openQA Infrastructure Project
openqa-review - Closed tickets last updated by openqa-review, last 30 days
QA roadmap long-term
QA SLE functional
QA SLE Functional - closed in last 14 days
QA SLE Functional - High, need to be refined
QA SLE Functional - over cycle time median
QA SLE u
QA SLE y
QA tools (tag not necessary in openQA and subprojects)
QA tools tag (tag not necessary in openQA and subprojects; excluding tickets in "Ready" version as they are already on the backlog)
QAC - Backlog
QE tools team - backlog (ready issues)
QE tools team - backlog (w/o infra)
QE tools team - backlog SLA high
QE tools team - backlog SLA immediate
QE tools team - backlog SLA no immediate/urgent in feedback/blocked
QE tools team - backlog SLA normal
QE tools team - backlog SLA urgent
QE tools team - backlog SLO high
QE tools team - backlog SLO normal
QE tools team - backlog SLO urgent
QE tools team - backlog, high-level view (epics and higher)
QE tools team - backlog, non-reactive work, needs parent
QE tools team - backlog, top-level view (all sagas)
QE tools team - closed within last 14 days
QE tools team - closed within last 60 days
QE tools team - closed yesterday
QE Tools Team - Collaborative Session
QE tools team - due date forecast
QE tools team - exceeding due-date
QE tools team - infrastructure backlog
QE tools team - next - sorted by update time
QE tools team - next issues
QE tools team - non-estimated (unblocked) issues (w/o infra)
QE tools team - ready issues - Workable
QE tools team - ready, not assigned/blocked/low
QE tools team - update forecast
QE tools team - updated by priority
QE tools team - what members of the team are working on - Feedback (not-low)
QE Tools Team Backlog By Assignee
Tools Team Retrospective
Tools Team Retrospective (not estimated or assigned)

Actions

Copy link

action #133490

closed

QA - coordination #121720: [saga][epic] Migration to QE setup in PRG2+NUE3 while ensuring availability

QA - coordination #123800: [epic] Provide SUSE QE Tools services running in PRG2 aka. Prg CoLo

Migration of o3 VM to PRG2 - Fix o3 bare metal hosts iPXE booting size:M

Added by okurz 10 months ago. Updated 8 months ago.

Status:

Resolved

Priority:

Urgent

Assignee:

dheidler

Category:

Target version:

openQA Project - Ready

Start date:

Due date:

% Done:

Estimated time:

Tags:

ipmi, o3, infra, nue1, prg2, bare metal

Description

Motivation¶

The openQA webUI VM for o3 has moved to PRG2. For now we don't have bare metal tests working, see https://openqa.opensuse.org/tests/3446336#step/ipxe_install/2 and it will take some time until hardware is available in PRG2. Let's fix it with o3 VM in PRG2 and hardware in NUE1 for the time being

Expected result¶

Last good: https://openqa.opensuse.org/tests/3411759

Acceptance criteria¶

AC1: bare metal tests can work on o3 after move as long as hardware is in NUE1

Suggestions¶

DONE Wait for the move
DONE After move connect NUE1 based workers over https to PRG2 based o3, see #132134 for details
Fix o3 bare metal hosts iPXE booting, see https://openqa.opensuse.org/tests/3446336#step/ipxe_install/2, i.e.
- try to either point to new-ariel as PXE server in /etc/dnsmasq.d/openqa.conf from old-ariel
- As alternative keep the /srv/tftpboot in sync with new-ariel, e.g. mount that directory remotely over the SSH tunnel

Related issues 1 (1 open — 0 closed)

Copied from openQA Infrastructure - action #132647: Migration of o3 VM to PRG2 - bare-metal tests size:M

Workable

okurz

Actions

Issue # Delay: days Cancel

History
Notes
Property changes

Actions

Copy link

Updated by okurz 10 months ago

Copied from action #132647: Migration of o3 VM to PRG2 - bare-metal tests size:M added

Actions

Copy link

Updated by okurz 10 months ago

Status changed from New to In Progress
Assignee set to nicksinger

Actions

Copy link

Updated by okurz 10 months ago

Subject changed from Migration of o3 VM to PRG2 - Fix o3 bare metal hosts iPXE booting to Migration of o3 VM to PRG2 - Fix o3 bare metal hosts iPXE booting size:M
Description updated (diff)

Actions

Copy link

Updated by openqa_review 10 months ago

Due date set to 2023-08-12

Setting due date based on mean cycle time of SUSE QE Tools

Actions

Copy link

Updated by nicksinger 10 months ago

Assignee deleted (~~nicksinger~~)

We tried to apply some forwarding from the old network to new ariel but for some reason tftp doesn't like this. It needs further debugging if e.g. some special ports are required to make it work (we assumed no because it should be "trivial"). I haven't worked on this and put it back into our queue if somebody else wants to give it a shot

Actions

Copy link

Updated by nicksinger 10 months ago

Status changed from In Progress to Workable

Actions

Copy link

Updated by nicksinger 10 months ago

Assignee set to nicksinger

Actions

Copy link

Updated by okurz 9 months ago

Project changed from QA to openQA Infrastructure

Actions

Copy link

#10

Updated by livdywan 9 months ago

Due date deleted (~~2023-08-12~~)
Assignee deleted (~~nicksinger~~)
Start date deleted (~~2023-07-28~~)

I haven't worked on this and put it back into our queue if somebody else wants to give it a shot

Unassigning for clarity and because someone really should pick this up if possible.

Actions

Copy link

#11

Updated by dheidler 9 months ago

Status changed from Workable to In Progress
Assignee set to dheidler

The issue appears because openqa.opensuse.org used to be not terminating SSL.
Now we get a redirect to ssl which iPXE is not compiled with.

Actions

Copy link

#12

Updated by dheidler 9 months ago

Reconfigured o3 to serve this asset paths (/assets and /images) directly without redirect even on unencrypted http.
This is needed because iPXE HTTPS support is not very good.

2do:

Update ipxe menu on old-ariel to load config from new-ariel.
Update ipxe module used in referenced o3 test to work with updated ipxe menu.

Actions

Copy link

#13

Updated by vojha 9 months ago

Hello Team,
For https://openqa.opensuse.org/tests/overview?distri=opensuse&version=Tumbleweed&build=20230809&groupid=38
virt-guest-installation-kvm@64bit-ipmi

virt-guest-installation-xen@64bit-ipmi
above mentioned tests are failing at installation.

I've identified the root cause of the test failures and the associated error. The issue seems to stem from an outdated file at the following location: http://openqa.opensuse.org/assets/other/autoyast_opensuse_kvm_sshd.xml. This file wasn't updated during the O3 migration triggered initially and I was working on this autoyast file at that time, even though you may have made updates to the "ipxe menu on old-ariel to load config from new-ariel."

However, please note that the file at this location on old Ariel is not being updated. O3: http://openqa.opensuse.org/assets/other/autoyast_opensuse_kvm_sshd.xml.
To access and update the file on old Ariel, you can use the following path: vojha@old-ariel:> sudo vi /var/lib/openqa/share/factory/other/autoyast_opensuse_kvm_sshd.xml

It's important that both locations have the same file content to ensure consistency. This will allow us to make necessary changes to the autoyast profile in the future, in line with our requirements.

Thank you for your attention to this matter. I hope this helps

For the time being I would request to please overwrite the file completely from https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/virt_autotest/host_unattended_installation_files/autoyast_opensuse_kvm_sshd.xml

OR FROM old-ariel:> sudo vi /var/lib/openqa/share/factory/other/autoyast_opensuse_kvm_sshd.xml

and copy to http://openqa.opensuse.org/assets/other/autoyast_opensuse_kvm_sshd.xml

Actions

Copy link

#14