Project

General

Profile

action #93381

[O3]request to add an IPMI SUT to O3

Added by Julie_CAO 5 months ago. Updated 3 days ago.

Status:
Feedback
Priority:
Normal
Assignee:
Target version:
Start date:
2021-06-02
Due date:
2021-10-30
% Done:

0%

Estimated time:

Description

Hi,

Virtualization team requested infra to move a IPMI SUT to vlan 663 from OSD for temporary use in O3. Now the infra is ready, refer to https://infra.nue.suse.com/SelfService/Display.html?id=189224.

the server has been added to racktables and was located in SRV2 not the labs.
https://racktables.suse.de/index.php?page=object&tab=default&object_id=14353
Both LAN port and BMC are untagged to Vlan662: openqa-worker.
662 f4:02:70:b9:3a:66 gi6 dynamic

Please check the mac @ on the DHCP server to get an IP@.

Now what should I do is to add the MAC of the machine to O3 dhcp server to get an IP? Is the O3 host the dhcp server? ie. request to add my ssh pub key to O3 and add the mac to /etc/dnsmasq.d/openqa.conf and /etc/hosts?(I got the idea from https://progress.opensuse.org/projects/openqav3/wiki/#Moving-worker-from-osd-to-o3) then access the IPMI machine over ssh by using the O3 server as the jump host?

Could someone from openqa infra team help? or guide me what to do. thanks!


Related issues

Blocked by openQA Infrastructure - action #97658: many (maybe all) jobs on rebel within o3 run into timeout_exceeded "setup exceeded MAX_SETUP_TIME" size:MIn Progress2021-08-302021-11-05

History

#1 Updated by Julie_CAO 5 months ago

  • Subject changed from [O3]add a IPMI SUT to O3 to [O3]request to add an IPMI SUT to O3

#2 Updated by VANASTASIADIS 5 months ago

  • Target version set to Ready

#3 Updated by nicksinger 5 months ago

  • Status changed from New to In Progress
  • Assignee set to nicksinger

I checked ariel (the o3 host) and apparently it is the dhcp server and I see requests from this machine. I will take care of adding it.

#4 Updated by Julie_CAO 5 months ago

Thank you nick for taking care. I need two fixed IPs for both IPMI BMC and machine NIC.

my ssh key:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDI/mD7o5XnT6X6bVN+sZgFBkfAdA7/wisXOnGYSdhzIECK6OCxf06VD4rFjDoAJuhviixBeG0h8tu4Zu49qXiXBq3BsXUJdaNlHCBxmrF0oxqPKZRprF8tCn4dCFECVOfWNj4B51/9I7A76Qq+QkSxc3qSH2hzZBj1PIXBsCnb7YyUVSq3ZLWyZDY/ts1p5G+R7xWhgwunoaDM78RfTK2HZpXUtidLy5RVhp8wZM6vyOd8nKd5oWKxlD2T8IobkPHyLb3VuoGxhOqIj2mUYIBkxDs5qXmv3e17TMT2Lj2FuQCk/a912ozuHlJwPuI+zCKv86ealFTPLEUeXlBEcxtfiMIciD/LRHYYx7iRQ8HnJ7z9ld31G1xkwWwYfXscF4w+m0k8sgVWWLa69zyCXibbAfUARNIOLG2UDOEOniwj1xMeB1zEAobHi62nNLwjwn4PIYihz8LKo3zd+Alkwx4PhLn1X7ekAvU+Rq39A6+C1JyFS4I8lKkMclgrtb6c5DWS+8q3kDbykLCdgvZRVeWzmhaoANuSjDk9RqFABSdyWh4qWYBV/hRbrhYvOVsa5YDwvD76IkUu41lhk1fy3ouy2n7rtMg5QfCcIk8HgJuCiRwW+40UT9522UgBq+l4IhcjbU7W3osdzN1Q9H9KMcjvP9IOUNhgXFqgMb0zif6qqQ== julie_caohe@163.com

Let me know if any information are needed.

#5 Updated by openqa_review 5 months ago

  • Due date set to 2021-06-17

Setting due date based on mean cycle time of SUSE QE Tools

#6 Updated by nicksinger 4 months ago

I've added the machine on ariel to the dnsmasq config.
SUT can be reached under: blackbauhinia.openqanet.opensuse.org
IPMI can be reached under: blackbauhinia-ipmi.openqanet.opensuse.org

I've also added it as worker to openqaworker7 with the following config in /etc/openqa/workers.ini:

[20]
WORKER_CLASS=64bit-ipmi,64bit-ipmi-large-mem,64bit-ipmi-amd,blackbauhinia
IPMI_HOSTNAME=blackbauhinia-ipmi.openqanet.opensuse.org
IPMI_USER=ADMIN
IPMI_PASSWORD=ADMIN
SUT_IP=blackbauhinia.openqanet.opensuse.org
SUT_NETDEVICE=em1
IPMI_SOL_PERSISTENT_CONSOLE=1
IPMI_BACKEND_MC_RESET=1

#7 Updated by okurz 4 months ago

I suggest to add to "rebel" which runs without apparmor validation which likely prevents the ipmi backend to work

#8 Updated by nicksinger 4 months ago

  • Status changed from In Progress to Feedback
  • Assignee changed from nicksinger to Julie_CAO

Thanks for the suggestion, I moved the config to rebel as worker instance 5 - see: https://openqa.opensuse.org/admin/workers/382
I also added Julies ssh key from https://progress.opensuse.org/issues/93381#note-4 to all machines with

for i in aarch64 openqaworker1 openqaworker4 openqaworker7 power8 rebel imagetester; do echo $i && sshpass -p opensuse ssh root@$i "echo '$(cat .ssh/authorized_keys | grep "julie_caohe")' >> '/root/.ssh/authorized_keys'"; done

@Julie_CAO please test if the machine is working for you as expected and feel free to raise any problems and reassign back to me :)

#9 Updated by Julie_CAO 4 months ago

Hi nicksinger, thank you for adding the machine to O3 network and openQA worker pool. Is the correct way to access the machine over rebel as the jump host from SUSE? My following procedure does not seem correct as password is needed.

jcao@linux-brmq:~/.ssh> cat config
Host ariel
HostName gate.opensuse.org
Port 2213
Host *.opensuse.org
ProxyCommand ssh -q -A -x ariel -W %h:%p

jcao@linux-brmq:~/.ssh> ssh -X rebel.opensuse.org
jcao@gate.opensuse.org's password:

Another question, is there a PXE server in O3 network? I'd like to install openSUSE Tumbleweed in each test, the command we used is:
/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot2021xxxx/boot/x86_64/loader/linux initrd=/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot2021xxxx/boot/x86_64/loader/initrd install=http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210521

#10 Updated by nicksinger 4 months ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Julie_CAO to nicksinger

Julie_CAO wrote:

Hi nicksinger, thank you for adding the machine to O3 network and openQA worker pool. Is the correct way to access the machine over rebel as the jump host from SUSE? My following procedure does not seem correct as password is needed.

jcao@linux-brmq:~/.ssh> cat config
Host ariel
HostName gate.opensuse.org
Port 2213
Host *.opensuse.org
ProxyCommand ssh -q -A -x ariel -W %h:%p

jcao@linux-brmq:~/.ssh> ssh -X rebel.opensuse.org
jcao@gate.opensuse.org's password:

Yes, sorry I forgot to create your user on gate.opensuse.org and just added your key to the root account. Now it is fixed and you should be able to ssh into ariel (gate.opensuse.org). From there you can ssh into root@rebel.openqanet.opensuse.org (we don't have users on workers, just root). Please feel free to ask here or in Rocket if you have further questions how this jump host stuff is done - it can be confusing :)

Hi Nick, I am still unable to log in 'ariel'. Is the correct ssh key used?
jcao@linux-brmq:~> ssh gate.opensuse.org
Password:

jcao@linux-brmq:~> ssh jcao@gate.opensuse.org
Password:

jcao@linux-brmq:~/.ssh> pwd
/home/jcao/.ssh
jcao@linux-brmq:~/.ssh> cat id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDI/mD7o5XnT6X6bVN+sZgFBkfAdA7/wisXOnGYSdhzIECK6OCxf06VD4rFjDoAJuhviixBeG0h8tu4Zu49qXiXBq3BsXUJdaNlHCBxmrF0oxqPKZRprF8tCn4dCFECVOfWNj4B51/9I7A76Qq+QkSxc3qSH2hzZBj1PIXBsCnb7YyUVSq3ZLWyZDY/ts1p5G+R7xWhgwunoaDM78RfTK2HZpXUtidLy5RVhp8wZM6vyOd8nKd5oWKxlD2T8IobkPHyLb3VuoGxhOqIj2mUYIBkxDs5qXmv3e17TMT2Lj2FuQCk/a912ozuHlJwPuI+zCKv86ealFTPLEUeXlBEcxtfiMIciD/LRHYYx7iRQ8HnJ7z9ld31G1xkwWwYfXscF4w+m0k8sgVWWLa69zyCXibbAfUARNIOLG2UDOEOniwj1xMeB1zEAobHi62nNLwjwn4PIYihz8LKo3zd+Alkwx4PhLn1X7ekAvU+Rq39A6+C1JyFS4I8lKkMclgrtb6c5DWS+8q3kDbykLCdgvZRVeWzmhaoANuSjDk9RqFABSdyWh4qWYBV/hRbrhYvOVsa5YDwvD76IkUu41lhk1fy3ouy2n7rtMg5QfCcIk8HgJuCiRwW+40UT9522UgBq+l4IhcjbU7W3osdzN1Q9H9KMcjvP9IOUNhgXFqgMb0zif6qqQ== julie_caohe@163.com

Another question, is there a PXE server in O3 network? I'd like to install openSUSE Tumbleweed in each test, the command we used is:
/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot2021xxxx/boot/x86_64/loader/linux initrd=/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot2021xxxx/boot/x86_64/loader/initrd install=http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210521

Yes but as far as I understand it currently only supports Leap. I will check if I can enable TW installations on there.

Thank you, Nick. We need not an actual PXE entry for TW installation. The PXE server can support TW installation with boot parameters the way we do in OSD is ok.

#11 Updated by okurz 4 months ago

  • Status changed from In Progress to Workable

#13 Updated by cdywan 4 months ago

  • Due date changed from 2021-06-17 to 2021-07-02

#14 Updated by okurz 4 months ago

  • Status changed from Workable to New

moving all tickets without size confirmation by the team back to "New". The team should move the tickets back after estimating and agreeing on a consistent size

#15 Updated by cdywan 4 months ago

  • Due date deleted (2021-07-02)

#16 Updated by nicksinger 3 months ago

  • Assignee changed from nicksinger to Julie_CAO

@Julie_CAO I checked the key again on ariel and everything looks like it should. Please use the command ssh ariel to login into that host. If you use "gate.opensuse.org" your config is not used and therefore it does not work.

#17 Updated by Julie_CAO 3 months ago

nicksinger wrote:

@Julie_CAO I checked the key again on ariel and everything looks like it should. Please use the command ssh ariel to login into that host. If you use "gate.opensuse.org" your config is not used and therefore it does not work.

Thanks, Nick. ssh ariel works for me. I successfully login blackbauhinia over ssh on ariel.

So the only remained part is the PXE support. :-)

#18 Updated by Julie_CAO 3 months ago

Hi Nick,

Could you kindly give a general time when the PXE server would be ready? I will have a vacation leave in August, so I'd like to arrange my tasks ahead based on different situations.

#19 Updated by nicksinger 3 months ago

  • Status changed from New to In Progress
  • Assignee changed from Julie_CAO to nicksinger

I've added a kernel and initrd from the latest tw-netinstall ISO onto the PXE server on ariel. A first try didn't seem to work despite ariel serving the required files:

Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: error 0 TFTP Aborted received from 192.168.112.13
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: failed sending /srv/tftpboot/pxelinux.0 to 192.168.112.13
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: sent /srv/tftpboot/pxelinux.0 to 192.168.112.13
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/44454c4c-3100-1059-8052-b9c04f463733 not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/01-f4-02-70-b9-3a-66 not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/C0A8700D not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/C0A8700 not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/C0A870 not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/C0A87 not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/C0A8 not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/C0A not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/C0 not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: file /srv/tftpboot/pxelinux.cfg/C not found
Jul 27 08:10:07 ariel dnsmasq-tftp[1999]: sent /srv/tftpboot/pxelinux.cfg/default to 192.168.112.13

I saw something hinting at syntax errors but the IPMI connection in the train is not good enough. Will check if I can make it work at home with a stable connection.

#20 Updated by openqa_review 3 months ago

  • Due date set to 2021-08-11

Setting due date based on mean cycle time of SUSE QE Tools

#21 Updated by nicksinger 3 months ago

  • Status changed from In Progress to Blocked

I can't reach the machine nor the ipmi interface. I created an infra ticket now to check this machine: https://infra.nue.suse.com/Ticket/Display.html?id=193192

#22 Updated by Julie_CAO 3 months ago

Thank you for open the infra ticket, Nick. The machine is in SERV2, it was shutdown by me days ago when AC had a water leaking. But the ipmi is inaccessible either, It may have been implicated in the cooling system accident?

#23 Updated by nicksinger 3 months ago

  • Status changed from Blocked to In Progress

#24 Updated by nicksinger 3 months ago

  • Status changed from In Progress to Feedback

I've fixed the PXE setup in O3 and was able to start a tw installer. Please let me know if anything is not working as expected for you.

#25 Updated by Julie_CAO 3 months ago

Thank you, Nick.
I just tried with new PXE server from blackbauhinia-ipmi. It finds out the PXE server and the pxe menu shows up, TW can be installed with the 'openSUSE tumbleweed' entry. but when I press 'tab' to input the boot command below, then type 'enter', the screen went back to the pxe entry, No installation began or no any error popped up. Anything wrong?

/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210804/boot/x86_64/loader/linux initrd=/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210804/boot/x86_64/loader/initrd install=http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210804

#26 Updated by cdywan 2 months ago

  • Due date changed from 2021-08-11 to 2021-08-31

#27 Updated by nicksinger about 2 months ago

  • Status changed from Feedback to Resolved

Sorry for the long delay. I didn't notice your answer. I think you need to add the proper console parameter. Something like this should make the console show up over SOL: console=tty console=ttyS1,115200 (maybe ttyS0 depending on the machine). Please have a look what other IPMI tests on e.g. OSD supply as their command line:

[2021-07-28T20:31:29.995 CEST] [debug] <<< testapi::type_string(string="  Y2DEBUG=1 vga=791 video=1024x768 console=ttyS1,115200 linuxrc.log=/dev/ttyS1 linuxrc.core=/dev/ttyS1 linuxrc.debug=4,trace reboot_timeout=0  ", max_interval=4, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)

(from https://openqa.suse.de/tests/6599035/logfile?filename=autoinst-log.txt)

I'm resolving this now as this is an test issue and not caused by the PXE setup which apparently works for your case :) Feel free to reopen if you discover other issues

#28 Updated by Julie_CAO about 2 months ago

  • Due date changed from 2021-08-31 to 2021-08-11

nicksinger wrote:

Sorry for the long delay. I didn't notice your answer. I think you need to add the proper console parameter. Something like this should make the console show up over SOL: console=tty console=ttyS1,115200 (maybe ttyS0 depending on the machine). Please have a look what other IPMI tests on e.g. OSD supply as their command line:

[2021-07-28T20:31:29.995 CEST] [debug] <<< testapi::type_string(string="  Y2DEBUG=1 vga=791 video=1024x768 console=ttyS1,115200 linuxrc.log=/dev/ttyS1 linuxrc.core=/dev/ttyS1 linuxrc.debug=4,trace reboot_timeout=0  ", max_interval=4, wait_screen_changes=0, wait_still_screen=0, timeout=30, similarity_level=47)

(from https://openqa.suse.de/tests/6599035/logfile?filename=autoinst-log.txt)

I'm resolving this now as this is an test issue and not caused by the PXE setup which apparently works for your case :) Feel free to reopen if you discover other issues

Hi @Nick, I have to reopen this ticket as the PXE server does not work in O3 actually. Even with "console=tty console=ttyS1,115200", when I press 'tab' to input the boot command, then type 'enter', the screen went back to the pxe entry, No installation began on any console. It should not be console problem because it returns to the PXE menu.

Or let's try it as a SUT in O3, the 'boot_from_pxe' will type the correct command since the machine worked in OSD. Please help me add this SUT in O3 or point me what I shall do(it seems that I have no permission to login rebel).

    WORKER_CLASS: 64bit-ipmi,64bit-ipmi-amd
    IPMI_HOSTNAME: blackbauhinia-ipmi.openqanet.opensuse.org
    IPMI_USER: ADMIN
    IPMI_PASSWORD: ADMIN
    SUT_IP: blackbauhinia.openqanet.opensuse.org
    SUT_NETDEVICE: eno1
    IPMI_SOL_PERSISTENT_CONSOLE: 1
    IPMI_BACKEND_MC_RESET: 1

#29 Updated by Julie_CAO about 2 months ago

  • Status changed from Resolved to New

#30 Updated by cdywan about 2 months ago

  • Status changed from New to Feedback

@Julie_CAO In the future, please use Feedback when re-opening tickets.

#31 Updated by nicksinger about 2 months ago

  • Due date changed from 2021-08-11 to 2021-09-11

#32 Updated by nicksinger about 1 month ago

  • Status changed from Feedback to Blocked

I once again tried it out manually.

First, I set the bootdev:
ipmitool -I lanplus -C 3 -H blackbauhinia-ipmi.openqanet.opensuse.org -U ADMIN -P ADMIN chassis bootdev pxe

Attaching to the machine from ariel by using IPMI:
ipmitool -I lanplus -C 3 -H blackbauhinia-ipmi.openqanet.opensuse.org -U ADMIN -P ADMIN sol activate

After that it boots into the PXE menu where I just select "install-tumbleweed" and the kernel starts to load (and give me output on SOL). Just a small example from the beginning:

Loading linux/openSUSE/tumbleweed/linux.............
Loading linux/openSUSE/tumbleweed/initrd........................................
................................................................................
................ready.

[    0.000000] Linux version 5.13.4-1-default (geeko@buildhost) (gcc (SUSE Linux) 11.1.1 20210625 [revision 62bbb113ae68a7e724255e17143520735bcb9ec9], GNU ld (GNU Binutils; openSUSE Tumbleweed) 2.36.1.20210326-4) #1 SMP Thu Jul 22 15:55:06 UTC 2021 (91a0cca)
[    0.000000] Command line: initrd=linux/openSUSE/tumbleweed/initrd vga=normal console=tty0 console=ttyS1,115200 linemode=1 ignore_loglevel unknown_nmi_panic insecure=1 install=http://download.opensuse.org/tumbleweed/repo/oss/ nameserver=192.168.112.100 BOOT_IMAGE=linux/openSUSE/tumbleweed/linux
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
[    0.000000] BIOS-provided physical RAM map:

So manually I can't reproduce your issue. However, as rebel is currently down (https://progress.opensuse.org/issues/97658) and oli indicated that other workers will most likely won't work due to apparmor I think we're currently blocked there to get the IPMI machine into o3 and I can't do further testing for now.

#33 Updated by nicksinger about 1 month ago

  • Blocked by action #97658: many (maybe all) jobs on rebel within o3 run into timeout_exceeded "setup exceeded MAX_SETUP_TIME" size:M added

#34 Updated by Julie_CAO about 1 month ago

  • Due date changed from 2021-09-11 to 2021-08-11

nicksinger wrote:

After that it boots into the PXE menu where I just select "install-tumbleweed" and the kernel starts to load (and give me output on SOL). Just a small example from the beginning:

Yes, the pxe entry worked for me either. But I meant that the command below(press 'tab' key to type the command the way boot_from_pxe module does in openqa) does not work:
/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210830/boot/x86_64/loader/linux initrd=/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210830/boot/x86_64/loader/initrd install=http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210830 console=ttyS1,115200

I'ed like to install the latest snapshot, such as 20210830 that day I tried. Press 'enter' after typing the command, pxe menu screen showed up, no any console output.

#35 Updated by Julie_CAO about 1 month ago

  • Due date changed from 2021-08-11 to 2021-09-11

#36 Updated by nicksinger about 1 month ago

Ah, now I understand. Sorry for the confusion. Of course this can not work as we don't have the "/mnt/openqa" mountpoint on o3. I asked in our team chat for some help on what we could mount instead to always have the latest snapshot of TW available to boot from.

#37 Updated by Julie_CAO about 1 month ago

nicksinger wrote:

Ah, now I understand. Sorry for the confusion. Of course this can not work as we don't have the "/mnt/openqa" mountpoint on o3. I asked in our team chat for some help on what we could mount instead to always have the latest snapshot of TW available to boot from.

Thank you, Nick. that's exactly what I want!

#38 Updated by okurz about 1 month ago

  • Due date changed from 2021-09-11 to 2021-09-17

please take a look into the blocking ticket #97658 which we consider still "urgent" and will exceed the due-date tomorrow.

#39 Updated by cdywan about 1 month ago

  • Due date deleted (2021-09-17)

Still blocked by #97658 as I understand it, although Redmine won't show that for some reason.

#40 Updated by nicksinger about 1 month ago

While #97658 is still blocked I try to figure out what to mount to get the most recent TW snapshot in the opensuse network. I asked in #opensuse-factory:

14:58 <sqozz> hey all! I'm currently looking for a way to enable openqa.opensuse.org to test the most recent TW snapshot on bare hardware. For this I'd need a mountpoint to the most recent snapshot reachable from within the opensuse-network. Any hints where I could find such thing?

and in #team-buildops in slack: https://suse.slack.com/archives/C02BX1X92HM/p1632144642253300

#41 Updated by Julie_CAO about 1 month ago

Thank you nick for asking help from outside. I saw the thread in #team-buildops but the result seems not very optimistic. I am not familiar with http boot or EFI boot. Let's see if the factory folks can give more clue.

#42 Updated by Julie_CAO 25 days ago

Hi Nick, do you get any positive feedback from factory folks?

#43 Updated by Julie_CAO 11 days ago

Hi Nick,

regarding mounting to get lastest snapshot, we always run test this way in OSD. what's the difference between O3 and OSD? it sounds there are obstacles in O3 network. I want to know what the specific problem is, and then I'll manage to find a way to get it around on my side.

boot: /mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210830/boot/x86_64/loader/linux initrd=/mnt/openqa/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210830/boot/x86_64/loader/initrd install=http://openqa.opensuse.org/assets/repo/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20210830

Are there extracted installer DVD directories in O3 server? for example, "openqa.opensuse.org:/share/factory/assets/repo/openSUSE-Tumbleweed-DVD-x86_64-SnapshotXXXX/", we have this path for SLE in OSD.

If the DVD is present, what we need is to mount the repo to our pxe server, right?

If the DVD is not present, we have to extract from openSUSE-Tumbleweed-DVD-x86_64-Snapshot20211008-Media.iso in O3?

I don't know how OSD implements these requirements. we hope O3 can provide the same functions as OSD, If it is not possible on O3, it is acceptable to have workaround.

Let me know what I can help on my side.

#44 Updated by Julie_CAO 8 days ago

Hi okurz, Nick seems to have been out of work these days, can we manage to mount the TW snapshot in PXE server without Nick?

#45 Updated by nicksinger 8 days ago

  • Status changed from Blocked to In Progress

Hello @Julie_CAO - sorry I was on a sick-leave yesterday and before didn't update the ticket here. I was able to boot files from download.opensuse.org directly which means you could always boot the latest snapshot. However, the architecture inside the o3 infrastructure is different. Therefore the tests would need to be adjusted accordingly. I will prepare a boot environment for you today so you can see how it would work. Afterwards you would need to adjust the tests.

#46 Updated by openqa_review 7 days ago

  • Due date set to 2021-10-30

Setting due date based on mean cycle time of SUSE QE Tools

#48 Updated by nicksinger 3 days ago

  • Status changed from In Progress to Feedback

So I created an IPXE config which offers several options to boot a machine. I tested it with blackbauhinia manually from ariel with the following command: ipmitool -I lanplus -C 3 -H blackbauhinia-ipmi.openqanet.opensuse.org sol activate. It always loads the most recent TW snapshot from download.opensuse.org over http since we don't have any mountpoint with the most recent release like on OSD.

I realized that the IPMI output is quite slow but still was able to start an installer. Please test it manually to see if you could adjust the tests accordingly or if anything is missing. I also attach the IPXE-script below so you can see with which options the installer gets started. Let me know if any option would need to be added.

#!ipxe
dhcp

set menu-timeout 5000

:start
menu iPXE boot menu for openqa.opensuse.org
item --gap --             ------------------------- Operating systems ------------------------------
item --key l leapS1       Boot openSUSE Leap15.3 from download.opensuse.org / ttyS1 (http)
item --key l leapS2       Boot openSUSE Leap15.3 from download.opensuse.org / ttyS2 (http)
item --key t tumbleweedS1 Boot openSUSE Tumbleweed from download.opensuse.org / ttyS1 (http)
item --key t tumbleweedS2 Boot openSUSE Tumbleweed from download.opensuse.org / ttyS2 (http)
item --gap --             ------------------------- Advanced options -------------------------------
item --key c config       Configure settings
item shell                Drop to iPXE shell
item reboot               Reboot computer
item
item --key x exit         Exit iPXE and continue BIOS boot
choose --timeout ${menu-timeout} --default exit selected || goto cancel
set menu-timeout 0
goto ${selected}

:leapS1
kernel http://download.opensuse.org/distribution/leap/15.3/repo/oss/boot/x86_64/loader/linux usessh=1 sshpassword=linux network=1 install=http://download.opensuse.org/distribution/leap/15.3/repo/oss/ console=ttyS1,115200n8 root=/dev/ram0 initrd=initrd textmode=1
initrd http://download.opensuse.org/distribution/leap/15.3/repo/oss/boot/x86_64/loader/initrd
boot

:leapS2
kernel http://download.opensuse.org/distribution/leap/15.3/repo/oss/boot/x86_64/loader/linux usessh=1 sshpassword=linux network=1 install=http://download.opensuse.org/distribution/leap/15.3/repo/oss/ console=ttyS2,115200n8 root=/dev/ram0 initrd=initrd textmode=1
initrd http://download.opensuse.org/distribution/leap/15.3/repo/oss/boot/x86_64/loader/initrd
boot

:tumbleweedS1
kernel http://download.opensuse.org/distribution/openSUSE-current/repo/oss/boot/x86_64/loader/linux usessh=1 sshpassword=linux network=1 install=http://download.opensuse.org/distribution/openSUSE-current/repo/oss/ console=ttyS1,115200n8 root=/dev/ram0 initrd=initrd textmode=1
initrd http://download.opensuse.org/distribution/openSUSE-current/repo/oss/boot/x86_64/loader/initrd
boot

:tumbleweedS2
kernel http://download.opensuse.org/distribution/openSUSE-current/repo/oss/boot/x86_64/loader/linux usessh=1 sshpassword=linux network=1 install=http://download.opensuse.org/distribution/openSUSE-current/repo/oss/ console=ttyS2,115200n8 root=/dev/ram0 initrd=initrd textmode=1
initrd http://download.opensuse.org/distribution/openSUSE-current/repo/oss/boot/x86_64/loader/initrd
boot

:cancel
echo You cancelled the menu, dropping you to a shell

:shell
echo Type 'exit' to get the back to the menu
shell
set menu-timeout 0
set submenu-timeout 0
goto start

:failed
echo Booting failed, dropping to shell
goto shell

:reboot
reboot

:exit
exit

:config
config
goto start

#49 Updated by Julie_CAO 3 days ago

Hi Nick,

Thank you for setting it up, I'll try and adapt my test.

I reinstalled my laptop with Leap 15.3 on a new disk and the ssh keys changed. I am unable to login ariel to
access the SUT to try the new PXE now, could you help re-add my keys? sorry for bringing you the trouble.

jcao@localhost:~/.ssh> cat id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIvkuGQu2YkXbCMNnmtG/cyFLqUlYgrhJ0xFu0p1xjcY9gGtIlPzXZoEhNA5xxbsa0ydlW5GW/Msr+0zktNYzMksXKLQO4NQ0yAYrW5zaGpHZqrMImGCSe4JW0wG1w06Vpfj0IziqYpALdLch8sA+WEXMfe+58qW+1q+QsLL1rSvZ5ZlD18Z3PIKcMzJTgzxlSyInjqUwDzWaBnrBc8u4RfbGIZpq3USowDjX0VEGAJ+FEn6pWLbyfMkKaEQle0L5lSWLPxOPVr2XNVg1Fzy9eELfnqU23wMQdl8yGQU1WmHJrWV/jurcMbemMXo3od62bTIMO6sxooBp9pUSQFB2W9xdzmMZxJaaEQWHEe+LmVGNGogUqCbIPDstNewb8A5StavJh4iXQUIlVXsWuVJTCXzPqqAfymGR4au4UK1uggilB/rFTiYgVWM7eEO/Damb67qYAsCwh5/PTAFpWbIUnOZ/OVZ+mhMv6oXjbWe9NwSG4Kg2IAJG3Sic3gj8aFQnrrQMcUUE53QCN+akre2K61sI9DpQ8OjrbLeDNytCWCg5SpyzDbZHfpS9q2DEjRiN7BcKkijPkWzVV3cRpMPhH614CDbo4rCHs1clBKk2THuOtqGOnFVB4igNpiPdsfte/okLOBix+kbYckCmbi71JvWeyG0/R5h5KtDMPohI46w== julie_caohe@163.com

Also available in: Atom PDF