action #113794
closedUse prepared OVMF image with expected settings size:M
Added by livdywan almost 2 years ago. Updated over 1 year ago.
Description
Observation¶
With QEMU 7, the default video resolution was changed which caused #111992 since tests end up with an unexpected resolution. #111992#note-16 suggests using a prepared NVRAM image.
This might also be useful in preparation for #113357.
Acceptance criteria¶
- AC1: Tests can boot with a custom NVRAM image
Suggestions¶
- Maybe nvram tooling can be used to change the settings (efitools)
- The following can be used to manually generate a file. Create ovmf code+vars file, spawn VM, enter menu manually, then terminate the VM to save the file
/usr/bin/qemu-img create -f qcow2 -F raw -b /usr/share/qemu/ovmf-x86_64-ms-code.bin pflash-code-overlay0 1966080
/usr/bin/qemu-img create -f qcow2 -F raw -b /usr/share/qemu/ovmf-x86_64-ms-vars.bin pflash-vars-overlay0 131072
/usr/bin/qemu-system-x86_64 -vga qxl -only-migratable -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -audiodev none,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 -global isa-fdc.fdtypeA=none -m 2048 -cpu host -netdev user,id=qanet0 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -device qemu-xhci -device usb-tablet -smp 1 -enable-kvm -no-shutdown -vnc :101,share=force-shared -drive id=pflash-code-overlay0,if=pflash,file=pflash-code-overlay0,unit=0,readonly=on -drive id=pflash-vars-overlay0,if=pflash,file=pflash-vars-overlay0,unit=1
- See if we can generate the image in openQA or GitLab and automatically upload it as an asset to openQA's asset storage. Maybe small changes are necessary how we handle those special UEFI variables.
Updated by livdywan almost 2 years ago
- Copied from action #111992: Deal with QEMU and OVMF default resolution being 1280x800, affecting (at least) qxl size:M added
Updated by livdywan almost 2 years ago
- Subject changed from Use prepared OVMF image with expected settings to Use prepared OVMF image with expected settings size:M
- Description updated (diff)
- Status changed from New to Workable
- Assignee deleted (
nicksinger)
Updated by okurz almost 2 years ago
- Related to action #114523: Deal with QEMU and OVMF default resolution being 1280x800, affecting (at least) qxl, but on aarch64 size:M added
Updated by mkittler almost 2 years ago
- Assignee deleted (
mkittler)
With all the worker recovery going on I haven't had a chance to work on this ticket further. On Monday I'll also not be able to work on it the first half on the day. So anybody is welcome to pickup the ticket meanwhile.
Yesterday I explored the possibility to use efivar or tools from efitools to update the pflash-vars file without having to spawn a VM. However, it didn't work (e.g. efivar --fromfile=… didn't allow to edit read/write to the file). So likely we cannot avoid spawning a VM for doing the settings.
By the way, to enter the boot menu manually without VNC, one can use the following command (after creating the images as shown in the description):
/usr/bin/qemu-system-x86_64 -vga qxl -only-migratable -chardev ringbuf,id=serial0,logfile=serial0,logappend=on -serial chardev:serial0 -audiodev none,id=snd0 -device intel-hda -device hda-output,audiodev=snd0 -global isa-fdc.fdtypeA=none -m 2048 -cpu host -netdev user,id=qanet0 -device virtio-net,netdev=qanet0,mac=52:54:00:12:34:56 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 -device qemu-xhci -device usb-tablet -smp 1 -enable-kvm -no-shutdown -drive id=pflash-code-overlay0,if=pflash,file=pflash-code-overlay0,unit=0,readonly=on -drive id=pflash-vars-overlay0,if=pflash,file=pflash-vars-overlay0,unit=1 -boot menu=on
Updated by livdywan almost 2 years ago
mkittler wrote:
Yesterday I explored the possibility to use
efivaror tools fromefitoolsto update thepflash-varsfile without having to spawn a VM. However, it didn't work (e.g.efivar --fromfile=…didn't allow to edit read/write to the file). So likely we cannot avoid spawning a VM for doing the settings.
Would you mind clarifying what didn't work? Which commands did you try? Did you get an error?
Updated by mkittler almost 2 years ago
I tried efivar --fromfile=pflash-vars-overlay0 --list and it just returned my normal EFI vars (and not the ones from the QEMU image). Specifying /usr/share/qemu/ovmf-x86_64-ms-vars.bin directly didn't work either. I've checked the binaries from efitools but their --help-text didn't look like specifying a file would be possible at all.
Updated by okurz almost 2 years ago
- Priority changed from High to Normal
- Target version changed from Ready to future
We have to focus on other urgent tasks with reduced capacity so moving this out of backlog
Updated by okurz almost 2 years ago
- Target version changed from future to Ready
adding back to the backlog. I think I overlooked that we have two blocked tickets in the backlog
Updated by tinita over 1 year ago
- Priority changed from Normal to High
We need this as a prerequisite for #111992
Updated by tinita over 1 year ago
- Status changed from Workable to In Progress
- Assignee set to tinita
Updated by openqa_review over 1 year ago
- Due date set to 2022-09-08
Setting due date based on mean cycle time of SUSE QE Tools
Updated by tinita over 1 year ago
I created a job which sets the resolution in the tianocore menu, then enters grub and types "halt".
It publishes the modified pflash file via PUBLISH_PFLASH_VARS=ovmf-x86_64-ms-vars-800x600.qcow2.
Now I need to test if I can use that in a second job.
openqa-clone-job --host localhost:9526 https://openqa.opensuse.org/tests/2536831 QEMUVGA=qxl UEFI=1 UEFI_PFLASH_CODE=/usr/share/qemu/ovmf-x86_64-ms-code.bin UEFI_PFLASH_VARS=/usr/share/qemu/ovmf-x86_64-ms-vars.bin DUALBOOT=1 BOOT_MENU=1 PAUSE_ON_SCREEN_MISMATCH=1 _SKIP_POST_FAIL_HOOKS=1 RETRY=0 SCHEDULE=tests/boot/boot_to_desktop _TRIGGER_JOB_DONE_HOOK=0 ISO=openSUSE-Tumbleweed-DVD-x86_64-Snapshot20220824-Media.iso HDD_1= PUBLISH_PFLASH_VARS=ovmf-x86_64-ms-vars-800x600.qcow2
...
Updated by livdywan over 1 year ago
Discussed in the Unblock. Marius suggested this could be done by moving the logic into a test module, come up with a jobs post command line and e.g. add this to a (masnual) gitlab pipeline.
Updated by tinita over 1 year ago
I created a verification run:
https://openqa.opensuse.org/tests/2600931
Interestingly it worked with the openSUSE-Tumbleweed-NET-x86_64-Snapshot20220829-Media.iso while locally only DVD worked.
Next step: use the generated pflash vars on o3
Updated by tinita over 1 year ago
This PR is needed as a first step, so that existing code does not fail if we use the prepared pflash vars image: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15471 - Allow already preselected resolution 800x600
Updated by tinita over 1 year ago
Updated by tinita over 1 year ago
Verification run: https://openqa.opensuse.org/tests/2636971#step/bootloader_uefi/19
Setting: UEFI_PFLASH_VARS ovmf-x86_64-ms-vars-800x600.qcow2
Confirming that it is accepeted that the correct resolution is already set.
Updated by tinita over 1 year ago
Verification run on openqaworker1 with an upgraded qemu-ovmf-x86_64-202202-150400.3.3.noarch: https://openqa.opensuse.org/tests/2637658
Downgraded and added zypper lock again
Updated by tinita over 1 year ago
Updated by tinita over 1 year ago
Updated by livdywan over 1 year ago
- Due date changed from 2022-09-08 to 2022-09-14
- Status changed from In Progress to Feedback
Changes merged, let's see if this works as expected
Updated by tinita over 1 year ago
I'm trying to figure out the right openqa-cli api -X POST jobs commandline
Marius pointed my to
https://github.com/os-autoinst/scripts/blob/master/trigger-openqa_in_openqa#L30
Then I need to create a new repo in gitlab and create a schedule.
Updated by tinita over 1 year ago
Current working command:
openqa-cli api -X POST jobs --host https://openqa.opensuse.org
TEST=ovmf-resolution QEMUVGA=qxl UEFI=1
UEFI_PFLASH_CODE=/usr/share/qemu/ovmf-x86_64-ms-code.bin UEFI_PFLASH_VARS=/usr/share/qemu/ovmf-x86_64-ms-vars.bin
SCHEDULE=tests/boot/tianocore_set_resolution.pm
PUBLISH_PFLASH_VARS=ovmf-x86_64-ms-vars-800x600.qcow2
DISTRI=openSUSE VERSION=Tumbleweed FLAVOR=NET
ISO=openSUSE-Tumbleweed-NET-x86_64-Snapshot20220911-Media.iso
Updated by tinita over 1 year ago
I moved the find_latest_published_tumbleweed_image from trigger-openqa_in_openqa into _common in smaller functions in order to test and reuse it. So far I pushed to a branch:
https://github.com/perlpunk/scripts/tree/trigger-pflash-test
Then I can use it in a new script to schedule the pflash job with the latest iso image.
Updated by tinita over 1 year ago
- Due date changed from 2022-09-14 to 2022-09-17
I created a trigger-pflash-test and a github workflow.
TODO:
- Change workflow to run once a day
- Create a dedicated user plus api key on o3 for scheduling the job
Updated by okurz over 1 year ago
We triggered a manual job from github actions which created an ovmf vars image with resolution 800x600. This created an image "ovmf-x86_64-ms-vars-800x600.qcow2" on o3 which I hard-linked to the "fixed" dir /var/lib/openqa/share/factory/hdd/fixed . We still plan to create an update on a weekly base.
tinita changed the definitions of openQA machines "uefi_win" and win10+win11 variants to use that file. I created a verification run with
openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org/tests/2664073 _GROUP=0 BUILD= TEST=gnome_dual_windows10_svga_ovmf_image
Created job #2669479: opensuse-Tumbleweed-DVD-x86_64-Build20220914-gnome_dual_windows10@uefi_win -> https://openqa.opensuse.org/t2669479
tinita removed the package locks on o3 workers and upgraded the package. I created https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15537 to only apply the resolution setting when necessary.
I downloaded the pflash vars image to OSD once manually with
cd /var/lib/openqa/factory/hdd/fixed
wget https://openqa.opensuse.org/tests/2669542/asset/hdd/ovmf-x86_64-ms-vars-800x600.qcow2
and tinita removed the locks from both o3+osd
EDIT: whoopsie, the above test job is missing the actual image setting. Retriggered with
openqa-clone-job --skip-chained-deps --within-instance https://openqa.opensuse.org/tests/2664073 _GROUP=0 BUILD= TEST=gnome_dual_windows10_svga_ovmf_image UEFI_PFLASH_VARS=ovmf-x86_64-ms-vars-800x600.qcow2
Created job #2670480: opensuse-Tumbleweed-DVD-x86_64-Build20220914-gnome_dual_windows10@uefi_win -> https://openqa.opensuse.org/t2670480
Updated by tinita over 1 year ago
https://github.com/os-autoinst/scripts/pull/187 - Trigger test to generate pflash vars image (merged)
- Added
toolsbotuser and api key in o3 - Added api key to https://github.com/os-autoinst/scripts
- Added a workflow which is run by cron, but can also be manually started: https://github.com/os-autoinst/scripts/actions/workflows/schedule.yml
- Removed zypper lock and upgraded qemu-ovmf-x86_64 on all o3 and osd workers (except power8)
- Changed setting
UEFI_PFLASH_VARS=ovmf-x86_64-ms-vars-800x600.qcow2for machines uefi_win, win10_uefi, win11_uefi on o3 and osd so now they should pass again without having to downgrade the package
Updated by okurz over 1 year ago
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15537 merged. https://openqa.opensuse.org/t2670542 shows that tests work now with the new image without setting any different resolution in the firmware menu. I suggest to wait until tomorrow to resolve the ticket unless objections or regressions have been met.
Updated by okurz over 1 year ago
- Due date deleted (
2022-09-17)
https://openqa.suse.de/tests/9526141#settings is using QEMUVGA=virtio. I suggested to jlausuch who saw that to try if the resolution will be correct if we use the updated pflash vars image.
Updated by jlausuch over 1 year ago
@okurz @tinita
I see most uefi jobs failing now, specially in JeOS and SLE Micro... I tried restarting some of them with UEFI_PFLASH_VARS=ovmf-x86_64-ms-vars-800x600.qcow2 and seems to work... should we update this in uefi machine in OSD or should I specify this variable in all uefi tests that are broken?
Updated by okurz over 1 year ago
Use the most downstream place where UEFI+QEMUVGA=qxl/virtio is set. So e.g. if UEFI is set by the machine and QEMUVGA in a job template then set in the job template. If QEMUVGA is set in a machine but UEFI in a job template (I advise against that combination setting) then in the job template. If UEFI+QEMUVGA are set in a machine definition (preferred solution) then please also set in the machine.
Please use the still open #111992 for any remaining problems.
Updated by jlausuch over 1 year ago
Hi, I have updated the machine uefi-virtio-vga in OSD (which already had QEMUVGA=virtio) with UEFI_PFLASH_CODE=ovmf-x86_64-ms-vars-800x600.qcow2 .
Those are used in JeOS tests where we had the most bunch of tests failing in the bootloader.
Updated by pstivanin over 1 year ago
@tinita:
I see that there still is a lock on the following worker:
pstivanin@openqa:~> sudo salt 'openqaworker-arm-3.suse.de' cmd.run 'zypper ll'
openqaworker-arm-3.suse.de:
# | Name | Type | Repository | Comment
--+-------------------+---------+------------+--------
1 | qemu-uefi-aarch64 | package | (any) |
is this expected or can we remove it?
Thanks a lot!
Updated by okurz over 1 year ago
pstivanin wrote:
@tinita:
I see that there still is a lock on the following worker:pstivanin@openqa:~> sudo salt 'openqaworker-arm-3.suse.de' cmd.run 'zypper ll' openqaworker-arm-3.suse.de: # | Name | Type | Repository | Comment --+-------------------+---------+------------+-------- 1 | qemu-uefi-aarch64 | package | (any) |is this expected or can we remove it?
This is still expected, see #114523 for details regarding aarch64.
Updated by okurz over 1 year ago
- Related to action #116914: [qe-core] recent uefi changes makes test fail added
Updated by okurz over 1 year ago
- Related to action #116812: [qe-core] Leap 15.5 uefi console switch fail size:M added
Updated by okurz over 1 year ago
- Related to action #116704: [qe-core] nvme@uefi: boot from DVD post install instead of from HDD added