QA (public) » openQA Project (public) » openQA Tests (public)

https://github.com/jknphy/os-autoinst-distri-opensuse/commit/780b957b90ad51d190e2949f22fe985d61ed9630
https://gitlab.suse.de/JERiveraMoya/os-autoinst-needles-sles/commit/82dad959a13b06477284a1a25e225c8425f65080
Need also needles for Tumbleweed, pending to send another PR to my fork and check with Sergio about these two above in my forks for being improved before to send to the official repo.
These two should cover scenarios in Text mode and Graphic for Tumbleweed. Successfully tested in my local four scenarios.

feel free to provide a PR and mark the subject line as "[WIP]" to prevent it from being merged but invite for early feedback. Until then I will take a look into your github commit and comment there.

EDIT: hm, I thought I could comment on a commit as well but I can't. Please provide a PR then

To make needling easier we should fully disable the timeout. Can you do that?

GRUB_TIMEOUT=-1 will cause the menu to be displayed until it is selected the boot entry manually.
Renamed tests and needles from change_grub_timeout to disable_grub_timeout.
This change does not make needling easier, given it is just type -1 instead of 60 seconds. Needles added just are need to ensure the path to the field where the value is typed (i.e.: correct tab selected, etc.)
Found missing needles in Tumbleweed text mode. Currently fixing it before sending [WIP] PR.

great news. So good change. Feel free to create a PR anyway when it works for just one scenario. We will comment on the PR and I am sure there is at least something to change which will have an impact. Better to use fast feedback

PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3715

Tested for Leap 42.3 Updates and Maintenance. New needles were created for Opensuse Leap. There are some subtle differences between Tumbleweed and Leap (i.e.: the lines to draw menu tabs have difference length, selected text has a square bigger in one than the other, etc.)

Added several improvements in the PRs based on review comments: syntax, better use of OpenQA libraries, visualization of the change with a screenshot, etc.

And it brokes all QAM workflow ...

As discussed with osukup, taking a look for re-needling in Maintenance: Test repo job group.

jrivera: looks like leap 15.0 uses alt-l? https://openqa.opensuse.org/tests/506564#step/disable_grub_timeout/6

Fixed several issues for other products:

• Test included in more test scenarios.
• SLE 12 GA: Fixed problem related with the range accepted for Timeout [0,300]. Instead of -1, for this product version is typed 60 seconds.
• SLE 12 SP1: missing needles.
• Opensuse Leap: different shortcut to access tab & corresponding needles.

But this seems to fail on Leap Maintenance?

disable_grub_timeout is failing in Leap Maintenance tests:

https://openqa.opensuse.org/tests/507382#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507383#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507384#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507385#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507386#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507387#step/disable_grub_timeout/7

https://openqa.opensuse.org/tests/507391#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507392#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507393#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507394#step/disable_grub_timeout/4
https://openqa.opensuse.org/tests/507395#step/disable_grub_timeout/7
https://openqa.opensuse.org/tests/507396#step/disable_grub_timeout/7

Looks like wrong hotkey for leap 42.3. https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3742 should be able to handle this.verified

-> #26868

New PR for fixing most of the found failures (hopefully): https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3749
Found today that sle-15 for aarch64 is 'alt + t': https://openqa.suse.de/tests/1224390#step/disable_grub_timeout/3

Last PR seems to work except for some needles in SLE-15 that are in different position in the screen, we might need another wait_still_screen(1) before sending key for selecting tab. BUT for leap 42.2 & 42.3 on machine 64bit-2G is not implemented the exception yet, so it is making fail a lot of tests, as expected. Added PR for workaround:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3755

PR to fix wrong hotkey on x86_64: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3760

Merged

Verified on OSD: https://openqa.suse.de/tests/1227894#step/disable_grub_timeout/4

• H1: REJECTED by E1-1 machines/workers with 2G, are causing the problem of different hotkey on module disable_grub_timeout.
• H2: Jobs without UEFI on Leap versions older than Leap 15 are causing the problem of different hotkey on module disable_grub_timeout.

• H3: Jobs without UEFI on SLE versions older than 12-SP2 are also affected by the problem of different hotkey on module disable_grub_timeout.

• E1-1: Find jobs with 2G machines where the module disable_grub_timeout works.

• R1-1: works fine for the same build, version and arch for jobs with UEFI:

  • o3#508722#step/disable_grub_timeout/7
  • o3#508723#step/disable_grub_timeout/7
  • o3#508725#step/disable_grub_timeout/7

• E2-1: Find all jobs with 2G machines where jobs haven't UEFI set and don't fail on the module disable_grub_timeout.

• R2-1: Not found.

• E2-2: Find all jobs with 2G machines where jobs haven't UEFI set and fail on the module disable_grub_timeout.

• R2-2: All jobs that execute the module disable_grub_timeout and doesn't have UEFI fail:

  • o3#508724#step/disable_grub_timeout/8
  • o3#508726#step/disable_grub_timeout/5

• E3-1: Find jobs with UEFI for SLE 12-SP1 that fails on module disable_grub_timeout.

PR to fix non-UEFI jobs for Leap 42.2: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3764

Closed.
Superseded by https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3762

So the failures I found which I think are most critical is the part which you mentioned as "not implemented yet" for openSUSE Leap maintenance tests. This is something that should also be done on the weekend, maintenance updates never sleep, especially not on openSUSE ;-)

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3762 is a two-fold approach, adds an explicit assert_screen to make sure the bootloader settings are shown before trying to press any key. Also, it accepts if the key to switch the tab could not be found and returns without failing.

tests fixed for the time being with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3765 by cancelling the configuration module when the hotkey did not match. I think a safer solution is to use send_key_until_needle_match 'installation-bootloader-options', 'tab'` which should work in case of both ncurses as well as GUI.

Investigating why is only typing 0 instead of -1 (this issue makes grub_test to fail). Not reproducible in my local machine. Logs shows -1 is typed. It seems the typing speed.
Textbox only detects "-" and after "ret" comes back to "0".
https://openqa.suse.de/tests/1230788#step/disable_grub_timeout/5
https://openqa.suse.de/tests/1231104#step/disable_grub_timeout/8
It might help to use: type_string_slow. PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3777

PR merged and active on both osd as well as o3. Now we seem to have the problem with the '0' sometimes so it seems we are not done yet. I will also try to run it more often locally.

type_string_slow would only wait between the characters and it's just two characters. Happened again in https://openqa.opensuse.org/tests/511736#step/disable_grub_timeout/8

I will try to reproduce locally as well. https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3778 as a temporary workaround to disable again.

Could not reproduce the problem with "0 typing" locally in 40 runs so only my best guess stays that the production environment is different here and something like a wait_still_screen(1) could help for production. This is included in my PR. I leave it to reviewers to decide about that.

Missing needles for SLE-12 GA: https://gitlab.suse.de/openqa/os-autoinst-needles-sles/merge_requests/544