QA » openQA Project » openQA Tests

favogt wrote:

For that, it needs to be made conditional on a specific staging first and then broadened to all of TW after it got accepted.

I suggest to dynamically detect the new situation and accept if there is only one password prompt instead of checking the version for the staging project name or something

okurz wrote:

favogt wrote:

For that, it needs to be made conditional on a specific staging first and then broadened to all of TW after it got accepted.

I suggest to dynamically detect the new situation and accept if there is only one password prompt instead of checking the version for the staging project name or something

That would make the test green, but not catch if the grub2 behavior regressed and the password prompt shows up twice again.

Oh this is a really nice change, actually very annoying to have to type it twice.

With yaml schedule would be as simple as unschedule a test module and will catch regressions.
For the change with main.pm I'm not so familiar as this module is everywhere ...

@favogt is there any staging build to get the ISO at least to get prepared from our side with yaml changes?
I think we could move to yaml some of those staging test suite as we did in SLE, or at least the encrypted one.

JERiveraMoya wrote:

Oh this is a really nice change, actually very annoying to have to type it twice.

With yaml schedule would be as simple as unschedule a test module and will catch regressions.
For the change with main.pm I'm not so familiar as this module is everywhere ...

@favogt is there any staging build to get the ISO at least to get prepared from our side with yaml changes?

Currently not, it was unstaged again due to size reasons. It should be possible to stage it again soon.

I think we could move to yaml some of those staging test suite as we did in SLE, or at least the encrypted one.

BTW, various tests were moved from YAML to main.pm based scheduling because YAML was not flexible enough.

There are quite a few jobs in the main group: https://openqa.opensuse.org/tests/overview?modules=boot_encrypt&version=Tumbleweed&build=20221019&groupid=1

Let me know so we can take a look when there is an ISO.

Regarding yaml schedule, it is on purpose not flexible enough to convert from previous test in main.pm to yaml schedule, I'm not surprise if something was moved back, tests need to be reworked a bit, otherwise you end up with too many conditional flows. For us works really well, we have too many files, that is true, but it is just configuration and now we are starting to move to some solution with yaml schedule based on default files to save files.
There is already some example in tw: https://openqa.opensuse.org/tests/2825203/settings/schedule/yast/tumbleweed/guided_ext4.yaml where we have a default file with all the default steps of the installation and you can just override, and files for default with architectures could be created, so the yaml at the end represent the test case, which is for that example, to select ext4 and validate it. We are still working on it. The example doesn't look perfect either because it requires some refactors in the code as I mentioned at the beginning. https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/declarative-schedule-doc.md#5-use-yaml_schedule_default-and-yaml_schedule_flows-to-yaml-schedule

JERiveraMoya wrote:

Let me know so we can take a look when there is an ISO.

Staging:M has the grub2 submission at the moment and openQA fails here: https://openqa.opensuse.org/tests/2837678#step/boot_encrypt/4

May I ask is there any ETA this can be fixed ? Thanks.

favogt wrote:

Looks like nobody is taking care of this, so I gave it a shot:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15861

This made Staging:M pass and grub2 has been checked-in to openSUSE:Factory
A followup PR to ask only once for the passphrase was submitted in https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15885

The product (Snapshot 20221109) looks reasonably ok with this, we have two tests failing though (upgrades from 15.0, resp 15.2)

• https://openqa.opensuse.org/tests/2866189#step/first_boot/18
• https://openqa.opensuse.org/tests/2866190#step/first_boot/18

After the upgrade the password is asked for twice - as opposed to all the other tests where we now only ask once.

Could you please email me the passphrase so I can check opensuse-15.2-x86_64-GM-cryptlvm@uefi.qcow2 ? It seems either /etc/crypttab is not used or unspecified encrypted volume other than root is in the way.

Two more tests have been identified as failing (they passed yesterday, as they were still using the old openQA code, so they expected the password entry at that point):

https://openqa.opensuse.org/tests/2867882#step/first_boot/18 (create_hdd_gnome_encrypt_separate_boot)
That one indicates that is_boot_encrypted is not telling the truth; as it seems to return 0 here, even though we intentionally did not encrypt the boot partition

https://openqa.opensuse.org/tests/2867881#step/first_boot/3 (crypt_no_lvm)

dimstar wrote:

Two more tests have been identified as failing (they passed yesterday, as they were still using the old openQA code, so they expected the password entry at that point):

https://openqa.opensuse.org/tests/2867882#step/first_boot/18 (create_hdd_gnome_encrypt_separate_boot)
That one indicates that is_boot_encrypted is not telling the truth; as it seems to return 0 here, even though we intentionally did not encrypt the boot partition

It's actually the opposite: /boot is encrypted, but nothing else. I suppose the key passing does not work for just mounting /boot.

https://openqa.opensuse.org/tests/2867881#step/first_boot/3 (crypt_no_lvm)

That looks like the key passing did not work.

dimstar wrote:

favogt wrote:

FWICT the openQA side is done - just product bugs left.

https://openqa.opensuse.org/tests/2880511#step/reboot_gnome/10

This one I'd rate as QA bug: it reboots, asks for the grub decrypt and then continues booting (as expected) - but openQA wants to see a decrypt dialog

Indeed. There's a better place to put the condition: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15926

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upgrade_Leap_15.0_cryptlvm@uefi
https://openqa.opensuse.org/tests/2917179#step/reboot_gnome/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upgrade_Leap_15.2_cryptlvm@uefi
https://openqa.opensuse.org/tests/2991474#step/reboot_gnome/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 56 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upgrade_Leap_15.0_cryptlvm@uefi
https://openqa.opensuse.org/tests/3136752#step/reboot_gnome/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 112 days if nothing changes in this ticket.

I missed the notification on this one (forgot to watch the ticket), sorry, glad that you moved forward.
Anyway this one was in the scope or anyone trying to boot the system,not only YaST, but it will be very useful.

Feel free next time to ping me or reach me in slack.
Can this ticket be resolved @favogt ? (not sure if the reminders are pointing to job correctly labeled).

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upgrade_Leap_15.0_cryptlvm@uefi
https://openqa.opensuse.org/tests/3201317#step/reboot_gnome/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 48 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upgrade_Leap_15.0_cryptlvm@uefi
https://openqa.opensuse.org/tests/3298368#step/reboot_gnome/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 96 days if nothing changes in this ticket.

@favogt could we close that ticket?

maritawerner wrote:

@favogt could we close that ticket?

Unfortunately not. This is still a broken case due to the way the test scheduling and conditions work.
Fixing this basically involves rewriting the functions handling the password entry for decryption, which is annoying mostly because of the many scenarios they have to cover.

@favogt what tests are affected? only the one failing, I don't have a clear idea of the scope of the issue after your answer. If you could give a bit more detail.

I will ping QE-Core to discuss, as it is a generic thing and we would not be fixing the world here, for example we would only fix the test scenarios using yaml schedule.
Also the maintainer belongs to that squad.
I will try to ping before un-assigning squad next time to make the process less sub-optimal.

JERiveraMoya wrote:

@favogt what tests are affected? only the one failing,

Not entirely sure. The current failures are a mix of test issues and product issues and IIRC there's even a case where a product bug cancels out a test bug...

I don't have a clear idea of the scope of the issue after your answer. If you could give a bit more detail.

What I remember is: There are two points during boot which ask for the passphrase:

1. GRUB asks if /boot is encrypted
2. Plymouth asks if / is encrypted and the passphrase entered at 1. wasn't reused (with the grub patches in TW)

The current logic in the test code mixes those up. What happens internally is that if the passphrase was reused on TW, 1 is skipped, not 2. They share needles and just do type_password anyway so it kind of works, but in some setups this causes issues. There is a mix between the boot_encrypt module getting used and boot_to_desktop taking care of both prompts.

I will ping QE-Core to discuss, as it is a generic thing and we would not be fixing the world here, for example we would only fix the test scenarios using yaml schedule.
Also the maintainer belongs to that squad.
I will try to ping before un-assigning squad next time to make the process less sub-optimal.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upgrade_Leap_15.0_cryptlvm@uefi
https://openqa.opensuse.org/tests/3414160#step/reboot_gnome/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upgrade_Leap_15.0_cryptlvm@uefi
https://openqa.opensuse.org/tests/3481164#step/reboot_gnome/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 56 days if nothing changes in this ticket.

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: upgrade_Leap_15.0_cryptlvm@uefi
https://openqa.opensuse.org/tests/3610433#step/reboot_gnome/1

To prevent further reminder comments one of the following options should be followed:

1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 112 days if nothing changes in this ticket.