Project

General

Profile

action #25376

coordination #28504: [sle][functional][epic] crypt lvm for SLE 12-SP3, SLE 12-SP4 and SLE 15

[sle][functional][ppc64le][hard] test fails in grub_test - stuck in grub screen, expected encrypted password prompt (was: boots completely)

Added by jorauch about 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Urgent
Category:
Bugs in existing tests
Start date:
2017-09-18
Due date:
2017-12-06
% Done:

0%

Estimated time:
Difficulty:

Description

Acceptance criteria

AC1: Product bugs related to cryptlvm are properly filed for SLE 15 on aarch64, ppc64 and x86_64

Tasks

  1. Determine and file product bugs related to cryptlvm for SLE 15 on aarch64, ppc64 and x86_64

Observation

Example for ppc: https://openqa.suse.de/tests/1173256#step/grub_test/1

Looks like the test boots straight to the login screen but expects to see Grub.
Maybe some needles are missing?

SLE 12-SP3 aarch

SLE 12-SP3 ppc

SLE 12-SP3 x86

SLE 15 349.1 aarch

SLE 15 349.1 ppc

SLE 15 349.1 x86

openQA test in scenario sle-15-Leanos-DVD-aarch64-yast_no_self_update@aarch64 fails in
grub_test

Reproducible

Fails since (at least) Build 260.4 (current job)

Acceptance criteria

  • AC1: Tests are green on all architectures again
  • AC2: Changes do not break tests for older versions (SLE12)

Tasks

  • Adapt test flow to architecture (aarch64 1x password, ppc64le 2x password)
  • Verify for all architectures

Expected result

Last good: 257.3 (or more recent)

All tests are either working or breaking in a product bug

Further details

Always latest result in this scenario: latest


Related issues

Related to openQA Tests - action #25388: [sle][functional][sle15][aarch64]test fails in grub_testResolved2017-09-182017-11-08

Related to openQA Tests - action #25472: [sles][functional][aarch64][sporadic] test fails in grub_test: openQA detects stall (cannot find grub2 needle) but system boots completely as seen in serial0.txtRejected2017-09-21

Related to openQA Tests - action #27397: [sle][functional][openqareview] Verification that AArch64 is able to boot from encrypted LVM - Password prompt appears on serial console instead of plymouth screenResolved2017-11-082018-01-30

Blocked by openQA Tests - action #25658: [sle][functional][migration][opensuse][virtualization]Increase/disable timeout of initial grub menue to ensure tests do not miss itResolved2017-09-292017-10-25

Blocked by openQA Tests - action #27082: [sle][functional] test fails in scc_registration - Since the beta version, modules are hidden by defaultResolved2017-10-27

History

#1 Updated by okurz about 4 years ago

  • Priority changed from Normal to High

the aarch64 test seems to have missed the grub menue completely, looking at the video. The ppc64le test is tricky because no videos are recorded on ppc64le by default. I recommend to manually retrigger one with NOVIDEO=0 and monitor that one.

#2 Updated by okurz about 4 years ago

  • Subject changed from [sle][functional][aarch64][ppc64le] test fails in grub_test - boots completely to [sle][functional][ppc64le] test fails in grub_test - boots completely

#3 Updated by okurz about 4 years ago

  • Related to action #25388: [sle][functional][sle15][aarch64]test fails in grub_test added

#4 Updated by okurz about 4 years ago

  • Target version set to Milestone 11

#5 Updated by mgriessmeier about 4 years ago

  • Related to action #25472: [sles][functional][aarch64][sporadic] test fails in grub_test: openQA detects stall (cannot find grub2 needle) but system boots completely as seen in serial0.txt added

#6 Updated by okurz about 4 years ago

  • Blocked by action #25658: [sle][functional][migration][opensuse][virtualization]Increase/disable timeout of initial grub menue to ensure tests do not miss it added

#7 Updated by jorauch about 4 years ago

Also happens in boot_encrypt
https://openqa.suse.de/tests/1224378#comments

#8 Updated by okurz about 4 years ago

hm, ok. so that's interesting because we now have "disable_grub_timeout" so we should not miss the grub screen. So this is a different issue then.

#9 Updated by okurz about 4 years ago

  • Due date set to 2017-11-08

#10 Updated by okurz about 4 years ago

  • Subject changed from [sle][functional][ppc64le] test fails in grub_test - boots completely to [sle][functional][ppc64le] test fails in grub_test - stuck in grub screen, expected encrypted password prompt (was: boots completely)

#11 Updated by JERiveraMoya about 4 years ago

  • Status changed from New to In Progress
  • Assignee set to JERiveraMoya

#13 Updated by SLindoMansilla about 4 years ago

  • Blocked by action #27082: [sle][functional] test fails in scc_registration - Since the beta version, modules are hidden by default added

#14 Updated by okurz about 4 years ago

PR merged, triggered one job manually:

openqa_clone_job_osd 1235824 _GROUP=0 TEST=okurz_crosscheck_cryptlvm_minimal_x

Created job #1239194: sle-15-Installer-DVD-ppc64le-Build321.5-cryptlvm_minimal_x@ppc64le -> https://openqa.suse.de/t1239194

#15 Updated by JERiveraMoya about 4 years ago

Test moved forward, but lines https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/installation/grub_test.pm#L76-L79 make it stuck again, checking for the grub screen again. Should be removed now with grub timeout disabled?

#16 Updated by JERiveraMoya about 4 years ago

After discussion with the team about the use of this test and also with the developer of the offending lines it seems reasonable to exclude them instead of delete them.
PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3837

#18 Updated by JERiveraMoya about 4 years ago

I got to a solution that is suppose to unblock staging and also fix the ppc64. PR modified (not closed, sorry for the previous confusion). It was successful with 64bit. Asked mgriessmeier for verification run in ppc. The problem that the grub appears before the passphrase need to be further discussed with developers I think.

#19 Updated by JERiveraMoya about 4 years ago

It still failing for ppc64: https://openqa.suse.de/tests/1242988# but at least unblock failures for 64bit: https://openqa.suse.de/tests/1242989#

#20 Updated by JERiveraMoya about 4 years ago

We can see failures in grub_test and boot_crypt tests. Sometimes the first does actions that the second should do due to the following differences in behavior for each architectures:

cryptlvm tests for SLE15:

Comparing with SLE-12-SP3:

Not sure what is the right approach to continue, meaning the correct workflow for each architecture. Set ticket on feedback.

#21 Updated by JERiveraMoya about 4 years ago

  • Status changed from In Progress to Feedback

#22 Updated by JERiveraMoya about 4 years ago

  • Status changed from Feedback to In Progress

#24 Updated by JERiveraMoya about 4 years ago

Verification failed because grub_test is typing passphrase for ppc. Excluding this part only for ppc: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3872
New verification seems successful: https://openqa.suse.de/tests/1244836#step/boot_encrypt/3

#25 Updated by SLindoMansilla about 4 years ago

  • Related to action #27397: [sle][functional][openqareview] Verification that AArch64 is able to boot from encrypted LVM - Password prompt appears on serial console instead of plymouth screen added

#26 Updated by JERiveraMoya about 4 years ago

Current status of this ticket:
ppc64le:

This task need to be picked up by someone else due to is required local verification for ppc to consolidate logic in tests grub_test and boot_encrypt.
That could be handled in a different ticket due to the main fix of this ticket is done, that is to pass the grub screen (due to disabled grub timeout) or to be continued in this one (asking Oliver about it).

Last re-triggered of the build: https://progress.opensuse.org/issues/25854 asked for password once, so both jobs are successful.

#27 Updated by okurz about 4 years ago

  • Due date changed from 2017-11-08 to 2017-11-22
  • Priority changed from High to Urgent
  • Target version changed from Milestone 11 to Milestone 12

We agree that this became a bigger issue now and we should work on this together. Scheduling for next sprint to re-evaluate.
We should understand the expected product behaviour, split the tasks, review open bugs, open fate issues, etc.

#28 Updated by okurz about 4 years ago

  • Due date changed from 2017-11-22 to 2017-12-06
  • Assignee deleted (JERiveraMoya)

really need to take a look in more detail and not wait for a single person to get lost here :)

#29 Updated by JERiveraMoya about 4 years ago

Regarding comments in related bug https://bugzilla.suse.com/show_bug.cgi?id=1062806#c25 seems to be reasonable that the first passphrase does not appear in ppc64le due to a dedicated /boot partition is present in this architecture outside the encrypted volume. In aarch64 and x86_64 there is no dedicated /boot partition, therefore grub prompts passphrase as it wants to load kernel and initrd from the LVM encrypted partition.
Current jobs related with encryption are successful so it might not be needed further actions from our side on ppc in this ticket:
https://openqa.suse.de/tests/1268728
https://openqa.suse.de/tests/1268722

#30 Updated by jorauch about 4 years ago

  • Subject changed from [sle][functional][ppc64le] test fails in grub_test - stuck in grub screen, expected encrypted password prompt (was: boots completely) to [sle][functional][ppc64le][hard] test fails in grub_test - stuck in grub screen, expected encrypted password prompt (was: boots completely)
  • Description updated (diff)

What is the current state of this ticket, I see that it works for ppc, but ARM is still broken

#31 Updated by jorauch about 4 years ago

  • Description updated (diff)

#32 Updated by SLindoMansilla about 4 years ago

  • Assignee set to SLindoMansilla

Further investigation is needed here. Collecting important information here.

SLE 12-SP3 aarch
https://openqa.suse.de/tests/1057976#step/partitioning_lvm/4
https://openqa.suse.de/tests/1057976#step/grub_test/1 no password asked
https://openqa.suse.de/tests/1057976#step/boot_encrypt/1 password asked

SLE 12-SP3 ppc
https://openqa.suse.de/tests/1058033#step/partitioning_lvm/4
https://openqa.suse.de/tests/1058033#step/grub_test/1 password asked
https://openqa.suse.de/tests/1058033#step/boot_encrypt/1 password asked

SLE 12-SP3 x86
https://openqa.suse.de/tests/1058314#step/partitioning_lvm/4
https://openqa.suse.de/tests/1058314#step/grub_test/1 no password asked
https://openqa.suse.de/tests/1058314#step/boot_encrypt/1 password asked

SLE 15 349.1 aarch
https://openqa.suse.de/tests/1268571#step/partitioning_lvm/7
https://openqa.suse.de/tests/1268571#step/grub_test/1 password asked
N/A
• openQA test fails in boot_encrypt - AArch64 cannot boot from encrypted LVM - https://bugzilla.suse.com/show_bug.cgi?id=1062806
• passphrase for the password is asked in text, instead of gui on arm and x86_64 due to dropped plymouth - https://bugzilla.suse.com/show_bug.cgi?id=1067481

SLE 15 349.1 ppc
https://openqa.suse.de/tests/1268728#step/partitioning_lvm/7
https://openqa.suse.de/tests/1268728#step/grub_test/1 no password asked
https://openqa.suse.de/tests/1268728#step/boot_encrypt/1 password asked

SLE 15 349.1 x86
https://openqa.suse.de/tests/1269048#step/partitioning_lvm/7
https://openqa.suse.de/tests/1269048#step/grub_test/1 password asked
https://openqa.suse.de/tests/1269048#step/boot_encrypt/1 password asked
passphrase is asked twice when encrypt root partition on arm and x86_64 - https://bugzilla.suse.com/show_bug.cgi?id=1067479

#33 Updated by SLindoMansilla almost 4 years ago

  • Description updated (diff)
  • Status changed from In Progress to Feedback

On aarch64 and x86_64 the behavior is expected because of the requirements:

But it is not wanted, so a bug is filed: https://bugzilla.suse.com/show_bug.cgi?id=1067479


On ppc64 the behavior is not wanted, because we want the boot partition to also be encrypted, so bug is filed: https://bugzilla.suse.com/show_bug.cgi?id=1070139


Then a new feature is expected for SLE 15 to only need to type the password once, based on: https://fate.suse.com/320901

#34 Updated by SLindoMansilla almost 4 years ago

  • Description updated (diff)

#35 Updated by SLindoMansilla almost 4 years ago

  • Parent task set to #28504

#36 Updated by SLindoMansilla almost 4 years ago

  • Description updated (diff)
  • Status changed from Feedback to Resolved

At the moment, all related product bugs are properly filed. There is no more work from QA to be done from this ticket.
Acceptance criteria for this planned ticket added and is already fulfilled.

Please, find next steps on parent task: https://progress.opensuse.org/issues/28504

#37 Updated by SLindoMansilla almost 4 years ago

  • Description updated (diff)

Also available in: Atom PDF