Project

General

Profile

Actions
Copy link

action #89716

closed

[qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped

Added by dzedro about 3 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
dzedro
Category:
Bugs in existing tests
Target version:
-
Start date:
2021-03-09
Due date:
% Done:

100%

Estimated time:
Difficulty:

Description

Observation

aarch64 is slower and sometimes reboot after installation is not stopped [1],
because default 10 seconds countdown is missed.
To increase reboot_timeout linuxrc option [3][4][5] would have to be backported to 15 & a5 SP1
This option is implemented and used on 15sp2 [2]

[1] https://openqa.suse.de/tests/5621766#step/await_install/25
[2] https://openqa.suse.de/tests/5625237#step/await_install/22
[3] https://github.com/yast/yast-installation/pull/823
[4] https://github.com/yast/yast-yast2/pull/977
[5] https://github.com/openSUSE/installation-images/pull/344

openQA test in scenario sle-15-SP1-Server-DVD-Updates-aarch64-mru-install-minimal-with-addons@aarch64-virtio fails in
logs_from_installation_system

Reproducible

Fails since (at least) Build 20210308-2 (current job)

Expected result

Last good: 20210308-1 (or more recent)

Further details

Always latest result in this scenario: latest

Actions
Copy link
 #1

Updated by dzedro about 3 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by openqa_review about 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: qam-minimal+base
https://openqa.suse.de/tests/5718781

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #3

Updated by okurz about 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: qam-minimal+base
https://openqa.suse.de/tests/5718781

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #4

Updated by tjyrinki_suse about 3 years ago

  • Status changed from New to Workable
Actions
Copy link
 #5

Updated by dzedro almost 3 years ago

  • Status changed from Workable to In Progress
  • Assignee set to dzedro
Actions
Copy link
 #6

Updated by dzedro almost 3 years ago

Created PRs to Backport linuxrc option reboot_timeout to 15 SP1

https://github.com/openSUSE/installation-images/pull/505

Actions
Copy link
 #8

Updated by dzedro almost 3 years ago

  • Status changed from In Progress to Blocked

Waiting until the change is released.

Actions
Copy link
 #9

Updated by dzedro almost 3 years ago

  • Subject changed from [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped to [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped auto_review:"(?s)aarch64.*inst-console timed out":retry
Actions
Copy link
 #10

Updated by dzedro almost 3 years ago

  • Subject changed from [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped auto_review:"(?s)aarch64.*inst-console timed out":retry to [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped auto_review:"(?s)aarch64.*(inst-console|install-shell) timed out":retry
Actions
Copy link
 #11

Updated by openqa_review almost 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_hpc_textmode
https://openqa.suse.de/tests/6205212

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #12

Updated by okurz almost 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_hpc_textmode
https://openqa.suse.de/tests/6309273

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #13

Updated by openqa_review almost 3 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: qam-minimal+base
https://openqa.suse.de/tests/6397926

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The label in the openQA scenario is removed
Actions
Copy link
 #14

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_hpc_textmode
https://openqa.suse.de/tests/6587014

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The label in the openQA scenario is removed
Actions
Copy link
 #15

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_hpc_textmode
https://openqa.suse.de/tests/6639300

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The label in the openQA scenario is removed
Actions
Copy link
 #16

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_hpc_textmode
https://openqa.suse.de/tests/6639300

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The label in the openQA scenario is removed
Actions
Copy link
 #17

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_hpc_textmode
https://openqa.suse.de/tests/6639300

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The label in the openQA scenario is removed
Actions
Copy link
 #18

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_hpc_textmode
https://openqa.suse.de/tests/6639300

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #19

Updated by dzedro over 2 years ago

  • Subject changed from [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped auto_review:"(?s)aarch64.*(inst-console|install-shell) timed out":retry to [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped auto_review:"(?s)aarch64.*(inst-console|install-shell|root-console) timed out":retry
Actions
Copy link
 #20

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: offline_sles15sp1_ltss_media_basesys-srv-desk-dev-contm-lgm-py2-wsm_all_full
https://openqa.suse.de/tests/7494437

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #21

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: offline_sles15sp1_ltss_media_basesys-srv-desk-dev-contm-lgm-py2-wsm_all_full
https://openqa.suse.de/tests/7657498

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #22

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: online_sled15sp3_smt_basesys-desk-we-phub-python2_def_full_y
https://openqa.suse.de/tests/7742694

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #23

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: mru-install-minimal-with-addons
https://openqa.suse.de/tests/7837769

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #24

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: jeos-fips@aarch64-HD24G
https://openqa.opensuse.org/tests/2110648

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #25

Updated by openqa_review over 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: offline_sles15sp2_ltss_pscc_lp-basesys-srv-desk-dev-contm-lgm-py2-tsm-wsm_all_full
https://openqa.suse.de/tests/7982872

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #26

Updated by openqa_review about 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: security_389ds_sssd_client
https://openqa.opensuse.org/tests/2169877

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #27

Updated by openqa_review about 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: security_389ds_sssd_client
https://openqa.opensuse.org/tests/2187589

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234
Actions
Copy link
 #28

Updated by openqa_review about 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: create_hdd_hpc_textmode
https://openqa.suse.de/tests/8323256

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 56 days if nothing changes in this ticket.

Actions
Copy link
 #29

Updated by szarate about 2 years ago

  • Subject changed from [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped auto_review:"(?s)aarch64.*(inst-console|install-shell|root-console) timed out":retry to [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped auto_review:"(?s)aarch64.*(inst-console|install-shell|root-console) timed out after 180":retry

Changing the regex for now,auto_review:"(?s)aarch64.*(inst-console|install-shell|root-console) timed out":retry was mislabeled: https://openqa.suse.de/tests/8503278#step/check_logs/35

Actions
Copy link
 #30

Updated by szarate about 2 years ago

I hope the regex is correct, idk how to test it at this moment

Actions
Copy link
 #31

Updated by openqa_review almost 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: sles4sap_hana_node02@ppc64le-sap-qam
https://openqa.suse.de/tests/8648408#step/check_after_reboot#1/1

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 28 days if nothing changes in this ticket.

Actions
Copy link
 #32

Updated by openqa_review almost 2 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: sles4sap_hana_node02@ppc64le-sap-qam
https://openqa.suse.de/tests/8904553#step/check_after_reboot#1/1

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released" or "EOL" (End-of-Life)
  3. The bugref in the openQA scenario is removed or replaced, e.g. label:wontfix:boo1234

Expect the next reminder at the earliest in 60 days if nothing changes in this ticket.

Actions
Copy link
 #33

Updated by tinita about 1 year ago

I can see the following in the osd logs:

grep failed: cmd=>grep -qPzo '(?s)aarch64.*(inst-console|install-shell|root-console) timed out after 180' '/tmp/tmp.HKYSbhPnjQ'< output='grep: exceeded PCRE's backtracking limit'
Actions
Copy link
 #34

Updated by dzedro about 1 year ago

  • Subject changed from [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped auto_review:"(?s)aarch64.*(inst-console|install-shell|root-console) timed out after 180":retry to [qe-core][qem] Fix occasional failres on aarch64 due to reboot after installation not being stopped
  • Status changed from Blocked to Feedback

I didn't see this failure so often lately, I remove the auto_review and if failure appear again, regex can be refined or the issue resolved.

Actions
Copy link
 #35

Updated by dzedro about 1 year ago

  • Status changed from Feedback to Resolved
  • % Done changed from 0 to 100

15-SP1 does not support reboot_timeout, but it's passing, never versions use reboot_timeout
I didn't see this failure on 15-SP1

Actions
Copy link

Also available in: Atom PDF