Project

General

Profile

Actions
Copy link

action #56045

closed

[qe-core][functional][sporadic] command 'dasd_configure 0.0.0150 0' timed out at /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/bootloader_s390.pm

Added by JRivrain over 5 years ago. Updated almost 4 years ago.

Status:
Rejected
Priority:
High
Assignee:
SLindoMansilla
Category:
Bugs in existing tests
Target version:
SUSE QA (private) - Milestone 30
Start date:
2019-08-28
Due date:
% Done:

0%

Estimated time:
42.00 h
Difficulty:

Description

Observation

Test times out at dasd_configure. This happens not every time but quite often, since at least build 294.

openQA test in scenario sle-12-SP5-Server-DVD-s390x-btrfs_libstorage@s390x-zVM-hsi-l2 fails in
bootloader_start

Acceptance Criteria

AC1: remove workaround and set device down again after bug is fixed

Reproducible

Fails since (at least) Build 0296

Expected result

Last good: 0296 (or more recent)

Further details

Always latest result in this scenario: latest

Actions
Copy link
 #1

Updated by mgriessmeier over 5 years ago

  • Subject changed from [functional] command 'dasd_configure 0.0.0150 0' timed out at /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/bootloader_s390.pm to [functional][u] command 'dasd_configure 0.0.0150 0' timed out at /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/bootloader_s390.pm
  • Status changed from New to Workable
  • Target version set to Milestone 27
Actions
Copy link
 #2

Updated by SLindoMansilla over 5 years ago

  • Estimated time set to 42.00 h

It looks like a problem with the serial device.
I was able to reproduce it on my local openQA setup at home 4 of 4 times.

Actions
Copy link
 #3

Updated by SLindoMansilla over 5 years ago

Sometimes it works: https://openqa.suse.de/tests/3321190#step/bootloader_s390/36
Sometimes it fails later: https://openqa.suse.de/tests/3321149#step/bootloader_s390/46
Does someone know how to debug what's happening on the serial device?
As this kind of failure always became the 'hated by all', we should document this for future references.

Actions
Copy link
 #4

Updated by SLindoMansilla over 5 years ago

  • Blocks action #48434: [functional][u] test Tumbleweed s390x again added
Actions
Copy link
 #5

Updated by SLindoMansilla over 5 years ago

  • Status changed from Workable to In Progress
  • Assignee set to SLindoMansilla
Actions
Copy link
 #8

Updated by mgriessmeier over 5 years ago

  • Status changed from Rejected to In Progress
  • Target version changed from Milestone 27 to Milestone 28
Actions
Copy link
 #9

Updated by mgriessmeier over 5 years ago

  • Assignee changed from SLindoMansilla to mgriessmeier
Actions
Copy link
 #11

Updated by mgriessmeier over 5 years ago

32 passed, 18 failed -> 36% failure rate in https://openqa.suse.de/tests/overview?version=12-SP5&build=mgriessmeier_poo56045&distri=sle

I will try to improve it - or better find the root cause and fix it

Actions
Copy link
 #12

Updated by SLindoMansilla over 5 years ago

Thanks! I actually have no idea how to fix it.

Actions
Copy link
 #13

Updated by mgriessmeier over 5 years ago

filed product bug https://bugzilla.suse.com/show_bug.cgi?id=1151394 against 12SP5, also happens on 15SP2 (will clarify if I should clone it)
will implement softfail workaround tomorrow

Actions
Copy link
 #14

Updated by okurz over 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: btrfs_libstorage@s390x-zVM-hsi-l2
https://openqa.suse.de/tests/3418584

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #16

Updated by mgriessmeier over 5 years ago

  • Description updated (diff)
  • Status changed from In Progress to Blocked

added AC to remove workaround again

Actions
Copy link
 #17

Updated by mgriessmeier about 5 years ago

  • Target version changed from Milestone 28 to Milestone 30

Workaround is still triggered sometimes
revisit next week

Actions
Copy link
 #18

Updated by mgriessmeier about 5 years ago

  • Assignee changed from mgriessmeier to SLindoMansilla
Actions
Copy link
 #19

Updated by SLindoMansilla about 5 years ago

Trying Berthold's suggestion: https://bugzilla.opensuse.org/show_bug.cgi?id=1156047#c5

hardened_usercopy=off
Actions
Copy link
 #20

Updated by SLindoMansilla about 5 years ago

  • Status changed from Blocked to In Progress
Actions
Copy link
 #22

Updated by SLindoMansilla about 5 years ago

Verified on OSD: https://openqa.suse.de/tests/3789298

Added WORKAROUND_BUGS=bsc1156047 to machine settings in OSD

  • s390x-zVM-ctc
  • s390x-zVM-hsi-l2
  • s390x-zVM-hsi-l3
  • s390x-zVM-vswitch-l2
Actions
Copy link
 #23

Updated by SLindoMansilla about 5 years ago

Verified on O3: https://openqa.opensuse.org/tests/1144907#step/bootloader_s390/45

Added WORKAROUND_BUGS=bsc1156047 to machine settings in OSD

  • s390x

(we should change the machine name to match their OSD equivalent)

Actions
Copy link
 #25

Updated by mgriessmeier about 5 years ago

hmm guess the setting was added to the machine after the job ran

Actions
Copy link
 #26

Updated by SLindoMansilla about 5 years ago

JRivrain wrote:

Seems to still happen :( https://openqa.suse.de/tests/3803881#step/bootloader_start/48

Sorry, my mistake, I didn't notice that there was a second page on the machines view with the machine this job uses.
Setting added.

Actions
Copy link
 #28

Updated by SLindoMansilla about 5 years ago

mgriessmeier wrote:

still happens :(
https://openqa.suse.de/tests/3822768#step/bootloader_s390/47

I realized that it is sporadic.

Actions
Copy link
 #29

Updated by okurz about 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: textmode@s390x-zVM-vswitch-l3
https://openqa.suse.de/tests/3869904

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #30

Updated by openqa_review about 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: textmode@s390x-zVM-vswitch-l3
https://openqa.suse.de/tests/3869904

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #31

Updated by SLindoMansilla about 5 years ago

  • Subject changed from [functional][u] command 'dasd_configure 0.0.0150 0' timed out at /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/bootloader_s390.pm to [functional][u][sporadic] command 'dasd_configure 0.0.0150 0' timed out at /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/bootloader_s390.pm
  • Description updated (diff)
  • Status changed from In Progress to New
  • Assignee deleted (SLindoMansilla)
Actions
Copy link
 #32

Updated by SLindoMansilla about 5 years ago

  • Assignee set to mgriessmeier
Actions
Copy link
 #33

Updated by mgriessmeier about 5 years ago

  • Status changed from New to Blocked

there was improved debug output PR by Jozef: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9664

still considering this as blocked by bsc#1156047: https://bugzilla.opensuse.org/show_bug.cgi?id=1156047

Actions
Copy link
 #34

Updated by SLindoMansilla almost 5 years ago

  • Blocks deleted (action #48434: [functional][u] test Tumbleweed s390x again)
Actions
Copy link
 #36

Updated by okurz almost 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: default@s390x-zVM-vswitch-l3
https://openqa.suse.de/tests/4028628

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #37

Updated by okurz almost 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: default@s390x-zVM-vswitch-l3
https://openqa.suse.de/tests/4028628

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #38

Updated by openqa_review almost 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: default@s390x-zVM-vswitch-l3
https://openqa.suse.de/tests/4093600

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #39

Updated by okurz almost 5 years ago

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: allpatterns@s390x-zVM-vswitch-l3
https://openqa.suse.de/tests/4170908

To prevent further reminder comments one of the following options should be followed:

  1. The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
  2. The openQA job group is moved to "Released"
  3. The label in the openQA scenario is removed
Actions
Copy link
 #40

Updated by SLindoMansilla almost 5 years ago

  • Status changed from Blocked to Workable
  • Assignee changed from mgriessmeier to SLindoMansilla

change the command from assert_script_run to script_run and if return code > 0 => record_soft_failure

Actions
Copy link
 #41

Updated by tjyrinki_suse over 4 years ago

  • Subject changed from [functional][u][sporadic] command 'dasd_configure 0.0.0150 0' timed out at /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/bootloader_s390.pm to [qe-core][functional][sporadic] command 'dasd_configure 0.0.0150 0' timed out at /var/lib/openqa/cache/openqa.suse.de/tests/sle/tests/installation/bootloader_s390.pm
Actions
Copy link
 #42

Updated by SLindoMansilla almost 4 years ago

  • Status changed from Workable to Rejected

Bug fixed for SLE15-SP3 installer.

For release products:

Actions
Copy link

Also available in: Atom PDF