Project

General

Profile

Actions
Copy link

action #44432

closed

[sle][functional][u][ppc64le] test fails in bootloader - smt is still on after reboot on qa-power8-5

Added by zluo over 5 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Low
Assignee:
szarate
Category:
-
Target version:
SUSE QA - Milestone 21
Start date:
2018-11-28
Due date:
% Done:

0%

Estimated time:

Description

Compared with successful test run of a couple days ago, I found that installation image is not ready or guest is not initialized the display.
So needle inst-bootmenu cannot be matched.

Observation

openQA test in scenario sle-15-SP1-Installer-DVD-ppc64le-create_hdd_gnome@ppc64le fails in
bootloader

Reproducible

Fails since (at least) Build 104.1 (current job)

Expected result

Last good: 102.1 (or more recent)

Further details

Always latest result in this scenario: latest

Related issues 1 (0 open — 1 closed)

Related to openSUSE admin - tickets #25170: openQA ppc64le workers bad kvm setupResolvedokurz

Actions
Actions
Copy link
 #1

Updated by okurz over 5 years ago

  • Subject changed from [sle][functional][u] test fails in bootloader - Guest hast not initialized the display (yet) to [sle][functional][u][ppc64le] test fails in bootloader - Guest hast not initialized the display (yet)

Hm, we only wait 30s. I suggest to bump the timeout but selectively, e.g. only for ppc64le under the assumption that the ppc64le bootloader is "special".

Actions
Copy link
 #2

Updated by okurz over 5 years ago

  • Priority changed from Urgent to High
  • Target version set to future

rescheduled as https://openqa.suse.de/tests/2286135, which passed that step already. As retriggering works as workaround I lower the prio to "High".

Actions
Copy link
 #3

Updated by riafarov over 5 years ago

I guess we might reconsider priority, ~20 jobs failed in YaST job group, for both scenarios, booting into the iso or qcow image.

Actions
Copy link
 #4

Updated by okurz over 5 years ago

and the workaround does not work for you?

Actions
Copy link
 #5

Updated by szarate over 5 years ago

Well looking at the logs helped a little bit...

However, smt is disabled for that worker... 

[2018-11-28T08:52:52.582 UTC] [debug] QEMU: Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
[2018-11-28T08:52:52.582 UTC] [debug] QEMU: error: kvm run failed Device or resource busy
[2018-11-28T08:52:52.582 UTC] [debug] QEMU: This is probably because your SMT is enabled.
[2018-11-28T08:52:52.582 UTC] [debug] QEMU: VCPU can only run on primary threads with all secondary threads offline.
[2018-11-28T08:52:52.582 UTC] [debug] QEMU: NIP 0000000000000100   LR 0000000000000000 CTR 0000000000000000 XER 0000000000000000 CPU#0
[2018-11-28T08:52:52.582 UTC] [debug] QEMU: MSR 8000000000000000 HID0 0000000000000000  HF 8000000000000000 iidx 3 didx 3
[2018-11-28T08:52:52.582 UTC] [debug] QEMU: TB 00000000 00000000 DECR 00000000
[2018-11-28T08:52:52.582 UTC] [debug] QEMU: GPR00 0000000000000000 0000000000000000 0000000000000000 000000003fef0000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: GPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: GPR08 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: GPR12 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: GPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: GPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: GPR24 0000000000000000 0000000000000000 0000000000000000 0000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: 000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: GPR28 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: CR 00000000  [ -  -  -  -  -  -  -  -  ]             RES ffffffffffffffff
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: FPR00 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: FPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.583 UTC] [debug] QEMU: FPR08 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: FPR12 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: FPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: FPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: FPR24 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: FPR28 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: FPSCR 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU:  SRR0 0000000000000000  SRR1 0000000000000000    PVR 00000000004d0200 VRSAVE 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: SPRG0 0000000000000000 SPRG1 0000000000000000  SPRG2 0000000000000000  SPRG3 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: SPRG4 0000000000000000 SPRG5 0000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: 000000000  SPRG6 0000000000000000  SPRG7 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU: HSRR0 0000000000000000 HSRR1 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU:  CFAR 0000000000000000
[2018-11-28T08:52:52.584 UTC] [debug] QEMU:  LPCR 000000000004f001
[2018-11-28T08:52:52.584 UTC] [debug] QEMU:  SDR1 0000000000000005   DAR 0000000000000000  DSISR 0000000000000000
[2018-11-28T08:52:52.585 UTC] [debug] Snapshots are supported
Actions
Copy link
 #6

Updated by szarate over 5 years ago

  • Status changed from New to In Progress
  • QA-Power8-5-kvm was rebooted 3 days ago... this caused all the trouble, enabled the smt_off service, salt recipe fix WIP
Actions
Copy link
 #7

Updated by szarate over 5 years ago

  • Assignee set to szarate
Actions
Copy link
 #8

Updated by szarate over 5 years ago

  • Project changed from openQA Tests to openQA Infrastructure
  • Subject changed from [sle][functional][u][ppc64le] test fails in bootloader - Guest hast not initialized the display (yet) to [sle][functional][u][ppc64le] test fails in bootloader - smt is still on after reboot on qa-power8-5
  • Category deleted (Bugs in existing tests)
  • Priority changed from High to Low

Changing priority, main problem has been solved, salt recipe needs to be fixed

Actions
Copy link
 #9

Updated by szarate over 5 years ago

  • Target version changed from future to Milestone 21+
Actions
Copy link
 #10

Updated by okurz over 5 years ago

  • Target version changed from Milestone 21+ to Milestone 21
Actions
Copy link
 #11

Updated by szarate over 5 years ago

There's this PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6315 just starter to add failure detection but we need a proper ticket for adding the support for autoinst-log.txt

Actions
Copy link
 #12

Updated by szarate over 5 years ago

  • Status changed from In Progress to Feedback

MR created, waiting for approval :) ( I will just automerge later if nobody else does) https://gitlab.suse.de/openqa/salt-states-openqa/merge_requests/87

Actions
Copy link
 #13

Updated by okurz over 5 years ago

I merged, let's see if we receive any feedback, screams, fire, pitchforks, you know ;)

Actions
Copy link
 #14

Updated by okurz over 5 years ago

  • Related to tickets #25170: openQA ppc64le workers bad kvm setup added
Actions
Copy link
 #15

Updated by okurz over 5 years ago

If this works would be good to to evaluate regarding #25170 if we can do something for the ppc64le worker on o3

Actions
Copy link
 #16

Updated by szarate over 5 years ago

  • Status changed from Feedback to Resolved

So this is done...

QA-Power8-4-kvm.qa.suse.de:
    * smt_off.service - ppc64 set SMT off
       Loaded: loaded (/usr/lib/systemd/system/smt_off.service; enabled; vendor preset: disabled)
       Active: active (exited) since Wed 2018-12-05 13:08:02 CET; 4h 8min ago
     Main PID: 28449 (code=exited, status=0/SUCCESS)
        Tasks: 0 (limit: 512)
       CGroup: /system.slice/smt_off.service

    Dec 05 13:08:02 QA-Power8-4-kvm systemd[1]: Starting ppc64 set SMT off...
    Dec 05 13:08:02 QA-Power8-4-kvm systemd[1]: Started ppc64 set SMT off.
QA-Power8-5-kvm.qa.suse.de:
    * smt_off.service - ppc64 set SMT off
       Loaded: loaded (/usr/lib/systemd/system/smt_off.service; enabled; vendor preset: disabled)
       Active: active (exited) since Thu 2018-11-29 10:18:46 UTC; 6 days ago
     Main PID: 82083 (code=exited, status=0/SUCCESS)
        Tasks: 0 (limit: 512)
       CGroup: /system.slice/smt_off.service

    Nov 29 10:18:46 QA-Power8-5-kvm systemd[1]: Starting ppc64 set SMT off...
    Nov 29 10:18:46 QA-Power8-5-kvm systemd[1]: Started ppc64 set SMT off.
powerqaworker-qam-1:
    * smt_off.service - ppc64 set SMT off
       Loaded: loaded (/usr/lib/systemd/system/smt_off.service; enabled; vendor preset: disabled)
       Active: active (exited) since Wed 2018-12-05 13:09:49 CET; 4h 8min ago
     Main PID: 102735 (code=exited, status=0/SUCCESS)
        Tasks: 0 (limit: 512)
       CGroup: /system.slice/smt_off.service

    Dec 05 13:09:49 powerqaworker-qam-1 systemd[1]: Starting ppc64 set SMT off...
    Dec 05 13:09:49 powerqaworker-qam-1 systemd[1]: Started ppc64 set SMT off.
malbec.arch.suse.de:
    * smt_off.service - ppc64 set SMT off
       Loaded: loaded (/usr/lib/systemd/system/smt_off.service; enabled; vendor preset: disabled)
       Active: active (exited) since Wed 2018-12-05 13:09:58 CET; 4h 8min ago
     Main PID: 45744 (code=exited, status=0/SUCCESS)
        Tasks: 0 (limit: 512)
       CGroup: /system.slice/smt_off.service

    Dec 05 13:09:58 malbec systemd[1]: Starting ppc64 set SMT off...
    Dec 05 13:09:58 malbec systemd[1]: Started ppc64 set SMT off.
Actions
Copy link

Also available in: Atom PDF