Project

General

Profile

Actions
Copy link

action #88189

closed

Firmware upgrade of qa-power8-4.qa.suse.de

Added by nicksinger almost 4 years ago. Updated over 3 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
nicksinger
Category:
-
Target version:
openQA Project - Ready
Start date:
2021-01-25
Due date:
% Done:

0%

Estimated time:

Description

Motivation

We were told that we need to apply the most recent firmware on machines which might help with some problems

I upgraded qa-power8-4 to the newest firmware version available at Fix-Central. The machine is according to /proc/cpuid a 8348-21C-machine. I Flashed the hpm package according to the docs https://delivery04.dhe.ibm.com/sar/CMA/SFA/08ct3/0/8348_820.1923.20190613n.html#__RefHeading___Toc1210_1053759979 (See section 7.2). Afterwards I boot up the system again just to see:

5.83228|System shutting down with error status 0x90FF0003

Several resets of the system and BMC I don't see any message anymore. The webinterface reports errors with the "sensors" "OCC Active", "Boot Count" and "BIOS Golden Side".

Actions
Copy link
 #1

Updated by nicksinger almost 4 years ago

hpm check indicates that the BIOS seems to be untouched? (even though the previous "hpm upgrade" told me it updates it).

workstation ~ ยป ipmitool -I lanplus -C 3 -H qa-power8-4.qa.suse.de -U ADMIN -P admin hpm check ~/Downloads/8348_820.1923.20190613n_update.hpm

PICMG HPM.1 Upgrade Agent 1.0.9: 

Validating firmware image integrity...OK
Performing preparation stage...OK

Comparing Target & Image File version
--------------------------------------------------------------------------
|ID  | Name        |                     Versions                        |
|    |             |     Active      |     Backup      |      File       |
--------------------------------------------------------------------------
|*^ 2|BIOS         |   1.12 60000000 | ---.-- -------- |   1.12 96000000 |
|*  0|BOOT         |   2.16 51220300 | ---.-- -------- |   2.16 51220300 |
|*  1|APP          |   2.16 51220300 | ---.-- -------- |   2.16 51220300 |
--------------------------------------------------------------------------
(*) Component requires Payload Cold Reset
(^) Indicates component would be upgraded
Actions
Copy link
 #2

Updated by nicksinger almost 4 years ago

Flashing the same file resulted into a bootable system. Unfortunately the system still don't seem to find its OS in petitboot (as tracked in https://progress.opensuse.org/issues/81058). I will see if it somehow possible to update petitboot too

Actions
Copy link
 #3

Updated by nicksinger almost 4 years ago

I'm building the most recent version of openPOWER (https://github.com/open-power/op-build) with power8 support for "habanero" (the codename for the used platform in power8-4) now. This should include the newest available versions for skiboot, skiroot and petitboot (https://wiki.raptorcs.com/wiki/OpenPOWER_Firmware#Process has a nice overview how the different loaders interact with each other). Also a recommended read in that regard: https://sthbrx.github.io/blog/2016/05/13/tell-me-about-petitboot/ (how skiboot/skiroot and petitboot are linked)

Actions
Copy link
 #4

Updated by nicksinger almost 4 years ago

From "Petitboot System Information" before flashing the new openPOWER firmware:

 System type: 8348-21C
 System id:   684F75A

 Primary platform versions:
        IBM-habanero-OP8_v1.12_2.96
        op-build-v2.3-5-g98aa884
        buildroot-2019.02.1-16-ge01dcd0
        skiboot-v6.3.1
        hostboot-p8-c893515-pd6f049d
        occ-p8-a2856b7
        linux-5.0.7-openpower1-p8e31f00
        petitboot-v1.10.3
        machine-xml-c3998a5
        hostboot-binaries-hw041519a.opv23
        capp-ucode-p9-dd2-v4

BMC current side:
        Device ID: 0x20
        Device Rev: 0x1
        Firmware version: 2.16.5bff0000
        IPMI version: 2

 BMC golden side:
        Device ID: 0x20
        Device Rev: 0x1
        Firmware version: 2.08.d4470100
        IPMI version: 2

 Storage devices
 sda1:
  UUID:       f5de0a79-bfa8-41c9-ba86-c0d2df739acc
  mounted at: /var/petitboot/mnt/dev/sda1

sdb2:
  UUID:       eebe647f-e867-416e-a0fa-7a6732bfcf9d
  mounted at: /var/petitboot/mnt/dev/sdb2


 Management (BMC) interface
  MAC:  a0:42:3f:32:b4:71
Actions
Copy link
 #5

Updated by nicksinger almost 4 years ago

hrm, flashing the openPOWER firmware with pflash did not succeed as the kernel reports an IO issue as soon as you try to erase the old stuff

Actions
Copy link
 #6

Updated by nicksinger almost 4 years ago

  • Status changed from In Progress to Feedback

I've sent out a mail onto the openpower mailinglist asking for help. You can see a message of what I wrote on osd-admins@suse.de. It waits for moderators approval now.

Actions
Copy link
 #7

Updated by okurz almost 4 years ago

  • Target version set to Ready
Actions
Copy link
 #8

Updated by nicksinger over 3 years ago

I got quite a few answers from the list. Unfortunately none of them really worked for me. The best Idea I got from it was to use the bmc system (you can ssh onto it) to flash the firmware. Unfortunately it is some old arm (ARM926EJ-S rev 5 (v5l)) with an ancient glibc (libc-2.11.3.so) and I was absolutely not able to get the pflash utility (cross-)compiled for it. Somebody in the mail thread offered me a binary download in an attachment which is a little bit too shady for me to execute.

So I'm out of ideas here. Anybody else has an idea how we could get a running version for pflash on that old system?
The code to compile pflash can be found here: https://github.com/open-power/skiboot/tree/master/external/pflash

Actions
Copy link
 #10

Updated by okurz over 3 years ago

  • Description updated (diff)
  • Status changed from Feedback to Rejected

As discussed with nsinger we are not aware of further problems after we fixed the boot problems software side in #68053 so we can also close this ticket as rejected as we did not conduct the firmware upgrades.

Actions
Copy link

Also available in: Atom PDF