Project

General

Profile

Actions
Copy link

action #94438

closed

OSD deployment fails at 2021-06-21 because ' openqaworker (arm-3 and arm-2) Minion did not return'

Added by Xiaojing_liu almost 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
okurz
Category:
-
Target version:
openQA Project - Ready
Start date:
2021-06-22
Due date:
% Done:

0%

Estimated time:

Description

Observation

OSD deployment failed, reason showed:

    openqaworker-arm-2.suse.de:
    Minion did not return. [Not connected]
    openqaworker-arm-3.suse.de:
    Minion did not return. [Not connected]

See details in: https://gitlab.suse.de/openqa/osd-deployment/-/jobs/466302

I have rebooted arm-2 by using ipmitool power cycle, but arm-3 cannot be connected with ipmitool

   #ipmitool -I lanplus -C 3 -H openqaworker-arm-3-ipmi.suse.de chassis power status
    Error: Unable to establish IPMI v2 / RMCP+ session

Acceptance criteria

  • AC1: OSD deployment continued
  • AC2: both openqaworker-arm-2 and openqaworker-arm-3 are online again

Related issues 1 (1 open — 0 closed)

Related to openQA Infrastructure - action #94399: No alert when arm workers are offline, alert if telegraf throws errors size:MWorkable2021-06-22

Actions
Actions
Copy link
 #1

Updated by Xiaojing_liu almost 3 years ago

  • Related to action #94399: No alert when arm workers are offline, alert if telegraf throws errors size:M added
Actions
Copy link
 #2

Updated by okurz almost 3 years ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Target version set to Ready
Actions
Copy link
 #3

Updated by okurz almost 3 years ago

  • Status changed from Workable to Resolved
  • Assignee set to okurz

I brought back workers manually now as the automatic recovery was broken at the time. Deployment continued and finished successfully.

Actions
Copy link

Also available in: Atom PDF