action #39806

[OpenQA][workers] 4/6 64bit-ipmi workers are down

Added by xlai over 1 year ago. Updated over 1 year ago.

Status:ResolvedStart date:16/08/2018
Priority:UrgentDue date:
Assignee:nicksinger% Done:

0%

Category:-
Target version:Done
Difficulty:
Duration:

Description

On openqa.suse.de, only 2 64bit-ipmi workers are online, and they are openqaw1:1 and openqaworker2:24. The other 4 64bit-ipmi are down, openqaw1:2 and openqaworker2:23, openqaworker2:25, openqaworker2:26.

Because 4 workers are offline, virtualization jobs are quite delayed, still 22 jobs(nearly half) are untested yet.

Can anyone help to recover those workers?

History

#1 Updated by xlai over 1 year ago

  • Priority changed from Normal to Urgent

#2 Updated by szarate over 1 year ago

  • Target version set to Current Sprint

I think this is related to the network outtage as we still have some hosts that are still not alive. We' ll check once infra mentions that everything is back

#3 Updated by nicksinger over 1 year ago

  • Assignee set to nicksinger

openqaw1:2 (quinn) is back again and jobs are running (most likely caused by network outage)

#4 Updated by nicksinger over 1 year ago

  • Status changed from New to In Progress

Also checked the other workers. All of them currently have a job they are working on. Closing as this is hopefully resolved now.

#5 Updated by nicksinger over 1 year ago

  • Status changed from In Progress to Resolved

#6 Updated by xlai over 1 year ago

@nicksinger, @szarat, thanks for the help! Yes, all the workers are back now.

#7 Updated by coolo over 1 year ago

  • Target version changed from Current Sprint to Done

Also available in: Atom PDF