Project

General

Profile

Actions

action #109055

closed

Broken workers alert

Added by livdywan over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Start date:
2022-03-28
Due date:
% Done:

0%

Estimated time:

Description

Observation

Number of broken workers 2.000

Workers on OSD shows:

  • powerqaworker-qam-1:1 powerqaworker-qam-1 qemu_ppc64le,qemu_ppc64le-large-mem,powerqaworker-qam-1 ppc64le Broken 1 25
  • powerqaworker-qam-1:5 powerqaworker-qam-1 qemu_ppc64le,power8,powerqaworker-qam-1 ppc64le Broken 1 25

The alert is currently active, and was also active twice earlier today, and thrice yesterday.

Rollback steps

  • Unpause broken workers alert

Related issues 2 (0 open2 closed)

Related to openQA Infrastructure (public) - action #108845: Network performance problems, DNS, DHCP, within SUSE QA network auto_review:"(Error connecting to VNC server.*qa.suse.*Connection timed out|ipmitool.*qa.suse.*Unable to establish)":retry but also other symptoms size:MResolvednicksinger2022-03-24

Actions
Related to openQA Project (public) - action #109734: Better way to prevent conflicts between openqa-worker@ and openqa-worker-auto-restart@ variants size:MResolvedjbaier_cz2022-04-09

Actions
Actions

Also available in: Atom PDF