Project

General

Profile

Actions

action #166802

open

openQA Project (public) - coordination #110833: [saga][epic] Scale up: openQA can handle a schedule of 100k jobs with 1k worker instances

openQA Project (public) - coordination #139010: [epic] Long OSD ppc64le job queue

Recover worker37, worker38, worker39 size:S

Added by okurz 3 months ago. Updated about 1 month ago.

Status:
Blocked
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
QA (public, currently private due to #173521) - future
Start date:
Due date:
% Done:

0%

Estimated time:
Tags:

Description

Motivation

After #139103 we should ensure that all remaining currently offline machines in PRG2 oQA infra are up and running again

Acceptance criteria

  • AC1: All w37-w39 run OSD production jobs
  • AC2: non-x86_64, non-qemu jobs are still executed and not starved out by too many x86_64

Suggestions

  • Take care to apply the workarounds from #157975-12 to prevent accidental distribution upgrades
  • Read what was done in #139103, bring up all w37-w39 again into production
  • Monitor for the impact on qemu_ppc64le job age as well as other non-x86_64, non-qemu jobs

Related issues 7 (2 open5 closed)

Related to openQA Infrastructure (public) - action #157726: osd-deployment | Failed pipeline for master (worker3[6-9].oqa.prg2.suse.org)Resolvedokurz2024-03-18

Actions
Related to openQA Infrastructure (public) - action #167057: Run more standard, qemu OSD openQA jobs in CC-compliant PRG2 and none in NUE2 size:SResolvedokurz2024-09-19

Actions
Related to openQA Project (public) - action #134924: Websocket server overloaded, affected worker slots shown as "broken" with graceful disconnect in workers tableNew2023-08-31

Actions
Related to openQA Infrastructure (public) - action #167081: test fails in support_server/setup on osd worker37 size:SResolvedmkittler2024-09-192024-10-09

Actions
Related to openQA Project (public) - action #157690: Simple global limit of registered/online workers size:MResolvedmkittler2024-03-21

Actions
Related to openQA Project (public) - action #168178: Limit connected online workers based on websocket+scheduler load size:MWorkable

Actions
Copied from openQA Infrastructure (public) - action #139103: Long OSD ppc64le job queue - Decrease number of x86_64 worker slots on osd to give ppc64le jobs a better chance to be assigned jobs size:MResolvedokurz2023-11-04

Actions
Actions

Also available in: Atom PDF