Project

General

Profile

Actions
Copy link

coordination #157669

open

coordination #110833: [saga][epic] Scale up: openQA can handle a schedule of 100k jobs with 1k worker instances

websockets+scheduler improvements to support more online worker instances

Added by okurz about 1 year ago. Updated 14 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
QA (public) - future
Start date:
2023-08-31
Due date:
% Done:

50%

Estimated time:
(Total: 0.00 h)

Subtasks 8 (5 open — 3 closed)

action #134924: Websocket server overloaded, affected worker slots shown as "broken" with graceful disconnect in workers tableNew2023-08-31

Actions
action #157675: Optimize openqa-scheduler database queries, e.g. "SELECT value FROM worker_properties..."New2024-03-21

Actions
action #157681: Profiling using NYTProf for openqa-websockets and openqa-schedulerNew2024-03-21

Actions
action #157684: cycle execution health check in openqa-schedulerNew2024-03-21

Actions
action #157690: Simple global limit of registered/online workers size:MResolvedmkittler2024-03-21

Actions
openQA Infrastructure (public) - action #167557: OSD not starting new jobs on 2024-09-28 due to >1k worker instances connected, overloading websocket serverResolvedokurz2024-09-28

Actions
action #168178: Limit connected online workers based on websocket+scheduler load size:MIn Progressmkittler

Actions
action #168502: Check for high websockets load on o3 2024-10-20Resolvedokurz2024-10-20

Actions

Related issues 2 (0 open — 2 closed)

Related to openQA Infrastructure (public) - action #157666: OSD unresponsive and then not starting any more jobs on 2024-03-21Resolvedokurz2024-03-12

Actions
Related to openQA Infrastructure (public) - action #157726: osd-deployment | Failed pipeline for master (worker3[6-9].oqa.prg2.suse.org)Resolvedokurz2024-03-18

Actions
Actions
Copy link

Also available in: Atom PDF