Project

General

Profile

Actions
Copy link

action #158910

open

coordination #110833: [saga][epic] Scale up: openQA can handle a schedule of 100k jobs with 1k worker instances

coordination #158110: [epic] Prevent worker overload

typing issue on ppc64 worker - reconsider number of worker instances in particular on ppc64le kvm tests size:M

Added by okurz 7 months ago. Updated 5 months ago.

Status:
Blocked
Priority:
Low
Assignee:
okurz
Category:
Feature requests
Target version:
QA - Tools - Next
Start date:
Due date:
% Done:

0%

Estimated time:
Tags:
infra

Description

Motivation

We observed that e.g. petrol is running 8 openQA worker instance which with 16 logical cores leaves no headroom for the system. Probably we need to lower that number

Acceptance criteria

  • AC1: diesel+petrol+mania consistently do not alert on too high system load
  • AC2: diesel+petrol+mania use an efficient number of instances, i.e. the highest number possible

Suggestions

  • Try a lower load limit threshold
  • Reduce number of instances
  • Monitor

Files

Screenshot_20240421_184442_system_load_mania_load_limit_10_in_action.png (407 KB) Screenshot_20240421_184442_system_load_mania_load_limit_10_in_action.png okurz, 2024-04-21 17:13

Related issues 1 (0 open — 1 closed)

Copied from openQA Project - action #158125: typing issue on ppc64 worker - only pick up (or start) new jobs if CPU load is below configured threshold size:MResolvedmkittler

Actions
Actions
Copy link

Also available in: Atom PDF