Project

General

Profile

Actions
Copy link

action #158709

open

openQA Project - coordination #110833: [saga][epic] Scale up: openQA can handle a schedule of 100k jobs with 1k worker instances

openQA Project - coordination #158110: [epic] Prevent worker overload

typing issue on ppc64 worker - with automatic CPU load based limiting in place let's increase the instances on mania again

Added by okurz 23 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
QA - future
Start date:
Due date:
% Done:

0%

Estimated time:
Tags:
performance, infra, load, optimization, tweaking

Description

Motivation

With #158125 we have CPU load based limiting in place so we can try to increase again the number of worker instances on mania while ensuring that no typing issues crop up.

Acceptance criteria

  • AC1: mania is having significantly more worker instances enabled again
  • AC2: No typing issues show up on mania
  • AC3: Based on results from AC1+AC2 tickets exists with plans for all other o3+osd workers

Suggestions

  • Look into the reduction of worker instances done in https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/759/diffs, increase to a value even higher than the original 30 while carefully monitoring that the CPU load based limiting should prevent a system overload and related typing issues in openQA jobs
  • Based on results from AC1+AC2 create according tickets with plans for all other o3+osd workers

Related issues 1 (0 open — 1 closed)

Copied from openQA Project - action #158125: typing issue on ppc64 worker - only pick up (or start) new jobs if CPU load is below configured threshold size:MResolvedmkittler

Actions
Actions
Copy link
 #1

Updated by okurz 23 days ago

  • Copied from action #158125: typing issue on ppc64 worker - only pick up (or start) new jobs if CPU load is below configured threshold size:M added
Actions
Copy link

Also available in: Atom PDF