action #139100
closedopenQA Project - coordination #110833: [saga][epic] Scale up: openQA can handle a schedule of 100k jobs with 1k worker instances
openQA Project - coordination #139010: [epic] Long OSD ppc64le job queue
Long OSD ppc64le job queue - Move nue3 power8 machines to nue2
0%
Description
Motivation¶
Currently on OSD there is a longer job queue in particular for ppc64le for multiple reasons, see #139010. One idea to increase capacity is to move nue3 power8 machines to nue2
Acceptance criteria¶
- AC1: At least one more power8 machine is working on production jobs from FC Basement
Suggestions¶
- Evaluate based on https://netbox.suse.de/dcim/devices/?tag=move-to-marienberg-dr&tag=qe-lsg&manufacturer_id=138 which machine(s) to move to FC Basement
- Coordinate the move, e.g. ask BuildOps in particular wengel who helped in the past, or Facilities, e.g. ofecher
- Setup the machines as openQA worker hosts same as diesel+petrol
- Add to https://gitlab.suse.de/openqa/salt-pillars-openqa/-/blob/master/openqa/workerconf.sls with experimentation worker classes
- Confirm stability of the machines
- Add production worker classes
- Announce to stakeholders about increase of worker capacity
- Monitor performance and stability of machines
Updated by okurz 6 months ago
- Due date set to 2023-11-25
- Status changed from In Progress to Feedback
- Priority changed from Normal to Low
- Target version changed from Ready to Tools - Next
https://suse.slack.com/archives/C05UHQ49B7D/p1699015046191019
(Oliver Kurz) hi guys, how feasible would it be to move 1-2 machines from NUE3 "MB" to NUE2-FC_Basement?
(Wolfgang Engel) Hi, that should be doable. We will move stuff from Maxtorhof to NUE3 soon and can pick up machines from NUE3 for NUE2-FC. Or at least tell Olli and he can inform Opto-Copy
(Oliver Kurz) that would be cool. So to not duplicate work can we first try with you guys? So that would be "QA-Power8" https://netbox.suse.de/dcim/devices/3501/ and "QA-Power8-2" https://netbox.suse.de/dcim/devices/3598/ to move to FC Basement, or anywhere in FC where I have access to and I will move to FC Basement. Just tell me if it's not feasible for you and I will open a request with Facilities + Opto-Copy
(Wolfgang Engel) I will talk to Olli next week to set a date for the move of our stuff so I can tell him that two machines from NUE3 needs to be picked up and moved to NUE2-FC. I will get back to you next week when I talked to Olli (Oliver Fecher, Facilities)
Updated by okurz 6 months ago
- Copied to action #139103: Long OSD ppc64le job queue - Decrease number of x86_64 worker slots on osd to give ppc64le jobs a better chance to be assigned jobs size:M added
Updated by okurz 5 months ago
- Due date deleted (
2023-11-25) - Status changed from Feedback to Resolved
- Target version changed from Tools - Next to Ready
With mania in production I stated that this is less important to the point of not needed at all. So actually either a machine will pop up in FC Basement or not, fine etiher way :)