coordination #80142
open[saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
73%
Description
Motivation¶
Nowadays a container based deployment becomes industry standard which we should fully support and prominently feature as supported both for simple single-instance setups of individual persons as well as multi-node setups in clusters.
Also, single, production instances of the webui can cause longer downtimes and make upgrades of OS more risky, e.g. when we do not have management access to VMs that might fail to reboot. Also, load-balancing can help as well as having switch-over deployments possible for easier testing, staging, etc.
Acceptance criteria¶
- AC1: an openQA infrastructure deployed on kubernetes is part of our continuous testing setup
- AC2: documentation exists how to setup redundant load-balancing infrastructures
- AC3: The support for openQA on container management frameworks is prominently presented
- AC4: documentation exists for simple, personal single-instance setups, e.g. "get your openQA tests to run in less than 5 minutes"
Suggestions¶
Based on the spike conducted in #69355 we can streamline the support, add documentation, introduce proper testing, consider running that setup as part of our DevOps structure, etc.
state-of-the-art is k8s so we should aim for that. Maybe a "docker compose" file is a good intermediate step, then k8s with a helm chart, potentially also some setup based on gitlab, see
https://docs.gitlab.com/ee/ci/environments/incremental_rollouts.html#blue-green-deployment
Subtasks 154 (44 open — 110 closed)
Updated by okurz almost 4 years ago
- Subject changed from [saga][epic] redundant/load-balancing webui deployments of openQA to [saga][epic] redundant/load-balancing webui deployments of openQA (container on kubernetes)
- Status changed from Workable to Blocked
- Assignee set to okurz
I created this saga as apparently the whole "container" feature block was not apparent enough for some stakeholders.
Some subtasks are there, ready to be worked on. Setting to "Blocked" by subtasks.
Updated by okurz almost 4 years ago
- Subject changed from [saga][epic] redundant/load-balancing webui deployments of openQA (container on kubernetes) to [saga][epic] Scale out openQA: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
- Description updated (diff)
Updated by ilausuch almost 4 years ago
- Related to action #80466: docker: Base the webUI and worker Dockerfiles in Tumbleweed added
Updated by livdywan almost 4 years ago
- AC4: documentation exists for simple single-instance setups, e.g. "get your openQA tests to run in less than 5 minutes"
This seems oddly phrased considering how the Motivation states:
Single instances of the webui can cause longer downtimes and make upgrades of OS more risky
I would expect that as a side-effect of using kubernetes users decide what resources they want to use.
Updated by okurz over 3 years ago
- Description updated (diff)
cdywan wrote:
- AC4: documentation exists for simple single-instance setups, e.g. "get your openQA tests to run in less than 5 minutes"
This seems oddly phrased considering how the Motivation states:
Single instances of the webui can cause longer downtimes and make upgrades of OS more risky
I would expect that as a side-effect of using kubernetes users decide what resources they want to use.
Right. I rephrased it so that it's clear in the motivation we mean "single, not-load-balanced production instances" in contrast to "personal instances that are fast to setup"
Updated by livdywan over 3 years ago
- Related to action #89749: containers: Allow to the user to choose the source of images in the docker-compose added
Updated by okurz over 3 years ago
- Related to deleted (action #89749: containers: Allow to the user to choose the source of images in the docker-compose)
Updated by okurz over 3 years ago
- Subject changed from [saga][epic] Scale out openQA: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes to [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
Updated by mkittler almost 3 years ago
- Related to action #92893: containers, docker-compose: Ensure that the scheduler can connect to the websockets container size:M added
Updated by okurz over 2 years ago
- Related to action #110497: Minion influxdb data causing unusual download rates size:M added
Updated by okurz almost 2 years ago
- Copied to coordination #121723: [saga][epic] Scale out: Future uses of on-premise cloud or public cloud added
Updated by okurz over 1 year ago
- Related to coordination #80150: [epic] Scale out openQA: Easier openQA setup added