action #89752
closedcoordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
coordination #89842: [epic] Scalable and streamlined docker-compose based openQA setup
containers: Add a worker service as part of the docker-compose
Description
Motivation¶
To have a complete openQA deployment using the docker-compose at least one worker should be created
Acceptance Criteria¶
- AC 1: Worker service is part of the compose workflow
- AC 2: The number of workers is configurable.
Suggestions¶
- See: https://github.com/os-autoinst/openQA/pull/3755
- Add a section for the worker service to
container/webui/docker-compose.yaml - Add a variable like
$WORKER_COUNT - Add a corresponding healthcheck
Updated by livdywan over 3 years ago
- Description updated (diff)
- Status changed from New to Workable
Updated by livdywan over 3 years ago
- Related to action #76978: How to run an openQA test in 5 minutes size:M added
Updated by okurz over 3 years ago
- Status changed from Workable to New
- Target version set to future
- Parent task set to #89842
So far I can't see this ticket as "Workable" as part of the ACs reads more like "tasks to do", not real acceptance criteria. Please revisit the description.
Updated by livdywan over 3 years ago
- Description updated (diff)
- Status changed from New to Workable
Updated by ilausuch over 3 years ago
- Status changed from Workable to In Progress
Updated by ilausuch over 3 years ago
Updated by okurz over 3 years ago
Please stick to the plan and not arbitrarily pick up tickets in "future".
Updated by openqa_review over 3 years ago
- Due date set to 2021-03-26
Setting due date based on mean cycle time of SUSE QE Tools
Updated by okurz over 3 years ago
- Due date deleted (
2021-03-26) - Status changed from In Progress to Workable
- Assignee deleted (
ilausuch)
As discussed in chat. We should focus on the tasks that are in our backlog for now.
Updated by mkittler about 3 years ago
- Status changed from Workable to Feedback
- Assignee set to mkittler
I've just came across https://github.com/os-autoinst/openQA/pull/3781 because of the stale bot's comment.
With https://github.com/os-autoinst/openQA/pull/4160 I have already improved the web UI's and the worker's docker compose files so one can use them together. However, I refrained from adding the worker to the web UI docker-compose file itself (as it is suggested by this ticket) because I don't think that's something we generally like to have. It should be sufficient if both docker-compose setups can work together and see no need to tie them together that closely. To me it seems totally fine to start only the worker or only the web UI using docker-compose and doing so makes actually no difference on the surface, e.g. a "normal" worker can connect to a web UI started using docker-compose without any special considerations and vice versa. Before https://github.com/os-autoinst/openQA/pull/4160 the docker-compose setup was just too broken to make that work.
As documented the worker can be started using docker compose now and the number of replicas can be configured. So even though it is using a different docker-compose file than what's used for the web UI I would consider both ACs covered. There's still the "limitation" that all worker replicas use the same instance number so they are all configured the same way. However, maybe that's actually wanted here. At least too me it sounds like this kind of setup is more about redundancy of the same kind of worker than spawning many different ones.
I'm assigning this ticket to myself keeping it in feedback but if you agree that the ACs are implemented we can actually consider it resolved.
Updated by okurz about 3 years ago
mkittler wrote:
There's still the "limitation" that all worker replicas use the same instance number so they are all configured the same way. However, maybe that's actually wanted here. At least too me it sounds like this kind of setup is more about redundancy of the same kind of worker than spawning many different ones.
And what would be the implications of this?
I'm assigning this ticket to myself keeping it in feedback but if you agree that the ACs are implemented we can actually consider it resolved.
I would be ok with this. @ilausuch do you agree?
Updated by mkittler about 3 years ago
And what would be the implications of this?
It means that all the worker "replicas" one would spawn are "of the same kind" (their instance number is equal and therefore they'll use the same section of the configuration file). So if one spawns 10 workers via OPENQA_WORKER_REPLICAS=10 all workers will use the config in the [1] section of workers.ini. I suppose I could tweak the setup to use the sections [1] to [10]. However, I'm not sure what's wanted here. "replica" sounds like "more of the same" so just using section [1] seems actually logical to me but I also see use cases for using the different sections and maybe some users would expect that. I assume if we change it to use different sections one can simply just put everything into the global section so we wouldn't really disturb any use cases by that.
Updated by okurz about 3 years ago
- Status changed from Feedback to Resolved
- Target version changed from future to Ready
Discussed after daily 2021-09-23 and we agreed that the current state is good enough to resolve the ticket. Both ACs are fulfilled. Thanks.