Project

General

Profile

action #89752

coordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes

coordination #89842: [epic] Scalable and streamlined docker-compose based openQA setup

containers: Add a worker service as part of the docker-compose

Added by ilausuch 8 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2021-03-09
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

To have a complete openQA deployment using the docker-compose at least one worker should be created

Acceptance Criteria

  • AC 1: Worker service is part of the compose workflow
  • AC 2: The number of workers is configurable.

Suggestions


Related issues

Related to openQA Project - action #76978: How to run an openQA test in 5 minutesNew2020-11-04

History

#1 Updated by cdywan 8 months ago

  • Description updated (diff)
  • Status changed from New to Workable

#2 Updated by ilausuch 8 months ago

  • Description updated (diff)

#3 Updated by cdywan 8 months ago

  • Related to action #76978: How to run an openQA test in 5 minutes added

#4 Updated by okurz 8 months ago

  • Status changed from Workable to New
  • Target version set to future
  • Parent task set to #89842

So far I can't see this ticket as "Workable" as part of the ACs reads more like "tasks to do", not real acceptance criteria. Please revisit the description.

#5 Updated by cdywan 8 months ago

  • Description updated (diff)
  • Status changed from New to Workable

#6 Updated by ilausuch 8 months ago

  • Assignee set to ilausuch

#7 Updated by ilausuch 8 months ago

  • Status changed from Workable to In Progress

#9 Updated by okurz 8 months ago

Please stick to the plan and not arbitrarily pick up tickets in "future".

#10 Updated by openqa_review 8 months ago

  • Due date set to 2021-03-26

Setting due date based on mean cycle time of SUSE QE Tools

#11 Updated by okurz 8 months ago

  • Due date deleted (2021-03-26)
  • Status changed from In Progress to Workable
  • Assignee deleted (ilausuch)

As discussed in chat. We should focus on the tasks that are in our backlog for now.

#12 Updated by mkittler about 1 month ago

  • Status changed from Workable to Feedback
  • Assignee set to mkittler

I've just came across https://github.com/os-autoinst/openQA/pull/3781 because of the stale bot's comment.

With https://github.com/os-autoinst/openQA/pull/4160 I have already improved the web UI's and the worker's docker compose files so one can use them together. However, I refrained from adding the worker to the web UI docker-compose file itself (as it is suggested by this ticket) because I don't think that's something we generally like to have. It should be sufficient if both docker-compose setups can work together and see no need to tie them together that closely. To me it seems totally fine to start only the worker or only the web UI using docker-compose and doing so makes actually no difference on the surface, e.g. a "normal" worker can connect to a web UI started using docker-compose without any special considerations and vice versa. Before https://github.com/os-autoinst/openQA/pull/4160 the docker-compose setup was just too broken to make that work.

As documented the worker can be started using docker compose now and the number of replicas can be configured. So even though it is using a different docker-compose file than what's used for the web UI I would consider both ACs covered. There's still the "limitation" that all worker replicas use the same instance number so they are all configured the same way. However, maybe that's actually wanted here. At least too me it sounds like this kind of setup is more about redundancy of the same kind of worker than spawning many different ones.


I'm assigning this ticket to myself keeping it in feedback but if you agree that the ACs are implemented we can actually consider it resolved.

#13 Updated by okurz about 1 month ago

mkittler wrote:

There's still the "limitation" that all worker replicas use the same instance number so they are all configured the same way. However, maybe that's actually wanted here. At least too me it sounds like this kind of setup is more about redundancy of the same kind of worker than spawning many different ones.

And what would be the implications of this?

I'm assigning this ticket to myself keeping it in feedback but if you agree that the ACs are implemented we can actually consider it resolved.

I would be ok with this. ilausuch do you agree?

#14 Updated by mkittler about 1 month ago

And what would be the implications of this?

It means that all the worker "replicas" one would spawn are "of the same kind" (their instance number is equal and therefore they'll use the same section of the configuration file). So if one spawns 10 workers via OPENQA_WORKER_REPLICAS=10 all workers will use the config in the [1] section of workers.ini. I suppose I could tweak the setup to use the sections [1] to [10]. However, I'm not sure what's wanted here. "replica" sounds like "more of the same" so just using section [1] seems actually logical to me but I also see use cases for using the different sections and maybe some users would expect that. I assume if we change it to use different sections one can simply just put everything into the global section so we wouldn't really disturb any use cases by that.

#15 Updated by okurz about 1 month ago

  • Status changed from Feedback to Resolved
  • Target version changed from future to Ready

Discussed after daily 2021-09-23 and we agreed that the current state is good enough to resolve the ticket. Both ACs are fulfilled. Thanks.

Also available in: Atom PDF