action #73450
closedcoordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
POC: Create openQA worker container image (feature)
Added by plorinc over 3 years ago. Updated over 3 years ago.
Description
User story¶
openQA worker is preinstalled with all it's dependencies (os-autoinst, etc.)
The worker has access to the test database
The worker can start and evaluate tests orchestrated by the openQA Web App
The worker can run tests for a simple package (e.g. vi)
Acceptance criteria¶
- AC1: Container started based on this image is able to run on the Docker engine
- AC2: Full Green Stack: All software components are available in the (open)SUSE repositories
- AC3: An automated way is provided to fill the database with necessary tables and schemas, and a predefined set of test data
- AC4: Non-static data is stored on container volumes, if possible
- AC5: Deliverable is an image stored in a registry, a Dockerfile (with instructions on how to create a container from the image) or a Compose file
Tasks¶
Further details¶
Updated by okurz over 3 years ago
- Description updated (diff)
- Due date set to 2020-10-31
- Status changed from New to Feedback
- Assignee set to okurz
- Target version set to Ready
In principle I see this already fulfilled by https://github.com/os-autoinst/openQA/blob/master/docker/worker/Dockerfile but I want to clarify that your request is completely covered.
- I assume when you say "SUSE repositories" that "openSUSE" is ok?
- Can you clarify on "AC3: An automated way is provided to fill the database with necessary tables and schemas, and a predefined set of test data": What do you want to achieve with that and what do you mean with "predefined set of test data" in particular?
- Can you clarify on "AC4: Non-static data is stored on container volumes, if possible": Do you mean something like an external temporary directory passed into the container as the worker's cache and/or pool directory?
Updated by okurz over 3 years ago
- Related to action #73447: POC: Create openQA Web Application container image (feature) added
Updated by okurz over 3 years ago
- Related to coordination #43706: [epic] Generate "download&use" docker image of openQA for SUSE QA added
Updated by okurz over 3 years ago
- Related to action #43715: Update upstream dockerfiles to provide an easy to use docker image of workers added
Updated by ilausuch over 3 years ago
This case is similar to my comment on #73447 and a PR that is currently WIP PR https://github.com/os-autoinst/openQA/pull/3475 that covers some ACs
Updated by livdywan over 3 years ago
plorinc wrote:
Acceptance criteria¶
- AC1: Container started based on this image is able to run on the Docker engine
This is covered by #43715 already.
- AC5: Deliverable is an image stored in a registry, a Dockerfile (with instructions on how to create a container from the image) or a Compose file
This is covered by #43718 already.
Updated by livdywan over 3 years ago
plorinc wrote:
Acceptance criteria¶
- AC2: Full Green Stack: All software components are available in the (open)SUSE repositories
What (open)SUSE repositories are we talking about here?
The images are based on Leap 15.1 (worker and webui respectively), and use these repos on top:
zypper ar -f obs://devel:openQA/openSUSE_Leap_15.1 openQA
zypper ar -f obs://devel:openQA:Leap:15.1/openSUSE_Leap_15.1 openQA-perl-modules
Updated by okurz over 3 years ago
@cdywan is that you trying to answer the question to plorinc or are you just adding additional information for the current state?
Updated by okurz over 3 years ago
- Due date deleted (
2020-10-31) - Status changed from Feedback to New
- Assignee deleted (
okurz) - Priority changed from Normal to Low
- Target version changed from Ready to future
@plorinc can you please look into the questions from #73450#note-1
Updated by okurz over 3 years ago
Well, I prefer to not regard this as duplicate as the specific stakeholder expectations might be different. But this is what I hoped I could learn from plorinc when he has the possibility to answer.
Updated by plorinc over 3 years ago
okurz wrote:
In principle I see this already fulfilled by https://github.com/os-autoinst/openQA/blob/master/docker/worker/Dockerfile but I want to clarify that your request is completely covered.
The Dockerfile looks good. Do you have any guide on how to start a container from the image?
- I assume when you say "SUSE repositories" that "openSUSE" is ok?
Leap 15.2 is perfectly ok.
- Can you clarify on "AC3: An automated way is provided to fill the database with necessary tables and schemas, and a predefined set of test data": What do you want to achieve with that and what do you mean with "predefined set of test data" in particular?
The image will also be used in test/training scenarios completely isolated from the production system.
In these use cases we needed to put the system into a predefined state, for example by filling the database with a snapshot or a prepared dataset.
- Can you clarify on "AC4: Non-static data is stored on container volumes, if possible": Do you mean something like an external temporary directory passed into the container as the worker's cache and/or pool directory?
Persistent data should be stored on mounted docker volumes so if the container gets restarted (e.g. because of redeployment or software upgrade) the data won't lost.
Basically: configuration is stored inside the container, state is stored outside.
Updated by plorinc over 3 years ago
okurz wrote:
Well, I prefer to not regard this as duplicate as the specific stakeholder expectations might be different. But this is what I hoped I could learn from plorinc when he has the possibility to answer.
We don't plan to build the images with OBS so please don't regard this as a duplicate.
Updated by okurz over 3 years ago
- Status changed from New to Resolved
- Assignee set to ilausuch
- Target version changed from future to Ready
plorinc wrote:
okurz wrote:
In principle I see this already fulfilled by https://github.com/os-autoinst/openQA/blob/master/docker/worker/Dockerfile but I want to clarify that your request is completely covered.
The Dockerfile looks good. Do you have any guide on how to start a container from the image?
Yes, see https://github.com/os-autoinst/openQA/blob/master/docs/Installing.asciidoc#container-based-setup or simply:
podman run --rm -it registry.opensuse.org/devel/openqa/containers15.2/openqa_worker:latest
- Can you clarify on "AC3: An automated way is provided to fill the database with necessary tables and schemas, and a predefined set of test data": What do you want to achieve with that and what do you mean with "predefined set of test data" in particular?
The image will also be used in test/training scenarios completely isolated from the production system.
In these use cases we needed to put the system into a predefined state, for example by filling the database with a snapshot or a prepared dataset.
I have commented to a similar question regarding the webUI in #73447. The worker only needs simple command line parameters or configuration files which can be linked into the container.
- Can you clarify on "AC4: Non-static data is stored on container volumes, if possible": Do you mean something like an external temporary directory passed into the container as the worker's cache and/or pool directory?
Persistent data should be stored on mounted docker volumes so if the container gets restarted (e.g. because of redeployment or software upgrade) the data won't lost.
Basically: configuration is stored inside the container, state is stored outside.
The worker forwards all relevant output data to the web UI. The only relevant data for the worker is the configuration mentioned above. So I don't see this as problem.
Let's see if we can cover the ACs:
- AC1: Container started based on this image is able to run on the Docker engine
DONE: We tested with both "docker" and "podman". https://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3Ddevel%3AopenQA lists the containers along with example commands based on podman as well as docker.
- AC2: Full Green Stack: All software components are available in the (open)SUSE repositories
DONE: Available in openSUSE Tumbleweed as well as build.opensuse.org/project/show/devel:openQA
- AC3: An automated way is provided to fill the database with necessary tables and schemas, and a predefined set of test data
DONE: N/A for the openQA worker
- AC4: Non-static data is stored on container volumes, if possible
DONE: N/A for the openQA worker
- AC5: Deliverable is an image stored in a registry, a Dockerfile (with instructions on how to create a container from the image) or a Compose file
DONE: All of that with https://registry.opensuse.org/cgi-bin/cooverview?srch_term=project%3Ddevel%3AopenQA and https://github.com/os-autoinst/openQA/tree/master/docker
Same as in #73447 setting @ilausuch as he was the main driver in the latest work, I am just collecting facts as result :)