action #76978
coordination #127031: [saga][epic] openQA for SUSE customers
coordination #80150: [epic] Scale out openQA: Easier openQA setup
How to run an openQA test in 5 minutes
0%
Description
User story¶
As a potential user of openQA I want to run an openQA test in 5 minutes to be hooked
Acceptance criteria¶
- AC1: An openQA instance can be installed and is used to execute a new (empty) test within 5 minutes (not counting download times on slow connections)
Suggestions¶
- In your local openQA working copy, try out to build and run containers, e.g.
podman build -t openqa-webui docker/webui
and thenpodman run --rm -it openqa-webui
- We already have a quick bootstrap method but it might not be prominent enough and we can present it as a "first-class citizen"
- Have a workflow (or test or something) that uses the quick bootstrap or comparable approach and runs tests based on github.com/os-autoinst/os-autoinst-distri-example
- record a video and make some noiz about it :)
- maybe create/adapt some tool like
osc
but for openqa that will allow users to run something likeosc build
Further details¶
Inspired by talking to bzoltan :)
Related issues
History
#1
Updated by okurz over 2 years ago
https://github.com/os-autoinst/openQA/pull/3511 as a related first step.
EDIT: merged. Next steps as outlined in suggestions
#2
Updated by okurz over 2 years ago
- Description updated (diff)
- Status changed from New to Workable
- Target version changed from future to Ready
- Parent task set to #80142
#3
Updated by okurz over 2 years ago
- Parent task changed from #80142 to #80150
#4
Updated by okurz over 2 years ago
- Description updated (diff)
#5
Updated by ilausuch over 2 years ago
- Assignee set to ilausuch
#6
Updated by openqa_review over 2 years ago
- Due date set to 2021-02-13
Setting due date based on mean cycle time of SUSE QE Tools
#7
Updated by ilausuch over 2 years ago
I recorded a video to show how to deploy openqa_webui container
https://asciinema.org/a/x8UYkEysNsGlAfjCamvzLmjh0
git clone https://github.com/os-autoinst/openQA.git mkdir -p data/factory/{iso,hdd,other,tmp} data/{testresults,tests,conf} data/certs/{ssl.crt,ssl.key} cp openQA/container/openqa_data/data.template/conf/* data/conf cp openQA/container/webui/conf/database.ini data/conf/ cat /tmp/openqa.ini | sed 's/method = OpenID/method = Fake/' >data/conf/openqa.ini docker network create openQA docker run -d --network openQA -e POSTGRES_PASSWORD=openqa -e POSTGRES_USER=openqa -e POSTGRES_DB=openqa --net-alias=db --name openqa_db postgres docker logs -f openqa_db openssl req -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -subj '/CN=www.mydom.com/O=My Company Name LTD./C=DE' -out data/certs/ssl.crt/server.crt -keyout data/certs/ssl.key/server.key cp data/certs/ssl.crt/server.crt data/certs/ssl.crt/ca.crt volumes="-v $(pwd)/data:/data" certificates="-v $(pwd)/data/certs/ssl.crt:/etc/apache2/ssl.crt -v $(pwd)/data/certs/ssl.key:/etc/apache2/ssl.key" docker build openQA/container/webui -t openqa_webui docker run -d --network openQA $volumes $certificates -p 80:80 --net-alias=openqa_webui --name openqa_webui openqa_webui docker logs -f openqa_webui
#8
Updated by okurz over 2 years ago
To have videos is great but we should not encourage users to use the development repos. Also our goal should be "single-command" solutions. As openqa-bootstrap already supports to take jobs to clone as parameters I suggest to start with that
#9
Updated by cdywan over 2 years ago
This seems very similar to what openqa-bootstrap-container
does except that it uses docker as a runtime. Maybe we want that as a re-usable script?
The video teaches the user how to build and deploy openQA, and that's what this ticket asks for, so I like it. Not sure if this is exactly what I was expecting since the audience wasn't defined. Deplying a container probably makes it very widely accessible so I like that. My main concern would be the use of the git repo as already mentioned by @okurz... thanks to #43718 we presumably have versioned images that can be used here and avoid git?
#10
Updated by cdywan over 2 years ago
- Status changed from Workable to In Progress
#11
Updated by okurz over 2 years ago
cdywan wrote:
This seems very similar to what
openqa-bootstrap-container
does except that it uses docker as a runtime. Maybe we want that as a re-usable script?
Yes, that was mentioned as the second point within the suggestions.
The video teaches the user how to build and deploy openQA, and that's what this ticket asks for, so I like it. Not sure if this is exactly what I was expecting since the audience wasn't defined.
Well, the user story has an actor "potential user of openQA" and the goal is "to be hooked" :)
And yes, a container based solution was suggested. That is fine. But as we already have an approach as documented in http://open.qa/docs/#bootstrapping that shows how an openQA installation can be installed and immediately start a clone-job I consider the multiple steps shown in the video as a step backwards.
#12
Updated by cdywan over 2 years ago
okurz wrote:
Well, the user story has an actor "potential user of openQA" and the goal is "to be hooked" :)
And yes, a container based solution was suggested. That is fine. But as we already have an approach as documented in http://open.qa/docs/#bootstrapping that shows how an openQA installation can be installed and immediately start a clone-job I consider the multiple steps shown in the video as a step backwards.
Those instructions say, here's a command, go read a file to figure out how to actually use it. I think that's what we want to improve here.
- What concrete steps are missing here in the typical case?
- Is it desirable to customize the container like in #76978#note-7
- Certificate setup like in #76978#note-7
- Running an actual test, not covered yet
#13
Updated by kraih over 2 years ago
Might be fun to piggyback on k3s for this. https://k3s.io Then the user would need literally nothing but a Linux kernel and Bash to get started with openQA. You might not like curl | bash
one-liners for one reason or another, but that's what gets people hooked.
#14
Updated by okurz over 2 years ago
exactly :) That makes sense.
#15
Updated by cdywan over 2 years ago
ilausuch Take a look at openqa-bootstrap{,-container
for inspiration
- Avoid git repos - keep in mind the new user who just wants to run a test
- Pretend there's nothing installed, don't assume Tumbleweed/SLE (we cover that use case with existing scripts, no need to replicate that)
- Use
--device=/dev/kvm
- Keep it simple, one or two lines if possible
- Make sure any setup of e.g. qemu/isotovideo is inside the container
#16
Updated by ilausuch over 2 years ago
Thanks for all your help.
I recorded a video showing how to setup the worker https://asciinema.org/a/390320
The script to have the worker running is this: $ chmod a+w data/testresults In this section we are going to use only opensuse test suites, but if you want more add them following the example: $ git clone git@github.com:os-autoinst/os-autoinst-distri-opensuse.git $ ln -s os-autoinst-distri-opensuse opensuse $ git clone https://github.com/os-autoinst/os-autoinst-needles-opensuse.git $ mv os-autoinst-needles-opensuse/ opensuse/products/opensuse/needles Create a key/secret pair on your webUI and replace the example ones to configure the client.conf $ echo -e “[openqa_webui]\nkey = 5AD3533BB586FD54\nsecret = AE0A2A27FD82D530” > data/conf/client.conf Ensure that the variable volumes is set $ volumes=”-v $(pwd)/data:/data” Build the worker $ docker build openQA/container/worker -t openqa_worker Run the worker $ docker run -d —network openQA $volumes —device=/dev/kvm —privileged —name openqa_worker openqa_worker You can launch a test accessing to the web UI $ docker exec -ti openqa_webui /usr/share/openqa/script/clone_job.pl —apikey 5AD3533BB586FD54 —apisecret AE0A2A27FD82D530 https://openqa.opensuse.org/tests/1620808
#17
Updated by ilausuch over 2 years ago
Trying to figure out how to run the worker without being root
The group of the kvm is 128 that doesn't belongs to any group in the container
4a8e60738367:/ # ls -lh /dev/kvm crw-rw---- 1 root 128 10, 232 Feb 11 08:34 /dev/kvm
I tried to assign the group using the number, but doesn't work
usermod -g 128 _openqa-worker usermod: group '128' does not exist
#18
Updated by ilausuch over 2 years ago
thanks George we found a solution using groupmod
groupmod -g 128 kvm
#19
Updated by ilausuch over 2 years ago
cdywan wrote:
ilausuch Take a look at
openqa-bootstrap{,-container
for inspiration
- Avoid git repos - keep in mind the new user who just wants to run a test
I have this in mind, the previous recordings were tests for the final recording that will be using the container repository
- Pretend there's nothing installed, don't assume Tumbleweed/SLE (we cover that use case with existing scripts, no need to replicate that)
Right now, these steps are valid for in a Ubuntu too. So we could consider that is independent to the linux distro. Only, the user has to install docker and as far as I know kvm too
- Use
--device=/dev/kvm
Yes, the PR associated to this point is this https://github.com/os-autoinst/openQA/pull/3715
- Keep it simple, one or two lines if possible
There are several steps we cannot avoid, because we have different containers, in this case a progress, webUI and worker has to be deployed.
Additionally some aspects like volumes and certificates has to be provided from the user.
- Volumes: We need to set-up an initial directory structure and provide to the containers via volumes. For instance, the configuration on webui and worker container doesn't exists. Has to be provided.
- Certificates: In some previous discussions we agree that the certificates has to be provided by the user.
- Tests: If we are working a generic tool the tests and needles cannot be part of the worker container image because we don't know what tests the user is going to use. Then has to be downloaded and put in the right directory by the user
Update: I am preparing a simple script to help the user to run all of these in two or there lines
- Make sure any setup of e.g. qemu/isotovideo is inside the container
That works. The video of the job is stored on assets can can be played.
#20
Updated by cdywan over 2 years ago
- Due date changed from 2021-02-13 to 2021-02-19
Bumping the Due date since enabling kvm isn't fully solved yet and there were open questions with scripting installation. See the record of the last conversation above.
#21
Updated by cdywan over 2 years ago
- Due date changed from 2021-02-19 to 2021-02-26
Update: I am preparing a simple script to help the user to run all of these in two or there lines
#22
Updated by cdywan over 2 years ago
ilausuch wrote:
- Keep it simple, one or two lines if possible
There are several steps we cannot avoid, because we have different containers, in this case a progress, webUI and worker has to be deployed.
Additionally some aspects like volumes and certificates has to be provided from the user.
- Volumes: We need to set-up an initial directory structure and provide to the containers via volumes. For instance, the configuration on webui and worker container doesn't exists. Has to be provided.
- Certificates: In some previous discussions we agree that the certificates has to be provided by the user.
- Tests: If we are working a generic tool the tests and needles cannot be part of the worker container image because we don't know what tests the user is going to use. Then has to be downloaded and put in the right directory by the user
Update: I am preparing a simple script to help the user to run all of these in two or there lines
I commented on the PR but for the record also replying here.
Do we really want a whole CLI here? Can't we use docker-compose
for the setup of volumes, different containers etc.?
I could see if letting users supply certificates or openSUSE test repos is not trivial, but then that's the only things the script should be doing.
#23
Updated by ilausuch over 2 years ago
Working on the docker-compose file
Some considerations:
- Because all parts work in different containers, the OPENQA_SCHEDULER_HOST and OPENQA_WEB_SOCKETS_HOST env variables need to be set in the webui containers and the scheduler so that when they communicate properly
- The directory factory needs the subdirectories: iso,hdd,other,tmp
- The directories factory and subdirectories, and testresults has to be read-write
- To add a worker in the docker-compose a API key/secret is needed from the beginning
- The scheduler, webui, and others tries to create the tables on the database at the same time creating failures. Is needed to delay some of them to prevent this.
Preparing the PR. Now It can execute jobs, but doesn't show anything in the live view.
#24
Updated by cdywan over 2 years ago
ilausuch wrote:
- The directory factory needs the subdirectories: iso,hdd,other,tmp
- The directories factory and subdirectories, and testresults has to be read-write
Could we have those created on demand by the worker? It occurs to me this is something I'd also do as part of a development setup as well, ensure e.g. share/factory/hdd
exists because otherwise cloning won't work. Just empty folders are expected here. 🤔
#25
Updated by ilausuch over 2 years ago
- Due date changed from 2021-02-26 to 2021-03-05
#26
Updated by ilausuch over 2 years ago
cdywan wrote:
ilausuch wrote:
- The directory factory needs the subdirectories: iso,hdd,other,tmp
- The directories factory and subdirectories, and testresults has to be read-write
Could we have those created on demand by the worker? It occurs to me this is something I'd also do as part of a development setup as well, ensure e.g.
share/factory/hdd
exists because otherwise cloning won't work. Just empty folders are expected here. 🤔
This was my first approach, but it's more easy than that. The docker-compose can do it. I've just checked that work
#27
Updated by ilausuch over 2 years ago
Created the PR that uses the docker-container to deploy the entire openQA system
https://github.com/os-autoinst/openQA/pull/3755
Considerations:
- The user must install the tests on workdir/tests
- To run opensuse and other SUSE tests execute: docker exec -t openqa_worker zypper in -y os-autoinst-distri-opensuse-deps
#28
Updated by cdywan about 2 years ago
- Blocked by action #89719: docker-compose up fails on master added
#29
Updated by okurz about 2 years ago
- Status changed from In Progress to Blocked
We discussed in the daily 2021-03-09 that we encountered multiple issues with the current docker-compose.yaml file so that the current docker-compose.yaml which we should cover in individual tickets so that we can progress within cycle times. Also we need an automatic check for docker-compose, see #89722
#30
Updated by okurz about 2 years ago
- Due date deleted (
2021-03-05)
#31
Updated by cdywan about 2 years ago
- Blocked by action #89731: containers: The deploy using docker-compose is not stable and eventually fails added
#32
Updated by cdywan about 2 years ago
- Blocked by deleted (action #89731: containers: The deploy using docker-compose is not stable and eventually fails )
#33
Updated by okurz about 2 years ago
- Blocked by action #88451: Static validation for containers added
#34
Updated by cdywan about 2 years ago
- Related to action #89752: containers: Add a worker service as part of the docker-compose added
#35
Updated by cdywan about 2 years ago
- Related to action #89755: container: Fix missing shared directories and its permissions added
#36
Updated by ilausuch about 2 years ago
- Status changed from Blocked to In Progress
I unblocked because recent related tickets are closed and with what we have now it could be possible. I am going to explore if is it possible now run a full env with the web UI using the docker-compose and worker using a few commands. If is not possible I'll write down what are the reasons and then create new tickets if are necessary
#37
Updated by ilausuch about 2 years ago
- Status changed from In Progress to Blocked
During this new attempt to run a job I found a problem:
webui_2 | [warn] Unable to wakeup scheduler: Connection refused
I tested the access to the scheduler from this webui and works
$ ping scheduler
PING scheduler (192.168.80.8) 56(84) bytes of data.
64 bytes from webui_scheduler_1.webui_default (192.168.80.8): icmp_seq=1 ttl=64 time=0.126 ms
$ telnet scheduler 9529
Trying 192.168.80.8...
Connected to scheduler.
Escape character is '^]'.
So, maybe is a configuration problem or other env variables needed
To prepare the environment:
$ cd openQA/containers/webui
$ docker-compose build
# Edit openqa.ini and set auth method to Fake
$ docker-compose up -d
# Now you can access to the UI at http://localhost:9526 and login. If you want you can create a new credentials, there are ones by default
# go to the directory workdir/data/tests and download the test suites you want, for instance opensuse
$ sudo git clone https://github.com/os-autoinst/os-autoinst-distri-opensuse.git
$ sudo mv os-autoinst-distri-opensuse opensuse
$ sudo git clone https://github.com/os-autoinst/os-autoinst-needles-opensuse.git
$ sudo mv os-autoinst-needles-opensuse/ opensuse/products/opensuse/needles
# Move to the worker directory
$ ln -s ../webui/workdir/
# Edit conf/worker.ini and change HOST to http://nginx:9526
# Edit conf/client.conf and change [openqa_webui] to [nginx]
$ docker run -d --network=webui_default -v $(pwd)/workdir/data/factory:/data/factory:rw -v $(pwd)/workdir/data/tests:/data/tests:ro -v $(pwd)/conf:/data/conf:ro --device=/dev/kvm --privileged openqa_worker --name openqa_worker
# Check that the worker has connected
$ docker logs -f openqa_worker
To launch the test:
$ docker-compose exec webui bash
$ /usr/share/openqa/script/clone_job.pl --from https://openqa.opensuse.org --host http://nginx:9526 1738950
# This download all the assets correctly
I created a ticket to solve this problem #92833 and this one will be blocked until the other is solved
#38
Updated by ilausuch about 2 years ago
- Related to action #92833: containers: Web UI cannot connect to scheduler added
#39
Updated by ilausuch about 2 years ago
- Blocked by action #92893: containers, docker-compose: Ensure that the scheduler can connect to the websockets container size:M added
#40
Updated by okurz almost 2 years ago
- Related to coordination #81060: [epic] openQA web UI in kubernetes added
#41
Updated by ilausuch almost 2 years ago
- Assignee deleted (
ilausuch)
#43
Updated by okurz over 1 year ago
- Status changed from Blocked to New
- Assignee deleted (
okurz)
blocker #92893 resolved.
#44
Updated by okurz over 1 year ago
- Target version changed from Ready to future
#46
Updated by okurz 11 days ago
- Copied to action #129883: all-in-one openQA container solution added