Project

General

Profile

action #80520

coordination #80142: [saga][epic] Scale out openQA: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes

coordination #43706: [epic] Generate "download&use" docker image of openQA for SUSE QA

Automatic tests for our openQA containers - webUI only

Added by okurz about 2 months ago. Updated 7 days ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-11-27
Due date:
2020-12-24
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

Without automatic tests we would not call any contributions properly long-term supportable. These tests could be very simple, e.g. something like podman run --rm -it .... openqa-worker --help or something.

Acceptance criteria

  • AC1: The webUI server is started completely without errors from container within CI tests, i.e. can render a common page e.g. /tests

Suggestions

  • Find out if OBS can execute tests for container images
  • Consider kiwi vs Dockerfile
  • Setup tests as part of the openQA CI (e.g. CircleCI or GitHub Actions)

Related issues

Copied to openQA Project - action #80682: Automatic tests for our openQA containers - worker onlyFeedback2020-11-27

History

#1 Updated by cdywan about 2 months ago

  • Description updated (diff)

#2 Updated by ilausuch about 2 months ago

  • Status changed from New to In Progress
  • Assignee set to ilausuch

#3 Updated by ilausuch about 2 months ago

In order to fix several things, this PR is necessary to run the web UI without the load balancer
https://github.com/os-autoinst/openQA/pull/3607

#4 Updated by okurz about 2 months ago

  • Description updated (diff)

#5 Updated by okurz about 2 months ago

  • Subject changed from Automatic tests for our openQA containers to Automatic tests for our openQA containers - webUI only
  • Description updated (diff)
  • Status changed from In Progress to Workable
  • Assignee deleted (ilausuch)

#6 Updated by okurz about 2 months ago

  • Copied to action #80682: Automatic tests for our openQA containers - worker only added

#7 Updated by okurz about 2 months ago

  • Target version changed from Ready to future

#8 Updated by ilausuch about 2 months ago

This PR fix some problems with the SSL apache configuration when we run webUI container
https://github.com/os-autoinst/openQA/pull/3612

#9 Updated by cdywan about 2 months ago

ilausuch wrote:

This PR fix some problems with the SSL apache configuration when we run webUI container
https://github.com/os-autoinst/openQA/pull/3612

Where by fixing problems you mean dropping the SSL configuration for Apache, right? That's what the PR proposes.

#10 Updated by ilausuch about 1 month ago

  • Status changed from Workable to In Progress
  • Assignee set to ilausuch

#11 Updated by ilausuch about 1 month ago

I created this new PR following some comments on the previous PR (that is replaced by this new)
https://github.com/os-autoinst/openQA/pull/3632
In this new PR I created different dockerfiles to cover different current scenarios:

  • container with all the services executed inside with apache as a reverse proxy + SSL. User has to provide the certificates
  • container with all the services executed inside with apache as a reverse proxy without SSL
  • container without apache and the services individually-executed controlled by the MODE environment variable. To be used within docker-compose or kubernetes

#12 Updated by ilausuch about 1 month ago

for this test we will check that the web UI tries to connect to the DB

docker run -ti -v $(pwd)/t/data/database.ini:/data/conf/database.ini openqa_webui

With the result:

DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('dbname=openqa','',...) failed: could not connect to server: No such file or directory

Or we could try to provide a DB to the container and test this more properly.

#13 Updated by openqa_review about 1 month ago

  • Due date set to 2020-12-24

Setting due date based on mean cycle time of SUSE QE Tools

#14 Updated by ilausuch about 1 month ago

After a discussion today in a daily I will focus on create a test to cover the container without apache. My plan for the test is:

  • Build the web UI image
  • Create a DB container (postgresql)
  • Test that gru can start and listen on its port
  • Test that websockets can start and listen on its port
  • Test that livehandle can start and listen on its port
  • Test that webUI can start and listen on its port

This PR https://github.com/os-autoinst/openQA/pull/3632 will be postponed for the future if necessary

#15 Updated by ilausuch about 1 month ago

To cover the plan of my last commentary I provide this PR https://github.com/os-autoinst/openQA/pull/3633

#16 Updated by ilausuch about 1 month ago

I was exploring if we could use openQA jobs to create tests of our containers. Please see this job https://openqa.suse.de/tests/5173044
I think could be interesting because we could launch regular tests of our containers from VM created by openQA jobs. Yesterday I played with that and I achieved to run a container inside a job and also with privileged mode, so there are no barriers for that.
There are some considerations to take in consideration, and please, correct me if I am wrong: openQA is not integrated in the CI of the github so, if we want to test last changes on the dockerfiles in the last PR won't be useful, only could be use to regular testing.

On the other hand, we should consider two kind of testing:

  • Unit tests: For instance, that the web UI starts and is listening
  • Integration tests: we can launch a job, and all the parts work together til the end of the job.

#17 Updated by ilausuch about 1 month ago

Seems that circleCI doesn't allows to run containers in privilege mode
https://circleci.com/docs/2.0/executor-types/#using-docker

And there is not configuration options to do that
https://circleci.com/docs/2.0/configuration-reference/#docker--machine--macos--windows-executor

#18 Updated by ilausuch about 1 month ago

I am thinking in these solutions

Inside the github CI workflow we create a new step/action than launch a script that launches a job in openqa.suse.de / openqa.opensuse.org.
This script passes to this tests what is the branch we want to test as a parameter will be get by the perl test. The original script will remain waiting until the job has finished with a success, softfail or a failure reporting the result as result to the CI
The perl test clones the openQA repository with the specified branch, builds the container image, create the containers, and runs the tests of connection, etc.

There are other alternatives with kubernetes to this one, but I would like to explore this one.

#19 Updated by okurz about 1 month ago

We discussed different options in today's daily meeting which can all in principle be followed independently:

  1. Same as in os-autoinst run static style checks on Dockerfiles within PR checks and run build+run steps when running on master, compare to https://github.com/os-autoinst/os-autoinst/pull/1602
  2. Extend the openQA-in-openQA tests in https://openqa.opensuse.org/group_overview/24 with a build+run test of our container definitions, e.g. boot a Tumbleweed system, build container images with podman and run them
  3. Similar as the existing openQA-in-openQA tests create a new trigger script which looks for new container image files defined in http://download.opensuse.org/repositories/devel:/openQA/containers15.2/ and trigger according tests in openQA . For this I suggest to not trigger the same as we do in Jenkins but instead use github actions

#20 Updated by ilausuch 9 days ago

Trying to run the test this in OpenQA. I had always the same issue with the timeouts with the command docker run and podman run.
The problem of these VMs are that we don't have hardware virtualization, so this is really very slow

QEMU: Checking for hardware virtualization : FAIL (Only emulated CPUs are available, performance will be significantly limited)

With @Geor helping we are exploring baremetal machines to do this test.

Also available in: Atom PDF