QA » openQA Project

In order to fix several things, this PR is necessary to run the web UI without the load balancer
https://github.com/os-autoinst/openQA/pull/3607

This PR fix some problems with the SSL apache configuration when we run webUI container
https://github.com/os-autoinst/openQA/pull/3612

ilausuch wrote:

This PR fix some problems with the SSL apache configuration when we run webUI container
https://github.com/os-autoinst/openQA/pull/3612

Where by fixing problems you mean dropping the SSL configuration for Apache, right? That's what the PR proposes.

I created this new PR following some comments on the previous PR (that is replaced by this new)
https://github.com/os-autoinst/openQA/pull/3632
In this new PR I created different dockerfiles to cover different current scenarios:

• container with all the services executed inside with apache as a reverse proxy + SSL. User has to provide the certificates
• container with all the services executed inside with apache as a reverse proxy without SSL
• container without apache and the services individually-executed controlled by the MODE environment variable. To be used within docker-compose or kubernetes

for this test we will check that the web UI tries to connect to the DB

docker run -ti -v $(pwd)/t/data/database.ini:/data/conf/database.ini openqa_webui

With the result:

DBIx::Class::Storage::DBI::catch {...} (): DBI Connection failed: DBI connect('dbname=openqa','',...) failed: could not connect to server: No such file or directory

Or we could try to provide a DB to the container and test this more properly.

After a discussion today in a daily I will focus on create a test to cover the container without apache. My plan for the test is:

• Build the web UI image
• Create a DB container (postgresql)
• Test that gru can start and listen on its port
• Test that websockets can start and listen on its port
• Test that livehandle can start and listen on its port
• Test that webUI can start and listen on its port

This PR https://github.com/os-autoinst/openQA/pull/3632 will be postponed for the future if necessary

To cover the plan of my last commentary I provide this PR https://github.com/os-autoinst/openQA/pull/3633

I was exploring if we could use openQA jobs to create tests of our containers. Please see this job https://openqa.suse.de/tests/5173044
I think could be interesting because we could launch regular tests of our containers from VM created by openQA jobs. Yesterday I played with that and I achieved to run a container inside a job and also with privileged mode, so there are no barriers for that.
There are some considerations to take in consideration, and please, correct me if I am wrong: openQA is not integrated in the CI of the github so, if we want to test last changes on the dockerfiles in the last PR won't be useful, only could be use to regular testing.

On the other hand, we should consider two kind of testing:

• Unit tests: For instance, that the web UI starts and is listening
• Integration tests: we can launch a job, and all the parts work together til the end of the job.

Seems that circleCI doesn't allows to run containers in privilege mode
https://circleci.com/docs/2.0/executor-types/#using-docker

And there is not configuration options to do that
https://circleci.com/docs/2.0/configuration-reference/#docker--machine--macos--windows-executor

I am thinking in these solutions

Inside the github CI workflow we create a new step/action than launch a script that launches a job in openqa.suse.de / openqa.opensuse.org.
This script passes to this tests what is the branch we want to test as a parameter will be get by the perl test. The original script will remain waiting until the job has finished with a success, softfail or a failure reporting the result as result to the CI
The perl test clones the openQA repository with the specified branch, builds the container image, create the containers, and runs the tests of connection, etc.

There are other alternatives with kubernetes to this one, but I would like to explore this one.

We discussed different options in today's daily meeting which can all in principle be followed independently:

1. Same as in os-autoinst run static style checks on Dockerfiles within PR checks and run build+run steps when running on master, compare to https://github.com/os-autoinst/os-autoinst/pull/1602
2. Extend the openQA-in-openQA tests in https://openqa.opensuse.org/group_overview/24 with a build+run test of our container definitions, e.g. boot a Tumbleweed system, build container images with podman and run them
3. Similar as the existing openQA-in-openQA tests create a new trigger script which looks for new container image files defined in http://download.opensuse.org/repositories/devel:/openQA/containers15.2/ and trigger according tests in openQA . For this I suggest to not trigger the same as we do in Jenkins but instead use github actions

Trying to run the test this in OpenQA. I had always the same issue with the timeouts with the command docker run and podman run.
The problem of these VMs are that we don't have hardware virtualization, so this is really very slow

QEMU: Checking for hardware virtualization : FAIL (Only emulated CPUs are available, performance will be significantly limited)

With @geor helping we are exploring baremetal machines to do this test.

The test PR
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/11755

Executed test: http://10.161.229.197/tests/138

Initial tests using the openQA repo instead of opensuse
PR: https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/60
Test: http://10.161.229.197/tests/184

Now I am using docker because I have to solve some problems with the podman version resolving the name of the DB

I change the status of the PR to be reviewed.

Last test https://openqa.opensuse.org/tests/1601650

This test is regular tested in openqa.opensuse.org
https://openqa.opensuse.org/tests/1607610

Yes, you're right

My understanding was, CI didn't work so well, and containers do the job. Otherwise we wouldn't have gone for openQA tests at this point.

cdywan wrote:

"CI didn't work so well, and containers do the job."

Hm, not sure what you mean by that. Maybe you can clarify what you mean. How I see the current situation is like this: With openQA-in-openQA tests we can do real functional tests which is good. However, we can still have more feedback before merge, part of the pull request testing CI tests. In #80520#note-19 I mentioned the options and referenced the os-autoinst pull request that tests container definitions as part of such CI tests. In one of the followup meetings we clarified (I think) that we should also have such tests for openQA. They will not do the same as the openQA-in-openQA tests but check that the Dockerfiles as versioned within our git repo are usable, i.e. buildable.

okurz wrote:

cdywan wrote:

"CI didn't work so well, and containers do the job."

Hm, not sure what you mean by that. Maybe you can clarify what you mean.

There were problems with the proposed tests on CI, with building and running containers, both with podman and docker, concerning basics like permissions and what user to run as.

How I see the current situation is like this: With openQA-in-openQA tests we can do real functional tests which is good. However, we can still have more feedback before merge, part of the pull request testing CI tests. In #80520#note-19 I mentioned the options and referenced the os-autoinst pull request that tests container definitions as part of such CI tests. In one of the followup meetings we clarified (I think) that we should also have such tests for openQA. They will not do the same as the openQA-in-openQA tests but check that the Dockerfiles as versioned within our git repo are usable, i.e. buildable.

If you're indeed talking about static validation of container files I consider that out of scope for this ticket. The webUI server is started completely without errors is about running the container successfully.

cdywan wrote:

okurz wrote:

cdywan wrote:

"CI didn't work so well, and containers do the job."

Hm, not sure what you mean by that. Maybe you can clarify what you mean.

There were problems with the proposed tests on CI, with building and running containers, both with podman and docker, concerning basics like permissions and what user to run as.

How I see the current situation is like this: With openQA-in-openQA tests we can do real functional tests which is good. However, we can still have more feedback before merge, part of the pull request testing CI tests. In #80520#note-19 I mentioned the options and referenced the os-autoinst pull request that tests container definitions as part of such CI tests. In one of the followup meetings we clarified (I think) that we should also have such tests for openQA. They will not do the same as the openQA-in-openQA tests but check that the Dockerfiles as versioned within our git repo are usable, i.e. buildable.

If you're indeed talking about static validation of container files I consider that out of scope for this ticket. The webUI server is started completely without errors is about running the container successfully.

Well, the "static validation" is merely an implementation consequence when we can not have full functional tests within the PR CI tests.

I think the addition to the openQA-in-openQA tests are great. If you want to split out the mentioned idea into another ticket and call this one done, sure, we can do that. I would just prefer to have some tests for Dockerfiles being executed as part of every PR test.

okurz wrote:

I think the addition to the openQA-in-openQA tests are great. If you want to split out the mentioned idea into another ticket and call this one done, sure, we can do that. I would just prefer to have some tests for Dockerfiles being executed as part of every PR test.

Fully agree. Between the two of us, I really want to see those static checks, too. I'm just trying to keep ACs in focussed and workable chunks.

I filed #88451