QA » openQA Project

Working on the investigation, I found something that could be only circumstantial but maybe not.

The run_openqa.sh before we adapt that to have two versions (one for all services together and other to have this split and ready for HA) didn’t execute livehandle (see https://github.com/os-autoinst/openQA/commit/e4e4d4973b2dcc2e02ae136da91d688ea6417c8d#diff-601001db02292ee9c9a3c7f4dbf8699b9465ee448277cef81ecebd2bb83b7a58)

start_daemon -u geekotest /usr/share/openqa/script/openqa-scheduler &
start_daemon -u geekotest /usr/share/openqa/script/openqa-websockets &
start_daemon -u geekotest /usr/share/openqa/script/openqa gru -m production run &
apache2ctl start
start_daemon -u geekotest /usr/share/openqa/script/openqa prefork -m production –proxy

And now this runs the livehandler also

start_daemon -u geekotest /usr/share/openqa/script/openqa-scheduler &
start_daemon -u geekotest /usr/share/openqa/script/openqa-websockets &
start_daemon -u geekotest /usr/share/openqa/script/openqa-livehandler &
start_daemon -u geekotest /usr/share/openqa/script/openqa gru -m production run &
apache2ctl start
start_daemon -u geekotest /usr/share/openqa/script/openqa prefork -m production --proxy

At the same time we can see in the errors of https://openqa.opensuse.org/tests/1837881#step/single_container_webui/108

# Test died: output not validating at openqa/lib/utils.pm line 101.

This line (https://github.com/os-autoinst/openQA/blob/1248f9f088268759cbed5c91104c7ddb29d8c9d8/lib/OpenQA/Utils.pm#L101) refers to a this piece of code

# returns the url to the web socket proxy started via openqa-livehandler
sub determine_web_ui_web_socket_url {
my ($job_id) = @_;
return "liveviewhandler/tests/$job_id/developer/ws-proxy";
}

So this could be a coincidence or not

The directory is only created by the web UI but not other services like the livehandler. So I don't think the introduction of livehandler makes a difference.

More importantly, I'd like to note that "Permission denied" when tries to create the directory "/var/lib/openqa/share/factory/tmp" is not the reason why the test fails. It is just a log message which even appears on successful runs: https://openqa.opensuse.org/tests/1839850#step/single_container_webui/8
(The same counts for AH00558: httpd-prefork: Could not reliably determine the server's fully qualified domain name, using 172.18.0.3. Set the 'ServerName' directive globally to suppress this message.)

The last log message is Mojolicious::Plugin::AssetPack::Pipe::Sass requires ruby. https://ruby-lang.org/en/documentation/installation which is also absent in successful runs. So maybe that's the culprit? Of course it leaves the question why ruby is attempted to be used in some cases. I would assume it is only a build/development requirement as the asset cache is supplied by the rpm package.

For now I created this PR to ensure that the permissions are good for geekotest user
https://github.com/os-autoinst/os-autoinst-distri-openQA/pull/71

I tested locally that I the permissions were incorrect and was imposible to create these directories. But I don't know how in openqa only eventually this failed. So independent that is the real problem or not here, I think this solution could help if in the future we test more deeply

Now I am going to investigate your suggestions

About the message

Mojolicious::Plugin::AssetPack::Pipe::Sass requires ruby. https://ruby-lang.org/en/documentation/installation

The url is wrong, should be
https://www.ruby-lang.org/en/documentation/installation/

https://github.com/os-autoinst/openQA/pull/4050
I am preparing a test as a script in tools to try to reproduce this locally and to have a sooner than in openqa and before merge

Now the PR is merged. Waiting for some completed tests in https://openqa.opensuse.org/group_overview/24