QA » openQA Project

I've just ran the following command again from my notes on #162647 as it has all the bits required to understand the current behavior:

script/openqa-cli schedule --host http://localhost:9526 --monitor --param-file SCENARIO_DEFINITIONS_YAML=/hdd/openqa-devel/openqa/share/tests/example/scenario-definitions.yaml DISTRI=example VERSION=0 FLAVOR=DVD ARCH=x86_64 TEST=simple_boot BUILD=test-poo-162647 _GROUP_ID=0 CASEDIR=https://github.com/Martchus/os-autoinst-distri-example.git#test-poo-162647 NEEDLES_DIR=%%CASEDIR%%/needles PAUSE_AT=tests-boot

This test definitely ran tests from the Git branch specified via CASEDIR as it failed not finding need-yet-another-needle (a needle that is missing on purpose on that branch). It was however generally able to locate needles because the bootloader needle matched (which is the only needle these tests have so far). The symlink created by the worker was pointing to os-autoinst-distri-example/needles which is the needles directory checked out via CASEDIR. So everything works as expected.

Note that specifying NEEDLES_DIR=%%CASEDIR%%/needles would not have been necessary. However, without it, needles would not be loaded from the Git repo and branch specified via CASEDIR (and checked out by os-autoinst). They would instead be loaded from the default branch (checked out by the web UI and synced via the worker cache). I also verified that by leaving out NEEDLES_DIR=%%CASEDIR%%/needles. Then the symlink created the the worker was pointing to /hdd/openqa-devel/openqa/share/tests/example/needles (the location of the web UI checkout in my setup). This makes also sense because the CASEDIR often does not even contain needles. So setting CASEDIR should by default not influence NEEDLES_DIR.

All of this means that the documentation (as added in #94735) still matches the current behavior.

For #97112 we need to change the following behavior:

1. Make the behavior we currently get via NEEDLES_DIR=%%CASEDIR%%/needles the default if CASEDIR actually contains a needles directory. The "if" is important because otherwise we'll break the most common use case (when CASEDIR does not contain a needles directory). Unfortunately it isn't straight forward to check for this "if" because the openQA worker is evaluating these variables but only later os-autoinst is doing the actual checkouts.
   • Make the code for checking out Git repositories for the CASEDIR/NEEDLES_DIR from os-autoinst available to the openQA worker. The easiest approach would be to ensure the code is in a separate module (not some big utils.pm) and simply add that module to the openQA worker's Perl module search path. We can also make os-autoinst's install script install the module to the normal Perl search path.
   • Make the openQA worker (the worker code in openQA, distributed as openQA-worker package) handle CASEDIR/NEEDLES_DIR reusing code from os-autoinst so it has the required checkouts available and we can test for the "if". This will not disturb os-autoinst as it'll just skip the checkouts if already present. This will also not break os-autoinst-only workflows as os-autoinst remains the capability to do the Git checkouts as needed.
2. Invent yet another syntax (e.g. NEEDLES_DIR=%%DEFAULT_BRANCH%%) to load needles from the default branch as checked out by the web UI (even though CASEDIR contains a needles directory). I'm not sure whether that is an actual use case but it is a behavior we currently support (as the default behavior) and by just implementing the first point we'd remove it.

Further remarks:

1. We should probably keep the logic for %%CASEDIR%%-syntax to not break backwards compatibility. Besides, depending on the next point we still need it to be able to point to a different sub directory within CASEDIR then needles.
2. We should consider what we do if NEEDLES_DIR contains a relative path. So far it is treated relative to the default/web-UI checkout which makes also sense as this is so far also our default in the absence of the setting. Now we might want to change it so the %%CASEDIR%%/-prefix is the default if that directory exists (and e.g. the new prefix %%DEFAULT_BRANCH%%/ would be used for the current default behavior).
3. I'm personally not sure whether all of this is worth it or even a good idea. Specifying NEEDLES_DIR=%%CASEDIR%%/needles doesn't really hurt.

https://github.com/os-autoinst/openQA/pull/5799 merged. With this we can conclude.