Project

General

Profile

Actions
Copy link

action #164577

closed

coordination #58184: [saga][epic][use case] full version control awareness within openQA

coordination #94750: [epic] Support needles from git-cloned or separate repo if casedir points to a git-repo

[timeboxed:6h][research] Understand the current behaviour of openQA test distributions including needles

Added by okurz 18 days ago. Updated 17 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
mkittler
Category:
Feature requests
Target version:
Ready
Start date:
2024-07-29
Due date:
% Done:

0%

Estimated time:

Description

Motivation

See the unexpected current default behaviour in #94735. We plan to follow up in in #97112. As that is an older ticket we identified the need to learn what is the actual current behaviour.

Acceptance criteria

  • G1: For tests with CASEDIR pointing to a git repo including tests+needles we understand what is necessary

Suggestions

Related issues 1 (0 open — 1 closed)

Copied from openQA Project - action #97112: Support relative needle directories together with tests checked out from gitResolvedokurz

Actions
Actions
Copy link
 #1

Updated by okurz 18 days ago

  • Copied from action #97112: Support relative needle directories together with tests checked out from git added
Actions
Copy link
 #2

Updated by mkittler 18 days ago

  • Status changed from New to In Progress
  • Assignee set to mkittler
Actions
Copy link
 #3

Updated by mkittler 18 days ago · Edited

  • Status changed from In Progress to Feedback

I've just ran the following command again from my notes on #162647 as it has all the bits required to understand the current behavior:

script/openqa-cli schedule --host http://localhost:9526 --monitor --param-file SCENARIO_DEFINITIONS_YAML=/hdd/openqa-devel/openqa/share/tests/example/scenario-definitions.yaml DISTRI=example VERSION=0 FLAVOR=DVD ARCH=x86_64 TEST=simple_boot BUILD=test-poo-162647 _GROUP_ID=0 CASEDIR=https://github.com/Martchus/os-autoinst-distri-example.git#test-poo-162647 NEEDLES_DIR=%%CASEDIR%%/needles PAUSE_AT=tests-boot

This test definitely ran tests from the Git branch specified via CASEDIR as it failed not finding need-yet-another-needle (a needle that is missing on purpose on that branch). It was however generally able to locate needles because the bootloader needle matched (which is the only needle these tests have so far). The symlink created by the worker was pointing to os-autoinst-distri-example/needles which is the needles directory checked out via CASEDIR. So everything works as expected.

Note that specifying NEEDLES_DIR=%%CASEDIR%%/needles would not have been necessary. However, without it, needles would not be loaded from the Git repo and branch specified via CASEDIR (and checked out by os-autoinst). They would instead be loaded from the default branch (checked out by the web UI and synced via the worker cache). I also verified that by leaving out NEEDLES_DIR=%%CASEDIR%%/needles. Then the symlink created the the worker was pointing to /hdd/openqa-devel/openqa/share/tests/example/needles (the location of the web UI checkout in my setup). This makes also sense because the CASEDIR often does not even contain needles. So setting CASEDIR should by default not influence NEEDLES_DIR.

All of this means that the documentation (as added in #94735) still matches the current behavior.

For #97112 we need to change the following behavior:

  1. Make the behavior we currently get via NEEDLES_DIR=%%CASEDIR%%/needles the default if CASEDIR actually contains a needles directory. The "if" is important because otherwise we'll break the most common use case (when CASEDIR does not contain a needles directory). Unfortunately it isn't straight forward to check for this "if" because the openQA worker is evaluating these variables but only later os-autoinst is doing the actual checkouts.
    • Make the code for checking out Git repositories for the CASEDIR/NEEDLES_DIR from os-autoinst available to the openQA worker. The easiest approach would be to ensure the code is in a separate module (not some big utils.pm) and simply add that module to the openQA worker's Perl module search path. We can also make os-autoinst's install script install the module to the normal Perl search path.
    • Make the openQA worker (the worker code in openQA, distributed as openQA-worker package) handle CASEDIR/NEEDLES_DIR reusing code from os-autoinst so it has the required checkouts available and we can test for the "if". This will not disturb os-autoinst as it'll just skip the checkouts if already present. This will also not break os-autoinst-only workflows as os-autoinst remains the capability to do the Git checkouts as needed.
  2. Invent yet another syntax (e.g. NEEDLES_DIR=%%DEFAULT_BRANCH%%) to load needles from the default branch as checked out by the web UI (even though CASEDIR contains a needles directory). I'm not sure whether that is an actual use case but it is a behavior we currently support (as the default behavior) and by just implementing the first point we'd remove it.

Further remarks:

  1. We should probably keep the logic for %%CASEDIR%%-syntax to not break backwards compatibility. Besides, depending on the next point we still need it to be able to point to a different sub directory within CASEDIR then needles.
  2. We should consider what we do if NEEDLES_DIR contains a relative path. So far it is treated relative to the default/web-UI checkout which makes also sense as this is so far also our default in the absence of the setting. Now we might want to change it so the %%CASEDIR%%/-prefix is the default if that directory exists (and e.g. the new prefix %%DEFAULT_BRANCH%%/ would be used for the current default behavior).
  3. I'm personally not sure whether all of this is worth it or even a good idea. Specifying NEEDLES_DIR=%%CASEDIR%%/needles doesn't really hurt.
Actions
Copy link
 #4

Updated by tinita 18 days ago · Edited

Make the code for checking out Git repositories for the CASEDIR/NEEDLES_DIR from os-autoinst available to the openQA worker. The easiest approach would be to ensure the code is in a separate module (not some big utils.pm) and simply add that module to the openQA worker's Perl module search path. We can also make os-autoinst's install script install the module to the normal Perl search path.

I don't understand what you have in mind here. Can you elaborate?

Especially add that module to the openQA worker's Perl module search path sounds hacky and I don't understand why we would manyally have to add something to the standard perl module paths.

Actions
Copy link
 #5

Updated by tinita 18 days ago

Note that specifying NEEDLES_DIR=%%CASEDIR%%/needles would not have been necessary. However, ...

Necessary for what?

Maybe I also don't get the goal For tests with CASEDIR pointing to a git repo including tests+needles we understand what is necessary where it says we should understand what's necessary, but it doesn't say necessary for what.

Actions
Copy link
 #6

Updated by mkittler 18 days ago · Edited

tinita wrote in #note-4:

Make the code for checking out Git repositories for the CASEDIR/NEEDLES_DIR from os-autoinst available to the openQA worker. The easiest approach would be to ensure the code is in a separate module (not some big utils.pm) and simply add that module to the openQA worker's Perl module search path. We can also make os-autoinst's install script install the module to the normal Perl search path.

I don't understand what you have in mind here. Can you elaborate?

Especially add that module to the openQA worker's Perl module search path sounds hacky and I don't understand why we would manyally have to add something to the standard perl module paths.

The Perl modules provided by os-autoinst are installed under /usr/lib/os-autoinst so we have to add this path to the search path in openQA worker to use them. Or we make sure the required subset of the Perl modules provided by os-autoinst is installed under the standard Perl module paths, e.g. /usr/lib/perl5/vendor_perl/5.40.0/os_autoinst.

tinita wrote in #note-5:

Note that specifying NEEDLES_DIR=%%CASEDIR%%/needles would not have been necessary. However, ...

Necessary for what?

It is not necessary to run the test. However, if you want to load needles also from the branch specified via CASEDIR=… you have to add it.

Maybe I also don't get the goal For tests with CASEDIR pointing to a git repo including tests+needles we understand what is necessary where it says we should understand what's necessary, but it doesn't say necessary for what.

Yes, it doesn't say that. Hence I was trying to explain when you need NEEDLES_DIR=%%CASEDIR%%/needles and when you don't. It really depends on your use case. But the first paragraphs of my previous comment aren't that important. What's important is everything as of "For #97112 we need to change the following behavior:".

When I think more about it I'm really not sure whether I'm liking #97112. Why should specifying CASEDIR=…#some-branch change the branch we load needles from? After all, the variable is called CASEDIR and if I wanted needles from a custom branch as well then for this we have NEEDLES_DIR=…#some-branch or the also very explicit NEEDLES_DIR=%%CASEDIR%%/needles.

Actions
Copy link
 #7

Updated by mkittler 17 days ago

PR to improve the documentation: https://github.com/os-autoinst/openQA/pull/5799

Actions
Copy link
 #8

Updated by okurz 17 days ago

  • Status changed from Feedback to Resolved

https://github.com/os-autoinst/openQA/pull/5799 merged. With this we can conclude.

Actions
Copy link

Also available in: Atom PDF