coordination #108527
opencoordination #154777: [saga][epic] Shareable os-autoinst and test distribution plugins
[epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions
63%
Description
Motivation¶
A common problem within test distributions is that they need common functionalities shared but often resort to "copy-pasting" which is bad. E.g. #106922 shows a problem related to the helper function lib::utils::ensure_unlocked_desktop which we also have in os-autoinst-distri-opensuse. We want to keep os-autoinst-distri-openQA as separate distribution so that we have a good showcase of a realistic openQA test distribution that we as SUSE QE Tools care about ourselves without being as simple as a plain "example" test distribution and also not as complicated or "monolithic" as os-autoinst-distri-opensuse. We should look for a way to better share code among test distributions, e.g. could be git submodules, git subrepos, perl helper packages, etc.
Considered approaches for improving scalability of code reuse within the scope of os-autoinst and test distributions¶
Discussing different approaches how to "share code" or functionality.
- copy-paste: Easiest but hard to maintain, that's what we want to avoid
- What we currently do in many cases
- Include more in os-autoinst directly,
- We already do that within the "distribution.pm", and e.g. os-autoinst-distri-opensuse inherits with its "susedistribution.pm"
- e.g. comparable to os-autoinst/distribution.pm ensure_installed which already has install functions for Fedora and Debian (last changed 7 years ago, maybe not even used by them at all). The current if/else is not nice, better would be proper inheritance
- Easiest might be code that does not use needle tags
- If we include helper code which is relying on needle tags
- then the needle tags are becoming part of the API so on changes proper deprecation process should be conducted.
- New needles are stored by default in test distribution code, so not "automatically" forwarded to os-autoinst
- We could still try to include just a single needle, like an "example" or "template"
- We needed support for multiple needle repositories
SELECTED helper functions in plugin repositories that can be loaded into os-autoinst or the test distribution by
- git submodule
- git subrepo
- SELECTED package, e.g. dependencies.yaml same as we do for os-autoinst and os-autoinst-distri-opensuse possible, and then provide as openSUSE package and others, e.g. Debian, can easily maintain their version on top or contribute back upstream
os-autoinst pulls specified plugin dependencies, like salt formulas or "collections" from https://galaxy.ansible.com/ or https://github.com/junegunn/vim-plug
- similar to our custom git ref loading we could specify even tags, versions, etc., see CASEDIR in https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc
- cache "most used plugins and versions" on a central openQA instance, similar to assets, then included in worker cache -> 5. we need a catchy name similar to "galaxy/formulas/collections", maybe "colors" (you know, chameleon, changing colors), branches, leaves, roots, rockets, wheels, magic with scrolls, spells, potions, "something-to-sleep-over"
- similar to our custom git ref loading we could specify even tags, versions, etc., see CASEDIR in https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc
Maybe we just need to provide better and more "customization" points within os-autoinst which can be used for overriding and encourage test writers to contribute there
Functions should be small and simple for easier overriding
Implementation suggestions¶
- Create a folder using perl modules, e.g. x11_start_program from https://github.com/os-autoinst/os-autoinst-distri-openQA/blob/master/lib/susedistribution.pm#L8 without the actual needle and leave it to the test distribution to provide it.
- Start with a local git repository, e.g. file://…
- Afterwards provide in github repo
- DONE Provide the example as part of os-autoinst-distri-openQA, e.g. if a YAML file plugins.yaml exists next to a main.pm load all plugins from there -> https://github.com/os-autoinst/os-autoinst/pull/2133 + https://github.com/os-autoinst/os-autoinst-wheel-launcher
- Provide template project with template README and template test definitions
- Copy-pasting template project can make updating and maintaining hard, better provide a helper like "os-autoinst-plugin" with commands, like "ansible-galaxy". This tool can create initial folder structure ("os-autoinst-plugin" and a very bare Makefile where e.g. "make test" just calls "os-autoinst-plugin test")
- Start with "os-autoinst-plugin" which only supports one subcommand, "help"
- Provide os-autoinst-plugin in openSUSE subpackage
- folder structure "lib" is automatically added to perl library search path, "needles" is added to needle search path, "tests" is added to test search path
- Include the one example needle as part of the plugin, e.g. https://github.com/os-autoinst/os-autoinst-needles-openQA/blob/master/desktop-runner-20191108.json
- Prefix needle names with plugin name to prevent conflicts
- What if a plugin wants to use another plugin? For now I would not allow that. Otherwise plugin maintainer needs to spell out a list of dependencies to be pulled in, at best in machine-readable format
Provide a github action
We also need to care about needles
Updated by okurz over 2 years ago
- Copied from coordination #106922: [epic][sporadic] openqa_from_git fails in dashboard due to ensure_unlocked_desktop not expecting password entry screen in case of locked desktop auto_review:"match=desktop-runner,screenlock timed out.*":retry added
Updated by okurz over 2 years ago
- Status changed from New to Blocked
- Assignee set to okurz
ok, we have one specific subtask. Let's start with that and see where it leads us.
Updated by livdywan about 2 years ago
- Subject changed from [epic] os-autoinst plugins (or wheels or leaves or scrolls) for scalable code reuse of helper functions and segmented test distributions to [epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions
- With the conclusion of #108530 wheel-launcher becomes the first wheel including a minimal GHA self-test and a handy makefile for local development. It could serve as an example for other cases.
- Provide a github action is an existing suggestion I already looked into on the side that could reduce copypasta and make wheels easier to develop, see my isotovideo-action.
- A follow-up task I would additionally suggest: Use a wheel in distri-opensuse. This is where wheels will get more exposure and potentially be more beneficial compared to the small distri-openqa.
I'll also publish and share a blog post to help get people started with wheels.
Updated by okurz about 2 years ago
- Status changed from Blocked to Workable
- Assignee deleted (
okurz)
To create more subtasks
Updated by okurz about 2 years ago
- Status changed from Workable to Blocked
- Assignee set to okurz
Updated by okurz almost 2 years ago
- Related to coordination #58184: [saga][epic][use case] full version control awareness within openQA added
Updated by livdywan over 1 year ago
- Status changed from Blocked to Workable
With the resolution of #81899 this is probably unblocked again e.g. we can pick other subtasks.
Updated by okurz over 1 year ago
- Status changed from Workable to Blocked