Project

General

Profile

Actions

coordination #108527

open

coordination #154777: [saga][epic] Shareable os-autoinst and test distribution plugins

[epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions

Added by okurz over 2 years ago. Updated 6 months ago.

Status:
Blocked
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2021-01-08
Due date:
% Done:

63%

Estimated time:
(Total: 0.00 h)

Description

Motivation

A common problem within test distributions is that they need common functionalities shared but often resort to "copy-pasting" which is bad. E.g. #106922 shows a problem related to the helper function lib::utils::ensure_unlocked_desktop which we also have in os-autoinst-distri-opensuse. We want to keep os-autoinst-distri-openQA as separate distribution so that we have a good showcase of a realistic openQA test distribution that we as SUSE QE Tools care about ourselves without being as simple as a plain "example" test distribution and also not as complicated or "monolithic" as os-autoinst-distri-opensuse. We should look for a way to better share code among test distributions, e.g. could be git submodules, git subrepos, perl helper packages, etc.

Considered approaches for improving scalability of code reuse within the scope of os-autoinst and test distributions

Discussing different approaches how to "share code" or functionality.

  1. copy-paste: Easiest but hard to maintain, that's what we want to avoid
    1. What we currently do in many cases
  2. Include more in os-autoinst directly,
    1. We already do that within the "distribution.pm", and e.g. os-autoinst-distri-opensuse inherits with its "susedistribution.pm"
    2. e.g. comparable to os-autoinst/distribution.pm ensure_installed which already has install functions for Fedora and Debian (last changed 7 years ago, maybe not even used by them at all). The current if/else is not nice, better would be proper inheritance
    3. Easiest might be code that does not use needle tags
    4. If we include helper code which is relying on needle tags
      1. then the needle tags are becoming part of the API so on changes proper deprecation process should be conducted.
      2. New needles are stored by default in test distribution code, so not "automatically" forwarded to os-autoinst
      3. We could still try to include just a single needle, like an "example" or "template"
      4. We needed support for multiple needle repositories
  3. SELECTED helper functions in plugin repositories that can be loaded into os-autoinst or the test distribution by

    1. git submodule
    2. git subrepo
    3. SELECTED package, e.g. dependencies.yaml same as we do for os-autoinst and os-autoinst-distri-opensuse possible, and then provide as openSUSE package and others, e.g. Debian, can easily maintain their version on top or contribute back upstream
    4. os-autoinst pulls specified plugin dependencies, like salt formulas or "collections" from https://galaxy.ansible.com/ or https://github.com/junegunn/vim-plug

      1. similar to our custom git ref loading we could specify even tags, versions, etc., see CASEDIR in https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc
        1. cache "most used plugins and versions" on a central openQA instance, similar to assets, then included in worker cache -> 5. we need a catchy name similar to "galaxy/formulas/collections", maybe "colors" (you know, chameleon, changing colors), branches, leaves, roots, rockets, wheels, magic with scrolls, spells, potions, "something-to-sleep-over"
  4. Maybe we just need to provide better and more "customization" points within os-autoinst which can be used for overriding and encourage test writers to contribute there

  5. Functions should be small and simple for easier overriding

Implementation suggestions

  • Create a folder using perl modules, e.g. x11_start_program from https://github.com/os-autoinst/os-autoinst-distri-openQA/blob/master/lib/susedistribution.pm#L8 without the actual needle and leave it to the test distribution to provide it.
    • Start with a local git repository, e.g. file://…
    • Afterwards provide in github repo
  • DONE Provide the example as part of os-autoinst-distri-openQA, e.g. if a YAML file plugins.yaml exists next to a main.pm load all plugins from there -> https://github.com/os-autoinst/os-autoinst/pull/2133 + https://github.com/os-autoinst/os-autoinst-wheel-launcher
  • Provide template project with template README and template test definitions
  • Copy-pasting template project can make updating and maintaining hard, better provide a helper like "os-autoinst-plugin" with commands, like "ansible-galaxy". This tool can create initial folder structure ("os-autoinst-plugin" and a very bare Makefile where e.g. "make test" just calls "os-autoinst-plugin test")
  • Start with "os-autoinst-plugin" which only supports one subcommand, "help"
  • Provide os-autoinst-plugin in openSUSE subpackage
  • folder structure "lib" is automatically added to perl library search path, "needles" is added to needle search path, "tests" is added to test search path
  • Include the one example needle as part of the plugin, e.g. https://github.com/os-autoinst/os-autoinst-needles-openQA/blob/master/desktop-runner-20191108.json
  • Prefix needle names with plugin name to prevent conflicts
  • What if a plugin wants to use another plugin? For now I would not allow that. Otherwise plugin maintainer needs to spell out a list of dependencies to be pulled in, at best in machine-readable format
  • Provide a github action

  • We also need to care about needles


Subtasks 28 (11 open17 closed)

action #81899: Move code from isotovideo to a module size:MResolvedlivdywan2021-01-08

Actions
action #108530: os-autoinst wheels: x11_start_program from os-autoinst-distri-openQA dynamically loaded from another git repo size:MResolvedlivdywan2022-03-17

Actions
openQA Infrastructure - action #114914: Container devel/openqa/containers/isotovideo:qemu-x86 is outdatedRejectedokurz2022-03-17

Actions
action #115001: Add documentation how to use/develop wheels visible on open.qa/docsNew2022-08-04

Actions
action #115004: Use github.com/os-autoinst/os-autoinst-wheel-launcher in os-autoinst-distri-opensuse as wellRejectedszarate2022-09-16

Actions
action #115013: os-autoinst-plugin as a wheel helper toolNew2022-08-04

Actions
action #115016: Conduct a workshop to introduce wheels and discuss how to go forwardNew2022-08-04

Actions
action #115019: os-autoinst/wheel-boot which can be used to test os-autoinst/wheel-launcher size:MWorkable2022-08-04

Actions
action #115022: Complete unit test coverage (with coverage analysis) in os-autoinst/wheel-launcherResolvedmkittler2022-08-04

Actions
coordination #117097: Evaluate GitHub template repositories for wheelsNew2022-09-23

Actions
coordination #117673: [epic][tools] sporadic "Unable to clone Git repository" for wheelsNew2022-10-13

Actions
action #118633: Re-try on cloning of wheel repositories size:MResolvedlivdywan2022-10-13

Actions
action #118882: Make available wheels more discoverable and share knowledge size:SWorkable2022-10-13

Actions
action #118903: Repositories for wheels should be cachedNew2022-10-13

Actions
action #130369: [spike][timeboxed:20h] Reduce duplication of openQA-in-openQA tests in os-autoinst-distri-opensuse and os-autoinst-distri-openQA size:SResolvedtinita2023-06-01

Actions
action #133301: quick-fix for openQA-in-openQA lockscreen race conditionResolvedokurz2022-08-04

Actions
action #134390: Log proactively where wheel components are located to ease debugging size:MResolvedlivdywan2023-06-01

Actions
action #138029: [research][timeboxed:10h] How to cache "wheel" repositories which are stored on github size:MResolvedmkittler

Actions
action #138032: Find out most/least used testapi functions to decide about where to extend/cleanup size:MResolvedjbaier_cz

Actions
action #154237: [spike][timeboxed:10h] Ensure the worker cache doesn't duplicate git caching of test distributions on o3 size:SResolvedmkittler

Actions
action #154240: Ensure cloning openQA jobs with GIT_CACHE_DIR works in usual use casesResolvedmkittler2024-01-25

Actions
action #156859: Improve most used testapi functionsNew

Actions
action #156862: Consider removing/reworking least used testapi functionsNew

Actions
action #157339: os-autoinst t/14-isotovideo.t is again taking too long (>20s on my setup) size:MResolvedybonatakis2024-03-15

Actions
action #157540: [sporadic] ci openQA: t/33-developer_mode.t fails size:MResolvedmkittler2024-03-19

Actions
action #157543: [sporadic] ci openQA: t/ui/23-audit-log.t fails size:MResolvedmkittler2024-03-19

Actions
action #158422: flaky sporadic test failures t/ui/13-admin.tResolvedokurz2024-04-02

Actions
action #158511: Try to optimize the startup time of isotovideo moreNew

Actions

Related issues 1 (1 open0 closed)

Copied from openQA Project - coordination #106922: [epic][sporadic] openqa_from_git fails in dashboard due to ensure_unlocked_desktop not expecting password entry screen in case of locked desktop auto_review:"match=desktop-runner,screenlock timed out.*":retryBlockedokurz2022-02-16

Actions
Actions #1

Updated by okurz over 2 years ago

  • Copied from coordination #106922: [epic][sporadic] openqa_from_git fails in dashboard due to ensure_unlocked_desktop not expecting password entry screen in case of locked desktop auto_review:"match=desktop-runner,screenlock timed out.*":retry added
Actions #2

Updated by okurz over 2 years ago

  • Status changed from New to Blocked
  • Assignee set to okurz

ok, we have one specific subtask. Let's start with that and see where it leads us.

Actions #4

Updated by livdywan about 2 years ago

  • Subject changed from [epic] os-autoinst plugins (or wheels or leaves or scrolls) for scalable code reuse of helper functions and segmented test distributions to [epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions
  • With the conclusion of #108530 wheel-launcher becomes the first wheel including a minimal GHA self-test and a handy makefile for local development. It could serve as an example for other cases.
  • Provide a github action is an existing suggestion I already looked into on the side that could reduce copypasta and make wheels easier to develop, see my isotovideo-action.
  • A follow-up task I would additionally suggest: Use a wheel in distri-opensuse. This is where wheels will get more exposure and potentially be more beneficial compared to the small distri-openqa.

I'll also publish and share a blog post to help get people started with wheels.

Actions #5

Updated by okurz about 2 years ago

  • Status changed from Blocked to Workable
  • Assignee deleted (okurz)

To create more subtasks

Actions #6

Updated by okurz about 2 years ago

  • Description updated (diff)
Actions #7

Updated by okurz about 2 years ago

  • Status changed from Workable to Blocked
  • Assignee set to okurz
Actions #8

Updated by okurz almost 2 years ago

  • Related to coordination #58184: [saga][epic][use case] full version control awareness within openQA added
Actions #9

Updated by livdywan over 1 year ago

  • Status changed from Blocked to Workable

With the resolution of #81899 this is probably unblocked again e.g. we can pick other subtasks.

Actions #10

Updated by okurz over 1 year ago

  • Status changed from Workable to Blocked

cdywan wrote:

With the resolution of #81899 this is probably unblocked again e.g. we can pick other subtasks.

Agreed. We have enough open subtasks though, e.g. #115019 currently in the backlog so we can track this epic as "Blocked".

Actions #11

Updated by okurz over 1 year ago

  • Parent task changed from #80142 to #58184
Actions #12

Updated by okurz 12 months ago

  • Subtask #138029 added
Actions #13

Updated by okurz 12 months ago

  • Subtask #138032 added
Actions #14

Updated by okurz 9 months ago

  • Subtask #154156 added
Actions #15

Updated by okurz 9 months ago

  • Subtask #154237 added
Actions #16

Updated by okurz 9 months ago

  • Subtask #154240 added
Actions #17

Updated by okurz 9 months ago

  • Parent task changed from #58184 to #154777
Actions #18

Updated by okurz 9 months ago

  • Subtask deleted (#132335)
Actions #19

Updated by okurz 9 months ago

  • Subtask deleted (#154156)
Actions #20

Updated by okurz 9 months ago

  • Target version changed from Ready to Tools - Next
Actions #21

Updated by okurz 7 months ago

  • Subtask #156859 added
Actions #22

Updated by okurz 7 months ago

  • Subtask #156862 added
Actions #23

Updated by okurz 7 months ago

  • Subtask #157339 added
Actions #24

Updated by okurz 7 months ago

  • Subtask #157540 added
Actions #25

Updated by okurz 7 months ago

  • Subtask #157543 added
Actions #26

Updated by okurz 7 months ago

  • Target version changed from Tools - Next to future
Actions #27

Updated by okurz 6 months ago

  • Subtask #158422 added
Actions #28

Updated by okurz 6 months ago

  • Subtask #158511 added
Actions

Also available in: Atom PDF