Project

General

Profile

Actions

action #159171

open

coordination #58184: [saga][epic][use case] full version control awareness within openQA

coordination #152847: [epic] version control awareness within openQA for test distributions

Create and maintain up to date version of test distri/needles for webui size:M

Added by dheidler 13 days ago. Updated about 9 hours ago.

Status:
In Progress
Priority:
High
Assignee:
Category:
Feature requests
Target version:
Start date:
2024-04-17
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Consider transient openQA instances like openQA-single-instance-container but also new instances where we do not want to rely on admins to manually put stuff into /var/lib/openqa/share/tests. For those we want to ensure that openQA automatically clones git repos specified as part of tests but also keeps those working copies up to date.

Acceptance criteria

  • AC1: Given CASEDIR has a git repo And /var/lib/openqa/share/tests does not already have the corresponding distri checked out When tests are created Then openQA clones the test repo
  • AC2: Same as AC1 but for NEEDLE_DIR
  • AC3: Whenever new tests are created and repo already exists Then the git repo is updated
  • AC4: Given CASEDIR with specific commit in refspec When tests are created Then the git repo still stays on that refspec

Suggestions

The following describes something like a TOFU approach, where the first CASEDIR/NEEDLE_DIR for a DISTRI
is deemed to be the authoritative one in the future.

  • Job gets created
  • If job has CASEDIR / NEEDLE_DIR:
    • CASEDIR / NEEDLE_DIR is cloned or updated on the webui server
    • See HDD=1=http:// mechanism
    • Where to actually clone to? --> /var/lib/openqa/share/tests/DISTRI/products/DISTRI/needles
    • (- What to do with different forks? -> use https://git-scm.com/docs/git-worktree)
    • If there is not /var/lib/openqa/share/tests/DISTRI, clone CASEDIR to /var/lib/openqa/share/tests/DISTRI
    • If there is /var/lib/openqa/share/tests/DISTRI, only update it if git url is equal origin url in /var/lib/openqa/share/tests/DISTRI and branch in CASEDIR is equal checked out branch in /var/lib/openqa/share/tests/DISTRI
    • Analog with NEEDLE_DIR and /var/lib/openqa/share/tests/DISTRI/products/DISTRI/needles
  • Job is assigned

  • Also make sure that fetchneedles doesn't interfere.

  • The initial clone as well as updating should be handled in asynchronous tasks like ..._URL asset downloading


Related issues 2 (2 open0 closed)

Related to openQA Project - action #157273: Run os-autoinst-distri-openQA directly from git without anything related in /var/lib/openqa/share/tests - Follow-up with ideas not tried out in the spike solution #157270 size:MBlockeddheidler2024-04-17

Actions
Blocks openQA Project - action #159174: Decrease load on remote git servers by not cloning CASEDIR/NEEDLE_DIR from every workerNewokurz2024-04-17

Actions
Actions #1

Updated by dheidler 13 days ago

  • Blocks action #159174: Decrease load on remote git servers by not cloning CASEDIR/NEEDLE_DIR from every worker added
Actions #2

Updated by dheidler 13 days ago

  • Description updated (diff)
Actions #3

Updated by tinita 13 days ago

  • Target version set to Tools - Next
Actions #4

Updated by okurz 12 days ago

  • Parent task changed from #157273 to #152847
Actions #5

Updated by okurz 12 days ago

  • Related to action #157273: Run os-autoinst-distri-openQA directly from git without anything related in /var/lib/openqa/share/tests - Follow-up with ideas not tried out in the spike solution #157270 size:M added
Actions #6

Updated by okurz 12 days ago

  • Category set to Feature requests
  • Assignee set to okurz

will need to think about this

Actions #7

Updated by okurz 11 days ago

  • Assignee deleted (okurz)
  • Target version changed from Tools - Next to Ready

The initial clone as well as updating should be handled in asynchronous tasks like ..._URL asset downloading

Actions #8

Updated by jbaier_cz 5 days ago

  • Subject changed from Create and maintain up to date version of test distri/needles for webui to Create and maintain up to date version of test distri/needles for webui size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #9

Updated by okurz 5 days ago

  • Priority changed from Normal to High
Actions #10

Updated by dheidler about 9 hours ago

  • Status changed from Workable to In Progress
  • Assignee set to dheidler
Actions

Also available in: Atom PDF