action #159171
opencoordination #58184: [saga][epic][use case] full version control awareness within openQA
coordination #152847: [epic] version control awareness within openQA for test distributions
Create and maintain up to date version of test distri/needles for webui size:M
Description
Motivation¶
Consider transient openQA instances like openQA-single-instance-container but also new instances where we do not want to rely on admins to manually put stuff into /var/lib/openqa/share/tests. For those we want to ensure that openQA automatically clones git repos specified as part of tests but also keeps those working copies up to date.
Acceptance criteria¶
- AC1:
- Given CASEDIR has a git repo and /var/lib/openqa/share/tests does not already have the corresponding distri checked out
- When tests are created
- Then openQA clones the test repo
- AC2: Same as AC1 but for NEEDLE_DIR
- AC3:
- When new tests are created and repo already exists
- Then the git repo is updated
Suggestions¶
The following describes something like a TOFU approach, where the first CASEDIR/NEEDLE_DIR for a DISTRI
is deemed to be the authoritative one in the future.
- Job gets created
- If job has CASEDIR / NEEDLE_DIR:
- CASEDIR / NEEDLE_DIR is cloned or updated on the webui server
- See HDD=1=http:// mechanism
- Where to actually clone to? --> /var/lib/openqa/share/tests/DISTRI/products/DISTRI/needles
(- What to do with different forks? -> use https://git-scm.com/docs/git-worktree)- If there is not /var/lib/openqa/share/tests/DISTRI, clone CASEDIR to /var/lib/openqa/share/tests/DISTRI
- If there is /var/lib/openqa/share/tests/DISTRI, only update it if git url is equal origin url in /var/lib/openqa/share/tests/DISTRI and branch in CASEDIR is equal checked out branch in /var/lib/openqa/share/tests/DISTRI
- Analog with NEEDLE_DIR and /var/lib/openqa/share/tests/DISTRI/products/DISTRI/needles
Job is assigned
Also make sure that fetchneedles doesn't interfere.
The initial clone as well as updating should be handled in asynchronous tasks like ..._URL asset downloading
Updated by dheidler about 1 month ago
- Blocks action #159174: Decrease load on remote git servers by not cloning CASEDIR/NEEDLE_DIR from every worker added
Updated by okurz about 1 month ago
- Related to action #157273: Run os-autoinst-distri-openQA directly from git without anything related in /var/lib/openqa/share/tests - Follow-up with ideas not tried out in the spike solution #157270 size:M added
Updated by okurz about 1 month ago
- Category set to Feature requests
- Assignee set to okurz
will need to think about this
Updated by okurz about 1 month ago
- Assignee deleted (
okurz) - Target version changed from Tools - Next to Ready
The initial clone as well as updating should be handled in asynchronous tasks like ..._URL asset downloading
Updated by openqa_review 20 days ago
- Due date set to 2024-05-15
Setting due date based on mean cycle time of SUSE QE Tools
Updated by dheidler 7 days ago
- Status changed from In Progress to Workable
how to detect if a given string is a branch name, a tag name or a commit in a remote repository:
# git ls-remote --symref https://github.com/openSUSE/opi guoyunhe-patch-1
# -> \srefs/heads/ -> branch
# git ls-remote --symref https://github.com/openSUSE/opi v5.0.0
# -> \srefs/tags/ -> tag -> git fetch --tags --force
# else refspec is commit
# -> git fetch --all
Updated by livdywan 6 days ago
- Copied to action #160379: Support web UI checkouts of CASEDIR with a tag reference in a refspec added
Updated by livdywan 6 days ago
- Copied to action #160382: Support web UI checkouts of CASEDIR with a specific commit specified in a refspec added
Updated by dheidler about 3 hours ago
- Status changed from In Progress to Feedback