Project

General

Profile

Actions

action #154240

closed

coordination #154777: [saga][epic] Shareable os-autoinst and test distribution plugins

coordination #108527: [epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions

Ensure cloning openQA jobs with GIT_CACHE_DIR works in usual use cases

Added by okurz 9 months ago. Updated 9 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2024-01-25
Due date:
% Done:

0%

Estimated time:

Description

Motivation

As part of #138029 support for git caching was included in os-autoinst. We already have some special handling for CACHE directories so that people cloning without knowing about GIT_CACHE_DIR are not confronted with errors.

Acceptance criteria

  • AC1: Cloning jobs to a local openQA instance when jobs have GIT_CACHE_DIR set to a directory that does not exist locally works without errors

Acceptance tests

  • AT1-1: Given an openQA job on o3 with GIT_CACHE_DIR set When cloning that to a local instance And the specified GIT_CACHE_DIR does not exist locally And there is no locally specified GIT_CACHE_DIR Then no git caching is done And no error is given
  • AT1-2: Given an openQA job on o3 with GIT_CACHE_DIR set When cloning that to a local instance And the specified GIT_CACHE_DIR does not exist locally And there is a locally specified GIT_CACHE_DIR Then git caching is done into the locally specified GIT_CACHE_DIR

Suggestions

  • See what had been done in #154156
  • Lookup in our cloning tools and worker code how we handle cache directories
  • Add according handling for GIT_CACHE_DIR so that usual use cases for cloning to other openQA instances are handled
  • Set a local GIT_CACHE_DIR, and cross-check without it set and see what happens

Out of scope

And security considerations as we have apparmor and directory permissions for that


Related issues 2 (0 open2 closed)

Related to openQA Project - action #154156: [spike][timeboxed:10h] Cache test distributions from git on production size:SResolvedmkittler

Actions
Copied from openQA Project - action #154237: [spike][timeboxed:10h] Ensure the worker cache doesn't duplicate git caching of test distributions on o3 size:SResolvedmkittler

Actions
Actions #1

Updated by okurz 9 months ago

  • Copied from action #154237: [spike][timeboxed:10h] Ensure the worker cache doesn't duplicate git caching of test distributions on o3 size:S added
Actions #2

Updated by okurz 9 months ago

  • Subject changed from Ensure to Ensure cloning openQA jobs with GIT_CACHE_DIR works in usual use cases
Actions #3

Updated by okurz 9 months ago

  • Related to action #154156: [spike][timeboxed:10h] Cache test distributions from git on production size:S added
Actions #4

Updated by livdywan 9 months ago

  • Description updated (diff)
Actions #5

Updated by mkittler 9 months ago

  • Assignee set to mkittler

We haven't estimated the ticket yet but I think this small change will do it: https://github.com/os-autoinst/openQA/pull/5438
(I haven't conducted the acceptance tests yet.)

Actions #6

Updated by livdywan 9 months ago

  • https://github.com/os-autoinst/openQA/pull/5438
    • Change makes sense.
    • Worst case nothing will happen if someone tries to use the setting. Same for the video encoder.
  • Let's prepare a test job to check if e.g.
    • ASSETDIR=/tmp would break anything
    • CACHEDIRECTORY=~/.ssh ;-)

I'm just doing that now, so we can see what else we should look into.

Actions #7

Updated by mkittler 9 months ago

I'm just doing that now, so we can see what else we should look into.

Any results?

The PR has been merged meanwhile so we can move forward with #154156.

Actions #8

Updated by livdywan 9 months ago · Edited

Apparently my test job passed just fine despite openqa-clone-job --skip-download --skip-chained-deps --within-instance https://openqa.opensuse.org 3890816 _GROUP=0 BUILD+=poo#154240 GIT_CACHE_DIR=/tmp/ ASSETDIR=/tmp CACHEDIRECTORY=/tmp GENERAL_HW_CMD_DIR=/tmp overriding paths. CACHEDIRECTORY was replaced in vars.json whilst GIT_CACHE_DIR and GENERAL_HW_CMD_DIR are gone. Nothing exciting happened :-D (same for /home/geekotest/.ssh)

Actions #9

Updated by mkittler 9 months ago

  • Status changed from New to Resolved

The PR has been merged.

Since we don't have a job on o3 with GIT_CACHE_DIR I just used script/openqa-clone-job … https://openqa.opensuse.org/tests/3899547 GIT_CACHE_DIR=/var/lib/openqa/cache/git CASEDIR=https://github.com/os-autoinst/os-autoinst-distri-opensuse.git to test this. It works when the local worker config has GIT_CACHE_DIR (which is then used) and when it doesn't has that setting (then no caching happens). (The directory /var/lib/openqa/cache/git does not exist on my local machine.)

With that I consider this ticket resolved and we can continue with #154156.

Actions

Also available in: Atom PDF