action #154237
closedcoordination #154777: [saga][epic] Shareable os-autoinst and test distribution plugins
coordination #108527: [epic] os-autoinst wheels for scalable code reuse of helper functions and segmented test distributions
[spike][timeboxed:10h] Ensure the worker cache doesn't duplicate git caching of test distributions on o3 size:S
Description
Motivation¶
As part of #138029 support for git caching was included in os-autoinst covering test distributions as well as wheel repository. We can now look into caching test distributions from git fully enabled on a production openQA instance but also considering the worker cache service, e.g. circumvent the worker cache service or actually make use of it.
Acceptance criteria¶
- AC1: There is no redundant syncing in the worker cache service, e.g. no cloning from github but the worker cache syncs test data that is never used
Suggestions¶
- See what has been done in #154156
- Consider the impact of the worker cache service, e.g. take a look into logs of affected openQA jobs and see how caching and git caching interact or not interact
- One simple approach could be to simply not trigger syncing the worker test distribution directory caching in openQA jobs that have GIT_CACHE_DIR+CASEDIR+NEEDLESDIR includes a git URL set
- Checkout the documentation of the worker cache to get started: https://open.qa/docs/#asset-caching
- Consider adding something to the docs to make "worker cache" more discoverable
- Add GIT_CACHE_DIR to the docs
Out of scope¶
- Manage storage capacity long-term / clean-up
Updated by okurz 10 months ago
- Copied from action #154156: [spike][timeboxed:10h] Cache test distributions from git on production size:S added
Updated by okurz 10 months ago
- Copied to action #154240: Ensure cloning openQA jobs with GIT_CACHE_DIR works in usual use cases added
Updated by livdywan 9 months ago
- Subject changed from [spike][timeboxed:10h] Ensure the worker cache service is not calling redundant syncing for jobs using cached git cloned test and/or needle distributions to [spike][timeboxed:10h] Ensure the worker cache doesn't duplicate git caching of test distributions on o3 size:S
- Description updated (diff)
- Status changed from New to Workable
Updated by okurz 9 months ago
Reverted with https://github.com/os-autoinst/openQA/pull/5498
Updated by mkittler 9 months ago
- Status changed from Feedback to Resolved
The follow-up PR https://github.com/os-autoinst/openQA/pull/5499 was merged yesterday and hasn't caused any big regressions anymore so far. So I consider this ticket resolved.