action #169159
openAllow variable expansion incorporating worker settings size:S
0%
Description
Motivation¶
This would be useful so job settings referring to assets (e.g. QA_HEAD_REPO=https://download.suse.de/ibs/…) could be written in a more generic way (e.g. QA_HEAD_REPO=https://%REPO_MIRROR_HOST%/ibs/…). This way the download server could be replaced by a mirror more easily. Considering ideas like https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/924 it would be useful if worker settings are considered as well. See https://suse.slack.com/archives/C02CANHLANP/p1730383116044599 for the related discussion with @szarate.
This allows convenient variable evaluation in the worker so that we do not need to do that in test code, e.g. osado, in various place.
Acceptance criteria¶
- AC1: Placeholders in job settings are substituted taking settings from the worker config into account.
- AC2: The variable substitution introduced via AC1 is in-line with https://open.qa/docs/#_variable_expansion (and internally uses the same code as much as possible).
Suggestions¶
- Check what function is used to implement placeholders. Invoke that function in
isotovideo.pmin the openQA-worker as well before the asset handling is done. - The webUI already replaces variables. We could keep variables
- Consider a default value, e.g.
%REPO_MIRROR_HOST:my.default.host%so that we don't need to set variables on all workers. The default must then only be evaluated in the Last Instance (!) wherever that is, possibly worker. As alternative os-autoinst can be used.
Further details¶
- This feature is intended e.g. to be used with SLE maintenance tests so also qem-bot would need to be changed to only set a variable, possibly with default value
- For the purpose of OSD this feature only is usable if non-CC OSD workers have access to openqa.suse.de webUI, e.g. need other currently open tickets from https://progress.opensuse.org/issues?parent_id=~165282&set_filter=true&status_id=o
- The initial part about "before the worker cache asset download" does not make sense because worker asset caching only downloads assets from OSD, never from external resources so variable expansion is not useful there
Updated by szarate 23 days ago
- Related to action #168097: [qe-core] Make openqa.suse.de tests work with mirrors instead of dist.suse.de or download.suse.de added
Updated by okurz 19 days ago
- Assignee set to okurz
- Parent task set to #166598
mkittler wrote:
- AC1: Placeholders in job settings are substituted before the asset download taking settings from the worker config into account.
if that would be true how would worker specific settings be evaluated then?
Also, can you please explain the relation to already existing variable expansion as documented on https://open.qa/docs/#_variable_expansion ?
Updated by szarate 18 days ago
Please read the slack thread mentioned above.
okurz wrote in #note-3:
mkittler wrote:
- AC1: Placeholders in job settings are substituted before the asset download taking settings from the worker config into account.
if that would be true how would worker specific settings be evaluated then?
once the job is assigned to the worker, but before asset downloading starts, if that's not possible an alternative solution is needed
Also, can you please explain the relation to already existing variable expansion as documented on https://open.qa/docs/#_variable_expansion ?
Variable expansion is only supported at job creation time. See slack thread.
In general, we're looking to solve the problem described in the thread, at medium and test setting level.
Updated by okurz 18 days ago
- Subject changed from Allow variable expansion before a job starts to Allow variable expansion incorporating worker settings size:S
- Description updated (diff)
- Status changed from New to Workable
- Assignee deleted (
okurz) - Priority changed from Normal to Urgent
ok, so we clarified some points. I understood that szarate wants to urgently have that so let's give him that :)
@szarate please be aware that this will not help us to get tests running in NUE2 as currently not even NUE2 based OSD workers can connect to OSD webUI which would need the tickets from https://progress.opensuse.org/issues?parent_id=~166598&set_filter=true&status_id=o as well, at time of writing 7, e.g.
Updated by openqa_review 17 days ago
- Due date set to 2024-11-20
Setting due date based on mean cycle time of SUSE QE Tools
Updated by mkittler 16 days ago
- Status changed from In Progress to Feedback
The change has been deployed. So we could move https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/924 forward (although I'm currently checking whether we can do better than mentioning the default download.suse.de on all other workers).
Updated by szarate 16 days ago
Marius, I had
mkittler wrote in #note-12:
The change has been deployed. So we could move https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/924 forward (although I'm currently checking whether we can do better than mentioning the default
download.suse.deon all other workers).
I created: https://progress.opensuse.org/issues/169492 and crossed the respective line on Confluence
Updated by mkittler 12 days ago
- Blocked by action #169564: Configure wireguard tunnels on hosts located in the NUE2 server room size:S added