action #182063
closedcoordination #162539: [saga][epic] future ideas version for version control features within openQA
coordination #181886: [epic] Good git defaults for new instances while keeping compatibility for old instances
load-templates --clean cannot work with YAML job templates
Description
Trying to use openqa-load-templates --clean
if any job group has YAML templates doesn't work. This is because --clean attempts to delete all machines, test suites and products - but as soon as it tries to delete a machine that is referenced in any YAML template, this fails on the _verify_table_usage
check in the table API endpoint.
I hacked up a possible fix for this, but not sure if it's desired. The fix is to extend the job group update API to accept 'template' as a validated optional parameter, and if it is passed as an empty string, set it to undef in load_properties
, as we do already for size_limit_gb
. That allows us to do this in load-templates:
if ($options{'clean'}) {
# wipe existing job group template strings
my $job_groups_url = $url->clone->path($options{apibase} . '/job_groups');
my $existing_groups = $client->get($job_groups_url)->res->json;
for my $group (@$existing_groups) {
my $job_group_url = $url->clone->path($options{apibase} . '/job_groups/' . $group->{id});
my $yaml_res = $client->put(
$job_group_url,
form => {
template => '',
name => $group->{name},
})->res;
print_error($yaml_res) unless $yaml_res->is_success;
}
for my $table (@tables) {
next if ($table eq "JobGroups");
my $table_url = $url->clone->path($options{apibase} . '/' . decamelize($table));
...
which wipes the job group template strings as the first step of cleanup, then continues on. This allows --clean to work with YAML data. But it also introduces a new API endpoint for writing arbitrary unchecked template string data. We could also restrict it to only accept the empty string as a special hidden case. Or instead add a new API endpoint for wiping job group template strings. Or just not fix this (but then I don't see how you can ever keep your templates under version control and use the YAML format?)
I'd rather not go too far down a path that would not be approved, so filing this to look for feedback.
Updated by okurz 6 days ago
AdamWill wrote in #note-3:
Do you have any thoughts on the best approach for fixing this, @okurz ?
err, sorry, don't really know. But can I assume you are fixing this with https://github.com/os-autoinst/openQA/pull/6446 or at least working in a related area?
Updated by AdamWill 5 days ago ยท Edited
Well, I came up with something and made it a new PR: https://github.com/os-autoinst/openQA/pull/6455 . I realized we don't need the new API endpoint as we can just post whatever YAML template string we want via job_templates_scheduling
. So if we implement 'post empty template to any group with a non-empty template' and move JobGroup cleaning to the front of the list, I think it works as it should.
Updated by okurz about 12 hours ago
- Status changed from New to Resolved
- Assignee set to okurz
https://github.com/os-autoinst/openQA/pull/6455 is merged and caused problems as of now. I assume we are good with that.