Project

General

Profile

Actions

action #182063

closed

coordination #162539: [saga][epic] future ideas version for version control features within openQA

coordination #181886: [epic] Good git defaults for new instances while keeping compatibility for old instances

load-templates --clean cannot work with YAML job templates

Added by AdamWill 11 days ago. Updated about 12 hours ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2025-05-08
Due date:
% Done:

0%

Estimated time:

Description

Trying to use openqa-load-templates --clean if any job group has YAML templates doesn't work. This is because --clean attempts to delete all machines, test suites and products - but as soon as it tries to delete a machine that is referenced in any YAML template, this fails on the _verify_table_usage check in the table API endpoint.

I hacked up a possible fix for this, but not sure if it's desired. The fix is to extend the job group update API to accept 'template' as a validated optional parameter, and if it is passed as an empty string, set it to undef in load_properties, as we do already for size_limit_gb. That allows us to do this in load-templates:

if ($options{'clean'}) {
    # wipe existing job group template strings
    my $job_groups_url = $url->clone->path($options{apibase} . '/job_groups');
    my $existing_groups = $client->get($job_groups_url)->res->json;
    for my $group (@$existing_groups) {
        my $job_group_url = $url->clone->path($options{apibase} . '/job_groups/' . $group->{id});
        my $yaml_res = $client->put(
            $job_group_url,
            form => {
                template => '',
                name => $group->{name},
            })->res;
        print_error($yaml_res) unless $yaml_res->is_success;
    }
    for my $table (@tables) {
        next if ($table eq "JobGroups");
        my $table_url = $url->clone->path($options{apibase} . '/' . decamelize($table));
...

which wipes the job group template strings as the first step of cleanup, then continues on. This allows --clean to work with YAML data. But it also introduces a new API endpoint for writing arbitrary unchecked template string data. We could also restrict it to only accept the empty string as a special hidden case. Or instead add a new API endpoint for wiping job group template strings. Or just not fix this (but then I don't see how you can ever keep your templates under version control and use the YAML format?)

I'd rather not go too far down a path that would not be approved, so filing this to look for feedback.

Actions #1

Updated by okurz 11 days ago

  • Category set to Regressions/Crashes
  • Target version set to Ready
Actions #2

Updated by okurz 11 days ago

  • Parent task set to #181886
Actions #3

Updated by AdamWill 10 days ago

Do you have any thoughts on the best approach for fixing this, @okurz ?

Actions #4

Updated by okurz 6 days ago

AdamWill wrote in #note-3:

Do you have any thoughts on the best approach for fixing this, @okurz ?

err, sorry, don't really know. But can I assume you are fixing this with https://github.com/os-autoinst/openQA/pull/6446 or at least working in a related area?

Actions #5

Updated by AdamWill 5 days ago ยท Edited

Well, I came up with something and made it a new PR: https://github.com/os-autoinst/openQA/pull/6455 . I realized we don't need the new API endpoint as we can just post whatever YAML template string we want via job_templates_scheduling. So if we implement 'post empty template to any group with a non-empty template' and move JobGroup cleaning to the front of the list, I think it works as it should.

Actions #6

Updated by okurz about 12 hours ago

  • Status changed from New to Resolved
  • Assignee set to okurz

https://github.com/os-autoinst/openQA/pull/6455 is merged and caused problems as of now. I assume we are good with that.

Actions

Also available in: Atom PDF