This is a continuation of #87755

The problem¶

The product definitions in metadata are not updated in time when a new product shows up which results in templates not being generated properly.

Possible solution¶

As mentioned in https://gitlab.suse.de/qa-maintenance/metadata/-/issues/11 there is a possibility to use REST API from SMELT.

Acceptance criteria¶

• AC1: Product definitions in metadata do not need manual edits

Suggestions¶

• Read the proof of concept script from @jbaier_cz https://gitlab.suse.de/qa-maintenance/metadata/-/blob/master/product_definitions/get_products.pl and crosscheck it with https://gitlab.suse.de/qa-maintenance/metadata/-/blob/master/template_generator/productdefs.pm
• Metadata is generated from current SMELT state on the fly
• Metadata pipeline is triggered periodically or when data is missing

Further details¶

jbaier_cz created a small proof of concept with a simple script. The main issue here would be probably in the data transformation for the corner cases. I am able, quite reliably, fetch information about modules. Basically by invoking ./get_products.pl SLE-Module-Basesystem:15:SP4 I am able to get the same data structure as we have in the metadata. I can't do it for products (not sure why, maybe I am just missing something in the script, or maybe the products API is not searchable by product name) and there are corner cases like SLE-Module-Containers 15-SP4 which is named Containers SLE-15-SP4 in our metadata (for reasons unknown to me, the comment in the code suggests it could be connected to release channels not to be present in such cases, but it also could be some relic of the past and the transformation is no longer needed).

I see two possible approaches here, one is integrating that into template generator; the second would be integrating that into the metadata package (like introducing an automated pipeline for fetching and updating the product definitions in the metadata package). I would probably incline towards the second solution as that will allow to do some phase-out (and optionally still keep the possibility to do manual changes); and if that would work, we can then take it, integrate it as a module into template generator and delete all product definitions from the metadata forever.