action #115763
openAdd new feature to create a dependency that could be derived automatically from asset settings
0%
Description
Motivation¶
The use case is to be able to perform migrations between two products including unreleased maintenance updates for both products, so we can include the unreleased maintenance updates in the product from which we migrate and the unreleased maintenance updates in the product to which we migrated to. For example migration from SLE-15-SP3 to SLE-15-SP4. Both have different unreleased maintenance updates according to their job group and how it was created via isos post with bot-ng:
https://openqa.suse.de/tests/9384697#settings
https://openqa.suse.de/tests/9384871#settings
Technically dependencies can be between any jobs. When specifying them via test suite variables and scheduling an iso you're of course limited to jobs generated by that isos post call. If you post jobs manually you can have dependencies between them as you like.
With posting jobs manually it means using this API (and not the isos post API): https://open.qa/docs/#_spawning_single_new_jobs_jobs_post
When using the isos post API you'll create a bunch of jobs generated by certain settings (test suites, products, …). Dependencies can only be specified within that set of jobs. You cannot specify dependencies to jobs that were created or will be created by another isos post API call.
Note that technically every job can depend on any other job. It is just that the isos post API has no way of specifying dependencies outside of the scope of the current API call. (And that scope is called a "scheduled product" in openQA.).
Suggestions¶
(1) Modify the bot-ng to be able to publish setting from more than one product -> probably a total mess of settings in the final job, not worth it.
(2) Modify the bot-ng heavily so it can go posting job by job, line by line add corresponding settings with maintenance updates for each job -> even if it is feasible, unmaintainable by corresponding team in charge of this bot according to developer.
(3) Being able to establish dependencies between jobs in different product -> probably the best way as doesn't change openQA design neither bot-ng design.
Suggestion from Marius is:
"Maybe it would then be useful if the dependency could be derived automatically from asset settings"
which translate to be able "run a job only after asset actually exists".
Priorities regarding testing of unreleased maintenance updates:
https://confluence.suse.com/display/~hrommel1/Blue-Print+of+Online+Migration+Tests+Between+Two+Service+Packs+in+Maintenance
Updated by JERiveraMoya about 2 years ago
- Project changed from openQA Tests to 175
- Category deleted (
Infrastructure)
Updated by JERiveraMoya about 2 years ago
- Project changed from 175 to openQA Infrastructure
This feature doesn't resolve the problem fully, because there are migration as well with remote workers (ppc64le PowerVM & s390x).
I was expecting that something like START_DIRECTLY_AFTER_ASSET START_AFTER_ASSET would be feasible but according to @Marius:
"Out of my head I would say direct chaining breaks the idea. For that all jobs needed to be created in one API call. Otherwise we needed to reserve the worker slot just in case a new job would be scheduled that is supposed to run directly after which would be a whole new thing breaking the current architecture of directly chained jobs."
We might attempt to do partial migration first. instead of having the updates from both products.
I will update this ticket according to progress with migration if this feature could be key or not (perhaps partial migration has less value than full migration but limited to architecture where we can have assets to work for, but I don't think so).
Updated by mkittler about 2 years ago
Note that only START_DIRECTLY_AFTER_ASSET would be a problem (as explained in the previous comment). We could implement START_AFTER_ASSET. For START_DIRECTLY_AFTER_ASSET we could maybe come up with some kind of hack, though. E.g. when scheduling the first job we would specify some indication that further directly chained jobs are still being created (not sure how that would work exactly, though).
Updated by JERiveraMoya about 2 years ago
- Related to coordination #113644: [Epic] Add migration paths with unreleased maintenance updates added