action #124493
closedopenqa-clone-job --skip-deps behavior contradicts documentation size:M
Description
Observation¶
Both OpenQA documentation and openqa-clone-job --help say that --skip-deps and --skip-chained-deps should only prevent cloning of parent jobs. In reality, however, both options will prevent cloning of all (chained) dependencies regardless of parent/child relationship (even when you specify --clone-children). This means there is currently no way to clone a dependency subtree without parents using openqa-clone-job. The subtree can only be restarted in webUI which does not support modifying settings of the restarted jobs.
Acceptance criteria¶
- AC1: There is a way to clone a dependency subtree without parents using
openqa-clone-job(in accordance with the documentation).
Suggestions¶
- It probably worked in the past, maybe a regression?
- Create a set of dependent jobs locally (e.g. by setting dependencies manually within the database or by cloning a set of jobs from production) and run
openqa-clone-joblocally with parameters mention in description - Extend unit tests
Updated by mkittler almost 2 years ago
I'd say the documentation is correct and the code is broken. I suppose this has worked as documented at some point so likely a regression. This seems bad enough to consider adding this ticket to the backlog.
Updated by livdywan almost 2 years ago
- Target version changed from future to Ready
Ack. Thank you for verifying
Updated by mkittler almost 2 years ago
- Subject changed from openqa-clone-job --skip-deps behavior contradicts documentation to openqa-clone-job --skip-deps behavior contradicts documentation size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by kraih almost 2 years ago
Ok, here's a simple reproducer for O3:
$ openqa-clone-job --skip-download --host http://127.0.0.1:9526 --apikey KEY --apisecret SECRET --clone-children --from https://openqa.opensuse.org/tests/3148994
Cloning parents of opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome_libyui@aarch64
Cloning children of opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome@aarch64
Cloning children of opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome_libyui@aarch64
Cloning parents of opensuse-15.5-DVD-aarch64-Build414.4-yast2_firstboot@aarch64
Cloning parents of opensuse-15.5-DVD-aarch64-Build414.4-yast2_gui@aarch64
Cloning parents of opensuse-15.5-DVD-aarch64-Build414.4-yast2_ncurses@aarch64
Created job #2718511: opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome@aarch64 -> http://127.0.0.1:9526/t2718511
Created job #2718512: opensuse-15.5-DVD-aarch64-Build414.4-yast2_gui@aarch64 -> http://127.0.0.1:9526/t2718512
Created job #2718514: opensuse-15.5-DVD-aarch64-Build414.4-yast2_firstboot@aarch64 -> http://127.0.0.1:9526/t2718514
Created job #2718510: opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome_libyui@aarch64 -> http://127.0.0.1:9526/t2718510
Created job #2718513: opensuse-15.5-DVD-aarch64-Build414.4-yast2_ncurses@aarch64 -> http://127.0.0.1:9526/t2718513
$ openqa-clone-job --skip-download --host http://127.0.0.1:9526 --apikey KEY --apisecret SECRET --clone-children --skip-deps --from https://openqa.opensuse.org/tests/3148994
Created job #2718515: opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome_libyui@aarch64 -> http://127.0.0.1:9526/t2718515
Updated by kraih almost 2 years ago
Looking through the git history, this is actually an interesting case. When we estimated this ticket, we assumed that the documentation was right once and that not cloning children might be a regression. But it looks like the documentation referring to parent jobs actually predates the introduction of the ability to clone children by a few years. So it's the other way around, there was no regression, merely a feature was introduced that did not consider --skip-deps.
But since we all assumed the documentation was right, i'll still try to make it work the way it is described.
Updated by openqa_review almost 2 years ago
- Due date set to 2023-03-15
Setting due date based on mean cycle time of SUSE QE Tools
Updated by kraih almost 2 years ago
Updated by kraih almost 2 years ago
- Status changed from In Progress to Feedback
Considering that the script has grown without test coverage over a very long period of time, there's probably more flags with unintended interactions. We'll have to fix those as they come up and add unit tests along the way.
Updated by livdywan almost 2 years ago
- Due date changed from 2023-03-15 to 2023-03-17
Just to clarify since I apparently misunderstood the above comment. This should be "done". Martin said he's going to try it out and will let us know if it resolves the issue. So we'll wait until then.
Updated by MDoucha almost 2 years ago
I've tested it and cloning the desired jobs works. However, the resulting jobs are not properly linked so the children start running immediately, without waiting for the parent to finish and produce the required assets.
My test jobs:
https://openqa.suse.de/tests/10717398
https://openqa.suse.de/tests/10717399
Cloned from:
https://openqa.suse.de/tests/10717072#dependencies
Edit: created new test jobs to avoid interfering with active incident test results.
Updated by okurz over 1 year ago
- Due date changed from 2023-03-17 to 2023-03-31
- Status changed from Feedback to Workable
@kraih based on feedback from mdoucha you could look into that or provide us a status about what others should follow-up with
Updated by kraih over 1 year ago
This can indeed be easily replicated locally:
$ openqa-clone-job --skip-download --host http://127.0.0.1:9526 --apikey KEY --apisecret SECRET --clone-children --skip-deps --from http://openqa.suse.de/tests/10717072
...
$ openqa-clone-job --skip-download --host http://127.0.0.1:9526 --apikey KEY --apisecret SECRET --clone-children --from http://openqa.suse.de/tests/10717072
...
Only the latter shows up with actual dependencies.
Updated by kraih over 1 year ago
Looks like the problem is that the assignment of dependencies currently relies on the collection of parent dependency information, which is of course not happening if only children are cloned. I'll need some time to wrap my head around the current logic, since it uses recursion and goes back and forth between parents and children.
Updated by kraih over 1 year ago
Think i have a fix, but it requires some larger changes in the clone dependency code. Lets see how it looks once i add tests for this case. https://github.com/os-autoinst/openQA/commit/d7eb5fcc0789666179bc8f51067e89423a3551af