Project

General

Profile

Actions

action #124493

closed

openqa-clone-job --skip-deps behavior contradicts documentation size:M

Added by MDoucha over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2023-02-14
Due date:
2023-03-31
% Done:

0%

Estimated time:

Description

Observation

Both OpenQA documentation and openqa-clone-job --help say that --skip-deps and --skip-chained-deps should only prevent cloning of parent jobs. In reality, however, both options will prevent cloning of all (chained) dependencies regardless of parent/child relationship (even when you specify --clone-children). This means there is currently no way to clone a dependency subtree without parents using openqa-clone-job. The subtree can only be restarted in webUI which does not support modifying settings of the restarted jobs.

Acceptance criteria

  • AC1: There is a way to clone a dependency subtree without parents using openqa-clone-job (in accordance with the documentation).

Suggestions

  • It probably worked in the past, maybe a regression?
  • Create a set of dependent jobs locally (e.g. by setting dependencies manually within the database or by cloning a set of jobs from production) and run openqa-clone-job locally with parameters mention in description
  • Extend unit tests
Actions #1

Updated by okurz over 1 year ago

  • Target version set to future
Actions #2

Updated by mkittler about 1 year ago

I'd say the documentation is correct and the code is broken. I suppose this has worked as documented at some point so likely a regression. This seems bad enough to consider adding this ticket to the backlog.

Actions #3

Updated by livdywan about 1 year ago

  • Target version changed from future to Ready

Ack. Thank you for verifying

Actions #4

Updated by mkittler about 1 year ago

  • Subject changed from openqa-clone-job --skip-deps behavior contradicts documentation to openqa-clone-job --skip-deps behavior contradicts documentation size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #5

Updated by kraih about 1 year ago

  • Assignee set to kraih
Actions #6

Updated by kraih about 1 year ago

  • Status changed from Workable to In Progress
Actions #7

Updated by kraih about 1 year ago

Ok, here's a simple reproducer for O3:

$ openqa-clone-job --skip-download --host http://127.0.0.1:9526 --apikey KEY --apisecret SECRET --clone-children --from https://openqa.opensuse.org/tests/3148994
Cloning parents of opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome_libyui@aarch64
Cloning children of opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome@aarch64
Cloning children of opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome_libyui@aarch64
Cloning parents of opensuse-15.5-DVD-aarch64-Build414.4-yast2_firstboot@aarch64
Cloning parents of opensuse-15.5-DVD-aarch64-Build414.4-yast2_gui@aarch64
Cloning parents of opensuse-15.5-DVD-aarch64-Build414.4-yast2_ncurses@aarch64
Created job #2718511: opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome@aarch64 -> http://127.0.0.1:9526/t2718511
Created job #2718512: opensuse-15.5-DVD-aarch64-Build414.4-yast2_gui@aarch64 -> http://127.0.0.1:9526/t2718512
Created job #2718514: opensuse-15.5-DVD-aarch64-Build414.4-yast2_firstboot@aarch64 -> http://127.0.0.1:9526/t2718514
Created job #2718510: opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome_libyui@aarch64 -> http://127.0.0.1:9526/t2718510
Created job #2718513: opensuse-15.5-DVD-aarch64-Build414.4-yast2_ncurses@aarch64 -> http://127.0.0.1:9526/t2718513

$ openqa-clone-job --skip-download --host http://127.0.0.1:9526 --apikey KEY --apisecret SECRET --clone-children --skip-deps --from https://openqa.opensuse.org/tests/3148994
Created job #2718515: opensuse-15.5-DVD-aarch64-Build414.4-create_hdd_gnome_libyui@aarch64 -> http://127.0.0.1:9526/t2718515
Actions #8

Updated by kraih about 1 year ago

Looking through the git history, this is actually an interesting case. When we estimated this ticket, we assumed that the documentation was right once and that not cloning children might be a regression. But it looks like the documentation referring to parent jobs actually predates the introduction of the ability to clone children by a few years. So it's the other way around, there was no regression, merely a feature was introduced that did not consider --skip-deps.

But since we all assumed the documentation was right, i'll still try to make it work the way it is described.

Actions #9

Updated by openqa_review about 1 year ago

  • Due date set to 2023-03-15

Setting due date based on mean cycle time of SUSE QE Tools

Actions #11

Updated by kraih about 1 year ago

  • Status changed from In Progress to Feedback

Considering that the script has grown without test coverage over a very long period of time, there's probably more flags with unintended interactions. We'll have to fix those as they come up and add unit tests along the way.

Actions #12

Updated by livdywan about 1 year ago

  • Due date changed from 2023-03-15 to 2023-03-17

Just to clarify since I apparently misunderstood the above comment. This should be "done". Martin said he's going to try it out and will let us know if it resolves the issue. So we'll wait until then.

Actions #13

Updated by MDoucha about 1 year ago

I've tested it and cloning the desired jobs works. However, the resulting jobs are not properly linked so the children start running immediately, without waiting for the parent to finish and produce the required assets.

My test jobs:
https://openqa.suse.de/tests/10717398
https://openqa.suse.de/tests/10717399

Cloned from:
https://openqa.suse.de/tests/10717072#dependencies

Edit: created new test jobs to avoid interfering with active incident test results.

Actions #14

Updated by okurz about 1 year ago

  • Due date changed from 2023-03-17 to 2023-03-31
  • Status changed from Feedback to Workable

@kraih based on feedback from mdoucha you could look into that or provide us a status about what others should follow-up with

Actions #15

Updated by kraih about 1 year ago

  • Status changed from Workable to In Progress

I'll take a look.

Actions #16

Updated by kraih about 1 year ago

This can indeed be easily replicated locally:

$ openqa-clone-job --skip-download --host http://127.0.0.1:9526 --apikey KEY --apisecret SECRET --clone-children --skip-deps --from http://openqa.suse.de/tests/10717072
...
$ openqa-clone-job --skip-download --host http://127.0.0.1:9526 --apikey KEY --apisecret SECRET --clone-children --from http://openqa.suse.de/tests/10717072
...

Only the latter shows up with actual dependencies.

Actions #17

Updated by kraih about 1 year ago

Looks like the problem is that the assignment of dependencies currently relies on the collection of parent dependency information, which is of course not happening if only children are cloned. I'll need some time to wrap my head around the current logic, since it uses recursion and goes back and forth between parents and children.

Actions #18

Updated by kraih about 1 year ago

Think i have a fix, but it requires some larger changes in the clone dependency code. Lets see how it looks once i add tests for this case. https://github.com/os-autoinst/openQA/commit/d7eb5fcc0789666179bc8f51067e89423a3551af

Actions #20

Updated by kraih about 1 year ago

  • Status changed from In Progress to Feedback
Actions #21

Updated by kraih about 1 year ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF