Project

General

Profile

action #88521

Provide an easy way to clone multi machine jobs

Added by dheidler 8 months ago. Updated 7 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2021-02-10
Due date:
2021-02-26
% Done:

0%

Estimated time:
Difficulty:

Description

Currently it is very painful to work on multi machine tests as they can not easily be rescheduled.
They often need a special network setup that is not available locally and also some special configuration.

It should be possible to clone them somehow with a normal clone job action.

History

#1 Updated by okurz 8 months ago

  • Category set to Feature requests
  • Assignee set to okurz
  • Target version set to future

Hm, there is a way to clone existing multi-machine clusers, e.g. call clone-job on a parent with --clone-children. So I could just consider this ticket as "support"-ticket but when you have not found that I would like to look how we can improve the experience then so that you and others can find these options more easily. So, what would help you more?

#2 Updated by dheidler 8 months ago

kazhua:~/devel/openQA # openqa-clone-job  --skip-deps --skip-download --host https://openqa.suse.de --clone-children https://openqa.suse.de/tests/5444256
Created job #5449971: sle-15-SP3-Online-x86_64-Build145.1-wireguard_client@64bit -> https://openqa.suse.de/t5449971
kazhua:~/devel/openQA # openqa-clone-job  --skip-deps --skip-download --host https://openqa.suse.de --clone-children https://openqa.suse.de/tests/5443990
Created job #5449972: sle-15-SP3-Online-x86_64-Build145.1-wireguard_server@64bit -> https://openqa.suse.de/t5449972

Doesn't look like it actually works. As I wasn't sure which one was the parent job, I cloned from both.
But none of the calls spawned two new jobs.

#3 Updated by okurz 8 months ago

none, is the parent. The real parent is the HDD creation job: https://openqa.suse.de/tests/5443775

Only that one provides the complete cluster. Of course you don't want to run 40 jobs, you just want the "wireguard server+client". But openQA does not know what part of a cluster you want. That's making it hard. But of course it should be feasible to detect the siblings. Let me try to bring others into the discussion.

#4 Updated by dheidler 8 months ago

That's what I feared.

Of course I only want the two multimachine jobs and neither regenerate the hdd image nor run the other 40 children of that parent job.

Also it would be nice if it would be implemented in a way that allows downloading the hdd image to another openQA instance and running the multimachine jobs there without the need of having to re-run the parent job on the target instance again.

#5 Updated by mkittler 8 months ago

Doesn't look like it actually works. But none of the calls spawned two new jobs.

Considering you've specified --skip-deps that shouldn't be a surprise, though.

As I wasn't sure which one was the parent job, I cloned from both.

see #81859#note-7


I've just tested the following:

script/openqa-clone-job --dir /hdd/openqa-devel/openqa/share/factory --show-progress --apikey … --apisecret … --host http://localhost:9526 --clone-children https://openqa.suse.de/tests/5443990

It worked fine, both parallel jobs have been cloned. Of course the create HDD job has been cloned as well. This can be avoided via --skip-chained-deps, e.g.:

script/openqa-clone-job --dir /hdd/openqa-devel/openqa/share/factory --show-progress --apikey … --apisecret … --host http://localhost:9526 --skip-chained-deps --clone-children https://openqa.suse.de/tests/5443990

Also it would be nice if it would be implemented in a way that allows downloading the hdd image to another openQA instance

To my knowledge that's something that's really not implemented so far. Of course you can simply run openqa-clone-job on the target web UI. Obviously that would require some daemon on the target web UI to do the download. Not sure whether it is worth implementing that.

#6 Updated by openqa_review 8 months ago

  • Due date set to 2021-02-26

Setting due date based on mean cycle time of SUSE QE Tools

#7 Updated by okurz 7 months ago

mkittler wrote:

[...]
It worked fine, both parallel jobs have been cloned. Of course the create HDD job has been cloned as well. This can be avoided via --skip-chained-deps

Do you think something like

openqa-clone-job --skip-chained-deps --clone-children https://openqa.example.org/tests/1234

can be more prominently featured in the help text of openqa-clone-job or the docs? Can you do that?

#8 Updated by mkittler 7 months ago

  • Status changed from New to In Progress
  • Assignee changed from okurz to mkittler

#9 Updated by mkittler 7 months ago

  • Status changed from In Progress to Resolved

The PR has been merged. I suppose that should be enough for this ticket.

#10 Updated by okurz 7 months ago

That's great.

dheidler can you confirm this helps? Do you have something else in mind that could help?

#11 Updated by dheidler 7 months ago

Looks good.

Also available in: Atom PDF