QA » openQA Project

I think mkittler also wanted to take a look into it. Don't know if he's in the office today though.

I also added related tickets which can give some ideas on what's wished for.

Just (small) side note for who's gonna take this task: openqa-client fails when we specify an invalid json file or not existant badly:

Usage: JSON::XS::decode(self, jsonstr) at /usr/bin/openqa-client line 156.

Most of the new features requested are just features that require some repetition (like restarting all the tests, stopping all the tests), or improving some new existing features, like start a job without starting the parent job and so on...

Since some of the features overlap over the scripts we have, i thought of unifying the features.

This is my first proposal:

http://pastebin.nue.suse.com/20219/src

The invocation examples are not complete and not reflect the final result and are subject to change.
Please note that since json is returned we allow the users of the cli application to process the results in their scripts, like for example with the jq utility.

In your proposal I strongly recommend to not leave any unacompanied word "test" around but stick to the definition in https://github.com/os-autoinst/openQA/blob/master/docs/GettingStarted.asciidoc#concepts . I guess your "test" is most likely "job". But then I would also be careful about the word "restart" because strictly speaking the job is never changed as it is a static result, like for what is "build" in the jenkins domain. How about "retrigger"? The word "group" is a new definition or the "job group" as defined in openQA glossary? Your overall proposal looks like you basically want to rewrite everything or just extend? I really like your complete view on what could be helpful. I would suggest though to keep the interface and functionality as is of now, e.g. openqa-client jobs/<id> delete. The implementation is less important of course and may change.

okurz wrote:

In your proposal I strongly recommend to not leave any unacompanied word "test" around but stick to the definition in https://github.com/os-autoinst/openQA/blob/master/docs/GettingStarted.asciidoc#concepts . I guess your "test" is most likely "job". But then I would also be careful about the word "restart" because strictly speaking the job is never changed as it is a static result, like for what is "build" in the jenkins domain. How about "retrigger"? The word "group" is a new definition or the "job group" as defined in openQA glossary? Your overall proposal looks like you basically want to rewrite everything or just extend? I really like your complete view on what could be helpful. I would suggest though to keep the interface and functionality as is of now, e.g. openqa-client jobs/<id> delete. The implementation is less important of course and may change.

I used the terms as they are indicated in the webinterface. There's a small explanation of what i meant with each term:

• test one run of individual test cases Example of a test screen: https://openqa.suse.de/tests/1412943
• worker Test worker that will run the test Example of a worker screen: https://openqa.suse.de/admin/workers/936
• group a group of builds Example of a group screen: https://openqa.suse.de/group_overview/110
• build a group of tests Example of a build screen: https://openqa.suse.de/tests/overview?distri=sle&version=15&build=422.1&groupid=110

Your overall proposal looks like you basically want to rewrite everything or just extend

Lots of rewriting that's for sure. My idea was the client would only wrap the web calls to openqa (a dumb client). For example, the client could have a command that would call a webservice in openqa, filter the response, and do another call to another webservice with the filtered results and then displaying to the user the outcome. Eventually we could create a webservice that does those two calls, so the client would just do one call.

With the changes to the openQA client in pr#1571 and pr#1564, we can now extend the client to add features/functionalities self contained running in a similar way to what OpenQA::Client::Archive is doing.

On top of that, while the ideas proposed by David are a good starting point, there are few things that are going to be needed:

• Implement a generic wrapper that allows the client to group calls to the API The reasoning here is that what we currently have it's just a rest client that mainly just talks to the API, requiring the person to look at the api help. Having an help embedded within the client would greatly improve this, and since we have already work that generates help for the routes, this could be ported to the client, and generated while packaging.

New Functionality for the client

• Job monitoring
• Build monitoring

Extensions to Clone Job:

• Allow remote clonning (Dangerous and should be dropped :))
• Make it part of OpenQA::Client to reduce code that is repeated
• Allow using SKIPTO on tests, that basically the tests would be cloned with TEST_DEBUG=1 and then when finished, next one would start with SKIPTO already set to the initial value.

Build Administration

• Allow the client to trigger a build for a jobgroup

szarate wrote:

Extensions to Clone Job:

• Allow remote clonning (Dangerous and should be dropped :))

that already works, see https://github.com/okurz/scripts/blob/master/openqa_clone_job_o3 and if you "drop" it as in "remove the feature" then I would be very sad.

• Allow using SKIPTO on tests, that basically the tests would be cloned with TEST_DEBUG=1 and then when finished, next one would start with SKIPTO already set to the initial value.

That would be nice feature in case you do not actually want to fix something but repeatedly only run one test module, e.g. to gather better statistics.

that already works, see https://github.com/okurz/scripts/blob/master/openqa_clone_job_o3 and if you "drop" it as in "remove the feature" then I would be very sad.

What this does, is cloning a job without downloading assets, the actual suggestion was "I would like to be able to do a clone_job without needing to download the assets manually on the remote webUI.

What this does, is cloning a job without downloading assets, the actual suggestion was "I would like to be able to do a clone_job without needing to download the assets manually on the remote webUI."

uhm, what is the difference between these two? What do you mean with "needing to download the assets manually on the remote webUI." when would you need to do this?

I've started working on this, should have something to share soon.

I've opened a draft PR with a first proof of concept for a new openQA client. https://github.com/os-autoinst/openQA/pull/2859

After many more updates the PR is no longer a draft and will be merged soon.

kraih can you comment on what you see as done and what you think should be done next? E.g. compare to the suggestions given in the ticket description. We should keep in mind that this ticket is an "epic" and hence I see it as not "workable" by itself. So we can define subtickets for specific tasks.

I've started working on some documentation for openqa-cli. https://github.com/os-autoinst/openQA/pull/3009

See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html