Project

General

Profile

action #60173

Improve the openQA documentation

Added by dancermak over 1 year ago. Updated 5 months ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2019-11-22
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

The current openQA documentation is imho still lacking in some regards, as I (as a total beginner) am feeling completely lost when I need to get something done in openQA.

My problems with the documentation:

  • It needs a better introduction that explains the basics, a bit like http://open.qa/docs/#_basic_concepts but with less details (the needle description doesn't belong there, the glossary is just far too long).

  • It needs comprehensive tutorials for various use cases:
    1) I just want to modify a single test, how do I run it?
    2) I want to add a new test suite (or a test group, or something that shows up as a single item on o3), how can I do that and what do I need to modify?
    3) I want to write actual GUI tests and create new needles, let openQA click things & read stuff from the screen.

  • It is completely intransparent how tests are actually run: there are job groups and test suites. Test suites appear to contain job groups, but how is it decided which tests are run in a job group? E.g. I tried to find out what runs create_hdd_gnome. But grepping the whole source tree of os-autoinst-distri-opensuse yields 0 hits.

  • At least the openSUSE tests appear to be using a lot of "magic variables" where it is not apparent whether they are required, how they are set and what actually uses them. For example the REPO_n variables are automagically set on o3, but not if you start local tests. But your local tests just die if you don't have them set (or maybe it's because of something else, who knows).

My recommendation:

  1. Get someone fresh (= has never worked with openQA), give them a link to the docs and watch them read. Then after 10-20 minutes ask them whether they have any clue and if not, what's missing. Then try to address that, because most people will give up in far less than 10-20 min.
  2. Reorder the docs: http://open.qa/docs/#get-testing belongs definitely after http://open.qa/docs/#installing; http://open.qa/docs/#_triggering_tests_based_on_an_any_remote_git_refspec_or_open_github_pull_request is completely buried deep down in the docs and no one knows about it unless told by someone in #opensuse-factory.
  3. If possible, get a technical writer to help.

History

#1 Updated by okurz over 1 year ago

  • Priority changed from High to Low

And I thought I nailed it with the last doc rework ;P You are certainly right in most points. For a start, we will most likely not consider this as high priority so please understand when I set it to "Low" now. You have certain specific suggestions which one can start to work on right away, that helps. Thanks. Some topics we want to address by rethinking how tests are scheduled so it would not only be "documentation improvements".

#2 Updated by okurz 12 months ago

  • Target version set to future

#3 Updated by MSirringhaus 5 months ago

I agree with the points raised here.

  1. Get someone fresh (= has never worked with openQA), give them a link to the docs and watch them read. Then after 10-20 minutes ask them whether they have any clue and if not, what's missing. Then try to address that, because most people will give up in far less than 10-20 min.

That is/was basically me today. What I wanted to do was to "quickly get an openQA instance running locally to write ONE test".
After spending the morning in the docs, all I can give you is a blank stare, I'm sorry to say.

It suffers from what I call "man page syndrome". It describes every little option and switch in great detail. But after reading that I still have no idea whats going on.

Each installation method is described in detail, each configuration option as well. But not how they interact with each other.
Which config steps are already done in the container bootstrap?
Do I have to call /usr/share/openqa/script/fetchneedles after bootstrapping or not?
Etc.

Once I had the web-UI running on http://localhost, I had no idea what to do next. Everything in the UI was empty.
After some heavy scrolling in the docs, I figured out that I probably needed ISOs.
Doc says "However, openQA also supports a remote-download feature of assets from trusted domains.", but not how (its in there somewhere, I'm sure. Described in great detail :-) ).

I also found myself going in an infinite loop between different sections, which referenced each other with "here are more infos on this".

At some point I just gave up.

Sorry, I can't give you a nice list of things to improve. This rather incoherent comment is all I can manage right now.

Dans initial list of improvements already sound like they might have helped me.

#4 Updated by okurz 5 months ago

MSirringhaus wrote:

...

Thanks for your feedback. Appreciated :)

Also available in: Atom PDF