action #2246

Consistent vocabulary

Added by ancorgs almost 6 years ago. Updated almost 4 years ago.

Status:ResolvedStart date:04/04/2014
Priority:LowDue date:
Assignee:okurz% Done:

0%

Category:Feature requestsEstimated time:10.00 hours
Target version:-
Difficulty:
Duration:

Description

For historical reason, "test" means a lot of things (and nothing at the same time) in openQA. In the webUI it means "job". In the API is not used. Internally is both a job attribute and a special job_setting which is always present and equal to the job attribute. In the meantime, we introduced a new word "TestSuite" to avoid reusing "test" one more time.

In my opinion.
* We should start using "job" everywhere for... well, for the jobs. Removing the current "tests/xx" urls.
* TestSuite should be renamed to Test. Now that even the internal job_setting called TEST and the job attribute are always set to the name of the TestSuite, it looks like we finally have a clear definition of what a "test" is.

History

#1 Updated by ancorgs almost 6 years ago

  • Description updated (diff)

#2 Updated by ancorgs almost 6 years ago

  • Target version set to future

#3 Updated by ancorgs almost 6 years ago

  • Priority changed from Urgent to Normal

#4 Updated by bmwiedemann over 5 years ago

my opinion: we should also try to avoid using the word "test" in the future, as it will remain ambiguous. Will be better to use job, testrun, testresult, teststep, testsuite etc. to be more precise what is meant. We don't really want to start a glossar documentation section.

#5 Updated by coolo over 5 years ago

  • Category set to Feature requests

#6 Updated by okurz about 4 years ago

We need a name for the specific composition of <distri>-<version>-<flavor>-<arch>-<scenario> (e.g. openSUSE-Tumbleweed-DVD-x86_64-gnome).
I just checked and it seems that "scenario" is so far only used very sparingly within the openQA repository and inconsistently so i would like to call the above "scenario". I don't mind also giving it a cute nickname "koala", see #10212#note-12 for details (koalas are cute and it's a funny name to remember easily).

I second the opinion of bmwiedemann and therefore do not propose to implement ancorgs suggestion of renaming. But what we are doing here is actually starting a "glossar documentation section" ;-)

So we have

  • test modules: an individual test case in a single perl module file, e.g. "sshxterm". If not further specified a test module is denoted with its "short name" equivalent to the filename including the test definition. The "full name" is composed of the test group (TBC), which itself is formed by the top-folder of the test module file, and the short name, e.g. "x11-sshxterm" (for x11/sshxterm.pm)
  • test suite: a collection of test modules, e.g. "textmode". All test modules within one test suite are run serially
  • job: one run of individual test cases in a row denoted by a unique number for one instance of openQA, e.g. one installation with subsequent testing of applications within gnome
  • test run: equivalent to job
  • test result: the result of one job, e.g. "passed" with the details of each individual test module
  • test step: the execution of one test module within a job
  • distri: a test distribution but also sometimes referring to a product (CAUTION: ambiguous, historically a "GNU/Linux distribution"), composed of multiple test modules in a folder structure that compose test suites, e.g. "opensuse" (test distribution, short for "os-autoinst-distri-opensuse")
  • product: the main "system under test" (SUT), e.g. "openSUSE"
  • job group: equivalent to product, used in context of the webUI
  • version: one version of a product, don't confuse with builds, e.g. "Tumbleweed"
  • flavor: a specific variant of a product to distinguish differing variants, e.g. "DVD"
  • arch: an architecture variant of a product, e.g. "x86_64"
  • scenario: A composition of <distri>-<version>-<flavor>-<arch>-<scenario>, e.g. "openSUSE-Tumbleweed-DVD-x86_64-gnome", nicknamed koala
  • build: Different versions of a product as tested, can be considered a "sub-version" of version, e.g. "Build1234"; CAUTION: ambiguity: either with the prefix "Build" included or not)

Corrections? If not I will probably eventually put this on some wiki or a text document within openQA

#7 Updated by okurz about 4 years ago

  • Status changed from New to Feedback
  • Priority changed from Normal to Low

I put the proposal on https://progress.opensuse.org/projects/openqav3/wiki/Wiki#Glossary

What could be improved is to use more consistent naming whereever it occurs otherwise in source. So if anyone encounters locations where this can be improved either fix it or note it down here.

#8 Updated by okurz almost 4 years ago

  • Status changed from Feedback to Resolved
  • Assignee set to okurz
  • Target version deleted (future)

wiki has been updated after https://progress.opensuse.org/issues/10644 but no other big changes so regarding this as done

Also available in: Atom PDF