Wiki¶

Also see https://progress.opensuse.org/projects/openqav3/wiki

test organization on https://openqa.suse.de/¶

job group names¶

Job group names should be consistent and structured for easy (daily) review of the current status¶

template:

<product_group_short_name> <order_nr>.<product_variant>

e.g. "SLE 12 SP1 1.Server". Keep the whitespace for separation consistent, also see https://progress.opensuse.org/issues/9916

Released products should be named with a prefix 'x' to show up late in the overview page¶

This way we can keep track if tests fail even though the product does not produce new builds. This could help us crosscheck tests. E.g. "x-released SLE 12 SP1 1.Server".

lowercase "x" as all our product names start with capital letters. Sorting works regardless (or uppercase first?).

For now we do not retrigger tests on old builds automatically but any test developer may retrigger it manually, e.g. if he suspects the tests broke and he wants to confirm that local changes are not at fault.

needling best practices¶

There are also other locations where "needling best practices" can be found but we should also have the possibility to keep something on the wiki. Feel free to contact me (okurz) and tell me where it should be instead if there is a better place

applying "workaround" needles¶

If a test reveals a product issue of minor importance it can make sense create a needle with the property "workaround" set. This way, if the needle is matched, the test records this as a "soft-fail". To backtrack the product issue and follow on this and eventually delete the workaround needle if the product issue is fixed, the product issue should be recorded in the needle name itself and at best also in the git commit message adding the needle. If test changes are necessary the source code should have a corresponding comment referencing the issue as well as marking start and stop of the test procedure that is necessary for applying the workaround. Example for a needle name: "gdm-workaround-bsc962806-20160125" referencing bsc#962806

do not overwrite old needles because old date confuses people¶

With the needle editor a timestamp of the current day is automatically added to new neeedles. When updating a needle, don't overwrite a needle with the old date tag not to confuse people as it will look really weird in the needle editor.