QA » openQA Project

User story¶

As a QA tester I want to be able to create needles that make a test softfail if found to not need to change test code with a "check_screen and record_soft_failure"

acceptance criteria¶

• AC1: A test module softfails if a specified needle with a certain property fails
• AC2: The test module does not fail if the softfail needle is checked but not matching

tasks¶

• Add checkbox for property on needle editor
• Match with priority on softfail needle
• Make testmodule softfail if softfail needle matches, e.g. "record_soft_failure"

further details¶

There are a number of different 'classes' of softfails

Two obvious examples are:

Workarounds (either in terms of code or needles). ie. "openQA detected this, and did something to workaround it"

Acceptable failures ie. "openQA detected this, and we accept it as not serious, but it is a failure"

We need to handle these two conditions differently - I think we need different icons/colours (Orange and Yellow, not just yellow?) and possibly different API calls (record_soft_failure and record_workaround?).
Added by okurz: The problem is that if two needles match, one old one and one new one which is more specific and marked as "workaround", it will match the old one and not mark a softfail. What we need is a "priority match"

For motivation also see #12162