action #126704
closed[regression] developer mode does not allow to create needle after use of "skip timeout" size:M
Description
Observation¶
While working on #126680 I triggered an openQA test https://openqa.opensuse.org/tests/3194107 in which I used the developer mode in the attempt to create a needle. I could enter the development mode and tried the checkbox "pause on failure" as well as "skip timeout" when the openQA test has reached a step where no needle match would be expected as the job uses icewm instead of gnome and we do not have needles for icewm yet. But no action was offered to go to the needle editor instead the job continued into the post_fail_hook evaluation.
Steps to reproduce¶
- Trigger openQA job on https://openqa.opensuse.org with a test that will fail a needle match, e.g. clone an openQA-in-openQA job with an HDD image that does not match the expected, likely even just
HDD_1=will have the desired effect - Open the developer mode and activate
- Click on "skip timeout" as soon as the test is in an
check_screenorassert_screencall - Observe that no needle editor option is provided
Impact¶
I assume that the developer mode is actually not usable at all for dynamic needle creation for all use cases and on all openQA instances due to this but it might be o3 specific or it might be that I did not follow the expected flow.
Further notes¶
- This is basically the workflow documented on https://open.qa/docs/#_workflow_for_creating_or_updating_needles.
Updated by livdywan about 1 year ago
- Subject changed from [regression] developer mode does not allow to create needle after use of "skip timeout" to [regression] developer mode does not allow to create needle after use of "skip timeout" size:M
- Description updated (diff)
Updated by mkittler about 1 year ago
- Status changed from Workable to In Progress
I couldn't reproduce it locally with an arbitrary job. Then I've read the steps to reproduce more closely and noticed that they're wrong. This part specifically:
and tried the checkbox "pause on failure" as well as "skip timeout" when the openQA test has reached a step where no needle match would be expected
Checking "pause on failure" will not affect screen mismatch assertions as documented, e.g. click on the "?" next to it. You need to select e.g. "assert_screen timeout" under "Pause on screen mismatch".
But no action was offered to go to the needle editor instead the job continued into the post_fail_hook evaluation.
As it should.
The documentation linked from the developer mode is correct. I also think it is good that "Pause on failure" does not affect screen mismatches because for that we have a more specific option and having both options interfere with each other makes it harder to understand. I will create a PR to clarify the "Pause on failure" checkbox.
Updated by mkittler about 1 year ago
- Status changed from In Progress to Feedback