Project

General

Profile

Actions

action #126704

closed

[regression] developer mode does not allow to create needle after use of "skip timeout" size:M

Added by okurz over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2023-03-27
Due date:
% Done:

0%

Estimated time:

Description

Observation

While working on #126680 I triggered an openQA test https://openqa.opensuse.org/tests/3194107 in which I used the developer mode in the attempt to create a needle. I could enter the development mode and tried the checkbox "pause on failure" as well as "skip timeout" when the openQA test has reached a step where no needle match would be expected as the job uses icewm instead of gnome and we do not have needles for icewm yet. But no action was offered to go to the needle editor instead the job continued into the post_fail_hook evaluation.

Steps to reproduce

  • Trigger openQA job on https://openqa.opensuse.org with a test that will fail a needle match, e.g. clone an openQA-in-openQA job with an HDD image that does not match the expected, likely even just HDD_1= will have the desired effect
  • Open the developer mode and activate
  • Click on "skip timeout" as soon as the test is in an check_screen or assert_screen call
  • Observe that no needle editor option is provided

Impact

I assume that the developer mode is actually not usable at all for dynamic needle creation for all use cases and on all openQA instances due to this but it might be o3 specific or it might be that I did not follow the expected flow.

Further notes

Actions #1

Updated by okurz over 1 year ago

  • Description updated (diff)
Actions #2

Updated by livdywan over 1 year ago

  • Subject changed from [regression] developer mode does not allow to create needle after use of "skip timeout" to [regression] developer mode does not allow to create needle after use of "skip timeout" size:M
  • Description updated (diff)
Actions #3

Updated by okurz over 1 year ago

  • Status changed from New to Workable
Actions #4

Updated by mkittler over 1 year ago

  • Assignee set to mkittler
Actions #5

Updated by mkittler over 1 year ago

  • Status changed from Workable to In Progress

I couldn't reproduce it locally with an arbitrary job. Then I've read the steps to reproduce more closely and noticed that they're wrong. This part specifically:

and tried the checkbox "pause on failure" as well as "skip timeout" when the openQA test has reached a step where no needle match would be expected

Checking "pause on failure" will not affect screen mismatch assertions as documented, e.g. click on the "?" next to it. You need to select e.g. "assert_screen timeout" under "Pause on screen mismatch".

But no action was offered to go to the needle editor instead the job continued into the post_fail_hook evaluation.

As it should.


The documentation linked from the developer mode is correct. I also think it is good that "Pause on failure" does not affect screen mismatches because for that we have a more specific option and having both options interfere with each other makes it harder to understand. I will create a PR to clarify the "Pause on failure" checkbox.

Actions #6

Updated by mkittler over 1 year ago

  • Status changed from In Progress to Feedback
Actions #7

Updated by okurz over 1 year ago

  • Status changed from Feedback to Resolved

PR merged, looks fine.

Actions

Also available in: Atom PDF