action #49943
closed
[tools] Add new needle match type: "click"
Added by dheidler over 5 years ago.
Updated over 5 years ago.
Category:
Feature requests
Description
Currently the testapi function assert_and_click is using the last area of a needle with type=match to click.
When there are multiple areas in a needle they can easily mixed up - especially as there is no indication in the needle editor
showing which area is the last one.
Therefore a new type of area with type=click should be added.
When such an area is present, only this area should be used for clicking.
Other than that it should behave like type=match areas.
When using assert_screen then there shouldn't be a difference between type=match and type=click areas.
A needle must not have more than one area with type=click - the needle testsuite will be adapted to fail otherwise.
Acceptance Criterias¶
- AC1: The needle editor in the webui can handle such areas
- Idea: Replace the unused (and also not even working) yellow ocr areas with click areas in the editor
- AC2: In
assert_and_click if there is a type=click area present in a needle then this area is used.
- AC3: In
assert_and_click if there is NO type=click area present in a needle the current behavior doesn't change for backwards compatibility.
- Description updated (diff)
The needle type so far determines how the match is computed. I'd rather not mess with that and instead add this as an additional flag. It would make more sense to show this with a mouse icon in the needle editor (instead of using a different color).
So far always the center of the needle is clicked, right? At least I remember people complaining about that limitation. Keeping that in mind I really opt for the "additional flag" and "mouse icon" solution because it could be extended more easily to also support relative click coordinates within the area.
- Category changed from 130 to 124
- Target version set to Ready
It's still an area to be matched after all, so changing the type is wrong. We have needles where there are 2 intersecting areas in the hope that openQA would click on the smaller one. So defining a defined 'click point' sounds like the easier to scale option. If an area has a click point, it's the click area that this feature asks for. If not, we fall back to current logic.
- Assignee set to mkittler
- Target version changed from Ready to Current Sprint
Does it mean that each time a clickable area is added, the pixel position needs to be calculated? That would be painful.
Keep in mind that the main problem we encounter here is that when creating a new needle based on a old needle, the areas are cloned but in an arbitrary order, which causes the click area to be different. There should be a feature to select which area is to be clicked even when creating a new needle based on an old one.
So, the behavior of clicking on the center of the area is fine. Creating an extra little area to be the clickable one is easier to maintain than recalculating pixels.
I think mkittler's suggestion of an additional area-specific flag would be better than a click point
The click point can be easily added via the needle editor of course and will be by default the center of the area.
It should also be easy to allow both, click_point: {xpos: ..., ypos: ...} and click_point: "center", in case you create needles often without the help of the needle editor and don't want to calculate coordinates.
- Status changed from New to In Progress
- Status changed from In Progress to Feedback
PRs have been merged. The new needle editor UI for that feature could still be improved.
I really like this approach!
Just one thing that could be added:
It would be nice to (also?) use click_point coordinates relative to the containing match area.
This way it would be possible to handle different window positioning in different WMs using a single needle.
We could add an option for this in the needle editor and add some entry to click_point - eg: relative: true (should default to false if not present and omitted if not enabled).
What else would the click point be? It's always relative, if it's not, it's a bug
We already talked about it, but for the record: Of course it is relative. Everything else makes not sense.
- Status changed from Feedback to Resolved
Making this more fancy can be done when giving the needle editor a general overhaul. (But it is actually not that bad.)
- Has duplicate action #25712: feature request for assert_and_click - define exact area for click added
Also available in: Atom
PDF