https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842020-11-26T13:15:28ZopenSUSE Project Management ToolopenQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3547102020-11-26T13:15:28Ztjyrinki_susetjyrinki+redmine@suse.de
<ul><li><strong>Subject</strong> changed from <i>Needle masks do not handle matches with offsets</i> to <i>[qe-core][opensuse]Needle masks do not handle matches with offsets</i></li><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li></ul><p>This looks like not happening anymore in shutdown phase so probably needles were fixed.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3547322020-11-26T13:23:06ZStefanBruensstefan.bruens@rwth-aachen.de
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>New</i></li></ul><p>This is <strong>not</strong> about some specific needles, but a generic deficiency in the implementation.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3547682020-11-26T13:33:31Ztjyrinki_susetjyrinki+redmine@suse.de
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Workable</i></li><li><strong>Start date</strong> deleted (<del><i>2020-10-28</i></del>)</li></ul> openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3550722020-11-27T09:04:45Ztjyrinki_susetjyrinki+redmine@suse.de
<ul><li><strong>Status</strong> changed from <i>Workable</i> to <i>New</i></li></ul> openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3875472021-03-04T08:46:35Ztjyrinki_susetjyrinki+redmine@suse.de
<ul><li><strong>Subject</strong> changed from <i>[qe-core][opensuse]Needle masks do not handle matches with offsets</i> to <i>[tools] Needle masks do not handle matches with offsets</i></li></ul><p>The generic needle offset handling something to be evaluated by the openQA developers.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3881112021-03-05T07:35:31Zokurzokurz@suse.com
<ul><li><strong>Target version</strong> set to <i>Ready</i></li></ul> openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3883962021-03-05T13:31:58Zmkittlermarius.kittler@suse.com
<ul></ul><blockquote>
<p>This is not about some specific needles, but a generic deficiency in the implementation.</p>
</blockquote>
<p>Than it would be great to come up with steps to reproduce and expected vs. actual behavior. Note that at this point not even the original job is available anymore so I have really no idea for what kind of improvement you're asking. By the way, what exactly do you mean with "needle masks"?</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3884922021-03-07T21:13:03Zokurzokurz@suse.com
<ul></ul><p>mkittler wrote:</p>
<blockquote>
<p>By the way, what exactly do you mean with "needle masks"?</p>
</blockquote>
<p>I assume the needle match areas with defined size and also the initially defined position where IIRC we also start a search for a match.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3885612021-03-08T09:47:30ZStefanBruensstefan.bruens@rwth-aachen.de
<ul></ul><p>okurz wrote:</p>
<blockquote>
<p>mkittler wrote:</p>
<blockquote>
<p>By the way, what exactly do you mean with "needle masks"?</p>
</blockquote>
<p>I assume the needle match areas with defined size and also the initially defined position where IIRC we also start a search for a match.</p>
</blockquote>
<p>I am referring to "excluded" match areas. See needle.pm in os-autoinst.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3896712021-03-09T10:45:32Zokurzokurz@suse.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Feedback</i></li><li><strong>Assignee</strong> set to <i>okurz</i></li></ul><p>oh, so you want to have exclude areas to be defined relative to the match areas rather than absolute areas?</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3898422021-03-09T14:41:22ZStefanBruensstefan.bruens@rwth-aachen.de
<ul></ul><p>okurz wrote:</p>
<blockquote>
<p>oh, so you want to have exclude areas to be defined relative to the match areas rather than absolute areas?</p>
</blockquote>
<p>I want the exclude areas to be consistent with the match areas. Everything else doesn't make sense and even creates false matches.</p>
<p>Imagine an entry in a list, where the icon in front of the label is excluded. A new entry is added in a new version, and the original entry moves down by one. The exclude area now overpaints the (excluded) icon of the entry above the real match with a blue rectangle, and also the icon of the needle area (that how excluding currently works). This has the effect the old position becomes a fairly good match, and the new position is a bad match because nothing is excluded there (even worse, made significantly different). Combined with the preference of small match offsets and weighting of the error by needle area size, at best we get no sufficient match at all, or at worst a false match.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3902652021-03-10T13:11:26Zokurzokurz@suse.com
<ul><li><strong>Project</strong> changed from <i>openQA Tests</i> to <i>openQA Project</i></li><li><strong>Subject</strong> changed from <i>[tools] Needle masks do not handle matches with offsets</i> to <i>Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/390265/diff?detail_id=371221">diff</a>)</li><li><strong>Category</strong> changed from <i>Bugs in existing tests</i> to <i>Feature requests</i></li><li><strong>Status</strong> changed from <i>Feedback</i> to <i>New</i></li><li><strong>Assignee</strong> deleted (<del><i>okurz</i></del>)</li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Low</i></li><li><strong>Target version</strong> changed from <i>Ready</i> to <i>future</i></li></ul><p>Well, at what you describe sounds sensible but we have to understand that this would be a breaking change. The current behaviour might be badly described or not at all but this is how it currently works. Having the exclude relative to match areas sounds useful at least as an configurable alternative. We have to see what we can do eventually. Converted to feature request for openQA</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3905532021-03-11T11:17:33Zmkittlermarius.kittler@suse.com
<ul></ul><blockquote>
<p>Having the exclude relative to match areas sounds useful at least as an configurable alternative.</p>
</blockquote>
<p>Yes, it could be a property of the exclude area.</p>
<p>I assume the exclude area would be tied to the match area it intersects. The raises the question what should happen if an exclude area intersects multiple match areas. Maybe we better allow the user to tie an exclude area manually to a match area within the needle editor?</p>
<p>By the way, the exclude areas are currently implemented by simply filling the parts of the images with a green rectangle. I'm not sure how complicated it'll be to make it more dynamic.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3905592021-03-11T11:21:59Zcoolocoolo@suse.com
<ul></ul><p>The needle matching is happening in 2 phases: find the pattern and then judge the percentage. You can in the middle of it blank parts of the image. That's generally quite expensive, but doable.<br>
But if that area to be excluded breaks your pattern matching, you're screwed. So don't expect miracles. This will only work for relatively large areas where the area to be excluded is rather a nuance.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3907032021-03-11T15:46:28ZStefanBruensstefan.bruens@rwth-aachen.de
<ul></ul><p>coolo wrote:</p>
<blockquote>
<p>The needle matching is happening in 2 phases: find the pattern and then judge the percentage. You can in the middle of it blank parts of the image. That's generally quite expensive, but doable.<br>
But if that area to be excluded breaks your pattern matching, you're screwed. So don't expect miracles. This will only work for relatively large areas where the area to be excluded is rather a nuance.</p>
</blockquote>
<p>You <em>can</em> do matching with "moving" exclude areas in a comparatively cheap way. As error used for finding the best location is a linear sum over the area, you can calculate the error of on arbitrarily shaped area by adding/subtracting smaller rectangular areas. I.e., Err_excluded = Err_full - Err_excludearea.</p>
<p>The addition/subtraction in general is trivial (algorithmically and computationally, relative to the matching itself). Some care is necessary when the positions are restricted e.g. when the match area is near the screen border, we have to clamp the error matrices so all use the same offsets.</p>
<p>I think I can write up some opencv POC over the weekend and provide some test cases.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3907122021-03-11T16:11:03Zcoolocoolo@suse.com
<ul></ul><p>That means duplicating the template matching for every excluded region per needle. E_full-E_area might be cheap (in comparision), but calculating E_erea is not. And if I imagine that this becomes popular, we will get even more of "why is needle matching so slow?</p>
<p>I'm not sure the price is worth it.</p>
openQA Project - action #76747: Define exclude areas relative to match areas (not absolute position) (was: Needle masks do not handle matches with offsets)https://progress.opensuse.org/issues/76747?journal_id=3907182021-03-11T16:35:07ZStefanBruensstefan.bruens@rwth-aachen.de
<ul></ul><p>coolo wrote:</p>
<blockquote>
<p>That means duplicating the template matching for every excluded region per needle. E_full-E_area might be cheap (in comparision), but calculating E_erea is not. And if I imagine that this becomes popular, we will get even more of "why is needle matching so slow?</p>
<p>I'm not sure the price is worth it.</p>
</blockquote>
<p>It depends. We already have cases where people (ab)use multiple areas. One case I am aware of is the matching of "#" for the console tests, which uses one area expanding to the right, and one area expanding down. Handling these together will actually be (slightly) cheaper.</p>
<p>And there is also the possibility to make the matching faster. Very often, we match N needles over M frames. Some of the calculations happening inside openCVs template_match are invariant in respect to the needle or the haystack, so instead of doing these MxN times, we can do these only N resp. M times. (Examples: DFT as preparation for the fast convolution, Integral images for error calculation). Currently, we do the invariant preprocessing only for the part happening outside openCV, e.g. color-to-gray conversion, blurring.</p>