QA » openQA Project

This is not about some specific needles, but a generic deficiency in the implementation.

Than it would be great to come up with steps to reproduce and expected vs. actual behavior. Note that at this point not even the original job is available anymore so I have really no idea for what kind of improvement you're asking. By the way, what exactly do you mean with "needle masks"?

mkittler wrote:

By the way, what exactly do you mean with "needle masks"?

I assume the needle match areas with defined size and also the initially defined position where IIRC we also start a search for a match.

okurz wrote:

mkittler wrote:

By the way, what exactly do you mean with "needle masks"?

I assume the needle match areas with defined size and also the initially defined position where IIRC we also start a search for a match.

I am referring to "excluded" match areas. See needle.pm in os-autoinst.

okurz wrote:

oh, so you want to have exclude areas to be defined relative to the match areas rather than absolute areas?

I want the exclude areas to be consistent with the match areas. Everything else doesn't make sense and even creates false matches.

Imagine an entry in a list, where the icon in front of the label is excluded. A new entry is added in a new version, and the original entry moves down by one. The exclude area now overpaints the (excluded) icon of the entry above the real match with a blue rectangle, and also the icon of the needle area (that how excluding currently works). This has the effect the old position becomes a fairly good match, and the new position is a bad match because nothing is excluded there (even worse, made significantly different). Combined with the preference of small match offsets and weighting of the error by needle area size, at best we get no sufficient match at all, or at worst a false match.

Having the exclude relative to match areas sounds useful at least as an configurable alternative.

Yes, it could be a property of the exclude area.

I assume the exclude area would be tied to the match area it intersects. The raises the question what should happen if an exclude area intersects multiple match areas. Maybe we better allow the user to tie an exclude area manually to a match area within the needle editor?

By the way, the exclude areas are currently implemented by simply filling the parts of the images with a green rectangle. I'm not sure how complicated it'll be to make it more dynamic.

The needle matching is happening in 2 phases: find the pattern and then judge the percentage. You can in the middle of it blank parts of the image. That's generally quite expensive, but doable.
But if that area to be excluded breaks your pattern matching, you're screwed. So don't expect miracles. This will only work for relatively large areas where the area to be excluded is rather a nuance.

coolo wrote:

The needle matching is happening in 2 phases: find the pattern and then judge the percentage. You can in the middle of it blank parts of the image. That's generally quite expensive, but doable.
But if that area to be excluded breaks your pattern matching, you're screwed. So don't expect miracles. This will only work for relatively large areas where the area to be excluded is rather a nuance.

You can do matching with "moving" exclude areas in a comparatively cheap way. As error used for finding the best location is a linear sum over the area, you can calculate the error of on arbitrarily shaped area by adding/subtracting smaller rectangular areas. I.e., Err_excluded = Err_full - Err_excludearea.

The addition/subtraction in general is trivial (algorithmically and computationally, relative to the matching itself). Some care is necessary when the positions are restricted e.g. when the match area is near the screen border, we have to clamp the error matrices so all use the same offsets.

I think I can write up some opencv POC over the weekend and provide some test cases.

That means duplicating the template matching for every excluded region per needle. E_full-E_area might be cheap (in comparision), but calculating E_erea is not. And if I imagine that this becomes popular, we will get even more of "why is needle matching so slow?

I'm not sure the price is worth it.

coolo wrote:

That means duplicating the template matching for every excluded region per needle. E_full-E_area might be cheap (in comparision), but calculating E_erea is not. And if I imagine that this becomes popular, we will get even more of "why is needle matching so slow?

I'm not sure the price is worth it.

It depends. We already have cases where people (ab)use multiple areas. One case I am aware of is the matching of "#" for the console tests, which uses one area expanding to the right, and one area expanding down. Handling these together will actually be (slightly) cheaper.

And there is also the possibility to make the matching faster. Very often, we match N needles over M frames. Some of the calculations happening inside openCVs template_match are invariant in respect to the needle or the haystack, so instead of doing these MxN times, we can do these only N resp. M times. (Examples: DFT as preparation for the fast convolution, Integral images for error calculation). Currently, we do the invariant preprocessing only for the part happening outside openCV, e.g. color-to-gray conversion, blurring.