action #68474
closedsimilarity calculation can return NaN
0%
Description
I have updated my SLE15SP1 machine to current openQA and installed all SLE updates:
openQA-4.6.1592990132.f2d269d68-2896.1.noarch
os-autoinst-4.6.1592908950.5038d8c2-458.1.x86_64
libopencv3_4-3.4.10-2.1.x86_64
Previously installed openQA version was from January.
Then I got failures in tests that worked before:
after x11_start_program('xterm') on GNOME desktop the xterm did not appear.
With vnc client I checked that xterm actually appeared, just the openQA screen was not updated.
After more debugging I have found that
$sim = $lastscreenshot->similarity($image) if $lastscreenshot;
in backend/baseclass.pm method enqueue_screenshot returns NaN and this breaks all further updates.
The following change apparently fixed the issue, but I don't understand why norm() could return NaN. Maybe the image is somewhat broken.
--- a/ppmclibs/tinycv_impl.cc
+++ b/ppmclibs/tinycv_impl.cc
@@ -301,6 +301,10 @@ double getPSNR(const Mat& I1, const Mat& I2)
double noise = norm(I1, I2);
+ if (std::isnan(noise)) {
+ return VERY_DIFF;
+ }
+
if (!noise) {
return VERY_SIM;
}
Files
Updated by mkittler almost 4 years ago
I don't know why it would return NaN. Reading the documentation didn't help much.
How many tests are affected and is there any pattern among the affected tests?
The change you're proposing likely doesn't cause any harm so you could create a PR yourself.
Updated by nadvornik almost 4 years ago
I have configured only installation and SUMA tests, installation works fine, SUMA test always fails at the beginning on xterm.
I think my patch only hides the real bug. There were changes in video encoder, maybe it is related.
Updated by nadvornik almost 4 years ago
- File opencv_test.tar.gz opencv_test.tar.gz added
I have saved the images, see attachment. The issue can be reproduced with python opencv.
python3 test.py
nan
I have reproduced it on SLE15SP1 with both
libopencv3_4-3.4.10-2.1.x86_64.rpm
libopencv4_3-4.3.0-2.1.x86_64.rpm
I could not reproduce it on leap on another machine.
It looks like miscompiled opencv in devel:openQA:SLE-15/SLE_15_SP1
Updated by nadvornik almost 4 years ago
For completeness, when it works, it returns
python3 test.py
161538.91156622296
Updated by mkittler almost 4 years ago
Yes, that PR would be a workaround (and it would be nice to print at least a warning in that case).
It is nice that you could come up with such a simple test case.
It looks like miscompiled opencv in devel:openQA:SLE-15/SLE_15_SP1
That might be the right. I could also not reproduce it under Tumbleweed.
There were changes in video encoder, maybe it is related.
That's very unlikely because by default nothing should have changed. And even if one configures an external video encoder (which is now possible) the old video encoder is still enabled to produce the "last PNG". Besides, as far as I know, the video encoder only consumes the PPM images.
Updated by nadvornik almost 4 years ago
OK, created PR: https://github.com/os-autoinst/os-autoinst/pull/1451
Updated by mkittler almost 4 years ago
GitHub seems to have problems so I can not comment there. So I'll add them here:
std::isnanrequires#include <cmath>which is not already present. (I'd avoid relying on indirectly included headers.)- I'd just use
std::cerr << "WARNING: cv::norm() returned NaN (poo#68474)\n";becausestd::cerris flushed automatically.
Beside these nitpicks the PR looks good.
Updated by mkittler almost 4 years ago
The workaround PR has been merged in the meantime: https://github.com/os-autoinst/os-autoinst/pull/1451
Updated by ggardet_arm almost 4 years ago
- Related to action #68510: test fails in logpackages on some aarch64 workers added
Updated by ggardet_arm almost 4 years ago
- Status changed from New to Resolved
Verified as part of https://progress.opensuse.org/issues/68510
Updated by mkittler almost 4 years ago
As @nadvornik already said: This is just a workaround. So I didn't resolve the issue.
Apparently it also happens on @ggardet_arm's Tumbleweed machine. That means we can discard the idea that it is just a miscompiled opencv in devel:openQA:SLE-15/SLE_15_SP1.
Updated by okurz almost 4 years ago
Sounds like #68554 is related and can be used as followup
Updated by okurz almost 4 years ago
- Related to action #68554: [sporadic] some tests fail in incomplete state due to a 'cv::Exception' added