Project

General

Profile

Actions
Copy link

action #68474

closed

similarity calculation can return NaN

Added by nadvornik over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
High
Assignee:
nadvornik
Category:
Regressions/Crashes
Target version:
-
Start date:
2020-06-26
Due date:
% Done:

0%

Estimated time:

Description

I have updated my SLE15SP1 machine to current openQA and installed all SLE updates:
openQA-4.6.1592990132.f2d269d68-2896.1.noarch
os-autoinst-4.6.1592908950.5038d8c2-458.1.x86_64
libopencv3_4-3.4.10-2.1.x86_64

Previously installed openQA version was from January.

Then I got failures in tests that worked before:
after x11_start_program('xterm') on GNOME desktop the xterm did not appear.
With vnc client I checked that xterm actually appeared, just the openQA screen was not updated.

After more debugging I have found that

$sim = $lastscreenshot->similarity($image) if $lastscreenshot;

in backend/baseclass.pm method enqueue_screenshot returns NaN and this breaks all further updates.

The following change apparently fixed the issue, but I don't understand why norm() could return NaN. Maybe the image is somewhat broken.

--- a/ppmclibs/tinycv_impl.cc
+++ b/ppmclibs/tinycv_impl.cc
@@ -301,6 +301,10 @@ double getPSNR(const Mat& I1, const Mat& I2)

     double noise = norm(I1, I2);

+    if (std::isnan(noise)) {
+        return VERY_DIFF;
+    }
+
     if (!noise) {
         return VERY_SIM;
     }

Files

opencv_test.tar.gz (862 KB) opencv_test.tar.gz nadvornik, 2020-06-26 13:59

Related issues 2 (0 open — 2 closed)

Related to openQA Tests (public) - action #68510: test fails in logpackages on some aarch64 workersResolved2020-06-29

Actions
Related to openQA Project (public) - action #68554: [sporadic] some tests fail in incomplete state due to a 'cv::Exception'Resolvedmkittler2020-07-01

Actions
Actions
Copy link
 #1

Updated by mkittler over 4 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by mkittler over 4 years ago

I don't know why it would return NaN. Reading the documentation didn't help much.

How many tests are affected and is there any pattern among the affected tests?

The change you're proposing likely doesn't cause any harm so you could create a PR yourself.

Actions
Copy link
 #3

Updated by nadvornik over 4 years ago

I have configured only installation and SUMA tests, installation works fine, SUMA test always fails at the beginning on xterm.

I think my patch only hides the real bug. There were changes in video encoder, maybe it is related.

Actions
Copy link
 #4

Updated by nadvornik over 4 years ago

I have saved the images, see attachment. The issue can be reproduced with python opencv.

python3 test.py
nan

I have reproduced it on SLE15SP1 with both
libopencv3_4-3.4.10-2.1.x86_64.rpm
libopencv4_3-4.3.0-2.1.x86_64.rpm

I could not reproduce it on leap on another machine.

It looks like miscompiled opencv in devel:openQA:SLE-15/SLE_15_SP1

Actions
Copy link
 #5

Updated by nadvornik over 4 years ago

For completeness, when it works, it returns
python3 test.py
161538.91156622296

Actions
Copy link
 #6

Updated by mkittler over 4 years ago

Yes, that PR would be a workaround (and it would be nice to print at least a warning in that case).

It is nice that you could come up with such a simple test case.

It looks like miscompiled opencv in devel:openQA:SLE-15/SLE_15_SP1

That might be the right. I could also not reproduce it under Tumbleweed.

There were changes in video encoder, maybe it is related.

That's very unlikely because by default nothing should have changed. And even if one configures an external video encoder (which is now possible) the old video encoder is still enabled to produce the "last PNG". Besides, as far as I know, the video encoder only consumes the PPM images.

Actions
Copy link
 #8

Updated by mkittler over 4 years ago

GitHub seems to have problems so I can not comment there. So I'll add them here:

  • std::isnan requires #include <cmath> which is not already present. (I'd avoid relying on indirectly included headers.)
  • I'd just use std::cerr << "WARNING: cv::norm() returned NaN (poo#68474)\n"; because std::cerr is flushed automatically.

Beside these nitpicks the PR looks good.

Actions
Copy link
 #9

Updated by mkittler over 4 years ago

The workaround PR has been merged in the meantime: https://github.com/os-autoinst/os-autoinst/pull/1451

Actions
Copy link
 #10

Updated by ggardet_arm over 4 years ago

  • Related to action #68510: test fails in logpackages on some aarch64 workers added
Actions
Copy link
 #11

Updated by ggardet_arm over 4 years ago

  • Status changed from New to Resolved
Actions
Copy link
 #12

Updated by okurz over 4 years ago

  • Assignee set to nadvornik

thanks all!

Actions
Copy link
 #13

Updated by mkittler over 4 years ago

As @nadvornik already said: This is just a workaround. So I didn't resolve the issue.

Apparently it also happens on @ggardet_arm's Tumbleweed machine. That means we can discard the idea that it is just a miscompiled opencv in devel:openQA:SLE-15/SLE_15_SP1.

Actions
Copy link
 #14

Updated by okurz over 4 years ago

Sounds like #68554 is related and can be used as followup

Actions
Copy link
 #15

Updated by okurz over 4 years ago

  • Related to action #68554: [sporadic] some tests fail in incomplete state due to a 'cv::Exception' added
Actions
Copy link

Also available in: Atom PDF