QA

okurz wrote:

Sounds like a good idea. Do you have any further references regarding the "is not properly working". Do you have mailing list discussions or bug reports which describe what exactly did not work?

In my case the problem is a missing instruction decoding which you hit with newer kernel and newer glibc:
https://bugzilla.suse.com/show_bug.cgi?id=1086543

I can think of a simple valgrind -q --leak-check=full --show-reachable=yes $(which true) test as well as one executed on a faulty binary if that is necessary. WDYT?

I don't know if we want to do a functional test, like if it finds the errors in a faulty binaries. I think there are many corner cases and we can easily miss some cases.

Also I wonder about upstream test suites which we could also execute during build time in OBS as well as a potential post-install test, still executed in OBS, as I assume valgrind is neither about GUI testing nor system level integration tests. WDYT?

There is an upstream test suite. What I found from the README_PACKAGERS file:

-- Please test the final installation works by running it on something
huge. I suggest checking that it can start and exit successfully
both Firefox and OpenOffice.org. I use these as test programs, and I
know they fairly thoroughly exercise Valgrind. The command lines to use
are:

valgrind -v --trace-children=yes firefox

valgrind -v --trace-children=yes soffice

I think we can add the test suite in the OBS build, but I suppose that won't be enough.

mbrugger wrote:

I can think of a simple valgrind -q --leak-check=full --show-reachable=yes $(which true) test as well as one executed on a faulty binary if that is necessary. WDYT?

I don't know if we want to do a functional test, like if it finds the errors in a faulty binaries. I think there are many corner cases and we can easily miss some cases.

Sorry but I don't understand now, what do you suggest we should test in openQA?

valgrind -v --trace-children=yes firefox
valgrind -v --trace-children=yes soffice
I think we can add the test suite in the OBS build, but I suppose that won't be enough.

Ok, why would that not be enough? What else would you execute then?

okurz wrote:

Hi @mbrugger, could you help me answer my last two questions in #36751#note-3 ?

After a chat with Dirk I think the best would be to add a simple check as you describe in #36751#note-1
Although it will only help us to find a totally broken valgrind as we had in aarch64.
Unfortunately the valgrind testsuite is really fragile and most probably will throw a lot of false positives.

Added a selftest for valgrind in the %check section as part of the 3.14 update.

@zluo please keep in mind that this is not a request for an openQA test but still should not be too hard

I'll open a ticket about valgrind test module for openQA.