action #36751
closed[functional][u] test valgrind
Added by mbrugger almost 7 years ago. Updated over 6 years ago.
0%
Description
Motivation¶
I lately realized that upstream valgrind is not properly working on arm64 and therefore on openSUSE Leap 15. It would be good if we could run a test of valgrind on openQA to fetch this regressions early.
Suggestion¶
Implement a simple check for valgrind -q --leak-check=full --show-reachable=yes $(which true) as a package self-test comparable to https://build.opensuse.org/package/view_file/openSUSE:Factory/widelands/widelands.spec?expand=1 , line 133
Updated by okurz almost 7 years ago
- Subject changed from test valgrind with openQA to [functional][u] test valgrind with openQA
- Assignee changed from openSUSE-Team-at-SUSE to okurz
Sounds like a good idea. Do you have any further references regarding the "is not properly working". Do you have mailing list discussions or bug reports which describe what exactly did not work?
I can think of a simple valgrind -q --leak-check=full --show-reachable=yes $(which true) test as well as one executed on a faulty binary if that is necessary. WDYT?
Also I wonder about upstream test suites which we could also execute during build time in OBS as well as a potential post-install test, still executed in OBS, as I assume valgrind is neither about GUI testing nor system level integration tests. WDYT?
Updated by mbrugger almost 7 years ago
okurz wrote:
Sounds like a good idea. Do you have any further references regarding the "is not properly working". Do you have mailing list discussions or bug reports which describe what exactly did not work?
In my case the problem is a missing instruction decoding which you hit with newer kernel and newer glibc:
https://bugzilla.suse.com/show_bug.cgi?id=1086543
I can think of a simple
valgrind -q --leak-check=full --show-reachable=yes $(which true)test as well as one executed on a faulty binary if that is necessary. WDYT?
I don't know if we want to do a functional test, like if it finds the errors in a faulty binaries. I think there are many corner cases and we can easily miss some cases.
Also I wonder about upstream test suites which we could also execute during build time in OBS as well as a potential post-install test, still executed in OBS, as I assume valgrind is neither about GUI testing nor system level integration tests. WDYT?
There is an upstream test suite. What I found from the README_PACKAGERS file:
-- Please test the final installation works by running it on something huge. I suggest checking that it can start and exit successfully both Firefox and OpenOffice.org. I use these as test programs, and I know they fairly thoroughly exercise Valgrind. The command lines to use are:valgrind -v --trace-children=yes firefox
valgrind -v --trace-children=yes soffice
I think we can add the test suite in the OBS build, but I suppose that won't be enough.
Updated by okurz almost 7 years ago
mbrugger wrote:
I can think of a simple
valgrind -q --leak-check=full --show-reachable=yes $(which true)test as well as one executed on a faulty binary if that is necessary. WDYT?I don't know if we want to do a functional test, like if it finds the errors in a faulty binaries. I think there are many corner cases and we can easily miss some cases.
Sorry but I don't understand now, what do you suggest we should test in openQA?
valgrind -v --trace-children=yes firefox
valgrind -v --trace-children=yes soffice
I think we can add the test suite in the OBS build, but I suppose that won't be enough.
Ok, why would that not be enough? What else would you execute then?
Updated by okurz almost 7 years ago
- Target version changed from future to future
Updated by okurz over 6 years ago
- Status changed from New to Feedback
- Target version changed from future to Milestone 25
Hi @mbrugger, could you help me answer my last two questions in #36751#note-3 ?
Updated by mbrugger over 6 years ago
okurz wrote:
Hi @mbrugger, could you help me answer my last two questions in #36751#note-3 ?
After a chat with Dirk I think the best would be to add a simple check as you describe in #36751#note-1
Although it will only help us to find a totally broken valgrind as we had in aarch64.
Unfortunately the valgrind testsuite is really fragile and most probably will throw a lot of false positives.
Updated by okurz over 6 years ago
- Project changed from openQA Tests (public) to QA (public)
- Subject changed from [functional][u] test valgrind with openQA to [functional][u] test valgrind
- Description updated (diff)
- Category deleted (
New test) - Status changed from Feedback to Workable
- Assignee deleted (
okurz) - Target version changed from Milestone 25 to Milestone 22
yes, that makes sense, thank you. Let's try with a package based check. I do not see the need to do that within a VM within openQA for now.
Updated by dirkmueller over 6 years ago
Added a selftest for valgrind in the %check section as part of the 3.14 update.
Updated by zluo over 6 years ago
- Status changed from Workable to In Progress
- Assignee set to zluo
take over
Updated by okurz over 6 years ago
@zluo please keep in mind that this is not a request for an openQA test but still should not be too hard
Updated by zluo over 6 years ago
- Assignee deleted (
zluo)
after talked with mgriessmeier, I don't thin I won't be able do this manual testing regarding to workflow with OBS, spec changes etc. But mkittler is willing to try this...
Updated by zluo over 6 years ago
I'll open a ticket about valgrind test module for openQA.
Updated by mkittler over 6 years ago
- Status changed from In Progress to Resolved
The package already contains a "patent pending" self-test: https://build.opensuse.org/package/view_file/devel:tools/valgrind/valgrind.spec?expand=1
This test is also present in the openSUSE:Factory version. The package containing the test also builds for aarch64. So it looks like the ticket has been resolved with https://build.opensuse.org/request/show/652030. If that's not true, just reopen the ticket. However, then I'm not sure what is missing here.
Updated by okurz over 6 years ago
- Assignee set to mkittler
@mkittler what a nice coincidence! Yep, that should be enough, cool!
zluo wrote:
I'll open a ticket about valgrind test module for openQA.
Please don't. Why should we need that?