Running unit tests under valgrind

[John Ralls]

On May 21, 2012, at 8:23 PM, Donald Allen wrote:

> On Mon, May 21, 2012 at 11:10 PM, John Ralls <jralls at ceridwen.us> wrote:
>> 
>> On May 21, 2012, at 3:41 PM, Donald Allen wrote:
>> 
>>> On Mon, May 21, 2012 at 6:22 PM, Phil Longstaff <phil.longstaff at yahoo.ca> wrote:
>>>> Yes.  A lot of the leaks are in the unit test code.  But I've found 2 legitimate leaks as well.
>>> 
>>> Do you guys run the whole system under valgrind (John is obviously
>>> right about tests on the parts don't necessarily say anything about
>>> the whole) prior to release as part of your QA process?
>> 
>> Not routinely. Every so often someone will get motivated and run under valgrind for a while and chase down some of the leaks, but there are too many execution paths and not yet enough tests to be sure of exercising even a significant percentage of the program. We'll get that sorted eventually.
> 
> I would argue that waiting until you have what you think is adequate
> test coverage is not a good strategy. Releases don't happen that often
> and it sounds like running the whole thing under valgrind isn't that
> difficult, so the cost of doing it is not high. And it just might turn
> up something important, even without comprehensive test coverage. You
> could make the gnucash+valgrind package available to some volunteers
> whever you see fit during the release cycle (I'd be happy to be one of
> them) to exercise the system and report the problems it turns up. So I
> think it makes sense from a cost-benefit standpoint not to wait.

Pay closer attention to what I wrote, please. Merely launching Gnucash and doing a few basic operations under Valgrind won't accomplish much -- but it will take a long time, because Valgrind slows execution by a factor of 20-30x, according to their own documentation.

As for the comprehensive testing, I'm plugging away at that in the engine, and Phil started this thread by noting that he was finding leaks with tests -- which is what he's working on (in the backend code).

As a more general observation, your constant carping about a process that *you don't participate in* is annoying and unhelpful.
Please restrain yourself.

Regards,
John Ralls