Testing reports

John Layman john.layman at laymanandlayman.com
Fri Apr 13 11:20:23 EDT 2012


> Chasing down a single test failure, particularly in system tests (as
opposed to unit tests),
> takes far more time than writing a good test in the first place.

If there were actual data to support such a claim, who could disagree?  As
intuition, however, the claim sounds suspiciously like rationalization.  I'm
reminded of the old saw about the hazards of asking an engineer the time.
At what probability does is make no sense to provide against an imaginable
eventuality?  Good grief!

An important point the developers of GnuCash need to bear in mind is that
the economics of the software do not begin and end with them.  The time
required to investigate why a test failed may be dear to the developer,
personally, but the cost of poor quality multiplied across the user
community is far more consequential.  The point which I'd hope we could all
agree upon is that a delay in putting some degree of effective testing in
place cannot be justified on grounds of sparing developers menial work.  

-----Original Message-----
From: John Ralls [mailto:jralls at ceridwen.us] 
Sent: Thursday, April 12, 2012 12:05 PM
To: john.layman at laymanandlayman.com
Cc: gnucash at double-bars.net; warlord at MIT.EDU; gnucash-user at gnucash.org
Subject: Re: Testing reports


On Apr 12, 2012, at 7:06 AM, John Layman wrote:

>> Tests which fail for insignificant reasons (like case in an SGML tag)
> waste
>> developer time and are worse than useless.
> 
> One could argue that it's a bigger waste of developer time to 
> complicate test creation in the interest of discriminating that which 
> is significant from that which is not in every imaginable potential 
> scenario.  By implication, you appear to be saying that the trivial 
> work of updating a test oracle is a poor use of the programming 
> artiste's time.  Artists do clean their own brushes, you know.  Doing 
> so may be trivial, but it's part of the process.

One could argue that, but one would be wrong. Chasing down a single test
failure, particularly in system tests (as opposed to unit tests), takes far
more time than writing a good test in the first place. That's especially
true if someone else has already written a comparison function that
discriminates correctly between changes that matter and those that don't,
which is the case for HTML parsing.

Regards,
John Ralls





More information about the gnucash-user mailing list