Testing reports

[Colin Scott]

> In any other case, though, that level of change awareness is too 
> expensive

Not half as bloody expensive as a bug (ie, an incorrect change) introduced that you can't find because you make too many assumptions about the correctness of the output from others' modules/libraries.  BTDGTTS!  (And yes, I was working in commercial development at the time, and the incident was one of the reasons for developing an auto-test environment in the first place!)

> or the comparison document gets robotically changed to match the 
> new output.

Which is, of course, exactly what *should* happen - once one has established that all the output changes flagged are harmless and/or WAD

Colin

-------- Original Message --------

*Subject:* Re: Testing reports
*From:* John Ralls <jralls at ceridwen.us>
*To:* gnucash at double-bars.net
*CC:* warlord at MIT.EDU, clanlaw at googlemail.com, gnucash-user at gnucash.org
*Date:* Thu, 12 Apr 2012 09:17:51 -0700

On Apr 12, 2012, at 7:57 AM, Colin Scott wrote:

> 
>> ... Tests which fail for insignificant 
>> reasons (like case in an SGML tag) waste developer time and are 
>> worse than useless.
> 
> Changes made in external libraries are just as capable of screwing up your results as changes made anywhere else, and you need to know what changes have occurred, even where those changes are harmless - and if they are they need to be incorporated into your standard.  

Rubbish again. Well, maybe it's true if you're writing embedded controls for critical equipment (say, fly-by-wire for an airplane). In any other case, though, that level of change awareness is too expensive. In commercial work, it would drive the cost of the product beyond what customers would pay; in volunteer work (like Gnucash) it would mean that those tests are ignored -- or the comparison document gets robotically changed to match the new output.

Regards,
John Ralls