Testing reports

[John Layman]

> There are no circumstances I can envisage where it is proper to ignore
*any* change to the output.

Indeed.  Whether you test format and content jointly or separately, the
expected result is not a moving target.  In basic testing, all you need to
ensure is that nothing unexpected has changed from the test oracle (in the
case of reports, there could conceivably be expected changes such as "date
printed" or some such that you would purposely ignore.)  In my Windows
installation of GnuCash, the Net Worth Linechart simply produces "An error
occurred while running the report."  Since the barchart version of the same
information runs just fine, I'm guessing the linechart at one time did
produce the expected result.  It would not require a complicated test to
detect that something had gone amiss with this report.

-----Original Message-----
From: gnucash-user-bounces+john.layman=laymanandlayman.com at gnucash.org
[mailto:gnucash-user-bounces+john.layman=laymanandlayman.com at gnucash.org] On
Behalf Of Colin Scott
Sent: Wednesday, April 11, 2012 8:06 AM
To: warlord at MIT.EDU; clanlaw at googlemail.com
Cc: gnucash-user at gnucash.org
Subject: Re: Testing reports


> I would argue that the contents of the report are more important than 
> the formatting, at least in the context of how this thread started.

Hmmmmm.  I'm not at all sure that I agree.  *WHY* would the report
formatting change?

The fundamental point here is that even if the change is harmless it should
only occur as the result of a specific and specifically intended action -
anything else must be a bug!!!  Either way, any change in the output should
be flagged, either so it may be certified as correct (and then incorporated
into the standard against which subseqent tests will be made) or so the
cause of the change can be tracked down and fixed.

There are no circumstances I can envisage where it is proper to ignore *any*
change to the output.

Colin

-------- Original Message --------

*Subject:* spam,Spam:****, Re: spam,Re: Testing reports
*From:* Derek Atkins <warlord at MIT.EDU>
*To:* Colin Law <clanlaw at googlemail.com>
*CC:* gnucash at double-bars.net, gnucash-user at gnucash.org
*Date:* Tue, 10 Apr 2012 13:52:45 -0400

Colin Law <clanlaw at googlemail.com> writes:

> It is a bit of a philosophical point really, but one could argue that 
> the output from the software is a report rather than a chunk of html.
> What matters is what the report looks like, and if the Document Object 
> Model has not changed then it will look the same whatever the details 
> of the underlying html.  Using the DOM allows the test to relate to 
> the end result, with tests such as "there should be a paragraph of a 
> particular class containing particular text", rather then "there 
> should be a string of the form <p class =......".

I would argue that the contents of the report are more important than the
formatting, at least in the context of how this thread started.
It's important to see that a foo report on a data file would report that the
balance of account X is $y.zz at date 1, and that after operation O occurs,
the balance of that same account is $a.bb (which may be the same as y.zz).

> I don't know how the html is generated in the code, if it uses an html 
> library (or if at some point in the future it used an html library) 
> then another issue could be that a newer version of the library might, 
> for example, change the order of attributes in a tag.  A DOM based 
> test would not care about this but for a text based one it would be a 
> disaster.

It's done using a standalone scheme generator.

> Colin

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord at MIT.EDU                        PGP key available

_______________________________________________
gnucash-user mailing list
gnucash-user at gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.