Testing reports

Colin Scott gnucash at double-bars.net
Tue Mar 27 18:51:00 EDT 2012


> Colin Scott is the one who wanted the output format to be tested.

To nit-pick a little, I was less concerned that the output format (ie, the layout) and changed than that the formatting - the characters that are included to achieve the layout - had changed.  This is because the formatting has implications for how the report renders if imported into Excel.  Changes to the layout may or may not be an error, but  I would view it as problematical werea report changed in such a way that the layout (and especially the columnar layout) varies according to whther the report is being viewed natively or within Excel

> Better to get the basics covered first before we worry about that. 
> Unit testing the reports code would be an excellent start. 

Couldn't agree more!  

Colin

-------- Original Message --------

*Subject:* Re: Testing reports
*From:* John Ralls <jralls at ceridwen.us>
*To:* Yawar Amin <yawar.amin at gmail.com>
*CC:* "gnucash-user at gnucash.org List" <gnucash-user at gnucash.org>
*Date:* Tue, 27 Mar 2012 07:15:33 -0700

On Mar 26, 2012, at 8:27 PM, Yawar Amin wrote:

> Hi John,
> 
> On 2012-03-23, at 19:48, John Ralls wrote:
> 
>>> [_]
>>> 
>>> So, what's a good test harness for a Scheme report against the GnuCash
>>> backend?
>> 
>> For what value of "good"?
> 
> _Good_ meaning something that ensures the contents of a report stay consistent between different versions. For example, the original problem on this thread was that the Profit & Loss report wasn_t showing correct figures after the book was closed. So that was a regression. A test for this report might be able to ensure that its results are the same before and after book closing, no matter what date the closing was done on.
> 
> If I understand unit testing, that should be able to cover cases like these?
> 
> Btw, I wasn_t actually talking about the output_s HTML rendering. Obviously, we_re rendering as a web page; we don_t really expect the result to be pixel-perfect.
> 

A test harness doesn't ensure anything. It facilitates running tests, collecting results, generating reports, and telling build systems (e.g. make) that tests failed. Nicer ones will have extra features, like a way to organize tests hierarchically and a variety of comparison functions. 

No, unit testing is testing each function (unit) in a program, isolated as much as possible from the rest of the program (even to the extent of having stand-ins for functions called by the function-under-test to minimize dependencies) while varying the inputs to probe each execution path through the function. Testing a report's output falls under what is variously called integration, functional, or acceptance testing, which tests whole libraries or programs together to make sure that all of the pieces work together properly. You can use a unit-testing harness to do parts of integration testing, but testing the GUI requires a special harness which simulates user input and instruments in some way the GUI display. The Ruby on Rails testing tutorial [1] has a nice introduction to the various phases of testing (and separates functional and integration testing).

Colin Scott is the one who wanted the output format to be tested. Better to get the basics covered first before we worry about that. Unit testing the reports code would be an excellent start. 

So, back to your original question: There's already a rudimentary test harness in place, along with some even more rudimentary tests. The shortest path would be to use that, and for the moment that's the best choice for integration tests because it's already in place. On the C side, I'm using the GLib testing facility [2]. You can wrap that facility for use in Guile if you want a purer unit-testing experience. Guile 2.0 includes its own unit testing module, but AFAIK Gnucash still chokes on Guile 2, so that would have to be fixed before you could use it.

Regards,
John Ralls
[1] http://guides.rubyonrails.org/testing.html
[2] http://developer.gnome.org/glib/unstable/glib-Testing.html
_______________________________________________
gnucash-user mailing list
gnucash-user at gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.



More information about the gnucash-user mailing list