Testing reports

[John Ralls]

On Mar 27, 2012, at 1:39 AM, Colin Law wrote:

> On 26 March 2012 23:14, John Ralls <jralls at ceridwen.us> wrote:
>> 
>> On Mar 26, 2012, at 1:45 PM, Colin Law wrote:
>> 
>>> On 26 March 2012 15:15, John Ralls <jralls at ceridwen.us> wrote:
>>>> ...
>>>> Do you know how to write automated tests -- in any language -- that accurately check the output format of an HTML page? Automated in this case means the test program decides whether the formatting is correct, no human involved. If you do, or know where I can learn that, please share.
>>> 
>>> Ruby on Rails has various tools that allow automated testing of html
>>> output (designed for testing web apps), but I don't know that much
>>> about them or how they could be used with gnucash.  It would need a
>>> working Ruby environment I believe with the appropriate Ruby Gems
>>> installed and so may not be appropriate.
>> 
>> Got a more complete pointer? A particular Gem, maybe? What I found [1][2] don't appear to do post-render checks. Most of the search results focussed on testing the Ruby rather than the HTML. That said, validating the HTML for well-formedness would be a pretty good start. There are C, Perl, and Python tools for doing that. A Ruby tool would be OK, too -- I made the Git tools that we use for synching with the Subversion repository by translating a Ruby Gem and then elaborating it.
>> 
>> Regards,
>> John Ralls
>> 
>> [1] http://guides.rubyonrails.org/testing.html
>> [2] http://strugglingwithruby.blogspot.com/2010/04/testing-for-valid-html.html
> 
> Section 4.6 Testing Views in the rails guide, [1 above] gives a brief
> overview of how to check html elements.  Also there is rspec that is
> considered by many the best route at the moment (for use with Rails of
> course).  See [3] for a quick rundown.
> 
> [3] http://www.arailsdemo.com/posts/37

Ah. I learn from that link that Rspec is Ruby's implementation of "Behavior Driven Development" [1]. Since Gnucash is a "legacy"* project, it's a bit late -- or early -- to embrace that paradigm., but we can still apply the testing strategy.

 It doesn't appear to me that Rspec would work very well testing non-Ruby programs, but I'm willing to be convinced otherwise. But that might be unnecessary if we can find tools which are a better fit. For example, I quickly found cspec [2] (unfortunately not seriously maintained) and lettuce [3] for C and Python, respectively. Nothing directly for Scheme, but [4] points to some (Emacs) Lisp frameworks which could probably be adapted without too much work. Christian might find "c plus plus Behavior Driven Development tools" [5] interesting. I don't know that any of these tools have the direct support for html testing the Rspec does (I'd guess that they don't), but HTMLDiff [6], a Python program, might be helpful either as-is or as a guide for writing something in Scheme.

Regards,
John Ralls

* By "legacy", I take the meaning from Feathers, "Working Effectively with Legacy Code" [7]: Code developed without unit testing. Gnucash could be considered legacy in some other meanings, too, in that the original developers are no longer involved or that it isn't completely object-oriented (and even the parts that are aren't necessarily well designed). I highly recommend the book, by the way.

[1] http://en.wikipedia.org/wiki/Behavior_Driven_Development
[2] https://github.com/arnaudbrejeon/cspec
[3] http://lettuce.it/
[4] http://www.emacswiki.org/emacs/UnitTesting#toc5
[5] http://www.squidoo.com/cplusplus-behaviour-driven-development-tools
[6] http://www.aaronsw.com/2002/diff/
[7] http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052/ref=sr_1_1?s=books&ie=UTF8&qid=1332864930&sr=1-1