Testing reports

[John Ralls]

On Apr 10, 2012, at 10:56 AM, Derek Atkins wrote:

> John Ralls <jralls at ceridwen.us> writes:
>> 
>> You guys are forgetting about CSS's affect on the way a particular DOM
>> is rendered. That can make a huge difference in appearance. Play
>> around a bit with http://www.csszengarden.com/ to see what I mean (or
>> just for fun, it's an awesome demo!)
> 
> Personally I'm ignoring CSS, because I don't care about testing the
> layout.
> 
>> An easy way to test the HTML without worrying about tag case or
>> extraneous whitespace is to use HTML Diff [1] against a canned
>> comparison document. That will get us close enough for a first order
>> test, I think. Anyone got time to code something up?
> 
> Is there some way to automate the validation of HTML Diff output?  When
> I did a cursory look at HTML Diff options they all marked up the HTML
> with diff-tags, whereas what we want would be essentially a boolean
> output whether there are any differences or not.
> 
>> The talk about DOM gave me another idea: Since the html generator
>> seems to be outputting XHTML transitional (at least it *says* it is),
>> we can write a strict schema and then validate the output report
>> against the schema. If it validates, it passes.
> 
> Except that we don't care about the HTML formatting as much as we care
> about the contents of the report (see my response from a minute ago).  A
> Schema validation isn't going to tell you that the report output $10.01
> when it should have output $100.01.
> 

HTML Diff is a single python source file. We can fix it up to do whatever we want.

True, a schema validation won't check contents. One would then have to walk the DOM tree to test the values of each element and attribute.

A schema-and-content test would be less brittle than would an HTML Diff test, which in turn would be less brittle than a simple diff test. It's also easier to isolate changes when coding, so that any side effects are immediately apparent.

On the other hand, it's also a lot more work to implement.

Regards,
John Ralls