Testing reports

[Colin Scott]

> So basically what you would like to test is that the normalized HTML
> output is "the same".  This would require:

Actually, no.  Were I doing this I would automate a test for whether the HTML text has changed.  At the point a change is found, it would probably require human intervention to see what has changed and why - *any* change is an error unless there is a legitimate reason for it, and automating the legitimacy test is probably too expensive a task to be worth doing.

As for the rest of it, I think you are about where I would go (though you express it differently!).  I was automating the testing of, oddly enough, a very large and customisable financial and management accounting system.  However, it ran on text-based terminals rather than a graphical environment, so my task was somewhat simpler than it would be here.  It was made even easier because the system had been designed and implemented so that for the entire suite of several hundred programs, all keyboard input, and all screen and print output, was funnelled through a common interface layer.  This made it a relatively simple matter to insert shims to capture, character-by-character, all input and output.  Once that was done, everything else was pretty easy ...

Colin