how exactly to do unit testing in scheme...

[Robert Merkel]

Sorry for the tardy response - I went away for the long weekend and was
busy yesterday!

As Phil says, you can test both leaf functions and controller code with
unit tests. With controller code, you may need to write mocks for some or
all of the called code.  I don't know whether there are prewritten mocking
libraries for guile, but given the extreme flexibility of scheme it
shouldn't be difficult to write code to replace one function call with
another where necessary.

As long as xaccTransGetDate and xaccSplitGetParent are adequately tested,
there is no particular reason from a testing perspective to throw in an
intermediate function.

As far as which options to test, if testing all the combinations
exhaustively results in too many tests (and it sounds like it does) try
pairwise combinatoric testing, which works as follows:

Say you've got options {o_1, o_2, o_3, ... o_n}, each of which can take a
limited set of values {o_i^1, o_i_2, o_i^k} (for instance, if the option
can either be "on" or "off" it has two values).

For any pair of options o_x and o_y, for all the possible combinations of
option values there should be at least one test that has that combination.

To give a very simple case, for three options {a, b, c}  each of which can
take two values {off, on}, you could have the following tests:

     Option   a     b     c
Test
1                 off  off   off
2                 off   on  off
3                 on   off   on
4                 on    on  off
5                 off   off    on

For the pair (a,b) tests 1 through 4 give all the possible combinations,
for the pair (a, c) tests {1, 3, 4, 5} give all the possible combinations,
and for the pair (b, c) tests {1, 2, 3, 4} give all the possible
combinations.

5 is the minimum number of tests that can meet the pairwise combinatoric
criterion, as compared to 8 if you tried all the possible combinations.
However, it becomes an even bigger win if you're trying lots of options:
you only need 9 tests for 8 options with two values, whereas you would need
256 tests to try every possible option combination!

To generate optimal pairwise test sets, you can use "jenny" (this is a
really useful but unmaintained tool, I really should take over the
maintenance):

http://burtleburtle.net/bob/math/jenny.html

Hope this helps, and feel free to ask more questions.  I will try to
respond more promptly!


On Thu, Jan 25, 2018 at 3:03 AM, Christopher Lam <christopher.lck at gmail.com>
wrote:

> Dear Devel
>
>
>
> To rgmerk: Welcome back, and it was a nice to meet irl!
>
>
>
> While simplifying transaction.scm and thinking of unit testing, I now have
> a conundrum worthy of an expert view.
>
>
>
> The reports require 2 main functions – the options generator and the
> renderer; the options generator generates a options.scm controller object,
> and the renderer takes options and outputs html.
>
>
>
> I understand unit testing to handle testing of ‘leaf’ functions e.g.
> (split->date), rather than the controller code (e.g. renderer takes options
> and outputs html) – but to me this is rather silly because split->date only
> tests xaccTransGetDate and xaccSplitGetParent, whereas the controller tests
> actual functionality.
>
>
>
> With regards to unit testing I can see several issues
>
>
>
>    1. The refactored report has inlined most single-use functions into
>    lambda expressions – I figured that directly stating (xaccTransGetDate
>    (xaccSplitGetParent split)) is much more descriptive to a programmer than
>    to create a testable leaf function (split->date split). I can see the
>    benefits of both – leave as lambda expressions which will can be
>    understandable by anyone who is familiar with the API, or break them out
>    into 100s of single use functions which can be tested, but introduces a
>    whole layer of cognitive load to anyone hacking code – (what does
>    split->date actually do? Where is its definition). Also, breaking the
>    lambda functions into testable functions means the implementation is frozen
>    and the next hacker will have lesser scope to rework/optimise the report.
>
>
>
>    1. The refactored report is now flexible enough to accommodate derived
>    reports with a different multicolumn data function – eg
>    income-gst-statement.scm has been reworked into a transaction.scm
>    derivative which passes its own calculated-cells to report on GST sales and
>    purchases. This is not yet committed.
>
>
>
>    1. I think the most useful testing approach for a complex
>    transaction.scm will be to test functions of various combinations of
>    options values, and test the resulting html for satisfactory output. There
>    are now dozens of bools and multichoices that can be triggered, each
>    effecting html in various ways. How best to test?
>
>
>
>    1. My view would be the unit test would check that:
>       1. the TR actually exists
>       2. it can display empty-report
>       3. it can understand passing of custom-calculated-cells
>       4. each of the options can be toggled, and the resulting html
>       displays/hides cells/detail as expected
>       5. and sorting options generate sorted rows
>
>
>
> Comments welcome, I had no formal training ☹
>