how exactly to do unit testing in scheme...

[Christopher Lam]

Hi Robert and devel,

Thank you for pointers, this is useful. The examples below illustrate 
how to test multiple items together. But I think the complexity of the 
52(and counting) options (37 binary, the rest are multiples/multichoice) 
means, I think it'll be better to handwrite tests. I'll need to reuse 
existing test frameworks to attempt maximise coverage.

C

On 31/01/18 09:01, Robert Merkel wrote:
> Sorry for the tardy response - I went away for the long weekend and was
> busy yesterday!
>
> As Phil says, you can test both leaf functions and controller code with
> unit tests. With controller code, you may need to write mocks for some or
> all of the called code.  I don't know whether there are prewritten mocking
> libraries for guile, but given the extreme flexibility of scheme it
> shouldn't be difficult to write code to replace one function call with
> another where necessary.
>
> As long as xaccTransGetDate and xaccSplitGetParent are adequately tested,
> there is no particular reason from a testing perspective to throw in an
> intermediate function.
>
> As far as which options to test, if testing all the combinations
> exhaustively results in too many tests (and it sounds like it does) try
> pairwise combinatoric testing, which works as follows:
>
> Say you've got options {o_1, o_2, o_3, ... o_n}, each of which can take a
> limited set of values {o_i^1, o_i_2, o_i^k} (for instance, if the option
> can either be "on" or "off" it has two values).
>
> For any pair of options o_x and o_y, for all the possible combinations of
> option values there should be at least one test that has that combination.
>
> To give a very simple case, for three options {a, b, c}  each of which can
> take two values {off, on}, you could have the following tests:
>
>       Option   a     b     c
> Test
> 1                 off  off   off
> 2                 off   on  off
> 3                 on   off   on
> 4                 on    on  off
> 5                 off   off    on
>
> For the pair (a,b) tests 1 through 4 give all the possible combinations,
> for the pair (a, c) tests {1, 3, 4, 5} give all the possible combinations,
> and for the pair (b, c) tests {1, 2, 3, 4} give all the possible
> combinations.
>
> 5 is the minimum number of tests that can meet the pairwise combinatoric
> criterion, as compared to 8 if you tried all the possible combinations.
> However, it becomes an even bigger win if you're trying lots of options:
> you only need 9 tests for 8 options with two values, whereas you would need
> 256 tests to try every possible option combination!
>
> To generate optimal pairwise test sets, you can use "jenny" (this is a
> really useful but unmaintained tool, I really should take over the
> maintenance):
>
> http://burtleburtle.net/bob/math/jenny.html
>
> Hope this helps, and feel free to ask more questions.  I will try to
> respond more promptly!
>
>
> On Thu, Jan 25, 2018 at 3:03 AM, Christopher Lam <christopher.lck at gmail.com>
> wrote:
>
>> Dear Devel
>>
>>
>>
>> To rgmerk: Welcome back, and it was a nice to meet irl!
>>
>>
>>
>> While simplifying transaction.scm and thinking of unit testing, I now have
>> a conundrum worthy of an expert view.
>>
>>
>>
>> The reports require 2 main functions – the options generator and the
>> renderer; the options generator generates a options.scm controller object,
>> and the renderer takes options and outputs html.
>>
>>
>>
>> I understand unit testing to handle testing of ‘leaf’ functions e.g.
>> (split->date), rather than the controller code (e.g. renderer takes options
>> and outputs html) – but to me this is rather silly because split->date only
>> tests xaccTransGetDate and xaccSplitGetParent, whereas the controller tests
>> actual functionality.
>>
>>
>>
>> With regards to unit testing I can see several issues
>>
>>
>>
>>     1. The refactored report has inlined most single-use functions into
>>     lambda expressions – I figured that directly stating (xaccTransGetDate
>>     (xaccSplitGetParent split)) is much more descriptive to a programmer than
>>     to create a testable leaf function (split->date split). I can see the
>>     benefits of both – leave as lambda expressions which will can be
>>     understandable by anyone who is familiar with the API, or break them out
>>     into 100s of single use functions which can be tested, but introduces a
>>     whole layer of cognitive load to anyone hacking code – (what does
>>     split->date actually do? Where is its definition). Also, breaking the
>>     lambda functions into testable functions means the implementation is frozen
>>     and the next hacker will have lesser scope to rework/optimise the report.
>>
>>
>>
>>     1. The refactored report is now flexible enough to accommodate derived
>>     reports with a different multicolumn data function – eg
>>     income-gst-statement.scm has been reworked into a transaction.scm
>>     derivative which passes its own calculated-cells to report on GST sales and
>>     purchases. This is not yet committed.
>>
>>
>>
>>     1. I think the most useful testing approach for a complex
>>     transaction.scm will be to test functions of various combinations of
>>     options values, and test the resulting html for satisfactory output. There
>>     are now dozens of bools and multichoices that can be triggered, each
>>     effecting html in various ways. How best to test?
>>
>>
>>
>>     1. My view would be the unit test would check that:
>>        1. the TR actually exists
>>        2. it can display empty-report
>>        3. it can understand passing of custom-calculated-cells
>>        4. each of the options can be toggled, and the resulting html
>>        displays/hides cells/detail as expected
>>        5. and sorting options generate sorted rows
>>
>>
>>
>> Comments welcome, I had no formal training ☹
>>