[GSOC] Data model unit testing

[John Ralls]

On Mar 31, 2011, at 1:02 PM, Muslim Chochlov wrote:

> I estimate the effort in the following way:
> (AVG_of_branching * number_of_functions * time_to_test_one_branch_in_minutes)/(minutes_in_hour * number_of_working_days).
> Avg of branching in function is something that i get by dividing code complexity number (3729) by number of functions, time to test one condition (branch) takes me about 10 minutes. All above brings me to (4.2875 * 866 * 10)/(60*8) = 77.35 working days. Now google allocates 3 months of coding that is 66 working days (being realistic :). 77 and 66 don't match at all. What I propose is to change the goal to "cover at least 80% of the libqof module". Then we get (4.2875 * (866 * 0.8) * 10)/(60*8) = 61.88 working days. In the worst case this will still provide a pretty cool code coverage (as 80% is considered a standard in SW engineering) and I will be able to finish my project in time. In the best case the 100% coverage will be done.
> Let me know if such goal modification is ok with you guys. In that case I'll have all the requirements gathered from you and submit the project.

Like I said, you're the only one at this point who knows how fast you can code. But your calculations are a bit, um, overly precise. Here's how I'd do the estimate: You can get the actual number of branches with `egrep '(if|else|case)' src/libqof/qof/*.c | wc -l ` and get the actual number of branches (2674). Add the number of functions for the total number of simple tests (3580). Recognizing that we're estimating, call it 3600, which happens to be a very convenient number when dealing with minutes. A goodly number of those functions are simple getters and setters which don't need testing, but we can ignore that; the effect is less than an order of magnitude. Accepting your 6 tests/hour expectation (I think that that's ambitious, because it doesn't allow much time  for writing fixtures and mocks) it's around 600 hours, or 10 weeks at 60 hours/week. GSoC allows 13 weeks, so I don't really see the problem. ;-) 

Actually, I do see a problem, as I mentioned: Writing 6 tests/hour is realistic for well-designed C++ where the functions are small, have no side effects, make good use of polymorphism, have minimal dependencies, and have no local variables which need to be tested. The real world isn't like that, and Gnucash is infinitely worse. The best of it (only a couple of files in libqof) is written in C using an obtuse we-can-write-object-oriented-code-in-C library (GObject); there's a lot of "GObject is a pain to set up so I'll go halfway there with homemade vtables, but no inheritance or polymorphism", and a generous helping of not very well structured normal C code in huge functions with lots of branching.  There are internal and external dependencies, so tests will need fixtures and mocks to isolate the function-under-test from the outside. 

While I hope that you'll be inspired (and have time) to keep working on this after GSoC is over, we want to set you up for a successful GSoC project. I think we need to be looking at 25-30% slack in your goal to allow for fixturing, mocking, and refactoring... and while I don't have any real insight into what the Google folks are looking for, I think that it makes a stronger presentation (and a more measurable goal for writing your evals) to select some files in Qof and propose fully testing those files rather than to propose X% test coverage of the whole module. It's too easy to fudge a test coverage number (for example, by including silly tests, like testing the getters & setters). I'll troll through the LibQof sources tomorrow and propose a list of files; Christian may have some thoughts on which ones would be good candidates as well.

Regards,
John Ralls