Missing Guile Bindings and False Positive Unit tests
Chad Albers
calbers at neomantic.com
Wed Feb 22 13:52:25 EST 2017
Hi Derek,
Thanks for the history about Gnucash bindings. I might be asking for
trouble. I just would rather interact with GnuCash rather that
Python. Personal preference that is pretty arbitrary. I'm curious
how far I can take it.
After further inspection, I now understand what you meant my "swig
exports the functions for guile". I found the exactly location where
I need to add new exports in the swig interface file. I haven't
managed, though, to try it out yet.
--
On Tue, Feb 21, 2017 at 11:59 AM, Derek Atkins <warlord at mit.edu> wrote:
> Hi Chad,
>
> Please remember to CC gnucash-user on all replies so that everyone can
> remain "in the loop".
>
> Also, the gnc-modules should properly export swig-exported functions
> without a re-export. If it stopped doing that, that would be a bug
> IMHO.
>
> Note that these unit tests are probably 18 years old.
>
> <GnuCash-History>
> A long long time ago (well, around GnuCash 1.6 or so), /usr/bin/gnucash
> was a guile script. It loaded everything in as guile modules
> (technically it loaded GncModules) and then jumped back and forth
> between C and Scheme. Still, not everything was available from Scheme,
> but I don't believe anything has been *removed* from the guile
> bindings.
>
> Anyways, this structure was very hard to debug. You couldn't
> run it under gdb; you needed to "attach" gdb to the running process,
> which made it nearly impossible to debug issues during application
> loading. Eventually the Schemer developers left the project to pursue
> other endeavors, and those that remained eventually switched back to the
> "main() in C" structure that we have today.
>
> Of course a bunch of the functionality is still in GncModules.
> </GnuCash-History>
>
> Having said all that, you shouldn't need to export or re-export a
> wrapped API. It should "just work". And the unit tests should be
> properly failing when there are missing or broken APIs.
>
> -derek
>
> Chad Albers <calbers at neomantic.com> writes:
>
>> I'll file a bug report.
>>
>> But, for the record, swig isn't exporting. That's not what swig does.
>> Function exports are done by using 2 APIs - the Guile define-modules
>> API or Guiles' C API.
>>
>> For example,
>>
>> - Some guile functions are defined here: src/engine/swig-engine.c.
>> This is where qof-session-new is defined.
>> - swig-engine.c uses Guile's C API to create a module called sw_engine
>> - swig-engine.c exports the functions - including qof-session-new -
>> using scm_c_export(...)
>> - Several Guile scm files use this module: (use-modules (sw_engine)).
>> None of these scm files either (export...) or (re-export...) functions
>> from the sw_engine module.
>>
>> Therefore, none of the Guile defined modules expose sw_engine functions.
>>
>> There may be, though, one exception, but even that isn't exposing
>> anything from sw_engine. The following is speculation...
>>
>> - GnuCash seems to include, what I suspect, is another module system.
>> I think it's in the module (gnucash gnc-module). Calls such as these
>> - (gnc:module-system-init) - may be a part that system.
>> - Included in that system may be code from: src/engine/gncmod-engine.c
>> - This C code has a C function called 'libgncmod_engine_gnc_module_init(...)
>> - This includes the sw_engine module. It calls
>> scm_c_eval_string("(use-modules (sw_engine))");
>> - This scm_c_eval_string does not include any (exports...) or
>> (re-exports...) in the gncmode-engine.c file.
>>
>> Therefore, the sw_engine functions are not available in this other
>> module system.
>>
>> To make the sw_engine functions available (which is what I would
>> like), this may have been how it was once accomplished. The units
>> test that pass when they shouldn't include this line:
>>
>> (gnc:module-load "gnucash/engine" 0)
>>
>> Maybe, at one time gnc:module-load exported functions from
>> "gnucash/engine" modules. Or something like that. I'm still
>> investigating the what/why/how of the other module system.
>>
>> In the meantime, one solution is just to re-export sw_engine functions
>> from (gnucash engine) or engine.scm. That's what I'm doing for my
>> experiments, and it works. Perhaps I will submit a patch for that,
>> but I'm thinking that that's a hack. I'm opening up the discussion
>> here for a better solution, if exposing more of the Guile API is a)
>> fixing a bug and/or b) adding a new feature. Since much of GnuCash
>> was written in Guile, I thought I would use that instead of Python.
>> As is, though, there doesn't seem to be a way to even source a GnuCash
>> using its Guile bindings. It can be done using Python.
>>
>> Chad
>>
>> --
>>
>> On Sun, Feb 19, 2017 at 5:34 PM, Derek Atkins <derek at ihtfp.com> wrote:
>>> That srfi didn't exist when the tests were written, which was probably
>>> around 18 years ago. The fact the tests don't fail just means nobody tested
>>> the tests for false positives. Please file a bug report?
>>>
>>> There isn't a need to be-export. Swig exports do that automatically.
>>>
>>> -derek
>>>
>>> Sent from my mobile device. Please excuse any typos.
>>>
>>> ----- Reply message -----
>>> From: "Chad Albers" <calbers at neomantic.com>
>>> To: "Derek Atkins" <derek at ihtfp.com>
>>> Subject: Missing Guile Bindings and False Positive Unit tests
>>> Date: Sun, Feb 19, 2017 5:23 PM
>>>
>>> Definitely, though, the unit tests it is checking is
>>> 'qof-session-new'. It passes, but it hasn't been exported in the
>>> Guile code.
>>>
>>> At one time, I think it was exported from this file:
>>> 'src/mine/gnucash/src/engine/engine.scm' This file includes the
>>> (sw_engine) module (defined in Guile C API), but it does not "(export
>>> qof-session-new)" or "(re-export qof-session-new). As such, the unit
>>> test shouldn't pass. The function doesn't exist outside of the C code.
>>>
>>> I haven't figure out why the unit test is passing when it should fail.
>>> GnuCash seems to have created its own unit-testing framework for its
>>> guile functions. It doesn't use guile's unit-testing module (srfi
>>> srfi-64). I suspect that there are many other guile binding unit
>>> tests that are false positive. I haven't done a full audit. I just
>>> wanted to give someone a heads up about this problem.
>>>
>>> I would submit a patch to fix the unit testing framework, but I
>>> haven't actually worked out how it works. In the meantime, I'm
>>> currently exporting some of the qof-session functions as an
>>> experiment. My goal is to try to accomplish in Guile what can be
>>> accomplished with the Python bindings.
>>>
>>> --
>>>
>>> On Sun, Feb 19, 2017 at 5:07 PM, Derek Atkins <derek at ihtfp.com> wrote:
>>>> That's a different issue. I don't know why that happens.
>>>>
>>>> -derek
>>>>
>>>> Sent from my mobile device. Please excuse any typos.
>>>>
>>>> ----- Reply message -----
>>>> From: "Chad Albers" <calbers at neomantic.com>
>>>> To: "Derek Atkins" <derek at ihtfp.com>
>>>> Subject: Missing Guile Bindings and False Positive Unit tests
>>>> Date: Sun, Feb 19, 2017 5:01 PM
>>>>
>>>> True, Derek. However, GnuCash does have unit tests that exercise
>>>> Guile bindings that have not been properly exported. And those test
>>>> falsely pass, when they should not.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Chad Albers
>>>>
>>>>
>>>> On Sun, Feb 19, 2017 at 4:57 PM, Derek Atkins <derek at ihtfp.com> wrote:
>>>>> Hi,
>>>>> Except for a few functions in app-utils, most of QofSession is not
>>>>> wrapped
>>>>> in the scheme bindings like they are in the python bindings.
>>>>>
>>>>> -derek
>>>>>
>>>>> Sent from my mobile device. Please excuse any typos.
>>>>>
>>>>> ----- Reply message -----
>>>>> From: "Chad Albers" <calbers at neomantic.com>
>>>>> To: <gnucash-devel at gnucash.org>
>>>>> Subject: Missing Guile Bindings and False Positive Unit tests
>>>>> Date: Sun, Feb 19, 2017 4:46 PM
>>>>>
>>>>> Hi,
>>>>>
>>>>> I found some problems with some unit tests for the Guile bindings.
>>>>> Here's the background.
>>>>>
>>>>> Rather than use the Python bindings, I wanted to use guile to
>>>>> add/manipulate
>>>>> my GnuCash data. From the Python bindings, I deduced that the entry
>>>>> point into creating
>>>>> transactions is via a construct called a "session". Accordingly, I went
>>>>> to
>>>>> find
>>>>> the Guile equivalent: "qof-session-new". I wrote a Guile script which
>>>>> successfully
>>>>> imports GNUCash's guile bindings. However, I was never able to
>>>>> successfully
>>>>> use
>>>>> "qof-session-*" functions. They are considered "unbound".
>>>>>
>>>>> After further investigation, I found that GNUCash has unit tests that
>>>>> specifically exercise the
>>>>> 'qof-session-new' function.
>>>>> Here...'src/engine/test/test-create-account.scm'. I ran
>>>>> these units tests with 'make check' and they passed. I concluded that
>>>>> since the unit test
>>>>> passes, it must somehow have the binding for 'qof-session-new'.
>>>>>
>>>>> 'make check', though, creates a log for each unit test. I peeked in
>>>>> 'test-create-account.scm.log'.
>>>>> I found the following line:
>>>>>
>>>>> ;;; src/engine/test/./test-create-
>>>>> account.scm:30:18: warning: possibly
>>>>> unbound variable `qof-session-new'
>>>>>
>>>>> This is the same error that I have encountered in my own Guile script.
>>>>> But...I also see this line
>>>>> at the end of the test log.
>>>>>
>>>>> PASS test-create-account (exit status: 0)
>>>>>
>>>>> So the unit test cannot find 'qof-session-new' and yet the unit test
>>>>> framework thinks its
>>>>> passing. I wanted to let someone know that GNUCash has unit tests
>>>>> (perhaps only for guile)
>>>>> that are returning false positives. That's bug one. Would you like me
>>>>> to file a bug for this?
>>>>>
>>>>> It turns out that GnuCash is not properly exposing the Guile bindings.
>>>>> I consider this
>>>>> a "bug", but others may disagree. GnuCash seems to work pretty well
>>>>> without them. For my
>>>>> purposes, though, I know where they are defined, why they aren't
>>>>> exposed, and how to expose them.
>>>>> If someone wants a patch for that, I may be able to provide it.
>>>>>
>>>>> In the meantime, the heads up here is that there are some testing
>>>>> returning false positives.
>>>>>
>>>>> Let me know if you want any more information.
>>>>> _______________________________________________
>>>>> gnucash-devel mailing list
>>>>> gnucash-devel at gnucash.org
>>>>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>>
>>
>
> --
> Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
> Member, MIT Student Information Processing Board (SIPB)
> URL: http://web.mit.edu/warlord/ PP-ASEL-IA N1NWH
> warlord at MIT.EDU PGP key available
More information about the gnucash-devel
mailing list