Missing Guile Bindings and False Positive Unit tests
Derek Atkins
warlord at MIT.EDU
Fri Feb 24 10:20:04 EST 2017
Hi,
Chad Albers <calbers at neomantic.com> writes:
> Hi Derek,
>
> Thanks for the history about Gnucash bindings. I might be asking for
> trouble. I just would rather interact with GnuCash rather that
> Python. Personal preference that is pretty arbitrary. I'm curious
> how far I can take it.
Oh, I understand. Personally I would have rather seen Ruby bindings
instead of Python, but I didn't write them so ....
> After further inspection, I now understand what you meant my "swig
> exports the functions for guile". I found the exactly location where
> I need to add new exports in the swig interface file. I haven't
> managed, though, to try it out yet.
Understood. Good Luck!
-derek
>
> On Tue, Feb 21, 2017 at 11:59 AM, Derek Atkins <warlord at mit.edu> wrote:
>> Hi Chad,
>>
>> Please remember to CC gnucash-user on all replies so that everyone can
>> remain "in the loop".
>>
>> Also, the gnc-modules should properly export swig-exported functions
>> without a re-export. If it stopped doing that, that would be a bug
>> IMHO.
>>
>> Note that these unit tests are probably 18 years old.
>>
>> <GnuCash-History>
>> A long long time ago (well, around GnuCash 1.6 or so), /usr/bin/gnucash
>> was a guile script. It loaded everything in as guile modules
>> (technically it loaded GncModules) and then jumped back and forth
>> between C and Scheme. Still, not everything was available from Scheme,
>> but I don't believe anything has been *removed* from the guile
>> bindings.
>>
>> Anyways, this structure was very hard to debug. You couldn't
>> run it under gdb; you needed to "attach" gdb to the running process,
>> which made it nearly impossible to debug issues during application
>> loading. Eventually the Schemer developers left the project to pursue
>> other endeavors, and those that remained eventually switched back to the
>> "main() in C" structure that we have today.
>>
>> Of course a bunch of the functionality is still in GncModules.
>> </GnuCash-History>
>>
>> Having said all that, you shouldn't need to export or re-export a
>> wrapped API. It should "just work". And the unit tests should be
>> properly failing when there are missing or broken APIs.
>>
>> -derek
>>
>> Chad Albers <calbers at neomantic.com> writes:
>>
>>> I'll file a bug report.
>>>
>>> But, for the record, swig isn't exporting. That's not what swig does.
>>> Function exports are done by using 2 APIs - the Guile define-modules
>>> API or Guiles' C API.
>>>
>>> For example,
>>>
>>> - Some guile functions are defined here: src/engine/swig-engine.c.
>>> This is where qof-session-new is defined.
>>> - swig-engine.c uses Guile's C API to create a module called sw_engine
>>> - swig-engine.c exports the functions - including qof-session-new -
>>> using scm_c_export(...)
>>> - Several Guile scm files use this module: (use-modules (sw_engine)).
>>> None of these scm files either (export...) or (re-export...) functions
>>> from the sw_engine module.
>>>
>>> Therefore, none of the Guile defined modules expose sw_engine functions.
>>>
>>> There may be, though, one exception, but even that isn't exposing
>>> anything from sw_engine. The following is speculation...
>>>
>>> - GnuCash seems to include, what I suspect, is another module system.
>>> I think it's in the module (gnucash gnc-module). Calls such as these
>>> - (gnc:module-system-init) - may be a part that system.
>>> - Included in that system may be code from: src/engine/gncmod-engine.c
>>> - This C code has a C function called 'libgncmod_engine_gnc_module_init(...)
>>> - This includes the sw_engine module. It calls
>>> scm_c_eval_string("(use-modules (sw_engine))");
>>> - This scm_c_eval_string does not include any (exports...) or
>>> (re-exports...) in the gncmode-engine.c file.
>>>
>>> Therefore, the sw_engine functions are not available in this other
>>> module system.
>>>
>>> To make the sw_engine functions available (which is what I would
>>> like), this may have been how it was once accomplished. The units
>>> test that pass when they shouldn't include this line:
>>>
>>> (gnc:module-load "gnucash/engine" 0)
>>>
>>> Maybe, at one time gnc:module-load exported functions from
>>> "gnucash/engine" modules. Or something like that. I'm still
>>> investigating the what/why/how of the other module system.
>>>
>>> In the meantime, one solution is just to re-export sw_engine functions
>>> from (gnucash engine) or engine.scm. That's what I'm doing for my
>>> experiments, and it works. Perhaps I will submit a patch for that,
>>> but I'm thinking that that's a hack. I'm opening up the discussion
>>> here for a better solution, if exposing more of the Guile API is a)
>>> fixing a bug and/or b) adding a new feature. Since much of GnuCash
>>> was written in Guile, I thought I would use that instead of Python.
>>> As is, though, there doesn't seem to be a way to even source a GnuCash
>>> using its Guile bindings. It can be done using Python.
>>>
>>> Chad
>>>
>>> --
>>>
>>> On Sun, Feb 19, 2017 at 5:34 PM, Derek Atkins <derek at ihtfp.com> wrote:
>>>> That srfi didn't exist when the tests were written, which was probably
>>>> around 18 years ago. The fact the tests don't fail just means nobody tested
>>>> the tests for false positives. Please file a bug report?
>>>>
>>>> There isn't a need to be-export. Swig exports do that automatically.
>>>>
>>>> -derek
>>>>
>>>> Sent from my mobile device. Please excuse any typos.
>>>>
>>>> ----- Reply message -----
>>>> From: "Chad Albers" <calbers at neomantic.com>
>>>> To: "Derek Atkins" <derek at ihtfp.com>
>>>> Subject: Missing Guile Bindings and False Positive Unit tests
>>>> Date: Sun, Feb 19, 2017 5:23 PM
>>>>
>>>> Definitely, though, the unit tests it is checking is
>>>> 'qof-session-new'. It passes, but it hasn't been exported in the
>>>> Guile code.
>>>>
>>>> At one time, I think it was exported from this file:
>>>> 'src/mine/gnucash/src/engine/engine.scm' This file includes the
>>>> (sw_engine) module (defined in Guile C API), but it does not "(export
>>>> qof-session-new)" or "(re-export qof-session-new). As such, the unit
>>>> test shouldn't pass. The function doesn't exist outside of the C code.
>>>>
>>>> I haven't figure out why the unit test is passing when it should fail.
>>>> GnuCash seems to have created its own unit-testing framework for its
>>>> guile functions. It doesn't use guile's unit-testing module (srfi
>>>> srfi-64). I suspect that there are many other guile binding unit
>>>> tests that are false positive. I haven't done a full audit. I just
>>>> wanted to give someone a heads up about this problem.
>>>>
>>>> I would submit a patch to fix the unit testing framework, but I
>>>> haven't actually worked out how it works. In the meantime, I'm
>>>> currently exporting some of the qof-session functions as an
>>>> experiment. My goal is to try to accomplish in Guile what can be
>>>> accomplished with the Python bindings.
>>>>
>>>> --
>>>>
>>>> On Sun, Feb 19, 2017 at 5:07 PM, Derek Atkins <derek at ihtfp.com> wrote:
>>>>> That's a different issue. I don't know why that happens.
>>>>>
>>>>> -derek
>>>>>
>>>>> Sent from my mobile device. Please excuse any typos.
>>>>>
>>>>> ----- Reply message -----
>>>>> From: "Chad Albers" <calbers at neomantic.com>
>>>>> To: "Derek Atkins" <derek at ihtfp.com>
>>>>> Subject: Missing Guile Bindings and False Positive Unit tests
>>>>> Date: Sun, Feb 19, 2017 5:01 PM
>>>>>
>>>>> True, Derek. However, GnuCash does have unit tests that exercise
>>>>> Guile bindings that have not been properly exported. And those test
>>>>> falsely pass, when they should not.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Chad Albers
>>>>>
>>>>>
>>>>> On Sun, Feb 19, 2017 at 4:57 PM, Derek Atkins <derek at ihtfp.com> wrote:
>>>>>> Hi,
>>>>>> Except for a few functions in app-utils, most of QofSession is not
>>>>>> wrapped
>>>>>> in the scheme bindings like they are in the python bindings.
>>>>>>
>>>>>> -derek
>>>>>>
>>>>>> Sent from my mobile device. Please excuse any typos.
>>>>>>
>>>>>> ----- Reply message -----
>>>>>> From: "Chad Albers" <calbers at neomantic.com>
>>>>>> To: <gnucash-devel at gnucash.org>
>>>>>> Subject: Missing Guile Bindings and False Positive Unit tests
>>>>>> Date: Sun, Feb 19, 2017 4:46 PM
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I found some problems with some unit tests for the Guile bindings.
>>>>>> Here's the background.
>>>>>>
>>>>>> Rather than use the Python bindings, I wanted to use guile to
>>>>>> add/manipulate
>>>>>> my GnuCash data. From the Python bindings, I deduced that the entry
>>>>>> point into creating
>>>>>> transactions is via a construct called a "session". Accordingly, I went
>>>>>> to
>>>>>> find
>>>>>> the Guile equivalent: "qof-session-new". I wrote a Guile script which
>>>>>> successfully
>>>>>> imports GNUCash's guile bindings. However, I was never able to
>>>>>> successfully
>>>>>> use
>>>>>> "qof-session-*" functions. They are considered "unbound".
>>>>>>
>>>>>> After further investigation, I found that GNUCash has unit tests that
>>>>>> specifically exercise the
>>>>>> 'qof-session-new' function.
>>>>>> Here...'src/engine/test/test-create-account.scm'. I ran
>>>>>> these units tests with 'make check' and they passed. I concluded that
>>>>>> since the unit test
>>>>>> passes, it must somehow have the binding for 'qof-session-new'.
>>>>>>
>>>>>> 'make check', though, creates a log for each unit test. I peeked in
>>>>>> 'test-create-account.scm.log'.
>>>>>> I found the following line:
>>>>>>
>>>>>> ;;; src/engine/test/./test-create-
>>>>>> account.scm:30:18: warning: possibly
>>>>>> unbound variable `qof-session-new'
>>>>>>
>>>>>> This is the same error that I have encountered in my own Guile script.
>>>>>> But...I also see this line
>>>>>> at the end of the test log.
>>>>>>
>>>>>> PASS test-create-account (exit status: 0)
>>>>>>
>>>>>> So the unit test cannot find 'qof-session-new' and yet the unit test
>>>>>> framework thinks its
>>>>>> passing. I wanted to let someone know that GNUCash has unit tests
>>>>>> (perhaps only for guile)
>>>>>> that are returning false positives. That's bug one. Would you like me
>>>>>> to file a bug for this?
>>>>>>
>>>>>> It turns out that GnuCash is not properly exposing the Guile bindings.
>>>>>> I consider this
>>>>>> a "bug", but others may disagree. GnuCash seems to work pretty well
>>>>>> without them. For my
>>>>>> purposes, though, I know where they are defined, why they aren't
>>>>>> exposed, and how to expose them.
>>>>>> If someone wants a patch for that, I may be able to provide it.
>>>>>>
>>>>>> In the meantime, the heads up here is that there are some testing
>>>>>> returning false positives.
>>>>>>
>>>>>> Let me know if you want any more information.
>>>>>> _______________________________________________
>>>>>> gnucash-devel mailing list
>>>>>> gnucash-devel at gnucash.org
>>>>>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>>>
>>>
>>
>> --
>> Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
>> Member, MIT Student Information Processing Board (SIPB)
>> URL: http://web.mit.edu/warlord/ PP-ASEL-IA N1NWH
>> warlord at MIT.EDU PGP key available
>
>
--
Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
Member, MIT Student Information Processing Board (SIPB)
URL: http://web.mit.edu/warlord/ PP-ASEL-IA N1NWH
warlord at MIT.EDU PGP key available
More information about the gnucash-devel
mailing list