Missing Guile Bindings and False Positive Unit tests
Ted Creedon
tcreedon at easystreet.net
Tue Feb 21 12:36:26 EST 2017
FYI there is a swig problem in OpenSuse leap 4.2.
Its not the latest version.
Tedc
________________________________________
From: gnucash-devel <gnucash-devel-bounces+tcreedon=easystreet.net at gnucash.org> on behalf of Derek Atkins <warlord at MIT.EDU>
Sent: Tuesday, February 21, 2017 8:59:27 AM
To: Chad Albers
Cc: gnucash-devel at gnucash.org
Subject: Re: Missing Guile Bindings and False Positive Unit tests
Hi Chad,
Please remember to CC gnucash-user on all replies so that everyone can
remain "in the loop".
Also, the gnc-modules should properly export swig-exported functions
without a re-export. If it stopped doing that, that would be a bug
IMHO.
Note that these unit tests are probably 18 years old.
<GnuCash-History>
A long long time ago (well, around GnuCash 1.6 or so), /usr/bin/gnucash
was a guile script. It loaded everything in as guile modules
(technically it loaded GncModules) and then jumped back and forth
between C and Scheme. Still, not everything was available from Scheme,
but I don't believe anything has been *removed* from the guile
bindings.
Anyways, this structure was very hard to debug. You couldn't
run it under gdb; you needed to "attach" gdb to the running process,
which made it nearly impossible to debug issues during application
loading. Eventually the Schemer developers left the project to pursue
other endeavors, and those that remained eventually switched back to the
"main() in C" structure that we have today.
Of course a bunch of the functionality is still in GncModules.
</GnuCash-History>
Having said all that, you shouldn't need to export or re-export a
wrapped API. It should "just work". And the unit tests should be
properly failing when there are missing or broken APIs.
-derek
Chad Albers <calbers at neomantic.com> writes:
> I'll file a bug report.
>
> But, for the record, swig isn't exporting. That's not what swig does.
> Function exports are done by using 2 APIs - the Guile define-modules
> API or Guiles' C API.
>
> For example,
>
> - Some guile functions are defined here: src/engine/swig-engine.c.
> This is where qof-session-new is defined.
> - swig-engine.c uses Guile's C API to create a module called sw_engine
> - swig-engine.c exports the functions - including qof-session-new -
> using scm_c_export(...)
> - Several Guile scm files use this module: (use-modules (sw_engine)).
> None of these scm files either (export...) or (re-export...) functions
> from the sw_engine module.
>
> Therefore, none of the Guile defined modules expose sw_engine functions.
>
> There may be, though, one exception, but even that isn't exposing
> anything from sw_engine. The following is speculation...
>
> - GnuCash seems to include, what I suspect, is another module system.
> I think it's in the module (gnucash gnc-module). Calls such as these
> - (gnc:module-system-init) - may be a part that system.
> - Included in that system may be code from: src/engine/gncmod-engine.c
> - This C code has a C function called 'libgncmod_engine_gnc_module_init(...)
> - This includes the sw_engine module. It calls
> scm_c_eval_string("(use-modules (sw_engine))");
> - This scm_c_eval_string does not include any (exports...) or
> (re-exports...) in the gncmode-engine.c file.
>
> Therefore, the sw_engine functions are not available in this other
> module system.
>
> To make the sw_engine functions available (which is what I would
> like), this may have been how it was once accomplished. The units
> test that pass when they shouldn't include this line:
>
> (gnc:module-load "gnucash/engine" 0)
>
> Maybe, at one time gnc:module-load exported functions from
> "gnucash/engine" modules. Or something like that. I'm still
> investigating the what/why/how of the other module system.
>
> In the meantime, one solution is just to re-export sw_engine functions
> from (gnucash engine) or engine.scm. That's what I'm doing for my
> experiments, and it works. Perhaps I will submit a patch for that,
> but I'm thinking that that's a hack. I'm opening up the discussion
> here for a better solution, if exposing more of the Guile API is a)
> fixing a bug and/or b) adding a new feature. Since much of GnuCash
> was written in Guile, I thought I would use that instead of Python.
> As is, though, there doesn't seem to be a way to even source a GnuCash
> using its Guile bindings. It can be done using Python.
>
> Chad
>
> --
>
> On Sun, Feb 19, 2017 at 5:34 PM, Derek Atkins <derek at ihtfp.com> wrote:
>> That srfi didn't exist when the tests were written, which was probably
>> around 18 years ago. The fact the tests don't fail just means nobody tested
>> the tests for false positives. Please file a bug report?
>>
>> There isn't a need to be-export. Swig exports do that automatically.
>>
>> -derek
>>
>> Sent from my mobile device. Please excuse any typos.
>>
>> ----- Reply message -----
>> From: "Chad Albers" <calbers at neomantic.com>
>> To: "Derek Atkins" <derek at ihtfp.com>
>> Subject: Missing Guile Bindings and False Positive Unit tests
>> Date: Sun, Feb 19, 2017 5:23 PM
>>
>> Definitely, though, the unit tests it is checking is
>> 'qof-session-new'. It passes, but it hasn't been exported in the
>> Guile code.
>>
>> At one time, I think it was exported from this file:
>> 'src/mine/gnucash/src/engine/engine.scm' This file includes the
>> (sw_engine) module (defined in Guile C API), but it does not "(export
>> qof-session-new)" or "(re-export qof-session-new). As such, the unit
>> test shouldn't pass. The function doesn't exist outside of the C code.
>>
>> I haven't figure out why the unit test is passing when it should fail.
>> GnuCash seems to have created its own unit-testing framework for its
>> guile functions. It doesn't use guile's unit-testing module (srfi
>> srfi-64). I suspect that there are many other guile binding unit
>> tests that are false positive. I haven't done a full audit. I just
>> wanted to give someone a heads up about this problem.
>>
>> I would submit a patch to fix the unit testing framework, but I
>> haven't actually worked out how it works. In the meantime, I'm
>> currently exporting some of the qof-session functions as an
>> experiment. My goal is to try to accomplish in Guile what can be
>> accomplished with the Python bindings.
>>
>> --
>>
>> On Sun, Feb 19, 2017 at 5:07 PM, Derek Atkins <derek at ihtfp.com> wrote:
>>> That's a different issue. I don't know why that happens.
>>>
>>> -derek
>>>
>>> Sent from my mobile device. Please excuse any typos.
>>>
>>> ----- Reply message -----
>>> From: "Chad Albers" <calbers at neomantic.com>
>>> To: "Derek Atkins" <derek at ihtfp.com>
>>> Subject: Missing Guile Bindings and False Positive Unit tests
>>> Date: Sun, Feb 19, 2017 5:01 PM
>>>
>>> True, Derek. However, GnuCash does have unit tests that exercise
>>> Guile bindings that have not been properly exported. And those test
>>> falsely pass, when they should not.
>>>
>>>
>>>
>>>
>>> --
>>> Chad Albers
>>>
>>>
>>> On Sun, Feb 19, 2017 at 4:57 PM, Derek Atkins <derek at ihtfp.com> wrote:
>>>> Hi,
>>>> Except for a few functions in app-utils, most of QofSession is not
>>>> wrapped
>>>> in the scheme bindings like they are in the python bindings.
>>>>
>>>> -derek
>>>>
>>>> Sent from my mobile device. Please excuse any typos.
>>>>
>>>> ----- Reply message -----
>>>> From: "Chad Albers" <calbers at neomantic.com>
>>>> To: <gnucash-devel at gnucash.org>
>>>> Subject: Missing Guile Bindings and False Positive Unit tests
>>>> Date: Sun, Feb 19, 2017 4:46 PM
>>>>
>>>> Hi,
>>>>
>>>> I found some problems with some unit tests for the Guile bindings.
>>>> Here's the background.
>>>>
>>>> Rather than use the Python bindings, I wanted to use guile to
>>>> add/manipulate
>>>> my GnuCash data. From the Python bindings, I deduced that the entry
>>>> point into creating
>>>> transactions is via a construct called a "session". Accordingly, I went
>>>> to
>>>> find
>>>> the Guile equivalent: "qof-session-new". I wrote a Guile script which
>>>> successfully
>>>> imports GNUCash's guile bindings. However, I was never able to
>>>> successfully
>>>> use
>>>> "qof-session-*" functions. They are considered "unbound".
>>>>
>>>> After further investigation, I found that GNUCash has unit tests that
>>>> specifically exercise the
>>>> 'qof-session-new' function.
>>>> Here...'src/engine/test/test-create-account.scm'. I ran
>>>> these units tests with 'make check' and they passed. I concluded that
>>>> since the unit test
>>>> passes, it must somehow have the binding for 'qof-session-new'.
>>>>
>>>> 'make check', though, creates a log for each unit test. I peeked in
>>>> 'test-create-account.scm.log'.
>>>> I found the following line:
>>>>
>>>> ;;; src/engine/test/./test-create-
>>>> account.scm:30:18: warning: possibly
>>>> unbound variable `qof-session-new'
>>>>
>>>> This is the same error that I have encountered in my own Guile script.
>>>> But...I also see this line
>>>> at the end of the test log.
>>>>
>>>> PASS test-create-account (exit status: 0)
>>>>
>>>> So the unit test cannot find 'qof-session-new' and yet the unit test
>>>> framework thinks its
>>>> passing. I wanted to let someone know that GNUCash has unit tests
>>>> (perhaps only for guile)
>>>> that are returning false positives. That's bug one. Would you like me
>>>> to file a bug for this?
>>>>
>>>> It turns out that GnuCash is not properly exposing the Guile bindings.
>>>> I consider this
>>>> a "bug", but others may disagree. GnuCash seems to work pretty well
>>>> without them. For my
>>>> purposes, though, I know where they are defined, why they aren't
>>>> exposed, and how to expose them.
>>>> If someone wants a patch for that, I may be able to provide it.
>>>>
>>>> In the meantime, the heads up here is that there are some testing
>>>> returning false positives.
>>>>
>>>> Let me know if you want any more information.
>>>> _______________________________________________
>>>> gnucash-devel mailing list
>>>> gnucash-devel at gnucash.org
>>>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
>
--
Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
Member, MIT Student Information Processing Board (SIPB)
URL: http://web.mit.edu/warlord/ PP-ASEL-IA N1NWH
warlord at MIT.EDU PGP key available
_______________________________________________
gnucash-devel mailing list
gnucash-devel at gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
More information about the gnucash-devel
mailing list