[GNC-dev] Problems with python 3 and the gnucash python bindings.

[David Osguthorpe]

Hi All,


In upgrading to gnucash 3.2 from 2.6.18 and updating my python scripts I
have found an issue with the gnucash bindings and python 3.


I saw this with query runs that failed to produce any results when they
should have, and used to under 2.6.18.


So far it appears the issue is with the wrapping of the
qof_query_search_for call.

By inspection of the generated c code it appears by default swig wraps
python 3 unicode strings by converting to a new python 3 byte string object
(SWIG_AsCharPtrAndSize) and passes its char buffer pointer as the char *
pointer to the wrapped function call.

This object and pointer is freed at the end of the wrapper function.

Unfortunately it seems qof_query_search_for just stores the char * pointer
for the query which means by the time the query is run the pointer is
invalid (and indeed junk was printed using gnucash debug logging as the
Query Object Type: which is what lead me to this problem).


This is going to be a general problem for any gnucash function that simply
stores a char * pointer rather than copying the string.


According to the Python 3 documentation the PyUnicode_AsUTF8 function
creates a char * buffer which is cached in the unicode object so exists for
the lifetime of the unicode object.

I dont understand why this isnt the default SWIG wrapping for char pointers
but it does seem to solve the problem here.

(I havent been able to find any real discussion of this via Google. The
only hint Ive seen is that its a memory issue as PyUnicode_AsUTF8 needs to
store 2 versions of the string.)


This requires setting up swig typemaps.

So far I have used a very specific typemap for QofIdType and QofIdTypeConst
which solves the problem and python query runs produce results (and my
scripts work as under 2.6.18).


The question is where and how to implement this.

Currently Ive put the typemaps in gnucash_core.i - but they probably
should be in base-typemaps.i.

Then the question is how general to make this - it could be a typemap for
all char * pointers otherwise each explicit character type (eg gchar *)
would need its own typemap.

I think its clear that any const char * should map to PyUnicode_AsUTF8.


Also note that the GSList typemap in base-typemaps.i needs to be updated to
use unicode strings for python 3 - otherwise it requires python 3 byte
strings.


David


PS. Another solution would be to force byte string only arguments for
python 3 using a SWIG define SWIG_PYTHON_STRICT_BYTE_CHAR.

This would require a major re-write of the gnucash_core.py to perform the
unicode<->byte transformations.