Python API, book options - GObject, QofInstance etc.

[John Ralls]

On 22 Jun 2014, at 04:38, Christoph Holtermann <c.holtermann at gmx.de> wrote:

> Am 21.06.2014 00:10, schrieb John Ralls:
>> On Jun 20, 2014, at 2:23 PM, Christoph Holtermann <c.holtermann at gmx.de> wrote:
>> 
>>> Hello,
>>> 
>>> I had just the same question for a while. I have been successful to access
>>> that data. I had to make some modifications which I put in a branch on
>>> github: https://github.com/c-holtermann/gnucash/tree/python-kvp
>>> See thread "Working example of kvp acess in Python".
>>> 
>>> But the best way to access that data has yet to be found before I ask it
>>> to be pulled into master.
>> KVP is private to the engine and off limits for python bindings. We’re certainly not going to revert the private-kvp changes, which is what your fork partly does.
>> 
>> In master, all of the stuff stored in KVP is accessible via GObject properties. QofInstance seems not to be exported, but you can import GObject via gobject-introspection and use GObject.GObject.get/set_property(); if you write just be sure to do it in an edit/commit block and to mark the instance dirty.
> Thanks for pointing this way. I'll have a look at it. I guess using Python I'll need something like PyGObject to do that. ( I found GObject via
>>> from gi.repository import GObject
> )
> 
> but the c/c++ object hierarchy sems not yet to be accessible in Python. So to be able to use GObject.GObject.get/set_property()
> There would need to be a relation of that kind between the book/Transaction/whatever object and GObject.
> 
> This leads to the question of type casting macros (is that the right name ?):
> In the sources there is the makro QOF_INSTANCE which is widely used. I did not see a way to access it with python so I wrote
> QofInstance *qof_book_get_qof_instance (QofBook *book) {
>     return (QOF_INSTANCE (book));
> }
> in src/optional/python-bindings/gnucash_core.i
> 
> which gives me access to QofInstance in Python.
> 
> 1) Is it legitimate to access (the books) QofInstance from Python ?
> 2) Is there a better way for type casting ?
> 
> QofInstance on the other hand seems to be a descendant of GObject. Objects are being treated as GObject, Transaction (etc.), QofInstance.
> Is this varying access also useful to be accessed via python ?
> 
> (How) Can I access a book, a transaction, etc. as GObject in Python ?
> 
> I'm happy just to go the main road. No need to establish ways that diverge from the intended API. I just need to get the intention.
> 
> Just to get things right about KVP:
> 1) It's not intended to change KVPs from Python. So there should be no representation of class KVPFrame. ?
> 
> Sorry for that bunch of questions.

Its a pretty flat hierarchy: GObject->QofInstance-><engine class>. It's all C right now, and you're right that SWIG doesn't understand, and so doesn't export, that hierarchy. GObject access in python is afforded by GObject-Introspection and has been since GLib-2.28; before that it was hand-coded in the PyGtk project. The current naming conventions in the GnuCash engine won't work with GI; I did modify the engine classes to be GObject-introspectable a few years ago, but never committed the changes because Guile-GI was at the time poorly maintained and we have a lot of scheme interface code that's dependent on the SWIG way. 

If we weren't moving away from GLib I'd suggest that as a route. The whole mess is going to get re-written over the next few years with a different architecture so anything you do now is going in the bin anyway; best to find the least-effort approach to expose what you need without exposing too much.

As part of the g_object_get/set infrastructure each class has a get/set_property(object, guint, gvalue, GParamSpec*) function. They're static functions registered into the GObject vtable for the class. They could be exposed to the SWIG interface simply by making them not static and declaring them in the header. The other sensible route would be to move the guts of the property set/get block to public getter/setter functions, called from get/set_property. Those public getter/setter functions would be exported via SWIG. 

In a class hierarchy some things are implemented in the parent class; that's part of the point of having a class hierarchy. GObject makes it painful to write virtual function code so it's typical to call the parent class function directly, casting the struct to the parent (that's not quite what's going on in the macro, but it's easier to think of it that way). Unfortunately that makes interfacing to GObject class hierarchies in Python painful without GObject-Introspection's help. Your workaround, to get the parent class pointer in C is one way; the other would be to wrap the parent-class functions that you need in child-class member functions (as mark_dirty() is wrapped in several engine classes, for example). The latter is a bit safer as long as you're careful to wrap only functions that don't interact with the child class. For example, calling qof_instance_commit_edit_part2() outside of a class's own commit function would lead to data corruption.

KVP:
KVP is an implementation detail that was poorly thought out from an object-oriented design perspective. Having an object's slots publicly accessible allows storing state on the object for which the object itself has no code. There's no constraint that the KVP even has anything to do with the class, no guarantee that the state will be persisted, and the only way to discover it is by exhaustive examination of the whole code base, making it very difficult to maintain. It's the polar opposite of the OO "data hiding" principle: Instead of the class hiding its data from the outside, the outside is able to hide the data from the class! That's not exclusive to Python: It's a problem in C, it was widely abused, and it caused real data loss to users of the SQL backends. 

The rationale behind KVP storage is that it allows one to extend a class without breaking database/datafile backward compatibility: Older versions of GnuCash will load and save the data stored in slots even if they don't have code to implement whatever feature uses the KVP. That's true if and only if the feature touches only that KVP; if it uses the KVP in conjunction with other object state, or worse uses the KVP to change the way other object state is calculated, then the older versions that don't implement the feature will do the wrong thing and corrupt the data.

As an interim measure I've made KVP private to the object owning it; that at least allows one to understand everything about a class's state from the class implementation. As part of the C++ conversion I plan to move it down another layer, so that all of the class's state is represented in class member variables directly; some of those variables may be persisted via KVP for backwards compatibility -- we have a design requirement that any two adjacent minor versions say 2.4 and 2.6 -- be able to read each other's data files.

Regards,
John Ralls