Thoughts on architecture for the GDA backend

[Derek Atkins]

Phil Longstaff <plongstaff at rogers.com> writes:

> OK.  Let me try this again.  I'll send as a plain text e-mail instead of 
> HTML and see if that fixes the formatting problem.

YAY!  Thanks!  Looks MUCH better.

>>> Except for a number of bugs (and a lot of testing, and a load/save UI), 
>>> the GDA backend basically works for the objects outside of the business 
>>> objects.  I also want to add an audit trail for changes and version 
>>> support to detect when a db needs to be updated to a newer version.
>>
>> So we've reached feature-parity with the 1.6 Postgres backend?
>> Excellent!  Definitely good progress.  What's the time frame to get
>> the business objects done?
>>   
> Depends, of course, on how much time I can spend on the project. With 
> holidays coming up, I should be able to spend more time, and maybe get 
> the business objects done. I've never used them before, so there will be 
> a bit of a learning curve.

You can always use src/business/business-core/file as a basis for
the business GDA plugin.  And note that yes, it DOES need to be
a plug-in because it's perfectly reasonable to be running without
the business-core.

>>> - create a new GObject interface which has methods for 1) getting a 
>>> dictionary of persistent data for the object 2) getting/setting 
>>> persistent data.  The goal would be to have the engine object and its 
>>> class have a bigger role in defining the db table columns in order to 
>>> better encapsulate persistence information in the engine objects.  The 
>>> backend would then be required to transfer the persistence information 
>>> for an object to/from the db or file.
>>>
>>> Thoughts?
>>>     
>>
>> Do we really need this?  Is the current plug-in framework not
>> sufficient?  Or do you really believe that you can come up
>> with an interface that is backend agnostic?  I'd like to see
>> a design proposal before going along this route, because if it's
>> not backend agnostic then it's no better than what we currently
>> have.
>>   
> I'm thinking of something like a modified version of the memento design 
> pattern (http://en.wikipedia.org/wiki/Memento_pattern).  It may have 
> originally been designed to allow state modifications to be undone, but 
> it should also be usable here with the gnucash object being the object 
> whose state needs to be saved and the backend being the caretaker.  
> However, we wouldn't need an object to be the memento.  We *would* need 
> name/type information to allow the memento state info to be saved/restored.

Well, the BeginEdit()/CommitEdit() sequence is supposed to be very
similar to this.  But I'm not 100% sure this is what we want in
the data store per se.

> In one sense, if every object kept made all of its information available 
> via kvp, it would be easy to store since there is a flexible mechanism 
> there, and there is no maintenance problem keeping the backend 
> up-to-date with the schema.

This is true, but it also makes it hard to search through KVPs.
How would you search a KVP for Account.name = "X"?

> Take the account object as an example.
>
> The XML backend calls xaccAccountGetName(), xaccAccountGetCommodity(), 
> ... and then encodes each piece of information.  The GDA backend calls 
> the same functions, formats the SQL INSERT or UPDATE request and passes 
> it to the db to be executed and save the object.  If a new attribute is 
> added to the account engine object, each backend needs to be modified to 
> call a new function to get the attribute value, then each backend needs 
> to encode it properly in order to save it.

Yeah..  This is certainly an annoyance.   By the way, I hope that you
encode it all into a single SQL INSERT or UPDATE request for the whole
object instead of one per attribute?

> The Account engine object now has GObject properties defined for all of 
> its internal attributes.  This allows g_object_get() and g_object_set() 
> to be called to get/set attribute values respectively, but also allows 
> g_object_class_list_properties() to be called to get a list of the 
> attributes including type information.  A backend could call 
> g_object_class_list_properties() and then, for each attribute, call 
> g_object_get(), encode the value and save it.  It would need to to know 
> how to encode each type (for XML, strings are easy, numbers are easy, 
> commodities are a namespace/name pair, etc., while for GDA, strings are 
> easy, numbers are easy, commodities are a GUID, etc.).  One problem with 
> g_object_class_list_properties() is that the account has a number of 
> properties (such as ending balance) which don't need to be persistent.  

* nods *

> Therefore, an IGncEnginePersistenceInfo interface with a 
> gnc_engine_object_get_persistence_parameters() function to return the 
> same kind of info as g_object_class_list_properties() would be added (an 
> alternative is to define GncObject or GncEngineObject as a subclass of 
> QofObject and subclass all engine objects from it - the 
> gnc_engine_object_get_persistence_parameters() could then be defined as 
> a method for GncEngineObject). Actually, it would be defined for the 
> engine object class objects, not the engine objects themselves.

Or maybe extend QofObject with this information?  Just make sure we
dont need to duplicate a lot of code, or duplicate the list of
properties.  Too bad we can't apply a property property called
"persistent" ;)

> I think it would reduce the size of each backend.  In addition, since 
> the engine object classes would supply the information about what would 
> need to be saved, it would improve encapsulation and also maintenance in 
> the backends as new attributes are added.  Each backend would need to 
> know how to save each attribute type, and how to convert from the saved 
> value if the attribute type has changed (e.g. recent sx change from 
> FreqSpec to Recurrence).  The GDA backend would need to know how to map 
> attribute names to column names (I don't think '-' is allowed in a 
> column name, for example) and how to modify tables if the table column 
> structure doesn't match the types of the attributes to be saved 
> (attribute added or type changed).

I'm not sure it would, honestly.  The XML backend is pretty tight.
But maybe I'm wrong....  Maybe this would really help it.

> Phil

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord at MIT.EDU                        PGP key available