Books, sessions [was: Re: UI independance]

[Derek Atkins]

linas at linas.org (Linas Vepstas) writes:

> Good point.
> 
> I guess I slept through a lot of the changes in the last few years.

yea, I suspect so.

> I'm wondering if there's any point to having a conversation about
> whether freqspec should have been merely a bunch of stuff tucked away 
> in a kvp tree somewhere.  I'm gonna say (without really thinking about
> it) 'it should have been'.   That would have tremendously simplified 
> the problem of needing to extend file format and sql backend, etc.

Didn't we have this kind of discussion about 1.5 years ago?  I'd
prefer not to rehash it.

No, it wouldn't have -- because the KVPs are practically useless in
the existing SQL schema.  Searching for items with a particular kvp
entry is just plain HARD.

> KVP's were invented not just to store some occasional attribute, but
> really to provide a mechanism for holding honest-to-goodness objects.

Yea, and most first-class data are not occassional attributes,
and indeed Freqspec (and some other data) is not occasional at all,
but is attached to every instance of an object.

> The only limitations is that kvp is *not* high-performance:  so 
> thier use is limited to things which are infrequently used,
> or are localized, that don't cast a wide net.   

Right, but MOST of the objects or tags that you suggest to put in KVPs
dont meet these limitations.  Indeed, I would assert that a number of
the attributes that we _DO_ put into the KVPs shouldn't be there,
either.

> I don't know, off-hand, whether some of your business objects might be
> better impelmented in KVP. I haven't thought one whit about this,
> I'm hoping you have. 

Yes, I have, and for searchability in SQL I opted for a flat object,
one that more closely resembles a SQL Schema ;)

> I guess we could debate the pro's and and con's of C-vs-kvp; I've already
> started writing an agonizing essay where I try to reconcile these
> tensions for myself.  I'm pissed off that I have to (and this is not 
> gnucash, but programming in general, e.g. gtt/gnotime, or weblications,
> etc.) I'm pissed off that I have to create C structs, *and* I have to
> create SQL schema, *and* I have to create the glue code between these
> and glade files, and I have to create glue between that and scheme,
> and so-on.   And (to get nebulous) that and evo/ldap addressbooks.
> and (more nebulous) that and ofx files.  And so on.   It seems that
> so much effort is wasted on piddling shit that should be automatic. 
> And worse, the piddling shit is hard-coded, inflexible, hard-to-extend,
> etc.  Rant, rant ... 

"Gee, Mr. Peabody, where are going today?"  Haven't I seen this rant before?

Linas, at some level all data processing requires stabilizing on a
data model.  The point of libraries is to allow the re-use of these
data objects..  Corba and its ilk allow even more sharing of data
objects.  But trying to completely separate out the data model and its
processing leads to low-performance, higher-complexity applications.
For example, if GnuCash _just_ used SQL, it would be much less complex
than it is, because we could depend on the SQL engine to perform some
of the functions for us.  Similarly, we could optimize our data access
to how SQL prefers to be accessed.

But to appease your sense of global domination, the new Query
interface is based on an object+parameter+datatype model.  For any
particular type of object you define the list of parameters
(attributes?) and their associated data-type (and accessor function).
For example, for a Transaction I've got something like:

        ...
    { TRANS_NUM, QUERYCORE_STRING, (QueryAccess)xaccTransGetNum },
    { TRANS_DESCRIPTION, QUERYCORE_STRING, (QueryAccess)xaccTransGetDescription },
    { TRANS_DATE_ENTERED, QUERYCORE_DATE, (QueryAccess)xaccTransRetDateEnteredTS },
        ...

Yes, theoretically this could all be defined in some meta-data tool,
but what you're now doing is..... DEFINING A SCHEMA!

        Transaction:= {
                ...
                String  Num;
                String  Description;
                DateTime        Date_Entered;
                ...
        };

The obvious problem with schemas is the extensibility problem.
Matthew Vanneck and I discussed this on #gnucash a while ago (you
know, you REALLY should join us there -- a lot of development
conversation happens on #gnucash).  There is an obvious way around
that -- when you want to extend your OWN object, you update the
schema.  Yea, this means you have backwards-compatibility issues, but
that's what version numbers are for.  However, if you want to update
ANOTHER object (because you want to add some attributes to another
object) then you define a schema PLUG-IN (sort of like a KVP, but with
schemas instead of unstructured data).

For example, and Invoice wants to add attributes to a Transaction, so
the Invoice module defines an Invoice-Transaction-Plugin schema, and
attaches that schema to the transaction (via the Transaction guid).
This solves the extensibility problem but also allows "fast SQL" or
other structured-data storage.

> --linas

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord at MIT.EDU                        PGP key available