GncBusiness v. GNCSession

Derek Atkins warlord@MIT.EDU
21 Nov 2001 17:09:22 -0500


linas@linas.org (Linas Vepstas) writes:

[lots of stuff snipped]

> The gnucash module system does not seem to be a mechanism for managing
> a collection of pointers.   To manage a collection of pointers, you would 
> have an interface something like this:
> 
> void gnc_insert_global (const char * name, void * data);
> void * gnc_fetch_global (const char * name);
> 
> which you would use like so:
> 
> typedef struct { ...} AccountGroup;
> typedef struct { ...} GNCPriceDB;
> typedef struct { ...} GNCPhoneBook;
> 
>  AccountGroup *topgrp = xaccAccountGroupNew();
>  GNCPhoneBook *fones = gncPhoneookNew();
> 
>   gnc_insert_global ("accounts", (void *) topgrp);
>   gnc_insert_global ("phones", (void *) fones);
>   
> topgrp = gnc_fetch_global ("accounts");
> xaccAccountGroupForEach (topgroup, xxxx);
> 
> fones = gnc_fetch_global ("phones");
> char * fonenumber = gncPhoneQuery (fones, "Jimmy Joe Bob");
> 
> where each of these lines of could could occur in distant parts of the
> code.  Thus, you don't need to hard-code a global anywhere, you 
> can dynamically look it up, if you know its name.  Kind of like
> guids/entities, but with well-known human-readable names, instead 
> of assigned numbers.

Funny, this is almost EXACTLY what I did, and what I wanted to
move into the gncSession... :)   The only difference is that I
wanted to leave a two-level abstraction.  I wanted the code in
gnc-session.c to maintain a list of potential "globals", and
then each GNCSession* would have a list of actual "globals".

Take a look at src/business/business-core/gncBusiness.c, in particular
the gncBusinessRegister(), gncBusinessLookupGUID(), and the
soon-to-be-committed gncBusinessAddEntity() and gncBusinessRemoveEntity()

I'd kind of like to move these out of gncBusiness and into gnc-session
(where they probably belong).

[more snipped]

> > This is indeed the path I'd like to take.  Right now I only provide
> > a single search query: hidden or not-hidden.  This is the "get_list"
> > method.  The actual method is:
> > 	GList * (*get_list) (Session, show_all);
> 
> OK, now we are getting somewhere.  Except that the above isn't valid C
> code, so I am a bit confused still.
>
> 1) session is the *wrong* place to hang data.  GNCBook is the right
>    place.

We've still got a problem.  There is the Entity Table and there are
Data Stores.  What's the difference?  The Entity Table is stored in
the GNCSession and maintains a list of all GUIDs, object types, and a
pointer the object.  However, the GNCBook contains the AccountGroup,
etc.

I think it's this dichotomy that is still confusing me (us)?

> 2) 'show all' is dangerous: it doesn't scale.  If I have a million 
>    customer names, I certainly would almost never actually do a 
>    'show all'.

Yea, I know.  It was an example.

> 3) What does the GList data point at?  The following code doesn't
>    work for me:
> 
>    GList * mylist = bizniz->get_list (...);
>    for (node=mylist; node; node=node->next)
>    {
>       GenericSearchWidgetDisplay (node->data);   // huh ????
>    }

Actually, it does.  You're just missing a few pieces.  The code
actually looks something like this:

	GList * mylist = gncBusinessGetList (bus_session, "gncCustomer", ...);
	for (node=mylist; node; node=node->next)
	{
		GenericSearchWidgetDisplay ("gncCustomer", node->data);
	}

> what's data? its a void *. What do I cast it to?  How can I display 
> it in your "generic search widget" ?? How does your generic widget
> discover that I returned a list of phone numbers ???

The generic widget doesn't care.  It just returns the void* to the
caller which happens to know what it was looking for and can do the
conversion itself.  Basically, I hand you a deck of cards and ask you
to choose one; I also hand you a hook to convert the card to somethink
you can display; you hand me back the card.  Through all this you
never have to actually understand what a card is.

> > Ideally this should be more like:
> > 	GList * (*query) (Session, Query);
> 
> 
> Maybe more like 
>     GList * (*query) (GNCBook *, GNCQuery *);

Sure.  I was trying to be non-specific by using "Session" and "Query".
I'll certainly accept this. :)

[snip]

> > Rather, I want to change the code in src/engine once and then be able
> > to extend it through plug-ins.  
> 
> Yes,  I understand what you want to do, I just don't understand how you
> think you'll do it.

I'm not sure how I'll do it either.  If I did I would go ahead and
do it and tell you all. :)  Or at least I'd be presenting my ideas.

> > That way, as I implement new data
> > types (right now I'm working on Invoices),
> 
> This is very off-topic, but, for me, an 'invoice' is a kind of report
> that you would have in the gnc report infrastructure.   A kind of
> variation on the 'show transactions' report.  I don't understand how an
> invoice is a 'core object'.  But maybe we should save this conversation
> for later ?? 

An Invoice has dual meanings.  There is the invoice which is something
that is printed/viewed/mailed/etc.  Then there is the GncInvoice which
links a bunch of order entries to the posted transaction.

> > Invoices can be searched for:
> 
> I think you mean 'accounts payable' not 'invoices'.
> 
> > 	type
> > 	customer/vendor
> > 	due/duedate
> > 	paid	

Not exactly.  An invoice is one transaction in "accounts payable" or
"accounts receivable".  It's a step in the process; something to be
recorded.

> Right. Very good.  We agree. These would be a good set of Query topics.  
> GNCQuery already implenments a decent subset of them.  Why not add the
> rest?  Or rather, why not add the rest by hard-coding them in?  
> Do we need to invent yet another extension mechanism so that we can 
> add new query types ?  Inventing extension mechanisms is hard.
> Understanding someone else's extension mechanism is even harder.

Extending GNCQuery by hand probably is the best short-term "thing to
do"....

> > Part of the reason why was for the RPC Server -- each rpc client is in
> > a different session, so you wanted the entity table to exist within
> > the session, which is tied to the rpc client thread.
> 
> I don't understand this statement either.  If I have three clients
> and they are manipulating the same book inside the server, then all
> three of them should see the same identical set of guids.  If I have 
> one book inside the server, and three distinct sets of guid's refering
> to the contents of the book ... ouch makes my head hurt.  Surely
> this cannot be what you mean.  There must be soe other reason, but
> I am growing dubious ....

If you have three clients connected to the same RPC server and they
are manipulating DIFFERENT books, you don't want GUID merging inside
the server's engine.  Worse, if they ARE working on the same book, the
way the GNCEntityTable works, it wont work with a File Backend!
You'll wind up stomping on each other.

The RPC Server is trying to maintain each client in a separate
sandbox.  The GNCSession is that sandbox.  The fact that two clients
are maniulating the same data is irrelevant, and should always be
irrelevant.  They should remain sandboxed, with well-defined
interfaces to send events between clients when "shared" data is
modified.

Remember:  GLOBALS == BAD!

[snip]

> Hmm. Well, ahhh, uhhh... I'm not sure I want to ... begrudgingly ... 
> 
> OK, here's one: how do I do a 'table join' query: i.e. 
> select all phone numbers where transaction amount > y dollars.
> 
> If I am given only one beginEdit()/commitEdit() pair, then 
> I am force to separate out Accounts from Transactions.  And there
> are very real-world table-join queries involved:
> 
> select all accounts where split-memo='some string'.
> 

Hrm.  I'm not sure what the beginEdit/commitEdit pair has to do with
anything here.  Consider this is a vector and EACH OBJECT has its own
vector.  In other words the Account* would supply a vector,
Transaction* would supply a vector, GncOrder* would supply a vector,
etc.

As for the join query, one would think that each object type would
have it's own portion of the GNCQuery and could convert it into a
partial data lookup.  The way I was envisioning it for the SQL backend
was that the general backend would build a SQL query by calling the
query() method for each object type.

By the process, the Account* query-method would give:
	all accounts

the Split* query-method would give:
	where split-memo='some string'

And then you could run the query.

> > > a business object would call a vector.  When a business-object backend
> > > is initialized, it fills in the vector table.
> > 
> > Who initializes this business-object backend, and how?
> 
> This is based on the URL. If the URL is postgres://asdf.com then
> the postgres backend is loaded.

That's nice in the abstract.  Show me the code.  Somewhere this
URL has to be passed to the business objects.  Where and how
is that done?

> > > > Who calls "save"?
> > > 
> > > gnc-session does.
> > 
> > Right, and how does gnc-session:
> > 	a) know that business objects exist, and
> > 	b) know what business-object save function(s) to call?
> 
> Well, with proposal A), its hard coded. With proposal B) its some hack.
> With proposal C) its a vector in the struct.

Ok.

> > Right.  I propose that each business-object-type register this
> > "save_function" 
> 
> Please stop calling it 'save'.  Save is a meaningless concept for 
> the sql backend.  Save is peculirat to the file backend.

Indeed, but until we stop supporting the file backend it's an
important concept.. That's why I keep bringing it up.  I acknowledge
that for SQL, RPC, HTTP, etc it has no meaning.  That's fine, and a
good thing.  However unless file support goes away it's the
"odd-problem" that needs to be solved.

> > > So maybe I answered my own question?  A generic core-object is a
> ...
> 
> > I think a generic core-object is slightly more than this (see my
> > method-list above).  But yes, this is what I'm getting at..
> 
> OK, like I said, violent agreement.  But now, the next level:
> how much more?  You might talk me into init, destroy, finalize,
> and a few more.   But maybe not all of them.  This is where the 
> rubber hits the road.  What really is the valid set of generic 
> methods?  I think we now need to start getting specific about 
> details.

Ok.  I think that the GncBusinessObject structure is one place
to start.  My list of methods from my previous mail is a second.
I'll think on this a bit more and get back to you..

> There are days when I wonder: do we really need that many lines of code
> to do this?  Isn't there some way of doing more with less?  Isn't the
> newbie contributor to GnuCash going to be turned off by this?

If we used C++ we could reduce it ;)  (READ: JOKE ALERT.. I AM NOT SERIOUS)

> > > Next, the business backend will presumably have ...
> > Yea..  I know.  This is another reason I'd like to vectorize the
> 
> I'm tempted to defer discussion of the generic backend until we have
> a better idea of what the generic object is.  In particular, I really
> want to know whether the tangle of account/transaction/split maps
> to one object or two.  I sense vague arguments in either direction.

Fair enough.

> > > Gack. My view of the world is not split like this.  I view
> > > accounts/transacations/splits to all be tangled up in one and the same
> > > core object, the 'account heirarchy'.   You can't really untangle them.
> > 
> > But you can, and you have to.  There is a "list of accounts" and there
> > is a "list of transactions" and there is a "list of splits".  These
> > are three object types that have pointers back and forth between them.
> > However, they are still three different objects.  They each have their
> > own XML structure.  They each have their own SQL tables.
> 
> You seem to be saying that the mapping between core-object and sql-table
> is 1-1. This has several problems:

Not necessarily.  The current Core Objects I see are:
	Account
	Transaction
	Split
	SchedXAction
	Commodity (maybe?)
	Prices	(pricedb)

I may be missing something here.  I'm also not 100% convinced that a
Commodity is a core object.  For one thing, it doesn't have a GUID, so
perhaps it is a "special" core object?

Aren't there more SQL tables than this?

> -- designing a query interface that allows queries that join tables.

How programatic is SQL?  Isn't it basically of the form:

	SELECT <what you want to see> FROM <where you want to find it>
		WHERE <what you want to search on>

My SQL isn't very good, but this seems to lend itself fairly well
to building the query in pieces based on the Query, no?

> -- although accounts and transactions have begin/end commit pairs,
>    splits do not.  One cannot commit a split all by itself, because
>    that would break the contraint that the sum of splits totals zero.
>    So having a commit() method in the core object would be
>    confusing/misleading for splits. And, on the flip side, the
>    transaction commit() needs to lock both the transaction *and*
>    the split table; locking one is not enough.

So the split begin/commit methods are NULL.  Not all objects have to
support all methods.

> I am somewhat concerned that there will be other subtle couplings of this
> sort between core objects.  For example, accounts payable groups
> together splits with different dates. (viz. "this check drawn on
> this bank today pays for that bill received 30 days ago.")  After
> this coupling has been made, you can't just change one of the member
> transactions without throwing things out of kilter.  You'd need
> to add a balancing transaction.   So you're not just updating one
> sql table, you are updating several, and you have to update them
> together, atomically, i.e. use locks.  

Hrm, I wasn't thinking of tying it in quite like this.  This is partly
where the GncInvoice() comes in.  When a Receipt is posted to A/R, the
receipt transaction is tagged with the GncInvoice that is being paid
(and the Invoice is marked paid).

There are _two_ transactions in A/R.  The first is when the Invoice
is posted; the second is when the invoice is paid.  They are linked
by the GncInvoice.

> So, if the mapping between sql tables and core objects is 1-1,
> then it doesn't make sense (to me) to have commit() and query() be a
> part of teh core object.   But if the mapping is not 1-1, then 
> having only one set of commit() might also not make sense.

Hrm.. True.. Any code that is going to modify a GncFoo already has to
know about the GncFoo, thereby being able to call gncFooBeginEdit()
and gncFooCommitEdit() directly.  Then the GncFoo object call call its
own backend functions for committing itself.  Is that what you mean?

I still think you want to be able to lock/commit the backend on a
per-object basis.  That way if I'm going to be editing, say, a
Customer record, you don't over-write my changes.

> --linas

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord@MIT.EDU                        PGP key available