GncBusiness v. GNCSession

Linas Vepstas linas@linas.org
Wed, 21 Nov 2001 18:47:15 -0600


On Wed, Nov 21, 2001 at 05:09:22PM -0500, Derek Atkins was heard to remark:
> linas@linas.org (Linas Vepstas) writes:
> 
> >  To manage a collection of pointers, you would 
> > have an interface something like this:
...
> > Thus, you don't need to hard-code a global anywhere, you 
> > can dynamically look it up, if you know its name.  Kind of like
> > guids/entities, but with well-known human-readable names, instead 
> > of assigned numbers.
> 
> Funny, this is almost EXACTLY what I did, and what I wanted to
> move into the gncSession... :) 

Am awaiting Dave's reply to other email but I'm now convinced that 
the entity table ended up in gncSession by accident, that it really
was supposed to be in gnc-book.

>  The only difference is that I
> wanted to leave a two-level abstraction.  I wanted the code in
> gnc-book.c to maintain a list of potential "globals", and
> then each GNCBook* would have a list of actual "globals".

Pardon me for s/Session/Book/g just trying it on for size.

I don't understand the distinction between 'potential' and 'actual'.

I don't understand what you mean by 'each GNCsessbook'.  Typically,
there is only one book, it serves as the anchor for 'global' data.
I imagine it will now now hold the business-object-hash-table, so
I don't know what you mean by 'each'.

> I'd kind of like to move these out of gncBusiness and into gnc-session
> (where they probably belong).

gnc-book.   

I'm OK with the move.  Can we change the name to GNCCore or something
like that?   (GNCCommon, GNCObject ...)  It should be worth converting 
the topgroup, the pricelist, the commoditiy list and the sched txn's 
to all be 'core' objects.

off-topic .. anyone looked at the gnome-2.0 'gobj' library?  Any
opinions?  Should we start using it?  I suspect is provides at least some
of the mechanism that we are fishing for, and if we're lucky, Ariel Rios
will write the scheme bindings for it ....

> 	GList * mylist = gncBusinessGetList (bus_session, "gncCustomer", ...);
> 	for (node=mylist; node; node=node->next)
> 	{
> 		GenericSearchWidgetDisplay ("gncCustomer", node->data);
> 	}
> 
> The generic widget doesn't care.  It just returns the void* to the
> caller which happens to know what it was looking for and can do the
> conversion itself.  

I don't understand two things:
1) why a generic gui wrapper is needed. What services/value-add
   does it provide?
2) The mechanism of returning a hook back to the 'actual' gui code.

> An Invoice has dual meanings.  There is the invoice which is something
> that is printed/viewed/mailed/etc.  Then there is the GncInvoice which
> links a bunch of order entries to the posted transaction.

I guess I'll need to think about what an 'order entry' is.

off-topic again, but what happens when you fill half an order, and
back-order the rest?  Is the posted transactin modified? (hope not).
Are multiple invoiced linked ?  ??

> > Right. Very good.  We agree. These would be a good set of Query topics.  
> > GNCQuery already implenments a decent subset of them.  Why not add the
> > rest?  Or rather, why not add the rest by hard-coding them in?  
> > Do we need to invent yet another extension mechanism so that we can 
> > add new query types ?  Inventing extension mechanisms is hard.
> > Understanding someone else's extension mechanism is even harder.
> 
> Extending GNCQuery by hand probably is the best short-term "thing to
> do"....

There are days when I feel I'm reinventing SQL, badly ...

I wonder if we should track down some sql parser, and rip out the
guts, and just use the hooks to hook into our own code.

> If you have three clients connected to the same RPC server and they
> are manipulating DIFFERENT books, 

... and the book held onto the entities, there would not be any problem.

> if they ARE working on the same book, the

... and the book held onto the entities, there would still be no
problem. 

>  work with a File Backend!

using a file backend with a server would be pretty stupid for 
other reasons, but I digress ...

> You'll wind up stomping on each other.
> 
> The RPC Server is trying to maintain each client in a separate
> sandbox.  The GNCSession is that sandbox.  The fact that two clients
> are maniulating the same data is irrelevant, and should always be
> irrelevant.  They should remain sandboxed, with well-defined
> interfaces to send events between clients when "shared" data is
> modified.

Well, precisiely. And if the entity table was in the book, then all
three clients would see the same entity table, and there would be no
risk of clobbering anything.  Whenever any one of them said 'save',
then the one and only true copy of the data would be saved.  Which
is why the entity table belongs with the data, and not separate from 
it.

> As for the join query, one would think that each object type would
> have it's own portion of the GNCQuery and could convert it into a
> partial data lookup.  The way I was envisioning it for the SQL backend
> was that the general backend would build a SQL query by calling the
> query() method for each object type.

I'm not sue what you are saying.  The GUI builds a query, using 
the engine query api.  It then 'executes' the query, which causes
the the query to be passed to the engine.  The backend walks the 
tree of boolean joins, and converts it to a single SQL query.
It sends that to the server, and finishes by poking data back into the
engine.

The backend doesn't call back into the engine to discover what the query
is, it doesn't need to, it has full access to all engine structs.

> By the process, the Account* query-method would give:
> 	all accounts
> 
> the Split* query-method would give:
> 	where split-memo='some string'

The point is that you want to do the join inside the sql server, and
not in the backend.  You want to say 

SELECT * FROM gncSplit WHERE gncSplit.memo='asdf' AND
gncAccount.type='bank'.  

Because there might be only one split that matches this join, whereas
there might be millions of splits with ths memo (beloning to type
'creditcard').  SQL servers are highly optimized to do this, we 
couldn't begin to compete by doing it ourselves.

If every core-object has a (*query)() method  on it, then how do I 
do boolean joins, and nesting?

(account='xxx' AND invoice='yyy') OR (transaction.amount >333 AND 
account.balance < 222)

> > > Who initializes this business-object backend, and how?
> > 
> > This is based on the URL. If the URL is postgres://asdf.com then
> > the postgres backend is loaded.
> 
> That's nice in the abstract.  Show me the code.  

This is exactly what gnc-session does.  Its almost the only
thing that gnc-session does.

> Somewhere this
> URL has to be passed to the business objects.  Where and how
> is that done?

Quite the reverse.  In the current scheme of thngs,  gnc-session 
parses the URL and determines which backend to load.  It then 
tells the backend 'giddyap', and the backend creates all the 
objects, and anchors them into the book.

Although the association of url-type to backend is currently
hard-coded, we could get it out of a config file if needed.

What I do *not* want to do is to load each backend, and ask it 
if it knows how to handle this particular URL.  That would blow
out the RAM.

> > Please stop calling it 'save'.  Save is a meaningless concept for 
> > the sql backend.  Save is peculirat to the file backend.
> 
> Indeed, but until we stop supporting the file backend it's an
> important concept.. That's why I keep bringing it up.  I acknowledge
> that for SQL, RPC, HTTP, etc it has no meaning.  That's fine, and a
> good thing.  However unless file support goes away it's the
> "odd-problem" that needs to be solved.

OK, point taken.  But we need to call it 'sync' not save. 
The sql backend may be abhorent of 'save', but it does need 
a 'sync', if, for example, the engine was loaded from a file,
and now needs to be merged into the (presumably larger) sql db.
'save' implies 'clobber', which is wrong.   Maybe I'm being picky.

> Ok.  I think that the GncBusinessObject structure is one place
> to start.  My list of methods from my previous mail is a second.

well, lets just crawl through this field by field.

'version'. Why is this needed? How will it be used?  How will it be
documented so that some joker doesn't mis-use it?

'description'.  Why is this needed?  Is it supposed to be human-readable,
i18n gettext'ed?  Who does the gettext?  Or is this a 'machine readable
string', listing some kind of config info?

'destroy()'  OK, sounds reasonable... is it OK to call other methods
on this class after destroy ahs been called? If not, then when do we
call finalize?

finalize() ... what does this do? is it OK to call this before calling
destroy() ?  

'save()' OK, as long as it has sync not clobber semantics.  I'm
wishy-washy about whether there should be independent 'sync-from'
and 'syncto' routines.

'the_session_is_going_away()' -- I think we need one of these,
although none of the objects/backends currently have this.  
I think that this is supposed to mean 'close down the back end'.

'begin_edit() && commit_edit()'.  Debatable.  I think its a
straightjacket. I think this overpsecifies the problem.

query() -- highly debatable, because of the join issue.





> I'll think on this a bit more and get back to you..
> 
> > There are days when I wonder: do we really need that many lines of code
> > to do this?  Isn't there some way of doing more with less?  Isn't the
> > newbie contributor to GnuCash going to be turned off by this?
> 
> If we used C++ we could reduce it ;)  (READ: JOKE ALERT.. I AM NOT SERIOUS)
> 
> > > > Next, the business backend will presumably have ...
> > > Yea..  I know.  This is another reason I'd like to vectorize the
> > 
> > I'm tempted to defer discussion of the generic backend until we have
> > a better idea of what the generic object is.  In particular, I really
> > want to know whether the tangle of account/transaction/split maps
> > to one object or two.  I sense vague arguments in either direction.
> 
> Fair enough.
> 
> > > > Gack. My view of the world is not split like this.  I view
> > > > accounts/transacations/splits to all be tangled up in one and the same
> > > > core object, the 'account heirarchy'.   You can't really untangle them.
> > > 
> > > But you can, and you have to.  There is a "list of accounts" and there
> > > is a "list of transactions" and there is a "list of splits".  These
> > > are three object types that have pointers back and forth between them.
> > > However, they are still three different objects.  They each have their
> > > own XML structure.  They each have their own SQL tables.
> > 
> > You seem to be saying that the mapping between core-object and sql-table
> > is 1-1. This has several problems:
> 
> Not necessarily.  The current Core Objects I see are:
> 	Account
> 	Transaction
> 	Split
> 	SchedXAction
> 	Commodity (maybe?)
> 	Prices	(pricedb)

I think I can go for:

 	Account && AccountGroup
 	Transaction && Split
 	SchedXAction
 	Commodity (yes!)
 	Prices	

> I may be missing something here.  I'm also not 100% convinced that a
> Commodity is a core object.  

it should be.

> For one thing, it doesn't have a GUID, so
> perhaps it is a "special" core object?

That's because commodities have thier own way of getting a unique
special name.   No matter, if guid is a show-stopper, we should issue
them guid's.

> Aren't there more SQL tables than this?
Yes, but they are subsidiary tables, for KVP, versioning, auditing.

> How programatic is SQL?  Isn't it basically of the form:
> 
> 	SELECT <what you want to see> FROM <where you want to find it>
> 		WHERE <what you want to search on>

Yes, except as dave points out, you mostly don't need the FROM,
and WHERE can be not only a boolean tree of nested parens, AND, OR,
NOT, but can also be the results of nested sub-selects, expressions
containing +, -, *, /, < > = !, full-blown regexp's for strings, 
system-defined functions like sum(), avg(), count(), or any user 
defined pattern-matching 'functions'.  (written in sql, or C, or 
perl, or python, or...)

I use some of the fancier forms to compute the running balances.

> Hrm, I wasn't thinking of tying it in quite like this.  This is partly
> where the GncInvoice() comes in.  When a Receipt is posted to A/R, the
> receipt transaction is tagged with the GncInvoice that is being paid
> (and the Invoice is marked paid).

which means that you have to lock the transaction and the invoice tables
which means that the invoice backend needs to know about the transaction
table, which pretty much means that the invoice backend will be 
integrated with the transaction backend.

> I still think you want to be able to lock/commit the backend on a
> per-object basis.  That way if I'm going to be editing, say, a
> Customer record, you don't over-write my changes.

No what I mean is that sometimes I have to lock the split transactino
and account tables at the same time, because I have to modify all of
them atomically.  I cant just do one, and then do the next one.
They have to happen simultaneously.

(in sql, locks define what happens 'simulatneously' from the point of
view of other observers.)


--linas

-- 
pub  1024D/01045933 2001-02-01 Linas Vepstas (Labas!) <linas@linas.org>
PGP Key fingerprint = 8305 2521 6000 0B5E 8984  3F54 64A9 9A82 0104 5933