GncBusiness v. GNCSession

[Linas Vepstas]

On Wed, Nov 21, 2001 at 08:17:18PM -0500, Derek Atkins was heard to remark:
> linas@linas.org (Linas Vepstas) writes:
> 
> An order entry is a line-item on an Order (either a purchase order
> or sales order).  For example, a customer orders:
> 	Object	Qty	Price
> 	Widget1 1	1.00
> 	Widget2 1	2.00
> 	Widget3	1	1.50
> 
> Each of these is an Entry.  Each of these could be invoiced
> separately.

Hmm.  It suddenly hits me that maybe we really need to start having 
the 'workflow' conversation.

Workflow is the idea that you can represent processes in a configurable
amnner (for example, with XML-based config files).  With workflow,
you describe a set of objects: e.g. Object A consists of a date, a
description, a few other fields.  You also describe a set of states, a
set of state transition rules, and a set of permissions, so that you can
control who has what permission to move what object from one state to
another.

For example, a 'bug tracking tool' is a particularly easy thing to 
to represent as workflow.  A bug has several states: "Open", "Being
Fixed", "Being Tested", "Closed".  Anyone has the permission to 
create a new bug report.  Only programmers are allowed to change the
state from "Open" to "Being Fixed".  Only testers are allowed to 
move it to "Being Tested".  etc. Everyone has permission to add a note
(i.e. has permission to modify the "notes" field of the object).

Te beauty of workflow is that if you don't like this bug-tracking
process, you can easily modify it to suit your needs.  You can add 
more states.  You can alter the permissions graph.  You can add new
fields (e.g. to track a prt number, if this is a hardware design bug 
tracking system).

The other beauty of workflow is that it is powerful enough to describe 
very broad systems.  Corporate purchase orders, when done by computer,
are always workflow:  anyone can requisition a chair, but only the boss
can approve, and then it moves on to the accountant's to-do list for
final approval, and then it is routed to purchasing.  Its a lot like 
opening a bug report, and assigning the bug to someone, except that
the fields are different. 

You are now moving to an area that is uncomfortably close to workflow.
I think that I aleady bitched to you that I didn't like how you 
designed the "employee" object.  Well, wouldn't it be nice if I could 
modify the employee object as I please, only by editing an XML file?  
And the purchase order: I am quite certain that I could poke holes in 
it, that I could find something to complan about ... Now if I could 
easily, flexibly, trivially change it, without having to recompile, 
to re-work giant chunks of the app, without (gosh) having to be a 
programmer?

Think about it ..... like really, really, seriously think about it ...

> > There are days when I feel I'm reinventing SQL, badly ...
> 
> It would be nice if we could have just used SQL directly ;)

There's a reason for that. Many, actually.
1) modularity.  If I modified an sql table, I'd have to change gui code
   all over the planet.
2) optimization.  To get good performance, fancy features, the 
   actual SQL may not be the textbook-intro-101-sql that you'd 
   write.  For example, to get the account balance in gnucash
   is a majorly complicated thing. Huge. Thousands of lines of 
   code.  I want to hid this behind a simpel query "get me bank
   balance".
3) portability.  This follows from 2).  Soon as you start doing 
   complex things, they are not portable.  And if you are not 1)
   modular, you are not portable.
4) network.  blah blah, client server. blah..
5) non-sql backends.  we'd need to find a some sql parser, and tailor it
   to work for the file backend.  And or the rpc backend, etc.

Although, it may be soon be time to explore 5).

> > The backend doesn't call back into the engine to discover what the query
> > is, it doesn't need to, it has full access to all engine structs.
> 
> My proposal is to change that.  I don't want to require the backend
> to know about all engine structs, because I want to dynamically add
> new structs to the engine.

Its not that the backend *must* know the structs.  Its that the backend
has the *authority* to do whatever it wants.  The backend is the supreme
ruler over all the domain. 

> > If every core-object has a (*query)() method  on it, then how do I 
> > do boolean joins, and nesting?
> > 
> > (account='xxx' AND invoice='yyy') OR (transaction.amount >333 AND 
> > account.balance < 222)
> 
> The query isn't EXECUTED in the core-object; the query is BUILT in the
> core-object (i.e. a char* is returned).  The core-object builds up
> it's part of the query, which is eventually executed in the "Central
> Backend".

How do I build that query, using C?

say I have the following:

struct GNCPhoneBook {
    void (*query) (...);
    ...
}

struct GNCAccountHeirarchy {
    void (*query) (...);
    ...
}

What C code would I write to construct the query
(account='xyz') AND (phone='555-1212')


> > > > > Who initializes this business-object backend, and how?
> > > > 
> > > > This is based on the URL. If the URL is postgres://asdf.com then
> > > > the postgres backend is loaded.
> > > 
> > > That's nice in the abstract.  Show me the code.  
> > 
> > This is exactly what gnc-session does.  Its almost the only
> > thing that gnc-session does.
> 
> Which implies we need a generic method to tell gnc-session all of the
> core-objects...  Which is my suggested "core-object registration" :)

sounds good.  Its some user-configurable table listing url-types,
object names, and .so library names ...

> Fair enough.  What I really meant was:
> 
> 	user calls 
> 		gnc_open_session("url:/path/to/data");
> 
> 	how does some the backend know how to load some random core-object?

Hmm.  I guess we need to have something at the top of the dataset that
declares: 'this dataset requires the use of the following core objects:'.
This would be the book.  The book, when represented as an XML file,
would have a section that says "this dataset has sections for commodities, 
for transactions, for phone numbers and for invoices".  The book,
when set up in RAM, would have the API we previously discussed.

> > > Ok.  I think that the GncBusinessObject structure is one place
> > > to start.  My list of methods from my previous mail is a second.
> > 
> > well, lets just crawl through this field by field.
> > 
> > 'version'. Why is this needed? How will it be used?  How will it be
> > documented so that some joker doesn't mis-use it?
> 
> The version is to make sure that the module and 'core' are
> compiled against the same API.

I'm not being facetious; I meant this as a rhetorical question.  
When you write the design document, you need to add a sentance that 
says not only this, but also says "when you change the struct, you must 
bump the version number.  However changes to the list of enums do not 
change the version number.  f teh engine has an older version then the
object, it can still access the object, but not the other way around."
And so on.  Maybe you need a minor version number as well?

The sql db has three version numbers: major, minor and rev.  different
major numbers are fundamentally incompatible, period. Different minor
numbers are upwards-compatible (allowing older to talk to newer).
Different revs are inconsequential for compatibility, and list only
bug fixes and minor enhancements and the like.

> > 'description'.  Why is this needed?  Is it supposed to be human-readable,
> > i18n gettext'ed?  Who does the gettext?  Or is this a 'machine readable
> > string', listing some kind of config info?
> 
> The description is used for generic searches, to present a printable
> string to the user when the generic widget says "Choose FOO".

I think I will avoid responding to this until you say "yeah verily,
here is the design document, and this is the the formal documentation".

Is there a length limit on this string?  Who does the translation of the
string to other languages (the gui above the object, or does the object
itself hold teh different translations)?

> > 'begin_edit() && commit_edit()'.  Debatable.  I think its a
> > straightjacket. I think this overpsecifies the problem.
> 
> What exactly is the problem?  I thought that begin/commit we there to
> "lock" and "commit" the data structure?  Or, at least, it was there to
> allow multiple 'changes' to be done with only one write-behind in the
> backend.

If we group accounts, transactions and splits into one business object,
then I need two sets of begin/commits.  If we have three objects,
then the commit for splits is superflouous.  

> > query() -- highly debatable, because of the join issue.
> 
> Yea.. I'm not sure about this, either..  But there needs to be some
> way to extend the backend (which includes queries) without requiring
> the backend to know about all data structures.

No, quite the opposite.  There needs to be a way for the user to be able
to specify arbitrarly compilcated queries, including joins across 
objects.    

The backend is not "required" to know anything.  It has the permission
to know everything.

> It we could assume SQL, then the query() method could be used to build
> a char* query string 

In that case, there would be one query method, on the book, and not a
one query method per object.

> and then the real backend would do that.

As mentioned above, things like "query the bank balance" present formal
difficulties.  The mapping between the actual SQL tables and the 
conceptual structure defined by te engine objects is not 1-1.

Never mind that MySQL doesn't support many of these features; they
are availble only in Postgres, and in commercial DB's.

That is not to say that we shouldn't define some pseudo-sql-language,
and use that.   I'd also be open to some pseudo-scheme-form language.
That is, assuming that the result is less cumbersome than what's in
src/engine/Query.c  KISS

> Oooh, I think I see the problem -- you can cause deadlock if you try

Its not just deadlocks, which you can avoid by always locking in the
same order. Its that the backend for one object needs to know about
and perform operations on other objects: i.e. cross boundaries.

But maybe this isn't an issue.

=============================================================
Last word: *please* give the workflow issue some real thought.  I think
that even a basic, minimal implementation, so that I could define 
the fields in my addressbook to be whatever I want them to be, would
be a very good step in the right direction.

--linas


-- 
pub  1024D/01045933 2001-02-01 Linas Vepstas (Labas!) <linas@linas.org>
PGP Key fingerprint = 8305 2521 6000 0B5E 8984  3F54 64A9 9A82 0104 5933