sqlite file format, anyone?

[Derek Atkins]

linas at linas.org (Linas Vepstas) writes:

> Doing real, true multi-user adds to the complexity.  Someone may have
> changed the transaction the same time you did.  So before you even start
> writing to the DB, you have to read from it first, and compare to what you
> had.  If what you had is old, you have to rollback the GUI, and tell
> the poor user to try again.  Oh, and get the new stuff from the DB. 
> (Which may be new, modified and/or deleted splits). 

This is the purpose of beginEdit() -- it should lock the necessary
database items.  Once the lock is held it can check consistency with
the current version.

> I suppose we can argue if this complexity is needed. I did it this way
> because I wanted every transaction in the DB to *always* balance at
> all times.  If you think that its OK to allow breif imbalances, then
> you can avoid most of this complexity.  On the other hand, if you allow
> imbalances, and the client crashes during a write, then you have to
> do a psychological equivalent of fsck on the tables: scrub them for 
> imbalances.  This is potentially ugly for large DB's.  
> 
> (I like tht analogy: its like a journalluing FS: added complexity in
> exchange for eliminating fsck ..)

This is what transactions are for...  The problem (IMHO) is that
which objects hold what information is slightly wrong.  For example,
you begin/commit an ACCOUNT in order to place a Split into the account,
but it's the Split that needs to know that information.

> > > Besides, I want an embedded solution to replace XML.
> 
> Yes, lets not loose sight of that.

Trust me, I'm not losing sight!  :)

Luckily I don't have to think about all the multi-user excess for
the "XML replacement". 

> > I'm hoping my version of the PG backend is pluggable enough that you can
> > just plug-n-play the db access code.  The bulk of the work is in the
> > Query decoding and Engine loading.  Simple things like submitting
> > queries and parsing results are rather easily coded.
> >
> > I don't know how any of this works into the QOF discussion, though...
> 
> I'm hoping that your version of the backend will fit hand-n-glove with the
> qof code, i.e. not make any explicit references to any of the accounting
> things (i.e. no mention of splits, accounts, prices, etc), and just work 
> at the gncObject level.  
> 
> At some layer, you'll have to deal with the multi-user balancing transaction
> thing I describe above.   Speaking of balancing, you'll also need to 
> implement the balance checkpoints, or copy that code over. (Uhh, can
> we talk about 'stored proceedures'?)  But I'm hoping the code to 
> convert from object to table and back is 'generic qof code' that can 
> work for any object.

My hope is that "balance checkpoints" can coincide with Periods.
Instead of using stored procedures to compute balances, we merge
balance checkpoints in with Period support...  And balances are
computed at Period boundaries.

It does mean you need to pull in all transactions within a period to
get proper balances, but honestly I dont consider that a problem.  Do
you?

This approach also means we can toss away the stored procedure mess,
because balance computation is pushed in when we do period rollover.

> --linas

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord at MIT.EDU                        PGP key available