sqlite file format, anyone?

Ben & Michelle Carlyle fuzzybsc at optusnet.com.au
Sat Jun 21 22:14:53 CDT 2003


G'day,

I've just been discussing a few issues relating to gnucash on my local home user group lists[1]. The story so far is that I'm very impressed with the capabilities of gnucash, and although I've run into a large number of small issues they for the most part make me want to contribute rather than abandon gnucash. I've been coming from a windows/quicken background and have been considering gnucash as a replacement for my personal finance tracking currently done by that product. My experimental setup as a P133 laptop with 800x600 LCD display. The laptop runs a debian distrubution and it was the debian standard package I used to install.

I've seen plenty of problems working at this resolution with gnucash 1.8.4. In fact it's very difficult to use many features and impossible to use others. 1024x768 seems to be ok. I have several suggestions about improving the import mechanisms for QIF files. I think the loan druid and the mechanisms to do the scheduled transactions for loans could do with a little refinement. I have a full list of my early impressions which I may present at a later time, but I'd prefer to do so with patches in-hand. Anyway. That's not what I'm here to talk about.

Members of my user group have motivated me to ask about contributing to what I saw as the greatest current deficiency of gnucash going forwards as a personal finances application, and that is the load/save philosophy when it comes to files. Unlike quicken which saves transactions as they are entered, gnucash's file backend requires the entire file to be saved manually by a user. In my experimentation I encountered three seperated occasions under which gnucash locked up[2] and I lost valuble time associated with entering transactions. I've been running a side-by-side configuration with quicken and gnucash both keeping identical record sets during my experimentation. If I'd only been running gnucash I would have lost the transactions completely and would likely have had a hard time reconstructing where I was up to.

The lockups I can live with. They're to be expected in relatively new code. They occur with terrible regularity under Quicken. I have no problem with that. Such problems can be fixed on a case-by-case basis or design re-engineered as appropriate. The time wasted re-entering my data, though, made me lose much of my goodwill for the gnucash project. After a little prompting I downloaded the gnucash source and for the moment at least my goodwill is greatly increased as a result of seeing the backend module associated with PostgreSQL.

I don't want to run PostgresSQL. I'm not a database admin. I'm a senior software engineer for my company who has about four years experience developing rheems of code for a top-end civil engineering software project for railways in Hong Kong and other systems around the world. I do SCADA systems.

As part of my work I ran across a program called sqlite[3]. sqlite is a portable[4], fast, transaction-safe database engine which runs as a C library, and keeps each database in a single filesystem file. No special maitanence is required for the file. The library can be statically or dynamically linked. sqlite is widely used. There's a debian package, and many other packages available. sqlite is not just free software, but public domain software so can be used pretty much anywhere you like. I've used sqlite in largeish commercial applications and have been very pleased with the performance and simplicity, even when files became very large (64-bit file handles are supported).

This brings me to your PostgreSQL backend. I have a deep desire to mate the gnucash and sqlite technologies using a backend based-on or in a similar vein to the PostgreSQL backend. I know this is a bold statement, but my gut feeling at the moment is saying that it could become a better "standard" file format than the xml version (at least as long as the XML format doesn't support transactions). I don't want to lose my data, nor do I want others to lose their data.

The question is, if I do this work and it is good is there a chance of it being accepted into the base as an optional file format which doesn't require explicit saving? I suppose another obvious question is, "Have I correctly interpreted the use of the PostgreSQL as being a no-save data format".

Finally, I'd be very interested in all the advice that the gurus associated with this project could offer before I embark on such a major undertaking :)

If I do find the support to proceed I'll be working in my spare time outside my office hours and hours of unpaid overtime :) I don't want to do something that won't have the support of the general community.

Right now I'm quite fired up about the possibility of having a replacement for my Quicken accounting system that meets all of my needs. If I can defeat this hurdle I'll be very interested in contributing further work as gnucash will undoubtably become my new home accounting system. Pehaps I'll even be able to work together with some local business-people to help add an Australian bent to the system, both in terms of Australian personal tax law and business tax issues such as GST (an issue also found in Canada).

[1] www.humbug.org.au
[2] Locked up, as in became unrepsonsive and had to be killed. No cpu was being used, and it usually occured as the result of pressing a button or tab on a gnucash display
[3] www.sqlite.org
[4] At least windows, all the traditional unicies, linux etc, and various embedded environments
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/attachments/20030621/beb15cd6/attachment.htm


More information about the gnucash-devel mailing list