QOF, pilot-link and GnuCash was Re: LinuxFormat 60

Sat Oct 23 19:11:21 EDT 2004

On Saturday 23 October 2004 10:34 pm, you wrote:
> > (Moneydance with 56-bit DES encryption won praise for security but the
> > same mark overall.)
>
> 56-bit DES?  Gee, perhaps we should just rot13 our data to keep it from
> prying eyes?  ;)

:-))

>
> > GnuCash was deemed to best overall in the review

Let's not forget that GnuCash was praised too - it's nice to see hard work 
being acknowledged.

> If someone (else!  not you, Neil, you're busy with more important
> tasks!

:-)))

Actually . . . . No, I'm kidding. 

What I have done is outlined the benefits of QOF for pilot-link so that I can 
generate a genuine QofBook direct from my Palm (on the host machine via 
pilot-link, the Palm can't cope with QOF). Rather than building QOF into 
pilot-link as a dependency (worries about glib not being available for all 
pilot-link installations), the target is to build an optional QOF framework 
for pilot-link for those who want it, like me. Overall, QOF does fit with 
some of the development targets for pilot-link, especially in 'raising the 
game' from CSV type data to complex data like invoices, transactions etc.. 
The QofBackend support will also come in handy in creating an offline 
representation of the Palm data. Although I can backup the Palm databases 
already, the backup is only useful for restoring back to the Palm - it's not 
readable/usable on the host machine.

It follows on from my post on Thursday 3:33:58 pm about QOF (and GnuCash) and 
portable devices.

I'm building the QOF code around pilot-link because I need to have both sides 
of the 'map' before I can really work on how to translate a QofBook of 
pilot-book objects into a QofBook of GnuCash objects.

pilot-link is trying hard to get v0.12 out asap but the optional QOF support 
could be in as soon as 0.13.
:-))

That'll mean having the map ready at the same time.

Some of the mappings are clear, some require at least some regular expression 
pattern matching and some require quite involved calculations, including 
defaults and conditionals. The regular expressions themselves will be in 
code, I'd just need a label saying which one to call and when (and what to do 
with the result). e.g. checking that an email address isn't a telephone 
number.

I don't want to hard code this map because I want other applications to be 
able to create their own maps easily. This is where I might be overstretching 
things.

I've put a little info on my thinking here:
http://code.neil.williamsleesmill.me.uk/palm.html#CURRENT

It illustrates some of the problems with mapping one object to another. I'll 
need pattern matching with regular expressions, I'll need conditionals (e.g. 
if a parameter is empty, try another one) and some complex logic (the 
repeating Calendar event in Palm needs careful handling to identify the 
correct date for the invoice).

Take this simple example. PDA on the left, GnuCash on the right :
ADDR_PHONE_ONE,         <==>    ADDRESS_PHONE   (if regexp matches)
ADDR_PHONE_TWO,         <==>    ADDRESS_FAX     (if regexp matches)
ADDR_PHONE_THREE,       <==>    ADDRESS_EMAIL   (if regexp matches)
Sometimes the fax number will appear in 1 or 3, sometimes the email will be in 
1 or 2 or absent.

(These are all simple data types: integers, floats, strings and long int.)

There are one-to-one maps, many-to-one and one-to-many. There are even 
many-to-many maps.

From the pilot-link side:
"Repeating events may need care to identify the actual date of the event. 
Check the repeatType: repeatNone, repeatDaily, repeatWeekly, 
repeatMonthlyByDay, repeatMonthlyByDate or repeatYearly. Check the 
repeatForever value and the repeatEnd Timespec. Check the repeatFrequency, 
repeatDay and repeatWeekStart only if appropriate. If still within the 
repeatEnd, check if any dates are to be ignored and find out which ones. 
Finally, if the event survives all checks, map to gncInvoice::INVOICE_OPENED 
and gncEntry::ENTRY_DATE, gncEntry::ENTRY_DATE_ENTERED. (Other maps are free 
to map to just one or two of those.)"

My first map will be basic - only those three Palm databases, only via 
pilot-link and only to create an invoice in GnuCash. As long as it works, the 
map can be extended to other GnuCash objects, other Palm databases and other 
applications, like gnumeric and even other platforms like the Psion. Probably 
by using a separate file for each map, bi-directional if possible.

I'd like to ask for ideas on how to format such a thing.

Initially, I looked at XSL but wouldn't that need either an external program 
or a lot of new code? Plus, IMHO, I don't feel it's up to the conditional 
logic. (I may be wrong.)
I thought about using the same format as procmail - that's a clear procedural 
text format that is fairly easy to work but a long way from the kind of 
handling I'd need.
One suggestion (he says, taking his life in his hands - again) is Lua. OK, 
it's another dependency (oops) but it runs natively within C and provides 
ample logic handling for the conditional statements, pattern matching and 
general fiddling. The downside is that it isn't easy to write or configure 
for those who haven't used it before.

So nothing stands out so far, any other ideas?

Scheme? (Can only be as hard as learning Lua).
Another procedural text format - like httpd.conf in Apache?
Perl? (very quick to write but also very easy to edit / update / break)
(is Finance::Quote as far as Perl should go?)
(if QOF is available on platform whatever, wouldn't Perl usually be available 
too? n.b.not GnuCash, just QOF.)

(The Psion map would work as:
Suppose I work on a Psion that has a contacts, expenses and calendar applet 
running as discrete applications. The device also runs GnuCash. By using QOF, 
all the data from the databases can be exchanged with GnuCash in real time. 
However, that same data can now be synced with a Palm via pilot-link without 
any loss of data - even if some data types are not supported either on the 
Psion or the Palm. )

To achieve this, I don't want to have to load an intermediary program or 
interpreter just to read the map. I'd like QOF to be able to load the map 
natively, load the external QofBook in a QofSession (no need to save it so no 
backend defined) and process the map to create a temporary QofBook that 
replaces the external one and that now contains objects compatible with the 
output QofBook which then gets passed to qof_book_merge (as implemented by 
the receiving program).

Phew!

-- 

Neil Williams
=============
http://www.codehelp.co.uk/
http://www.dclug.org.uk/
http://www.isbn.org.uk/
http://sourceforge.net/projects/isbnsearch/

http://www.biglumber.com/x/web?qs=0x8801094A28BCB3E3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.gnucash.org/pipermail/gnucash-devel/attachments/20041024/d7dfa031/attachment.bin