QSF update

[Neil Williams]

I'm making good progress on the QSF formats that will allow simple XML data 
imports into any QOF application. I've found a way of encoding the 
calculations required to convert QOF objects between applications within the 
XML itself, instead of using external libraries like oaf or lua. The QSF map 
includes XML tags that outline the steps required to transform the objects, 
including the use of defaults, conditionals, enumerators and variables. 
Multiple input objects can provide data for one or more output objects, so 
that data from expenses can be merged with defaults and data from datebook 
and contacts to create multiple gncEntry's in a gncInvoice. Defining the map 
is the most involving part of the process.

There's masses of documentation available:
http://code.neil.williamsleesmill.me.uk/qsf.html

Note that the doxygen output for QSF on my site above is now out of date as it 
refers to sixtp. Once I have a fully working QSF map and example parser, I'll 
return to the gnucash/backend/file tree and implement the libxml2-only 
methods.

I have pilot-link examples, GnuCash examples, a map example and an example 
parser.

QSF - QOF Serialization Format now consists of two component formats:
qof-qsf for the QSF object data and
qsf-map to map sets of QSF objects between QOF applications.

QSF object files will contain user data and be created by users in userspace.
QSF maps require more background knowledge of both applications and are 
expected to remain in developer space - installed alongside the application 
in install_dir/share/qsf/ - with certain defaults requiring user-edited 
versions in ~/.gnucash etc from time to time. Each map ties together two 
different QOF applications, but different kinds of data interchange could use 
different maps, e.g. to exchange accounts and their transactions rather than 
invoices. Maps can be uni- or bi-directional - the map installed with the 
receiving application would take precedence. There's no reason why the QSF 
XML data cannot be sent as a data stream instead of to the filesystem, so 
online communication remains an objective.

QSF object files are simple XML - changing to only one default namespace 
proved worthwhile as did dropping sixtp and using libxml2 directly (as sixtp 
is not currently part of QOF, this has other advantages for smaller QOF 
applications like pilot-link).

QSF is generic - it can include QOF data from any QOF compliant application. 
With a suitable map, it can also accept XML files that are created outside 
QOF. This could prove a worthwhile format for all kinds of data imports.

This is a snippet from a GnuCash QSF file that should appear familiar:

<?xml version="1.0"?>
<qof-qsf xmlns="urn:qof-qsf-container">
<book count="1">
 <book-guid>c0dd5ca1b11338f3ceae57f6e0106d75</book-guid>
 <object type="account" count="8">
  <string type="name">Liabilities</string>
  <string type="type">LIABILITY</string>
  <string type="description">Liabilities</string>
  <guid type="guid">f68c39388a52f57319203b3fcc04f8e2</guid>
  <gint64 type="commodity-scu">100</gint64>
 </object>

Note the continuing absence of currency data, only the SCU is available to QOF 
at the moment.

A QSF map snippet looks like this:
(This one is for pilot-link to GnuCash invoice conversions).

<definition qof_version="3">
   <define e_type="qof-expenses">Pilot-link QOF expenses</define>
    <define e_type="qof_datebook">Pilot-link QOF datebook</define>
    <define e_type="gncInvoice">Invoice</define>
    <define e_type="Trans">Transaction</define>
    <define e_type="gncEntry">Order/Invoice/Bill Entry</define>
    <default name="mileage_rate" type="numeric" value="28/100"/>
  </definition>
  <object type="gncEntry">
  <calculate type="string" value="action">
   <if type="qof-expenses">
     <set>Material</set>
   </if>
   <else type="qof-datebook">
     <set>Hours</set>
   </else>
   </calculate>
   <calculate type="numeric" value="iprice">
     <if type="string" value="expense_type">
       <equals type="string" value="etMileage">
         <set>mileage_rate</set>
       </equals>
     </if>
   </calculate>

The calculations are expressed in XML but all the comparison, assignment and 
logical processing takes place in the QSF handler written in C. By making the 
handler generic, the calculations can be expressed outside the C source code. 
Simple mathematical operators can be added if required - <add> <multiply> 
etc.

All QSF data will be imported via qof_book_merge and writing an export filter 
that can handle selected transactions, invoices, or all data between set 
dates should be straightforward. Any GnuCash data that can be partitioned 
into a secondary QofBook using QOF will be exportable as QSF. Simply select 
the QSF backend for the new book, copy the data and write the file.

Maps will only be needed to convert QSF data between applications - QSF data 
written by GnuCash will be available to be imported back into GnuCash using 
qof_book_merge without maps.

When importing/exporting for other applications, any QSF objects that are not 
defined in the selected map will be ignored, so too much data is not a 
problem.

There is also a possibility of an XSLT stylesheet that can convert existing 
GnuCash XML (v1 or 2) into QSF. I haven't looked at this yet, it may be 
harder than anticipated.

The QSF schema files are now available as well. One for QSF object files and 
one for the QSF map files. The QSF code will expect every QSF file to 
validate against the schema before importing the data (part of the reason for 
using libxml2 directly).

I have a working example parser for QSF files that uses only libxml2 and may 
prove to be a useful tool, although the QOF version will benefit from better 
hashtable handling in glib.

I expect to have a GnuCash menu option available in the next month or two.

The pilot-qsf-GnuCashInvoice QSF map is not yet complete - none of the 
datebook or address data is currently handled. 

The QSF map Schema and format may change slightly during this process - 
development is ongoing.

-- 

Neil Williams
=============
http://www.codehelp.co.uk/
http://www.dclug.org.uk/
http://www.isbn.org.uk/
http://sourceforge.net/projects/isbnsearch/

http://www.biglumber.com/x/web?qs=0x8801094A28BCB3E3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.gnucash.org/pipermail/gnucash-devel/attachments/20041129/16fc7da8/attachment.bin