Importing and exporting revisited

[Benoit Grégoire]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sunday 01 August 2004 10:35 pm, blfs wrote:
> So the question becomes why is it so hard to import
> and export a check register?  Are there similar problems
> with importing and exporting Customer lists, vendor
> lists, bills and invoices, etc, etc?

No offense, but this has been explained to you over and over from many 
different angles.  Let me try to summarise so we can finally stop this thread 
or get something constructive out of it. 

Exporting data in tabular format is inherently trivial, and is usually done 
for reporting purposes.  Few have been implemented in GnuCash (TXF export is 
an exception) because most people feel the current reports are adequate for 
most purposes.  Those who don't and want to export something really custom 
have usually done it by running an XSLT transform on the XML file.  It isn't 
durable, but it certainly is simple.

On the other hand, importing tabular data is inherently hard and involves 
solving several problems.

1-You need a generic, robust and flexible CSV parser.  We couldn't find a 
ready made one in C, but someone has contributed one.  You also need a small 
interface to tell it the separator, quoting caracter and if the first row 
must be treated as column names.

2-You need a column matcher to map column name (or number) to their gnucash 
semantic type (amount, value, commodity, currency, reconciled state, customer 
name, account id, split memo, tax rate, etc, etc, etc.).  Since there are 
several dozens of those (way too much for the casual user to find it easy to 
select from a list), you probably need to do it in two steps.  First you pick 
an import profile that represents the type of data you wisht to import (Bank 
statement, customer list, check register, stock value report, etc).  After 
that your are presented with a list of relevent data type to match against 
the columns.

3-Unfortunately, you'd have to run this process every single time you import a 
file.  So you also need a file profile manager.  This will store the data 
types/column name or number match above in a file as well as some metadata so 
the format can be usefull to others (general file format category, finincial 
institution or software, packager, application notes, etc.).  You will off 
course need a GUI to look at all such file in some directory and present the 
metadata to the user so he can pick it instead of running the previous step.

4-Now you have the data nto gnucash, but that is still not enough.  Now you 
need several GUIs to fill in missing data.  Some of them exist in Gnucash 
(the generic transaction matcher, the account matcher, the commodity 
matcher), more would have to be written (customer matching, invoice matching, 
etc.).

5-You also need helper support for such things as date formats, number 
formats.

None of this is rocket science, but most of it is very long as it involves 
many, many pieces of user interface. Lots of people want to see it happen, 
but so far we had few contributors for the import/export process.  It's a pet 
project of mine, but I have no time to spend on it right now.

Now if you can't see why this is a LOT of code to write, you either:
- -Don't understand the implementation challenges of real commodity and currency 
support and double entry accounting like gnucash does.
or
- -Underestimate the time required for the GUI part of all this.
or
- -Like many people assume that since you know and understand the meaning of 
your file, a computer can as easely infer the meaning of the differents 
columns, date formats, currency labels, etc.

The gnucash developers are not interested in solving a problem for a single 
individual (barring financial contribution...), however the ARE interested in 
solving problems for a group of users (or solving problem for themselves), 
when time permits.

- -- 
Benoit Grégoire, http://benoitg.coeus.ca/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBDkVhmZ6zzPlLuwMRAsqQAJ9qHdqpNQGlgfWgxBpXeDU03HhW7wCg6Q/H
kRu+PSXe8BHvdCfXRpD9DsM=
=P+p/
-----END PGP SIGNATURE-----