CSV Import Development Summary

[Christian Stimming]

Hi Benjamin,

this sounds great! I'm excited about the progress after only one week into the 
program :-) well, officially. I'd love to see your code committed to SVN; 
once the svn access for you is up and running, I'd encourage you to commit 
your code into the gnucash svn early and often. You will probably get a 
branch of your own for now, but this gives you the benefit that you really 
don't have to watch out for any other pitfalls - just commit your code :-).

Am Montag, 4. Juni 2007 20:54 schrieb Benjamin Sperisen:
> The Gnumeric code that handles the actual parsing of the file, the STF
> library, will be placed in gnucash/lib/stf.

Sounds fine. Might be changed later, but good enough for now.

> This function will first prompt the user to select a file. The
> filename will be passed to the function
>     GPtrArray* gnc_csv_parse(char* filename, StfParseOptions_t*).
> (...) gnc_csv_parse will
> return a two-dimensional array of char*s. So far as I can tell, there
> isn't a way for this parsing to "fail," in the sense that even if the
> array isn't what the user wants there will be some way for STF to
> parse it (no matter how badly it comes out).

I wonder how some of the more weird error conditions need to be checked for. 
Examples: A completely binary file (no table/CSV structure at all), or one 
where the user has no read permission.  Some instructive error messages might 
be helpful for those cases (as opposed to simply showing an empty GtkTreeView 
widget). Of course this can be deferred to later in the summer.

> new data that gnc_csv_parse returns. The user also needs to be able to
> specify the type of each column (which columns contain the date,
> description, amount, etc.) in the file. A row of combo boxes would
> appear above the columns, one for each column. The user could select a
> type in each, which will be stored in an array.

Sounds very good.

> The preview widget was the cause of some disagreement with my
> application. I think it should eventually be a custom widget, (...) 
> In the interest of having 
> something that works sooner, I will use a tree view first. Moreover,
> I'm pretty sure far more people have CSV files than fixed-width files,
> so for most people this decision will have little impact (besides
> maybe getting the feature sooner). If I finish with a lot of time left
> before the end of summer of code, I think it might be worth spending
> that time working on a custom widget.

Absolutely. Sounds like a very good plan. There's always the possibility to 
polish up your solution with an extremely cool custom widget, later.

> (...) The user
> can then finally select the destination accounts for each transaction
> and finally commit them to the account.

Yes, all fine.

One more feature that comes to mind is how one selection of parsing options 
could be cached or saved for later re-use. Just imagine people who get one 
particular kind of CSV files regularly from other sources (their bank or 
other finance software). I wonder how the repeated CSV import for those 
regular CSV users could be made easier. This is also a feature for the second 
round of your work :-), but I think it would be useful for quite a number of 
users. One possible solution is some way of saving the import options by some 
name, and selecting that options on the next CSV import; however, from a GUI 
point of view this is probably sub-optimum. Maybe there are other solutions 
that would be more elegant. Whatever, I'd just suggest to keep this in mind 
for later.

> So far, I have completed the extraction of STF from Gnumeric, created
> a CSV module that adds a menu entry, and gotten the importer to import
> a test CSV file with hard-coded meanings for each of the columns. It
> appears to work fine (with the exception of the possibility that
> unbalanced transactions getting through, discussed in this thread:
> https://lists.gnucash.org/pipermail/gnucash-devel/2007-June/020669.html).
> I'm now starting work on the parsing dialog.
>
> Let me know if you have any comments or questions!

Way to go! Sounds very good. As I already wrote, I'd strongly encourage you to 
commit the code (even if it's intermediate) as early and often as possible to 
SVN, once we've got the account set up. Other than that, I think this is very 
good progress in your project.

Regards,

Christian