Gnucash 1.8 schedule, ofx and generic import infrastructure

[Benoit Grégoire]

On November 20, 2002 05:35 am, Christian Stimming wrote:
>
> Which parts are missing that are essential for 1.8 but will break String
> freeze and/or be a new feature? I know of:
>
> * Currency exchange transaction Dialog; responsible: Derek
>
> * Generic transaction import - Account matcher; responsible: ? (Benoit,
> cstim, Derek); but I would consider this as non-essential for 1.8, so if
> we cannot get it done until the next beta, I would propose to delay that
> until final 1.8 is out.

I agree this should wait until 1.8, otherwise we would only have 10 days to 
discuss and implement this feature, which is not realistic.  Since all the 
import functionality is now implemented in modules,  I think this feature can 
be implemented during a stable release cycle (1.8.1 or 1.8.2) without risking 
destabilizing GnuCash as a whole.

This would leave us a month to discuss a GUI for auto-split creation and 
revisit the rest of the GUI without release pressure.  I am even willing to 
revisit the API if it will lead to more code sharing.  I need to implement 
reconciliation using the balance (especially now that I agreed that 
transactions will be cleared by default until there is a preference page).

Speaking of which if I have time in the next ten days I would like to 
implement the preference page for the transaction matcher before release.

---------------------------

So, let's start the discussion:

Here are the GUI services currently offered by the generic import module (Not 
all of which will be used by every module, obviously):
-Source account matching and creation.
-Commodity matching and creation.
-Transaction matching and adding.
It would also need to add:
-Destination account autoselection (auto-split creation) as part of the 
transaction matching (There is no choice but to put it on the same page, 
there are plenty of examples where two successive transactions will need a 
different match) .
-Reconcile window invocation (I see it as a list of accounts for which a 
balance was downloaded.  You check those for which you want to invoque a 
reconcile).  Since HBCI supports it, is suppose Christian wrote some code for 
this.

Now, I've taken a lot of heat in the past for the event based, "pop in your 
face" design of the current generic importer.  Derek and Christian want it to 
be druid based.  I've spent much energy defending it, but to this day I still 
don't have a clear understanding of what kind of Druid would replace it that 
could do OFX, HBCI, QIF and others.

However, assuming the technical challenges can be resolved I would be willing 
to go thru the programming effort to convert it to a Druid, if a coherent and 
generic whole can be integrated into such an interface (skiping unneeded 
pages automatically).

However one thing is ESSENTIAL for me:  It must be very simple to understand 
how to write a simple import module (CSV import for example) that uses the 
generic import services (altough wether or not it is currently the case might 
be debatable).   I wrote the Transaction matcher as a public service because 
if I hadn't, there would now be three different GUI for transaction import.  
Considering how few GnuCash developers there are, that would have been 
ridiculous.  Furthermore, I didn't want the learning curve for the next 
developers to be as steep.

The way I currently do my accounting, I don't really have a need for a 
transaction matcher.  But I hoped the extra effort to write truly generic 
import infrastructure would be offset by coding help, since it would be used 
by HBCI and the new qif code.  Unfortunately so far I've been mostly hacking 
at this alone.

To be honest, my motivation for GnuCash isn't at an all time high, and I no 
longer have 20 hours a week to spend on ofx and GnuCash as I had in the last 
6 months.  Not that I don't like peer review, (on the contrary, I crave 
feedback, good of bad).  However I have a feeling a that the code I've 
written generates much insatisfaction, not all of which is openly expressed.  
Insatisfaction for me, because I took time to write extensive design docs 
BEFORE I wrote a single line of code, yet feel that now that it is 
implemented, it is not fully embraced.  Insatisfaction for longtime GnuCash 
developers, because the API is not in the usual GnuCash "style"  (And altough 
this wasn't said to me openly, neither is my programming style).

Note that most of the criticism I received (even those with which I did not 
agree) have been very usefull and lead to changes in the code or a better 
understanding by everyone involved of WHY some things were implemented the 
way they were.  Everyone have been fairly open to discussion.  (Which is why 
most of the issues were resolved).  

But now that we are in feature freeze, I would like everyone to make a list of 
what they don't like in the generic import GUI and API, why, and what they 
would replace it with.  Much of the conflict over this code was due to people 
having different understanding and expectations about what this code should 
do. It is important that we do this now if we are to work together to improve 
this code.

Also, PLEASE, I need testers to give me feedback on ofx before 1.7.4 even if 
everything is fine, especially with investment support which is brand new 
code.  OFX was broken in 1.7.2 tarball so I got no feedback from that 
release.  LibOFX, the transaction matcher and the ofx module all had major 
changes in the last month.

I hope I didn't offend anyone, but I had to vent this off,

Have a nice day,
-- 
Benoit Grégoire
http://step.polymtl.ca/~bock/