discussion generic import design

Christian Stimming stimming@tuhh.de
Mon, 21 Oct 2002 20:39:22 +0200


-----BEGIN PGP SIGNED MESSAGE-----

Hi,

on IRC we've already started an exhaustive discussion on the design of the 
generic transaction import. The framework and GUI provided by Benoit is nice 
and a good thing to start with, but there are still some issues to be 
improved. Especially I am unhappy about a few issues there, and since it is 
basically Benoit's OFX module and my HBCI module that are using this 
framework we better sort that out. 

Here's what the HBCI module wants: The HBCI code has retrieved a list of 
transactions and it knows the originating gnucash account. Those transactions 
should be passed to the (generic or other) transaction importer. This should 
perform two steps: 1. Step: Fill in the other split in the transaction, that 
is, offer the user an account choice list for each transaction where the user 
can choose the other account. This choice should be remembered and 
auto-filled as much as possible. By default, some default account should be 
filled in there. 2. Step: Detect duplicate transactions. If transactions have 
been detected to probably be duplicates of other transactions, show those 
other transactions and offer the user a choice to confirm whether this is 
indeed a duplicate or whether this is a new transaction.

In the long run, I want these step to be performed in a druid-style GUI, that 
is, with one druid page for each step. This is the way how this is done in 
the current QIF importer. 

Three issues with the current state of implementation:

The second step is already implemented in the generic importer almost exactly 
in the way I hoped for :-), except that you prefer to offer four different 
choices of what to do with each transaction, where I would definitely prefer 
to have only two choices (only a binary decision for the user). (If we can't 
get to an agreement here, we can still implement both possibilities within 
the same generic framework, so that's not too big of a deal)

The first step is not yet implemented in the generic importer. You said you 
prefer to implement that choice list to be in the same window, but I would 
definitely prefer to have a seperate druid page for that. Either way, we both 
agree that the user should be presented with some choice list from where the 
"other" account can be chosen, and that this choice should be remembered for 
the next import. The automatic matching will be done based on some strings of 
the transaction, where either we agree on a convention on which strings of 
the transactino should be used for the matching, or the protocol-code passes 
another string to the generic importer which is solely used for the automated 
transaction matching (and not displayed anywhere). 

There's one more issue with the generic importer, namely, the imported 
transactions are already created in the final accounts currently and they are 
already visible in that account's registers. I would rather like to avoid 
that. Instead, I would suggest that the generic importer should define its 
own "transaction" data type, which might be something like

struct ImportTrans {
  /* gnucash transaction with everything filled and with one split, but 
     which was not yet inserted into the account */
  Transaction *trans; 
  /* gnucash account where the transaction's split is to be inserted 
     once this transaction should be imported */
  Account *src_account;
  /* A string for which the matching should be remembered, so that it 
     can be used for automated matching of the other account */
  char *payee_memo;
};

so that the transaction is *not* added to the actual account unless the 
generic importer eventually has decided that this should indeed be done.

That's about what I want to achieve. Some of this might not be possible 
without a major design overhaul. Some of this is already present in the 
current qif-import module. In fact, all of this is already implemented in the 
qif-importer as part of its file import process, but one would need to 
extract the above parts and write a generic interface. Either way, to get a 
final generic importer with the properties described above, I started 
wondering whether eventually it might be less work to start off anew from the 
qif-importer, or to modify the existing generic importer in those ways. Since 
I recently finished all important features of the HBCI module, and polishing 
the transaction import dialog, I just thought about that a bit. I will 
probably have some time for coding this weekend, so I definitely hope we can 
find a consensus on how to proceed with the generic importer. Programming 
with the same concept in mind is just so much more fun :-))

Christian



PS: just to get the measures right (about the amount of code that I have 
written and that I would be ready to write for the importer) why not do a 
quick line count in the hbci directory, not to mention the openhbci 
library...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iQCVAwUBPbRJ32XAi+BfhivFAQEAdgQApcwMUNRJt39Mub0QOnjHYIuwHFN6KZpZ
yKUm2w8C7IwBtdQSLfMMhBxhCQxni+SRgkHpR/MWIflT8SAEGyiMu+a5Y7BMfyU7
RJazF2aMoRABnYtmc/e6lTlSE9NpvTapg4UDcHmYEb6VDWjyEDxyYWm8Kr8BIhrI
EFiNID3J9D4=
=geLm
-----END PGP SIGNATURE-----