OFX importer and transaction matcher problems

[Benoit Grégoire]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sunday 21 March 2004 07:42, you wrote:
> On Sun, 2004-03-14 at 18:56, Benoit Grégoire wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > On Sunday 14 March 2004 12:49, Jochen De Smet wrote:
> > > GNUCash 1.8.8 on RH9.
> > >
> > > I'm importing an OFX file with about 180 transactions and am
> > > experiencing some strange behaviour in the generic import
> > > transaction matcher.
> > >
> > > 1) There's a transaction that only gets a confidence score of
> > >     2 bars, even though I do have a manually entered transaction
> > >     in the correct account with the exact amount and date of the
> > >     transaction i'm importing.
> >
> > That is REALLY strange, a transaction with an exact matching amount and
> > date would have a score of at least 6, somehow the transaction is seen as
> > different by the matcher.  Is this gnucash 1.8.8?
>
> Yes, See above :) But...
>
> <brown paper bag>
> This one was my fault. I had debits in credits reversed on my
> manual transaction and only noticed just now when I was about
> to paste the transaction to this mail.
> </bpb>
>
> So all I can complain about now is that when I fixed the transaction
> while the import window was open, the score didn't get updated. But
> that's a nitpick.
>
> > > 2) Similar, but worse. For another transaction my manually entered
> > >     transaction doesn't even get shown in the list of possible
> > >     matches, even though other transactions from both earlier and
> > >     later dates (and widely varying amounts) are shown.
> >
> > That combined with the previous issue would seem to indicate that there
> > is a problem with date comparing.
>
> I swear I wasn't drunk when I saw this. But when I did the import
> again after fixing the first issue this worked fine too.
>
> > > And while I'm whining about that, let me add a couple of requests
> > > and questions too:
> > >
> > > 1) Is there a way (modifying the sources if necessary) for me to
> > >     set the matcher to never show or match anything where the amounts
> > >     differ more than $1 ? (Actually, exact match amount-wise would
> > >     be fine too)
> >
> > In src/import-export/import-backend.c, split_find_match():
> >
> > In the Amount heuristics, replace
> > prob = prob-1; by prob = prob-100;
>
> Great thanks. Any chance of something like this making it into
> the preferences UI ?

Unlikely, however the algorithm could be improved.  How about (pseudocode)

prob = prob - 10 * ((amount_trans_imported - amount_trans_considered) /  
amount_trans_considered)

If you think that's sensible, can you file a RFE at bugzilla (with ofx in the 
title), I'll do it when I rework the importer for libofx 2.

> > > 2) What's the beaviour if you match two of the imported transactions
> > >     to the same already existing transaction?
> >
> > Only the match of the second one will be remembered.  The firts
> > transaction will appear again if you process the same file again.
>
> It would be nice if there could be a column in the match candidates
> table to indicate whether it's already matched or not. Granted, the
> only time this is really an issue is when there's two identical
> transactions the same day. I just happen to have a few of those :)

Humm, I was under the impression that those transactions wouldn't even show 
once matched.  Unfortunately I don't have time to look into it right now.
  
> > > 3) Is there a way to have the importer show all transactions in the
> > >    file you're importing? Currently it won't show transactions you
> > >    already imported (and any transactions you left in red if you
> > >    tried importing it before)
> >
> > The only way to see all the transactions you already imported is to
> > remove the lines:
> >  ((gnc_import_get_trans_online_id(xaccSplitGetParent(split))==NULL) ||
> >        (strlen(gnc_import_get_trans_online_id(xaccSplitGetParent(split)))
> > == 0) ||
> >
> > in the source file above.
>
> A more practical alternative to what I said above would be to have some
> kind of status line on the importer dialog with a summary like
> "178 transactions in file. 150 skipped, 5 red, 10 yellow, 13 green"
> (though worded better).

That's doable, but I don't see how that info is usefull for the user.  And 
since there already an overload on information during matching...

> > Transactions in red should already ALWAYS be shown again when you import,
> > otherwise it's a bug.
>
> Thinking about it a bit more, I guess all I really want is an easy way
> to distinguish between the case where all transactions in a certain file
> have already been imported vs. no transactions found in the file
> (because of openSP problems for example)

Now THAT is a good point...

- -- 
Benoit Grégoire, http://step.polymtl.ca/~bock/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAXl5hmZ6zzPlLuwMRApFlAKCGDYdg2KnNlU9wYcfjc4dNVMa6fwCfV8jF
4iodkXg+FOSyfHmqwUnm5Ww=
=1kcL
-----END PGP SIGNATURE-----