[GNC] Import matcher shortcomings, OFX realm (at least)

[David Reiser]

Michael Fross said:
> I have to keep importing the same QFX file over and over until I get
> “nothing to import” message. If I don’t, it seems to miss transactions in
> the file. Not sure about QIF, but Maybe it’s similar.
> 
> Michael

Ok, I’ll split this out into another discussion.

The need for multiple attempts at importing the same ofx file to get all the transactions imported is probably a result of a shortcoming in the matcher code when multiple same-dollar-value transactions (or nearly the same if Commercial ATM fee threshold is set to anything greater than 0.00) appear in the ofx file. One very common cause of such cases is vending machine transactions.

If you never enter any of the same-value transactions manually, and only import them, then you’ll probably be OK, because the matcher will suggest that all the transactions should be Added rather than matched.

If, however, you have even one of the same-value transactions entered manually, and a set of 5 same-value transactions incoming in the import file, the matcher’s default behavior is to display all 5 incoming transactions as having a good candidate match. The problem is that all five of those incoming transactions are pointed at a single transaction in the gnucash file. If you blithely click OK in the Matcher window, the import process matches the first incoming transaction to the existing transaction. Then when the second same-value transaction gets examined, the matcher says “Oh, I already matched that existing transaction, I’ll ignore this one”. And all subsequent same-value transaction that had reported they had a match in the file are ignored because the candidate match is already taken.

Matching can be even messier if you have, say, 4 transactions of $2.00 entered in your data file, but 7 $2.00 transactions coming in with the import. 

The reason sequential imports work is that once a candidate is matched and the import process ends, the next time the import process is launched, that first transaction is no longer a candidate match because it now has an imported transaction ID associated with it (and the transaction ID prevents the incoming transaction from appearing at all anymore), and the matcher moves on (sometimes only one candidate transaction at a time).

I did file a bug on this several years ago, but the matcher’s default match identification has not changed. What was added is the ability to double click a transaction in the matcher dialog window to see alternative transactions to match against. If you see multiple transactions in your matcher window with the same dollar value, you must inspect the potential matches for each one and select a different one from the top candidate picked by default for all the same-value transactions.

I hope this explanation helps reduce the number of repeat imports you have to use.

Dave
--
Dave Reiser
dbreiser at icloud.com