QIF importer in 1.4.7 fails to detect duplicate transactions

[Derek Atkins]

grib@billgribble.com (Bill Gribble) writes:

> On Wed, Nov 01, 2000 at 01:56:53PM -0500, Derek Atkins wrote:
> > Um, I think this may be a bug, and it certainly explains why things
> > are failing for me.  The QIF names do NOT match across all my QIF
> > files, because they all came from different sources. 
> 
> It's not a bug, but it is a sign of incompletion.  You are trying to
> use the QIF importer in a way it was explicitly designed not to
> address yet.  The main purpose of the QIF importer is to import a
> user's records from Quicken; importing from on-line bank sources is
> tougher, for the reasons you have discovered on your own.  

Ahh.. Having never used quicken, and trying to 'seed' my Gnucash
accounts with my old bank information, this would be a nice addition.

> In general, you lose the ability to assume that a transaction you are
> importing is not already in your records (you might have entered it
> from a receipt or downloaded an overlapping segment), and so you have
> to look not just in the QIF files you are importing but your entire
> record database for duplicates if you want to do it right.

This is very true.  I had noticed that this was a problem, too, but
figured you already knew it.  Just a 'complete' matching would be a
welcome addition.  And I'm certainly willing to put some cycles into
it ;)

> Dates can be fuzzy, transfers not to the right account/category names
> or not there at all (my bank puts nothing in the QIF transfer/category
> line, even for transfers between accounts in the bank), amounts are
> possibly different (if you entered an amount off-by-a-penny by hand,
> then imported it correctly from the bank).

This would be good, too.  Indeed, Vanguard leaves transfers between
funds nameless, so there is not an easy way to match them.  "All
transfers go to 'default'" :) I suppose I could just create a
'transfer account'...  ;)

> All this means you need a more sophisticated matching/merging routine
> with heuristics and user intervention.  It's (still) on my list to do.
> I don't think it necessarily should be part of the QIF importer;
> rather, I think it should be called by the QIF importer instead of the
> current call to xaccMergeAccounts, and should probably have its own
> dialog and windows; it will probably be useful for other stuff too
> (any other importing or info-acquisition) since it will work with
> weeding duplicates from gnucash pairs.

This sounds like a reasonable approach.

> > Another idea... Perhaps we can add another step in the process, where
> > "potentially duplicate" transactions are displayed and the user has
> > the chance to mark them as duplicates or not?
> 
> Very similar to what I'm talking about.  I have some ideas about how
> to do this dialog if you want to talk about getting started on it.

Sure!

> > How am I supposed to get various organizations to agree on what they
> > call each other?
> 
> You can't, of course, and so you should import each one separately and
> rely on the all-singing all-dancing merge dialog to find the dupes.
> In the end, you are "supposed" to be doing this in separate import
> sessions, since these files aren't a consistent set (i.e. exported en
> masse from Quicken).

I suppose this would only work once we have the matches to all
existing transactions.  But yea, that would be much better, IMHO, than
requiring consistent QIF-files across all suppliers.

> b.g.

So, how can I help?

Should we take this to private mail and stop deluging the list?

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/      PP-ASEL      N1NWH
       warlord@MIT.EDU                        PGP key available