[GNC] citibank card OFX download FITID problems?

[incoming-gnucash at sabot.com]

I am using GnuCash 2.6.17 on Ubuntu 16.04, and I am having some
problems with importing OFX files from the citcard web download.  I'd
love it if I could solve the problem by upgrading to a newer gnucash
release, but I think the symptoms seem to point at the OFX file not
being quite right (I'd love to be proven wrong about that!)

I use the citibank web site every week or two to download an OFX file
with my credit card transactions.  OFX import works fine for me with
other banks, but I've had a small problem with citi for a year or two:
Occasionally, say once every other month or two, a transaction I see
on the web site (and which is present in the OFX file) does not show
up in the gnucash window for me to accept or match.  In the past few
weeks, citi has been making changes to their web site and possibly to
their OFX generating code, and this problem seems to be worse/more
frequent.  Figuring out what transaction(s) got lost and manually
entering them is of course annoying.

In addition, there is a new problem: The matching window often shows a
handful of transactions that I know have been downloaded before.
These at least automatically show the checkmark to match them, so
dealing with this problem isn't too bad.  

Is anyone else seeing this issue?  I think citi might be generating
incorrect FITIDs, as follows:

Looking at a freshly downloaded citi file with month-to-date
transactions, the FITIDS in the file look like this:

  $ egrep FITID file.OFX
  <FITID>20190207090001
  <FITID>20190208090002
  <FITID>20190208090003
  <FITID>20190208090004
  <FITID>20190208090005
  ...

So they appear to append the 8 character date to "09" and then a 4
digit intra-file serial number.  The date is the date of the charge
(which is earlier than the date the file shows as DTPOSTED).

I tried downloading an OFX from "last statement" rather than
month-to-date, and the exact same pattern is used.  I don't know what
the "09" is, maybe it is a software version number?

This naming convention seems safe enough if they generated a day worth
of transactions at a time, but they do not; transactions trickle in.
So sometimes a subsequent download will add transactions on a date
that already has downloaded transactions.  This is a problem because
the ordering does not seem to keep the first-to-be-downloaded
transactions first in subsequent downloads.

For example, if you download the month-to-date transactions, on say
the final date with any transactions, there might be just a single
transaction T1.  A day later you download, and an additional
transaction T2 might show up with the same date.  If that happens, if
the new transaction came first in the file, T2 would get the same
FITID that T1 had, ending in 0001, so T2 would incorrectly be ignored.
And T1 would get a new FITID ending in 0002 and would incorrectly show
up in the gnucash matching window (which a checkmark to match it).

I do not normally save my OFX files after I load them, so I can't
completely verify all this yet, but I tried googling and came up with
this year-old issue that might be related:

   https://community.quicken.com/discussion/7647910/citibank-qfx-file-problems-causing-quicken-to-erroneously-disregard-some-transactions

A few questions:

First, is there a way for me to somehow inspect the list of FITIDs that my
gnucash has seen before, and any meta data like when gnucash first
saw them?  If so, that would help me nail down cause of what I am
seeing better.  And possibly it could help me coming up with a
workaround if the citi file is indeed broken and staying broken for
the foreseeable future.

Next, any advice on workarounds better than deducing what
transaction(s) in the OFX file are being ignored and entering them
manually?

Finally, is this idea safe: I think I can save myself typing in the
transaction details for any hidden transactions by editing the OFX to
uniquify any new transaction that is hidden.  If the OFX file rules
allow, I could add a suffix like "2019021909xxxxforceload".  Or maybe
I should just edit the date to start with 1 instead of 2, i.e. change
"2019021909xxxx" to "1019021909xxxx", which seems safe in that it
won't collide with any future OFX download using that naming scheme.

Thanks.

--gary