Weird OFX file import problem

[David Carlson]

On 10/1/2014 7:34 PM, msalists at gmx.net wrote:
> So it seems that a quick fix would be to change the FITIDs in each
> file to something unique.
> Trying to reimport the same OFX file with changed FITIDs makes it list
> all transactions, but they are recognized as dups, already being
> marked green and set to "reconcile (auto) match".
>
> So if I did go ahead and ran the OFX files through a script to make
> the FITIDs unique before importing them, it looks like GnuCash would
> still catch potential duplicate transactions, e.g. if there was a date
> range overlap in some of the files.
> Does that sound right?
>
> Thank you!
>
>
> On 2014-10-01 17:25, msalists at gmx.net wrote:
>> Oops I just noticed: every single transaction has this line:
>>     <FITID>LLLLLLLLLL
>>
>> So that's probably the reason? It seems that gnuCash does not care
>> about all the transactions inside one upload file having the same
>> FITID, only when processing additional files, it matters...
>>
>> Mark
>>
>>
>> On 2014-10-01 17:19, msalists at gmx.net wrote:
>>> Hi John,
>>>
>>> no, it's definitely different transactions for different months in
>>> there .
>>>
>>> Looking at the OFX file in a text editor, which tag / element are
>>> the OFX transaction IDs you mentioned that need to be different?
>>> There is a <TRNUID>, which is different in both files (basically the
>>> file download/generation date), but that is in the outer envelope of
>>> the download, not inside each individual transaction.
>>>
>>> Thank you,
>>>
>>> Mark
>>>
>>>
>>> On 2014-10-01 17:00, John Ralls wrote:
>>>> On Oct 1, 2014, at 11:02 AM, msalists at gmx.net wrote:
>>>>
>>>>>    Hello,
>>>>>    I have a problem with importing OFX files.
>>>>>    The very first import into a newly created account works, but
>>>>> trying to
>>>>>    import additional OFX files for the same account comes up with no
>>>>>    transactions to be imported.
>>>>>    Using an entirely new GnuCash file (sqlite storage), I narrowed
>>>>> it down
>>>>>    to the following:
>>>>>    I have two OFX files from the same bank (Targobank Germany, ex
>>>>>    Citibank).
>>>>>    File A.ofx is from June, file B.ofx is from October.
>>>>>    I have only the basic account structure setup, no bank account
>>>>> set up
>>>>>    yet.
>>>>>    I import A.OFX and chose to create a new bank account "Targo"
>>>>> in the
>>>>>    process.
>>>>>    The transaction import dialog comes up with the list of all
>>>>>    transactions and I import them without changing anything.
>>>>>    Then I proceed to import B.ofx. The transaction import dialog
>>>>> comes up
>>>>>    empty with no transactions.
>>>>>    In order to show that file B.ofx in itself has no problems, I
>>>>> remove
>>>>>    the account "Targo" and chose to delete all transactions in it.
>>>>>    Then I import B.ofx, and chose to create a new account "Targo2"
>>>>> the
>>>>>    process.
>>>>>    This time, the transaction import dialog comes up with all the
>>>>>    transactions in B.ofx.
>>>>>    I import them without changing anything.
>>>>>    Then I try importing A.ofx, which fimported fine earlier. This
>>>>> time,
>>>>>    the transaction import dialog comes up empty.
>>>>>    The transaction that had been listed earlier when importing
>>>>> into a new
>>>>>    account "Targo" do no longer show up.
>>>>>    What could be the problem here?
>>>>>    Thank you in advance for any help...
>>>> That probably means either that the two files contain the same
>>>> transactions, so once you've imported them GnuCash won't import
>>>> them again. If that's not the case, it means that your bank is
>>>> reusing OFX transaction IDs. That would be bad, because it would
>>>> mean that you can't use OFX to import more than once.
>>>>
>>>> Regards,
>>>> John Ralls
>>>>
>>>>
>>>>
>>>
>>
>
>
> _______________________________________________
> gnucash-user mailing list
> gnucash-user at gnucash.org
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>

If the non-unique FITID's were generated by the bank, then they are not
rigorously following the OFX standard.  I am not sure whether a German
bank would use that format, since it is mostly used in the United
States, but I would imagine that the German standard would also have
some sort of unique transaction identifier requirement.

If the FITID's are generated by some post processor, it should be
generating unique FITID's. 

The OFX standard does not require the FITID's to be unique between
institutions.  ie Two different financial institutions could duplicate
the same FITID, and it would be up to the client to differentiate.  For
more information about OFX visit http://www.ofx.net/.

David C