Weird OFX file import problem

[msalists at gmx.net]

On 2014-10-01 18:27, David Carlson wrote:
> On 10/1/2014 7:34 PM, msalists at gmx.net wrote:
>> So it seems that a quick fix would be to change the FITIDs in each
>> file to something unique.
>> Trying to reimport the same OFX file with changed FITIDs makes it list
>> all transactions, but they are recognized as dups, already being
>> marked green and set to "reconcile (auto) match".
>>
>> So if I did go ahead and ran the OFX files through a script to make
>> the FITIDs unique before importing them, it looks like GnuCash would
>> still catch potential duplicate transactions, e.g. if there was a date
>> range overlap in some of the files.
>> Does that sound right?
>>
>> Thank you!
>>
>>
>> On 2014-10-01 17:25, msalists at gmx.net wrote:
>>> Oops I just noticed: every single transaction has this line:
>>>      <FITID>LLLLLLLLLL
>>>
>>> So that's probably the reason? It seems that gnuCash does not care
>>> about all the transactions inside one upload file having the same
>>> FITID, only when processing additional files, it matters...
>>>
>>> Mark
>>>
>>>
>>> On 2014-10-01 17:19, msalists at gmx.net wrote:
>>>> Hi John,
>>>>
>>>> no, it's definitely different transactions for different months in
>>>> there .
>>>>
>>>> Looking at the OFX file in a text editor, which tag / element are
>>>> the OFX transaction IDs you mentioned that need to be different?
>>>> There is a <TRNUID>, which is different in both files (basically the
>>>> file download/generation date), but that is in the outer envelope of
>>>> the download, not inside each individual transaction.
>>>>
>>>> Thank you,
>>>>
>>>> Mark
>>>>
>>>>
>>>> On 2014-10-01 17:00, John Ralls wrote:
>>>>> On Oct 1, 2014, at 11:02 AM, msalists at gmx.net wrote:
>>>>>
>>>>>>     Hello,
>>>>>>     I have a problem with importing OFX files.
>>>>>>     The very first import into a newly created account works, but
>>>>>> trying to
>>>>>>     import additional OFX files for the same account comes up with no
>>>>>>     transactions to be imported.
>>>>>>     Using an entirely new GnuCash file (sqlite storage), I narrowed
>>>>>> it down
>>>>>>     to the following:
>>>>>>     I have two OFX files from the same bank (Targobank Germany, ex
>>>>>>     Citibank).
>>>>>>     File A.ofx is from June, file B.ofx is from October.
>>>>>>     I have only the basic account structure setup, no bank account
>>>>>> set up
>>>>>>     yet.
>>>>>>     I import A.OFX and chose to create a new bank account "Targo"
>>>>>> in the
>>>>>>     process.
>>>>>>     The transaction import dialog comes up with the list of all
>>>>>>     transactions and I import them without changing anything.
>>>>>>     Then I proceed to import B.ofx. The transaction import dialog
>>>>>> comes up
>>>>>>     empty with no transactions.
>>>>>>     In order to show that file B.ofx in itself has no problems, I
>>>>>> remove
>>>>>>     the account "Targo" and chose to delete all transactions in it.
>>>>>>     Then I import B.ofx, and chose to create a new account "Targo2"
>>>>>> the
>>>>>>     process.
>>>>>>     This time, the transaction import dialog comes up with all the
>>>>>>     transactions in B.ofx.
>>>>>>     I import them without changing anything.
>>>>>>     Then I try importing A.ofx, which fimported fine earlier. This
>>>>>> time,
>>>>>>     the transaction import dialog comes up empty.
>>>>>>     The transaction that had been listed earlier when importing
>>>>>> into a new
>>>>>>     account "Targo" do no longer show up.
>>>>>>     What could be the problem here?
>>>>>>     Thank you in advance for any help...
>>>>> That probably means either that the two files contain the same
>>>>> transactions, so once you've imported them GnuCash won't import
>>>>> them again. If that's not the case, it means that your bank is
>>>>> reusing OFX transaction IDs. That would be bad, because it would
>>>>> mean that you can't use OFX to import more than once.
>>>>>
>>>>> Regards,
>>>>> John Ralls
>>>>>
>>>>>
>>>>>
>>
>> _______________________________________________
>> gnucash-user mailing list
>> gnucash-user at gnucash.org
>> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>> -----
>> Please remember to CC this list on all your replies.
>> You can do this by using Reply-To-List or Reply-All.
>>
> If the non-unique FITID's were generated by the bank, then they are not
> rigorously following the OFX standard.  I am not sure whether a German
> bank would use that format, since it is mostly used in the United
> States, but I would imagine that the German standard would also have
> some sort of unique transaction identifier requirement.
>
> If the FITID's are generated by some post processor, it should be
> generating unique FITID's.
>
> The OFX standard does not require the FITID's to be unique between
> institutions.  ie Two different financial institutions could duplicate
> the same FITID, and it would be up to the client to differentiate.  For
> more information about OFX visit http://www.ofx.net/.
>
> David C
> _______________________________________________
>

  MsMoney and Quicken are (or have been) pretty popular in Germany as 
well, and I would think that even German software probably uses those 
file formats, because that's what pretty much every bank offers.

The FITID is generated by the bank like that. I contacted them and they 
said they never had any complaints from Quicken or other vendors - maybe 
those don't check the FITID either...