[GNC] OFX Import with Investment Transactions

[Chris Good]

Hi Robin,

+1 for Jean's comments.

Re the crash after reverting to 4.7:

This should not happen. The maintainers try hard to ensure you can always
revert to a previous version, especially from a minor version upgrade. To
revert to a previous major version, provision is made so you can always
revert to the last minor revision of a previous major version. See
https://wiki.gnucash.org/wiki/FAQ#Using_Different_Versions.2C_Up_And_Downgra
de

There were some problems with trading accounts fixed in 4.7, see
https://bugs.gnucash.org/show_bug.cgi?id=798221

Do you use trading accounts?

If that is not your problem, perhaps you could open a bug and provide a
trace file (see
https://wiki.gnucash.org/wiki/Tracefile
or preferably a stacktrace.

Regards, Chris Good

Message: 5
Date: Sat, 2 Oct 2021 11:51:40 -0700
From: Jean L <ripngo at gmail.com>
To: gnucash-user at gnucash.org
Subject: Re: [GNC] OFX Import with Investment Transactions
Message-ID: <08cf656d-09e4-c1a8-054b-51edf5f309af at gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed

Yes, there's a new behavior with OFX import. The match dialog shows up 
for each target account, so if you're importing securities the dialog 
will show up for each security. This could be a real annoyance if there 
are many different securities, so we may want to change that behavior. 
The new behavior was introduced to fix an issue when a single OFX file 
had transfers between accounts.

I see two issues here:
1) The fact that the match dialog runs for each target account (see 
comment above)
2) The fact that some of your imported transactions were simply ignored.

About 2): Did you try to continue with the import after the first 
matching dialog? Did it then go to the next securities? Or did you just 
abort thinking something was wrong (not expecting the new behavior of 
seeing the matching dialog for each security instead of all together as 
before)

If that was not the case (i.e., you tried continuing but nothing else 
happened) one reason why this could happen is if the FITID used for the 
imported transactions were already present in the target account. This 
is new in the latest version: transactions that were previously imported 
are no longer shown the matching dialog. In your case this seems 
erroneous. So either it's a bug (i.e., the FITID of the imported 
transactions are NOT found in any of the transactions of the account 
you're imported into, so they should show) or it's a problem with the 
OFX data that the transactions seem to reuse existing FITIDs.
The fact that things work normally when you import into a blank account 
would indicate that indeed, the FITID of the imported transactions 
already exist in your target account.

SO, could you re-try importing with the latest version, but making sure 
you continue clicking OK for each match dialog? Does this work or not?
If not, then we'll need to check whether the FITID of the new imported 
transactions somehow have been used before in the previously imported 
transactions. To see that, you'd need to pick one that was not imported 
(looking at the OFX file), get its FITID, and see in your account 
database (save it as an xml file so it can be looked at with a regular 
editor) whether that FITID exists.

Jean

On 10/2/2021 8:58 AM, Robin Chattopadhyay wrote:
> Ubuntu 20.04 LTS (VMWare virtual machine in case that matters)
> Gnucash 4.8
> libofx 0.9.15
>
> I tried importing an OFX file from my 401K provider this morning and I got
> a message box that said it had imported 2 transactions for a single
> security in the file and there were no additional transactions to process.
> This was incorrect as there were 18 new transactions across nine
securities
> (along with a number of other transactions that had already been imported
> previously).
>
> I tried a number of things to resolve (after taking a backup, of course):
> 1 - Tools > Import Map Editor and deleted all of the associations for this
> file. When I re-imported, I was prompted to map each security in the file
> to the correct account. No issues there. But when importing the OFX file
> again, I had the same experience as described above
> 2 - Created a blank file and attempted to import the file there. Through
> the import process, I created new securities and new accounts, nothing
> unexpected there. Then the generic transaction importer dialog popped up,
> but with just the two transactions for the same security cited in the
> original problem dialog. I imported those, clicked OK and then the generic
> transaction importer dialog came up *again* but only with transactions for
> a single, different security. I imported those and repeated the process
for
> each security with transactions in the file. Each time the generic
> transaction importer dialog came up, it only had transactions for a single
> security. This is definitely new behavior that I didn't see in 4.6.
> Previously the import dialog had all the new transactions in the file
> 3 - Reverted to 4.7. Crashed when selecting Import from the File menu
> (Trace/breakpoint trap (core dumped). Not unexpected, but thought I would
> try anyway.
> 4 - Reverted to 4.6. This works as it used to with all the new
transactions
> in a single dialog
>
> Finally, I don't know if this matters, but I scanned stdout from the
> build/make/install process to see if anything looked obviously wrong and I
> found this (I don't know if it's relevant):
>
> -- Performing Test HAVE_OFX_BUG_39
> -- Performing Test HAVE_OFX_BUG_39 - Failed
>
> Thanks,
> Robin