[GNC] OFX Import with Investment Transactions

Sat Oct 2 20:19:42 EDT 2021

Hi Jean-

I understand the logic for #1 and have no concerns about that.

As for #2, I did continue past the first matching dialog and there were no
additional dialogs, the focus returned to the account list.

There are definitely FITIDs that are already in the database that are also
in the most recent OFX file. This is because the provider doesn't allow you
to specify a date range, only 'current quarter' or 'some other quarter'.

I tried something else to research...

Using 4.8, I created a blank file and then imported an OFX file from Sep.
25, 2021. This file had 66 transactions and imported as expected with one
matcher screen per security/account as you indicated would happen. I then
attempted to import the OFX file from Oct. 2. This file has 84 transactions
-- the same 66 transactions from the Sep. 25 file plus 18 new transactions
dated 9/24.

The program then presents the dialog for the first account. There are two
transactions in the most recent file for this account; however they're
already in the database, thus triggering the message "OFX file
/home/robin/Downloads.qfx' imported transactions for account 'xxx' 2
transactions processed, no transactions to match". That makes sense as
those are older transactions that are already in the database. I clicked
'Close' for that dialog and then the focus returns to the main application.
It's almost as if the program -- having determined that there were no new
transactions to import into the first account -- decided not to look at the
subsequent accounts.

In service to the greater good, I am attaching the test file I created and
the Oct. 2 OFX file. I have removed all of the position data (there are
limits to what I'm willing to share :-) ), but that will not negatively
affect the import process.

Something else that I just noticed...

If -- after importing the OFX file -- you open one of the account
registers, you'll see duplicate transactions! Although, if you close and
re-open the file, those duplicated transactions do not appear when you
re-open the account register. I don't know what to make of that.

On Sat, Oct 2, 2021 at 1:52 PM Jean L <ripngo at gmail.com> wrote:
>
> Yes, there's a new behavior with OFX import. The match dialog shows up
> for each target account, so if you're importing securities the dialog
> will show up for each security. This could be a real annoyance if there
> are many different securities, so we may want to change that behavior.
> The new behavior was introduced to fix an issue when a single OFX file
> had transfers between accounts.
>
> I see two issues here:
> 1) The fact that the match dialog runs for each target account (see
> comment above)
> 2) The fact that some of your imported transactions were simply ignored.
>
> About 2): Did you try to continue with the import after the first
> matching dialog? Did it then go to the next securities? Or did you just
> abort thinking something was wrong (not expecting the new behavior of
> seeing the matching dialog for each security instead of all together as
> before)
>
> If that was not the case (i.e., you tried continuing but nothing else
> happened) one reason why this could happen is if the FITID used for the
> imported transactions were already present in the target account. This
> is new in the latest version: transactions that were previously imported
> are no longer shown the matching dialog. In your case this seems
> erroneous. So either it's a bug (i.e., the FITID of the imported
> transactions are NOT found in any of the transactions of the account
> you're imported into, so they should show) or it's a problem with the
> OFX data that the transactions seem to reuse existing FITIDs.
> The fact that things work normally when you import into a blank account
> would indicate that indeed, the FITID of the imported transactions
> already exist in your target account.
>
> SO, could you re-try importing with the latest version, but making sure
> you continue clicking OK for each match dialog? Does this work or not?
> If not, then we'll need to check whether the FITID of the new imported
> transactions somehow have been used before in the previously imported
> transactions. To see that, you'd need to pick one that was not imported
> (looking at the OFX file), get its FITID, and see in your account
> database (save it as an xml file so it can be looked at with a regular
> editor) whether that FITID exists.
>
> Jean
>
> On 10/2/2021 8:58 AM, Robin Chattopadhyay wrote:
> > Ubuntu 20.04 LTS (VMWare virtual machine in case that matters)
> > Gnucash 4.8
> > libofx 0.9.15
> >
> > I tried importing an OFX file from my 401K provider this morning and I
got
> > a message box that said it had imported 2 transactions for a single
> > security in the file and there were no additional transactions to
process.
> > This was incorrect as there were 18 new transactions across nine
securities
> > (along with a number of other transactions that had already been
imported
> > previously).
> >
> > I tried a number of things to resolve (after taking a backup, of
course):
> > 1 - Tools > Import Map Editor and deleted all of the associations for
this
> > file. When I re-imported, I was prompted to map each security in the
file
> > to the correct account. No issues there. But when importing the OFX file
> > again, I had the same experience as described above
> > 2 - Created a blank file and attempted to import the file there. Through
> > the import process, I created new securities and new accounts, nothing
> > unexpected there. Then the generic transaction importer dialog popped
up,
> > but with just the two transactions for the same security cited in the
> > original problem dialog. I imported those, clicked OK and then the
generic
> > transaction importer dialog came up *again* but only with transactions
for
> > a single, different security. I imported those and repeated the process
for
> > each security with transactions in the file. Each time the generic
> > transaction importer dialog came up, it only had transactions for a
single
> > security. This is definitely new behavior that I didn't see in 4.6.
> > Previously the import dialog had all the new transactions in the file
> > 3 - Reverted to 4.7. Crashed when selecting Import from the File menu
> > (Trace/breakpoint trap (core dumped). Not unexpected, but thought I
would
> > try anyway.
> > 4 - Reverted to 4.6. This works as it used to with all the new
transactions
> > in a single dialog
> >
> > Finally, I don't know if this matters, but I scanned stdout from the
> > build/make/install process to see if anything looked obviously wrong
and I
> > found this (I don't know if it's relevant):
> >
> > -- Performing Test HAVE_OFX_BUG_39
> > -- Performing Test HAVE_OFX_BUG_39 - Failed
> >
> > Thanks,
> > Robin
> > _______________________________________________
> > gnucash-user mailing list
> > gnucash-user at gnucash.org
> > To update your subscription preferences or to unsubscribe:
> > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > If you are using Nabble or Gmane, please see
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> > -----
> > Please remember to CC this list on all your replies.
> > You can do this by using Reply-To-List or Reply-All.
>
> _______________________________________________
> gnucash-user mailing list
> gnucash-user at gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: robin.qfx
Type: application/octet-stream
Size: 56427 bytes
Desc: not available
URL: <http://lists.gnucash.org/pipermail/gnucash-user/attachments/20211002/4d9fdfdc/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-file.gnucash
Type: application/x-gnucash
Size: 10338 bytes
Desc: not available
URL: <http://lists.gnucash.org/pipermail/gnucash-user/attachments/20211002/4d9fdfdc/attachment-0001.bin>