[GNC] Import Transaction Matcher issue in v4
David Carlson
david.carlson.417 at gmail.com
Mon Jul 27 22:33:24 EDT 2020
Michael,
Exactly. The QFX file only contains the information that the bank would
put on your monthly account statement along with the necessary
infrastructure to identify which bank created the file and which account
and the time frame which the file describes and sometimes some additional
information that GnuCash may or may not use. GnuCash has to synthesize the
rest in order to create a list of full double-entry qualified
transactions. That is why the import wizard is so difficult for users to
work with and even more difficult for developers to code.
Unfortunately most of us are very sloppy technically in talking of the file
as if it is a transaction list when it is really more like a bank
statement.
On Mon, Jul 27, 2020 at 8:57 PM Fross, Michael <michael at fross.org> wrote:
> Hello David,
>
> In looking at the QFX file, I only see one side of the transaction. This
> file has 3 distinct transactions (<STMTTRN> records) that are not related
> and only one account ID is listed.
>
> Michael
>
> On Mon, Jul 27, 2020 at 5:56 PM David Carlson <david.carlson.417 at gmail.com>
> wrote:
>
>> There is another use case which GnuCash also needs to handle. Some OFX
>> files may contain both sides of an inter-account transfer between two
>> accounts within the same bank because the OFX file can include multiple
>> accounts. In fact, I do that on a regular basis at one of my banks. I do
>> not test GnuCash's handling of these because I want to get the separate
>> notes associated with each side and merge them together into the same
>> transaction in my data file. Thus I deliberately call both of them New
>> rather than try to match them within the import process. I have not
>> checked to see if they have unique FITID's, but they would be under
>> different ACCOUNTID's and both can be accepted as New in release 2.6.19
>> and prior, which is the latest version that I use. I suppose ideally I
>> would like gnuCash to detect this case and automatically apply the
>> corresponding notes to both sides of the transfer, so I don't have to do
>> it manually.
>>
>> David Carlson
>>
>> On Mon, Jul 27, 2020 at 5:23 PM Fross, Michael <michael at fross.org> wrote:
>>
>>> Got it. Thanks. Now I understand why it used to work. And I certainly
>>> wouldn't expect GnuCash to correct a bank's horrible behavior. I'll keep
>>> an eye on the FITID and see if that's what they are doing.
>>>
>>> Thanks everyone for your help. Much appreciated.
>>>
>>> Michael
>>>
>>> On Mon, Jul 27, 2020 at 5:13 PM Jean Laroche <ripngo at gmail.com> wrote:
>>>
>>> > Ah, OK I get it now!
>>> > Yes, you should check that from one day to the next, the FTID returned
>>> > by citibank remains the same (for the same transaction). If not, then
>>> > that's going to be a problem.
>>> > What has changed recently with GC is that it not will not match a new
>>> > imported transaction with one from the register that has an online id.
>>> > Previous versions allowed that, so in your case, it's quite possible
>>> > that from one day to the next, the same transaction would be given a
>>> > different ID, but GC matched it to the same register transaction that
>>> > was previously matched.
>>> > So, on day 1, you'd match OFX1->Reg1, and on day two, you'd still match
>>> > OFX2 -> reg1, even though reg1 had already been matched to OFX1 the
>>> > previous day. So you'd see OFX2 appear as "match" and not "Add".
>>> >
>>> > The correct behavior is that OFX2 actually has the same ID as OFX1, and
>>> > it's entirely skipped by GC. But it seems that your bank is messing
>>> > their online ID, and this is what's causing the issue for you, along
>>> > with the new GC behavior.
>>> >
>>> > I'm not sure how to fix your problem. It's clearly a bank issue but
>>> > that's not terribly helpful as far as your problem goes. We should
>>> > probably add an option to defeat the new GC import behavior,
>>> > specifically for users like you whose bank send unreliable FTIDs/
>>> >
>>> > FYI, the new GC behavior was added because the old behavior was causing
>>> > issues with people who had recurring daily transactions: imagine buying
>>> > a cup of coffee every day: in that case GC will match all daily cups of
>>> > coffee to the same register transaction (within a date range, of
>>> course)
>>> > instead of adding the new one every day (which really is the correct
>>> way
>>> > to do it).
>>> >
>>> >
>>> >
>>> > Jean
>>> >
>>> >
>>> > On 7/27/20 2:45 PM, Fross, Michael wrote:
>>> > > Hi Jean. Let me try to better explain my issue.
>>> > >
>>> > > On most days, I download QFX files from my Checking and Credit Card
>>> > > accounts and import them. In this way I clear transactions and I can
>>> > > see what's going on. The next day I'll do so again, and the importer
>>> > > skips those that have already been matched per your comment.s I've
>>> been
>>> > > doing this for years and it's worked well.
>>> > >
>>> > > Since I updated to V4 the transactions that would have matched
>>> > > previously now come up with "Match Missing!" in the importer. I
>>> don't
>>> > > want to add them as they would create duplicates. Transactions that
>>> are
>>> > > not cleared seem to be able to be matched. But those that are
>>> > > previously cleared get the "Match Missing!" issue.
>>> > >
>>> > > Now, based on what you told me, the QFX file for a problem
>>> transaction
>>> > > has a FITID that is one higher than the ONLINE_ID in the gnucash file
>>> > > for the same transaction. So the Match Missing items seems to be
>>> > correct.
>>> > >
>>> > > If I duplicate the transaction, delete the original, it matches
>>> which I
>>> > > assume is because there is not an ONLINE_ID associated with the new
>>> one.
>>> > >
>>> > > What I don't understand is if there is an issue with GNUCash, which
>>> is
>>> > > why I asked if others are having the same issue, or if it is/was an
>>> > > issue with CITIBANK QFX files. If they changed the FITID for some
>>> > > reason that would cause this. Since no one else has said they are
>>> > > having the issue, I'm assuming this is not a GNUCash issue.
>>> > >
>>> > > Does that better explain the issue?
>>> > >
>>> > > I really appreciate the time to respond and help me through this.
>>> > >
>>> > > Michael
>>> > >
>>> > >
>>> > >
>>> > >
>>> > >
>>> > >
>>> > > On Mon, Jul 27, 2020 at 3:50 PM Jean Laroche <ripngo at gmail.com
>>> > > <mailto:ripngo at gmail.com>> wrote:
>>> > >
>>> > > There's a misunderstanding here.
>>> > > This is what GC does:
>>> > > - It looks at the OFX transaction, which always has an FITID.
>>> > > - IF GC sees this FITID in one of the register transactions, it
>>> skips
>>> > > the OFX transaction, because it's already been imported (when a
>>> > > transaction gets imported from the ofx, the online_id is copied
>>> into
>>> > > the
>>> > > register transaction, that's how GC remembers it's already
>>> imported
>>> > the
>>> > > transaction).
>>> > > - If the FITID is not found in the register, GC assumes that
>>> this is
>>> > a
>>> > > new transaction and attempts to match it with existing
>>> transactions
>>> > in
>>> > > your register. BUT:
>>> > > . It only looks at register transactions that have NO FITID
>>> > (i.e.,
>>> > > register transactions that you entered manually, typically)
>>> > > . Among those, it looks for transactions that match pretty
>>> > closely
>>> > > for the date and the amount.
>>> > > . It picks the one with the highest matching score, if that
>>> > > score is
>>> > > above a user-adjustable threshold.
>>> > >
>>> > > So in the case you outline below, if the register transaction
>>> has the
>>> > > same FITID as your ofx transaction, the ofx transaction will NOT
>>> be
>>> > > imported (it will be just skipped).
>>> > >
>>> > > I think I don't quite understand the problem you're having. Is it
>>> > that
>>> > > you're importing OFX transaction and they're not matching
>>> register
>>> > > transactions that you entered manually?
>>> > > Or is it that they're being imported despite the fact they were
>>> > > imported
>>> > > previously?
>>> > >
>>> > > J.
>>> > >
>>> > > On 7/27/20 1:31 PM, Fross, Michael wrote:
>>> > > > Thanks Jean / John for your thoughts. There is a register
>>> entry
>>> > > that
>>> > > > matches, IMHO, very closely. I increased the Match Display
>>> > > Threshold
>>> > > > from 1 to 3, and then to 6 (which appears to be the highest
>>> value
>>> > > > allowed.) Every transaction from the import says "Match
>>> Missing."
>>> > > >
>>> > > > Digging around a bit, for the transaction in question, the QFX
>>> > file
>>> > > > contains the FITID of 202007210003:
>>> > > >
>>> > > > <STMTTRN>
>>> > > > <TRNTYPE>CREDIT
>>> > > > <DTPOSTED>20200721120000
>>> > > > <TRNAMT>54.00
>>> > > > <FITID>202007210003
>>> > > > <NAME>ACH Electronic Credit
>>> > > > <MEMO>Expenses
>>> > > > </STMTTRN>
>>> > > >
>>> > > > My GNUCash file contains, for the same transaction has the
>>> online
>>> > id
>>> > > > being 202007210002
>>> > > > <split:slots>
>>> > > > <slot>
>>> > > > <slot:key>online_id</slot:key>
>>> > > > <slot:value
>>> type="string">202007210002</slot:value>
>>> > > > </slot>
>>> > > >
>>> > > > The online_ID is ...002 instead of ...003. Changing the QFX
>>> file
>>> > to
>>> > > > match the online_id value seemed to work. Now my question is
>>> why
>>> > > would
>>> > > > this be different for *lots* of transactions. Everything
>>> worked
>>> > > > normally in v3, but this would not have changed as part of the
>>> > > release.
>>> > > > I'll check a few more problem transactions and see if I can
>>> > detect a
>>> > > > pattern. Perhaps Citibank is paying games....
>>> > > >
>>> > > > Michael
>>> > > >
>>> > > > On Sun, Jul 26, 2020 at 4:57 PM jean laroche <
>>> ripngo at gmail.com
>>> > > <mailto:ripngo at gmail.com>
>>> > > > <mailto:ripngo at gmail.com <mailto:ripngo at gmail.com>>> wrote:
>>> > > >
>>> > > > To get a match you have to have a transaction in the
>>> register
>>> > > that's
>>> > > > sufficiently similar to the one you're importing, and that
>>> > > has not been
>>> > > > imported/matched before.
>>> > > > In your case, it could be one of these reasons (I can't
>>> see
>>> > > the image):
>>> > > > - There's no matching transaction in your register (no
>>> > existing
>>> > > > transaction has amount close, and a date close to the
>>> > > imported one)
>>> > > > - There's a matching transaction but it's already been
>>> > > matched to an
>>> > > > imported transaction at some point so it's not available
>>> to
>>> > > be matched
>>> > > > to the new imported one.
>>> > > > - There's a matching transaction that's available, but the
>>> > > match score
>>> > > > is below the threshold that allows the transaction to be
>>> > > shown as a
>>> > > > potential match. Too large a date mismatch can cause that.
>>> > > >
>>> > > > Can you check whether you're in one of these 3 cases? If
>>> > > you're in case
>>> > > > 3, you can lower the minimum matching threshold in the
>>> > > preferences and
>>> > > > see if that helps.
>>> > > > J.
>>> > > >
>>> > > >
>>> > > > On 7/26/2020 2:44 PM, John Ralls wrote:
>>> > > > > If there's no matching transaction already in the
>>> account
>>> > then
>>> > > > there's nothing to clear. In that case only adding or not
>>> > > makes sense.
>>> > > > >
>>> > > > > Regards,
>>> > > > > John Ralls
>>> > > > >
>>> > > > >
>>> > > > >> On Jul 26, 2020, at 1:56 PM, Fross, Michael
>>> > > <michael at fross.org <mailto:michael at fross.org>
>>> > > > <mailto:michael at fross.org <mailto:michael at fross.org>>>
>>> wrote:
>>> > > > >>
>>> > > > >> Hello all,
>>> > > > >>
>>> > > > >> I sent this earlier this month and didn't see any
>>> reply
>>> > so I
>>> > > > thought I
>>> > > > >> would try again. Has anyone else seen these
>>> issues? I
>>> > use
>>> > > > Citibank and
>>> > > > >> perhaps it's a Citibank issue, but I did not have this
>>> > > problem
>>> > > > on v2 or v3.
>>> > > > >>
>>> > > > >> Thanks all. I appreciate the help.
>>> > > > >>
>>> > > > >> Michael
>>> > > > >>
>>> > > > >>
>>> > > > >> On Sat, Jul 4, 2020 at 9:48 AM Fross, Michael
>>> > > <michael at fross.org <mailto:michael at fross.org>
>>> > > > <mailto:michael at fross.org <mailto:michael at fross.org>>>
>>> wrote:
>>> > > > >>
>>> > > > >>> Hello all,
>>> > > > >>>
>>> > > > >>> I typically download QFX files from my banks every
>>> day
>>> > > or two,
>>> > > > import them
>>> > > > >>> to clear them in Gnucash. Worked great. However,
>>> ever
>>> > > since
>>> > > > upgrading to
>>> > > > >>> v4, the importer seems to have trouble matching.
>>> Most
>>> > > of the
>>> > > > imported
>>> > > > >>> transactions are listed in the importer as (A)dd, but
>>> > > when I select
>>> > > > >>> (C)lear for them it says match missing.
>>> > > > >>>
>>> > > > >>> This has occurred for several accounts. Here is a
>>> simple
>>> > > > credit card
>>> > > > >>> example, although for my checking account, there are
>>> > dozens
>>> > > > like this. The
>>> > > > >>> top portion shows the register with the Sprint bill
>>> > > cleared.
>>> > > > The date,
>>> > > > >>> amount, and name (mostly) match.
>>> > > > >>>
>>> > > > >>> [image: image.png]
>>> > > > >>>
>>> > > > >>> Not sure if there is just something wrong with my
>>> setup
>>> > or
>>> > > > not. Perhaps a
>>> > > > >>> bug? Are others experiencing this? Any ideas to
>>> get the
>>> > > > matcher matching
>>> > > > >>> again? Something need to get cleared out?
>>> > > > >>>
>>> > > > >>> For those of us in the US, happy Independence Day.
>>> > > Thank you
>>> > > > all for your
>>> > > > >>> assistance.
>>> > > > >>>
>>> > > > >>> Michael
>>> > > > >>>
>>> > > > >>
>>> <image.png>_______________________________________________
>>> > > > >> gnucash-user mailing list
>>> > > > >> gnucash-user at gnucash.org
>>> > > <mailto:gnucash-user at gnucash.org> <mailto:
>>> gnucash-user at gnucash.org
>>> > > <mailto:gnucash-user at gnucash.org>>
>>> > > > >> To update your subscription preferences or to
>>> unsubscribe:
>>> > > > >>
>>> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>>> > > > >> If you are using Nabble or Gmane, please see
>>> > > > https://wiki.gnucash.org/wiki/Mailing_Lists for more
>>> information.
>>> > > > >> -----
>>> > > > >> Please remember to CC this list on all your replies.
>>> > > > >> You can do this by using Reply-To-List or Reply-All.
>>> > > > > _______________________________________________
>>> > > > > gnucash-user mailing list
>>> > > > > gnucash-user at gnucash.org <mailto:
>>> gnucash-user at gnucash.org>
>>> > > <mailto:gnucash-user at gnucash.org <mailto:
>>> gnucash-user at gnucash.org>>
>>> > > > > To update your subscription preferences or to
>>> unsubscribe:
>>> > > > >
>>> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>>> > > > > If you are using Nabble or Gmane, please see
>>> > > > https://wiki.gnucash.org/wiki/Mailing_Lists for more
>>> information.
>>> > > > > -----
>>> > > > > Please remember to CC this list on all your replies.
>>> > > > > You can do this by using Reply-To-List or Reply-All.
>>> > > >
>>> > > > _______________________________________________
>>> > > > gnucash-user mailing list
>>> > > > gnucash-user at gnucash.org <mailto:gnucash-user at gnucash.org>
>>> > > <mailto:gnucash-user at gnucash.org <mailto:
>>> gnucash-user at gnucash.org>>
>>> > > > To update your subscription preferences or to unsubscribe:
>>> > > > https://lists.gnucash.org/mailman/listinfo/gnucash-user
>>> > > > If you are using Nabble or Gmane, please see
>>> > > > https://wiki.gnucash.org/wiki/Mailing_Lists for more
>>> information.
>>> > > > -----
>>> > > > Please remember to CC this list on all your replies.
>>> > > > You can do this by using Reply-To-List or Reply-All.
>>> > > >
>>> > >
>>> >
>>> _______________________________________________
>>> gnucash-user mailing list
>>> gnucash-user at gnucash.org
>>> To update your subscription preferences or to unsubscribe:
>>> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>>> If you are using Nabble or Gmane, please see
>>> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
>>> -----
>>> Please remember to CC this list on all your replies.
>>> You can do this by using Reply-To-List or Reply-All.
>>>
>>
>>
>> --
>> David Carlson
>>
>
--
David Carlson
More information about the gnucash-user
mailing list