QFX import problem and work around, possible bug or new feature?

[David Carlson]

If I recall, the examples given early in this and possibly related threads
not only had invalid elements, but they also had a lack of valid and
meaningful text to use to populate a description field in an imported
transaction.  This would not help the user if there are many transactions
with similar values.

At this point it is just another issue to watch for when the dust settles.

David C

On Thu, Sep 22, 2016 at 11:04 PM, John Ralls <jralls at ceridwen.us> wrote:

>
> > On Sep 23, 2016, at 3:28 AM, David Reiser <dbreiser at icloud.com> wrote:
> >
> >
> >> On Sep 22, 2016, at 8:35 PM, David Reiser <dbreiser at icloud.com> wrote:
> >>
> >>
> >>> On Sep 21, 2016, at 8:09 AM, John Ralls <jralls at ceridwen.us> wrote:
> >>>
> >>>
> >>>> On Sep 21, 2016, at 5:48 AM, Greg Feneis <mfeneis at gmail.com> wrote:
> >>>>
> >>>> Hi Folks,
> >>>>
> >>>> If I understand correctly, QFX is Intuit's proprietary take on OFX.
> >>>> Looking at the OFX.net, there appears a press release (April 2016)
> >>>> indicating there's a new standard 2.2 for OFX.  It's possible that
> whatever
> >>>> has changed about QFX involves Intuit's adaptation of the latest OFX.
> >>>>
> >>>
> >>> That's likely the problem then. LibOFX supports only OFX version 2.0.
> >>>
> >>> Regards,
> >>> John Ralls
> >>
> >> <CATEGORY> is not a valid tag in any version of OFX, including 2.2. All
> of the visible propriety tags in a QFX file are of the form
> <INTU.whatever>. While it’s possible Intuit could have reserved Category as
> a proprietary tag, it seems unlikely to me because it would make
> maintenance especially hard to keep track of.
> >>
> >> Use of "<MEMO>null" strikes me as just plain stupid.
> >>
> >> Version 2.2 of OFX has not been adopted yet. Draft 3 is dated July 4,
> 2016. Almost the whole point of 2.2 over 2.1.1 is to standardize a way for
> OAuth2 tokenization to be possibly substituted for ID/Password
> authentication. In 2.2, the effort is clearly aimed at aggregators, such as
> Yodlee and possibly Mint (I think Mint is still owned by Intuit, but I’m
> not sure). It’s perhaps possible that Express Web Connect is an
> implementation of tokenized logins, and Intuit is getting the method
> standardized to make it easier to defend to regulators.
> >>
> >> Another thing that popped up in 2.2 is the option for a server or
> client to insist on <APPKEY> in addition to <APPID> and <APPVER> —
> specifically to prevent the unwashed from using a supposedly open standard
> to retrieve their own data. <<grumble>> <<grumble>> They even spend a
> paragraph talking about not making <APPKEY> static, so that it can’t be
> easily discovered by those folks (us) who are “ghosting” other software.
> (Again, to gain access to their own data…)
> >>
> >> And the reason Chase doesn’t have any problems with Quicken users?
> Easy: All their quicken users have new enough versions to be using
> DirectConnect. Chase sends different data streams to DC clients than they
> send in WebConnect downloads from the online accounts pages. I downloaded
> my Chase credit card transactions via aqbanking (with the ofxlog enabled),
> and then downloaded the same date range after logging into the Chase
> account web page. No <CATEGORY> tags in the DirectConnect data stream.
> Present in every transaction of the WebConnect download.
> >>
> >> What we’d need in order to beat on Chase is someone with an old enough
> version of Quicken that isn’t allowed to make DirectConnect requests, but
> which is supposed to import QFX files. If that person had troubles
> importing the webconnect file, then there might be some action at Chase.
> >>
> >> Another probably related fact: Chase now appears in the list of
> copyright holders on the title page of the OFX 2.2 standard. That means
> both that they have a corporate reason to want to steer the direction of
> the standard, and that there is someone at Chase whose responsibilities
> include being well-versed enough in OFX to make contributions to the
> standardization revision process. I’m guessing some junior underling of
> said person was given the responsibility to spruce up Chase’s WebConnect
> stuff for the recent account web page update. And that person didn’t bother
> to verify that something as ‘obvious' as “Category” was indeed an allowed
> tag.
> >>
> >> And in unrelated news: Back at the beginning of the year when Chase was
> breaking everyone’s directconnect sessions, they were actually giving us a
> valid error. Error 15510 is “CLIENT UID invalid”. Fancy that, Chase was
> attempting to be useful.
> >> --
> >> Dave Reiser
> >> dbreiser at icloud.com
> >>
> > It’s even worse:
> > "An OFX element must contain data (not just white space) and may not
> contain other elements. This is a refinement to the XML definition of an
> element which is more generic. An XML element containing other elements is
> defined in OFX as an aggregate. OFX specifically disallows empty elements
> and elements with mixed content.”
> > (This is from the 2.1.1 standard, but I’m pretty sure I could find it in
> the 1.x series)
> >
> > So, not only is <CATEGORY> not a valid tag, leaving it empty is a
> another direct violation of the OFX spec.
>
> So it would seem that at some point we'll need to get whatever is needed
> to generate APPKEYs and complain to the Comptroller of Currency (the
> ultimate regulator for banking) if the OFX committee won't provide it.
>
> In the meantime, it sounds like you're saying that we shouldn't support
> Chase QFX downloads because they're not compliant with the OFX spec. That
> doesn't seem reasonable to me if the solution is simply to disregard
> invalid elements.
>
> BTW, I don't get the point of the aside about XML. If it's in the spec
> they're blowing smoke: OFX, like HTML, is not XML compliant because it
> doesn't require closing tags for inner elements: An element's value is the
> string between the element's start tag and the next element's start tag.
>
> Regards,
> John Ralls
>
>
> _______________________________________________
> gnucash-user mailing list
> gnucash-user at gnucash.org
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>