QFX import problem and work around, possible bug or new feature?

John Ralls jralls at ceridwen.us
Fri Sep 23 00:04:53 EDT 2016


> On Sep 23, 2016, at 3:28 AM, David Reiser <dbreiser at icloud.com> wrote:
> 
> 
>> On Sep 22, 2016, at 8:35 PM, David Reiser <dbreiser at icloud.com> wrote:
>> 
>> 
>>> On Sep 21, 2016, at 8:09 AM, John Ralls <jralls at ceridwen.us> wrote:
>>> 
>>> 
>>>> On Sep 21, 2016, at 5:48 AM, Greg Feneis <mfeneis at gmail.com> wrote:
>>>> 
>>>> Hi Folks,
>>>> 
>>>> If I understand correctly, QFX is Intuit's proprietary take on OFX.
>>>> Looking at the OFX.net, there appears a press release (April 2016)
>>>> indicating there's a new standard 2.2 for OFX.  It's possible that whatever
>>>> has changed about QFX involves Intuit's adaptation of the latest OFX.
>>>> 
>>> 
>>> That's likely the problem then. LibOFX supports only OFX version 2.0.
>>> 
>>> Regards,
>>> John Ralls
>> 
>> <CATEGORY> is not a valid tag in any version of OFX, including 2.2. All of the visible propriety tags in a QFX file are of the form <INTU.whatever>. While it’s possible Intuit could have reserved Category as a proprietary tag, it seems unlikely to me because it would make maintenance especially hard to keep track of. 
>> 
>> Use of "<MEMO>null" strikes me as just plain stupid.
>> 
>> Version 2.2 of OFX has not been adopted yet. Draft 3 is dated July 4, 2016. Almost the whole point of 2.2 over 2.1.1 is to standardize a way for OAuth2 tokenization to be possibly substituted for ID/Password authentication. In 2.2, the effort is clearly aimed at aggregators, such as Yodlee and possibly Mint (I think Mint is still owned by Intuit, but I’m not sure). It’s perhaps possible that Express Web Connect is an implementation of tokenized logins, and Intuit is getting the method standardized to make it easier to defend to regulators.
>> 
>> Another thing that popped up in 2.2 is the option for a server or client to insist on <APPKEY> in addition to <APPID> and <APPVER> — specifically to prevent the unwashed from using a supposedly open standard to retrieve their own data. <<grumble>> <<grumble>> They even spend a paragraph talking about not making <APPKEY> static, so that it can’t be easily discovered by those folks (us) who are “ghosting” other software. (Again, to gain access to their own data…)
>> 
>> And the reason Chase doesn’t have any problems with Quicken users? Easy: All their quicken users have new enough versions to be using DirectConnect. Chase sends different data streams to DC clients than they send in WebConnect downloads from the online accounts pages. I downloaded my Chase credit card transactions via aqbanking (with the ofxlog enabled), and then downloaded the same date range after logging into the Chase account web page. No <CATEGORY> tags in the DirectConnect data stream. Present in every transaction of the WebConnect download.
>> 
>> What we’d need in order to beat on Chase is someone with an old enough version of Quicken that isn’t allowed to make DirectConnect requests, but which is supposed to import QFX files. If that person had troubles importing the webconnect file, then there might be some action at Chase.
>> 
>> Another probably related fact: Chase now appears in the list of copyright holders on the title page of the OFX 2.2 standard. That means both that they have a corporate reason to want to steer the direction of the standard, and that there is someone at Chase whose responsibilities include being well-versed enough in OFX to make contributions to the standardization revision process. I’m guessing some junior underling of said person was given the responsibility to spruce up Chase’s WebConnect stuff for the recent account web page update. And that person didn’t bother to verify that something as ‘obvious' as “Category” was indeed an allowed tag.
>> 
>> And in unrelated news: Back at the beginning of the year when Chase was breaking everyone’s directconnect sessions, they were actually giving us a valid error. Error 15510 is “CLIENT UID invalid”. Fancy that, Chase was attempting to be useful.
>> --
>> Dave Reiser
>> dbreiser at icloud.com
>> 
> It’s even worse:
> "An OFX element must contain data (not just white space) and may not contain other elements. This is a refinement to the XML definition of an element which is more generic. An XML element containing other elements is defined in OFX as an aggregate. OFX specifically disallows empty elements and elements with mixed content.” 
> (This is from the 2.1.1 standard, but I’m pretty sure I could find it in the 1.x series)
> 
> So, not only is <CATEGORY> not a valid tag, leaving it empty is a another direct violation of the OFX spec.

So it would seem that at some point we'll need to get whatever is needed to generate APPKEYs and complain to the Comptroller of Currency (the ultimate regulator for banking) if the OFX committee won't provide it.

In the meantime, it sounds like you're saying that we shouldn't support Chase QFX downloads because they're not compliant with the OFX spec. That doesn't seem reasonable to me if the solution is simply to disregard invalid elements. 

BTW, I don't get the point of the aside about XML. If it's in the spec they're blowing smoke: OFX, like HTML, is not XML compliant because it doesn't require closing tags for inner elements: An element's value is the string between the element's start tag and the next element's start tag.

Regards,
John Ralls




More information about the gnucash-user mailing list