[GNC-dev] [GNC] Failed - import of QIF files from Quicken

[Christopher Lam]

Thank you for your rant.

Unfortunately we want all messages to be public.

FYI all developers are volunteers, and this duplicate-transaction bug will
be fixed for 3.10.

On Fri, 3 Apr 2020 at 20:31, Tom Hatzigeorgiou <tomhatz at hotmail.com> wrote:

> Sorry Jim but I do not agree
>
> For me the problem is with GNU and not with Quicken.
>
> Here is my rationale.
>
> The problem that (at least) I have with the importer started after version
> 3.1.
>
> Every time I have to import my data I downgrade to GNU 3.1 import the data
> and then re-upgrade to the latest one and use the 3.1 imported file.
>
> Did it today and worked like a charm. NO DUPLICATE ENTRIES.
>
>
>
> I think that the distribution needed to modernize the importer, but
> whoever did it, he screw it up. He did not had a good enough test sample
> data.
>
> I don’t mind because as I said I have a solution to my problem.
>
> *The problem is with so many who want to adapt GNUCASH and cant because
> they can’t import their data.*
>
>
>
> I would say, if the program is modular, please substitute into 3.9 the
> importer of 3.1, until someone fixes this problem with the current one.
>
> Or remove the 3.9 importer and create a standalone importer using the 3.1
> module, so people like me don’t have to jump through hoops to import their
> data.
>
> The only part of the program that I don’t care how modern it is, is the
> importer, since I would use it only once. BUT it has to work right.
>
>
>
> Thank you for your emails.
>
> Tom
>
>
>
>
>
> *From: *James Peterson <lyle at austin.rr.com>
> *Sent: *Friday, April 3, 2020 12:44 PM
> *To: *Tom Hatzigeorgiou <tomhatz at hotmail.com>; Christopher Lam
> <christopher.lck at gmail.com>
> *Subject: *Re: [GNC-dev] [GNC] Failed - import of QIF files from Quicken
>
>
>
> Christopher indicated to me that what you report would be
> the case.
>
> The problem is double entry bookkeeping.  Quicken does it
> by making a debit in one account and a credit in another
> account.  Both of these entries are in the QIF file.
> Gnucash has to then find these two entries and combine
> them back into one.  If it fails to recognize they are
> the same transaction, it will then fill in the other end
> of the "missing" transaction.  So a transfer of $5000 from
> one account to another may show up in GNC as two deposits
> of $5000 in the receiving account:  one from Quicken and
> another one from GNC doing double entry, but having missed
> that Quicken already did that.
>
> There is a lot of code in GNC to recognize the double entries,
> but apparently only in the case of data from separate QIF files.
> So if you have one big QIF file, then you will get duplicate
> entries.  But if you have each account in its own QIF file,
> then the inter-account actions will be (should be) properly
> recognized and things should be okay.
>
> But I have, like, 100 accounts over 30 years, so that would
> be a real pain for me.  All those QIF files, generated by
> hand, even just once for migration purposes.  Maybe it would
> be easier for you.
>
> And it generally just doesn't like transactions where both
> the source and destination are the same account (why would
> you do that?)  which I have 133 or so of.
>
> That's what I understand, but I could be wrong.
>
> jim
>
>
>
>
>
> On Fri, 2020-04-03 at 15:44 +0000, Tom Hatzigeorgiou wrote:
> > Hi guys
> > If you remember we were talking about the failure on the import of the
> QIF
> > file.
> > I’m happy to report that the error is fixed with version 3.9. I can now
> > import my whole QIF data file without a failure on the import process. I
> > would like to believe that I had some input into this fix.
> >
> > The problem now is that when the file is imported (successfully) the
> data in
> > the account are NOT correct.
> > At a first glance I see a lot of transfers that are duplicated.
> > I hope that is not a huge issue and that you will fix it soon at a later
> > release.
> >
> > Thank you.
> >
> >
> >
> >
> > From: Tom Hatzigeorgiou
> > Sent: Saturday, February 29, 2020 7:38 PM
> > To: Christopher Lam; crazylyle
> > Subject: Re: [GNC-dev] [GNC] Failed - import of QIF files from Quicken
> >
> > thank you Chris
> > If I could understand what you said I could apply it on my whole file and
> > test if it fixed the issue on the whole file.
> > Any way this can be done as a preprocess of the latest importer
> > That could fix the problem for everyone.
> >
> > Thanks.
> >
> >
> >
> > From: Christopher Lam <christopher.lck at gmail.com>
> > Sent: Saturday, February 29, 2020 9:52 AM
> > To: Tom Hatzigeorgiou <tomhatz at hotmail.com>; crazylyle <
> lyle at austin.rr.com>
> > Cc: gnucash-devel <gnucash-devel at gnucash.org>
> > Subject: Re: [GNC-dev] [GNC] Failed - import of QIF files from Quicken
> >
> > I think I've nailed this bug. Apply this modification to qif-to-gnc.scm
> and
> > it should handle empty S fields. C
> >
> > modified   gnucash/import-export/qif-imp/qif-to-gnc.scm
> > @@ -584,13 +584,11 @@
> >                                          (hash-ref qif-memo-map
> qif-memo)))
> >                                 (and (string? memo)
> >                                      (not (string=? memo ""))
> > -                                    (hash-ref qif-memo-map memo))))
> > -                     (if (not far-acct-info)
> > -                                (set! far-acct-info
> > -                                      (hash-ref qif-memo-map
> > -
> (default-unspec-acct))))))
> > +                                    (hash-ref qif-memo-map memo))))))
> >
> > -                   (set! far-acct-name (qif-map-entry:gnc-name far-acct-
> > info))
> > +                   (set! far-acct-name (if far-acct-info
> > +                                           (qif-map-entry:gnc-name far-
> > acct-info)
> > +                                           (default-unspec-acct)))
> >                     (set! far-acct (hash-ref gnc-acct-hash
> far-acct-name))
> >
> > On Fri, 28 Feb 2020 at 15:41, Christopher Lam <christopher.lck at gmail.com
> >
> > wrote:
> > > (attachment snipped)
> > >
> > > To help you import your old quicken datafile, you would best revert to
> > > 3.1, import completely then upgrade to 3.8. The offending code has been
> > > identified but is very difficult to understand because the author left
> > > nearly 20 years ago. Meanwhile the code base and the language have both
> > > moved on.
> > >
> > > On Tue, 25 Feb 2020 at 18:34, Tom Hatzigeorgiou <tomhatz at hotmail.com>
> > > wrote:
> > > > Hi I have attached the first year of my data that fails as I try to
> > > > import into GC.
> > > > I have data since '89, the first year that I have an issue with is
> 1997.
> > > > It was about the time that I started trading, so maybe the problem
> has
> > > > something to so with my trading account.
> > > >
> > > > Again, the QIF file loads with no problem using GC (for windows)
> version
> > > > 3.1 (gnucash-3.1-2.setup.exe), but it fails with the importer of
> version
> > > > 3.8 (gnucash-3.8.setup.exe).
> > > >
> > > > Thank you.
> > > >
> > > >
> > > >
> > > >
> > > > From: Tom Hatzigeorgiou <tomhatz at hotmail.com>
> > > > Sent: Tuesday, February 18, 2020 11:18 PM
> > > > To: Frank H. Ellenberger <frank.h.ellenberger at gmail.com>
> > > > Cc: Christopher Lam <christopher.lck at gmail.com>; GnuCash users
> group <
> > > > gnucash-user at gnucash.org>; gnucash-devel <gnucash-devel at gnucash.org>
> > > > Subject: Re: [GNC-dev] [GNC] Failed - import of QIF files from
> Quicken
> > > >
> > > > If what I asked cannot be done i will try to produce a file with the
> > > > errorsome other way
> > > > I would suggest to take these emails offline.  I think they confuse
> the
> > > > user community
> > > >
> > > > I will be in touch
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Feb 18, 2020 5:34 PM, "Frank H. Ellenberger" <
> > > > frank.h.ellenberger at gmail.com> wrote:
> > > > >
> > > > > Am 18.02.20 um 22:52 schrieb Christopher Lam:
> > > > > > Sorry. Please ask other devs on public mailing list.
> > > > > >
> > > > > > On Tue, 18 Feb 2020 at 15:51, Tom Hatzigeorgiou <
> tomhatz at hotmail.com
> > > > > > wrote:
> > > > > >
> > > > > >> To do this someone will need to help me.
> > > > > >> I don't know how projects like this work.
> > > > > >> I have no idea where the program source exists and the tools I
> will
> > > > > need
> > > > > >> to manipulate it.
> > > > > >>
> > > > > >> I was thinking that a very simple way to debug the code is, for
> > > > > someone to
> > > > > >> place a display on a version of the new importer where every
> record
> > > > > that it
> > > > > >> processes gets displayed (in a log file) before it gets
> imported.
> > > > > >> This way by running the importer I will see at what record(s) it
> > > > > failed.
> > > > > >> I would run it multiple times until I find and eliminate ALL the
> > > > > records
> > > > > >> of my file that it fails on.
> > > > > >> When all bad records get eliminated and my file gets loaded
> > > > > correctly by
> > > > > >> the current importer, I don't think I would have a problem
> sending
> > > > > you all
> > > > > >> those bad records for you to analyze and see where the new
> importer
> > > > > has
> > > > > >> gone bad.
> > > > > >>
> > > > > >> DO you think something like the above can be done?
> > > > >
> > > > > At least it should - instead of "Failed" - say "Failed to read
> line nr
> > > > > %d".
> > > > >
> > > > > That would reduce the burden of bisect.
> > > > >
> > > > > ~Frank
> > > > >
> >
> >
> >
>
>
>