[GNC] Fwd: [GNC-dev] Fwd: Failed - import of QIF files from Quicken

[Christopher Lam]

You're right that scheme code logging is useful.

The scheme way is to use pk, such as

(pk "qif acct map is" qif-acct-map)

But often you'll find that gnc objects are opaque; you can use (gnc:pk "qif
acct map is"  qif-acct-map) instead.

You can combine multiple objects with pk or gnc:pk (gnc:pk "in
qif-import:qif-to-gnc. File list is " qif-files-list " acct map is "
qif-acct-map) etc.

Good luck!




On Wed, 19 Feb 2020, 8:36 am James Peterson, <lyle at austin.rr.com> wrote:

> Well, with no hint of what the problem is, even smaller
> test sets won't necessarily find anything, so it could be
> a lot of work for no benefit.  For example, I could export
> each account separately (although there are about 80 of them
> so that's a lot of work), and each account could load
> okay separately, because maybe the problem is an interaction
> between two different accounts. So then I have to do 80*79
> possible pairs of accounts, and so on.
>
> My poking around in the
> source code to try to find the problem suggests that the
> problem is in the Scheme part that seems to be working with
> all the accounts at the same time:
>
>  ;; Build a local account tree to hold converted transactions.
>
>  ;; Sort the account list on the depth of the account path.
>
>  ;; Make all the accounts.
>
>  ;; Run through the markable transactions marking any
>       ;; duplicates.  marked transactions/splits won't get imported.
>
>                ;; Convert into a GnuCash transaction.
>                ;; rebalance and commit everything
>
> So this looks like multiple runs thru the data, and even printing
> the line number won't narrow it down -- you have to know what
> step of the process we are doing, and where we are in the data
> at that point.
>
> As things stand, we don't even know what the immediate problem
> is -- a divide by zero?  Looking for an account that should exist,
> but doesn't?, an account type that it doesn't know about?
>
> The code is not written to allow it to even explain what it is
> doing. If we are going to use this code base, someone who actually
> understands, and is comfortable with the code will need to add
> a "verbose input processing" level or something that will create
> a log file explaining what it is going to try to do, then do it,
> then explain what it got and how it interprets that. I've done
> that with code before, but only C code, where, if need be, there is
> the equivalent of a printf to a log file between every major step,
> or each iteration of a loop, explaining what it thinks it is doing
> and with what input and with what result.  I just don't know how to
> add that to scheme.
>
> jim
>
>
> On Tue, 2020-02-18 at 16:53 -0600, Tommy Trussell wrote:
> > On Tue, Feb 18, 2020 at 4:07 PM crazylyle <lyle at austin.rr.com> wrote:
> > > So you are suggesting reducing the size of our QIF files to a small
> > > debuggable size.
> > >
> > > My QIF file is 653,808 lines long.  About 2^20.  So just using a binary
> > > search would
> > > take at least 20 trials to find the first line that it fails on.  Not
> > > exactly something that
> > > seems practical.  And that's just for my input file.
> > >
> > > And of course, just feeding a partial file might produce a failure,
> but a
> > > failure of a
> > > different kind, so this won't necessarily get us anywhere.
> >
> > What bisections or limited imports have you tried?
> >
> > If you haven't already, try going into Quicken and exporting one day, one
> > month, one year, one account... a smaller test batch and see if you get
> the
> > same error.
> >
> > I know years ago (maybe 2012 or earlier) when I switched from Quicken to
> > GnuCash I found that Quicken created many odd little problems that were
> just
> > not worth the effort to clean up. SO I exported all my data (going back
> to
> > about 1989) into one QIF file that I never bothered to import, but the
> data
> > is still there if I need it. As you know, QIF is a text file, so you can
> > search for transactions with a text editor.
> >
> > Then I went back and exported just the previous one or two years from
> > Quicken (so for you it might be 2019 and 2020-to-date) and imported only
> > that data into GnuCash. In order to make the import work its best, I
> used a
> > multi-step process, where I created the most-used accounts first, did a
> few
> > test imports, fixed a few small things in Quicken, then did my import. I
> > cleaned up the few remaining import problems in GnuCash and haven't
> looked
> > back.
> >
> > UNFORTUNATELY for you, most of us who make the conversion never go back
> and
> > try the latest (there's no need) so the developers depend upon folks who
> > encounter problems to fix them as they arise.
> >
> >
> > > Please remember to CC this list on all your replies.
> > > You can do this by using Reply-To-List or Reply-All.
>
> _______________________________________________
> gnucash-user mailing list
> gnucash-user at gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>