[GNC] Changing account tree on large database

Sat Apr 27 03:05:35 EDT 2019

David T and Cricket,

I am still using release 2.6.19 in either Windows 7 or Ubuntu 18.04 but I
like to keep several reports and 30 0r so accounts open when I close the
program a few times a week.  When I heard that 3.x was much slower and even
now still has other bugs that would frustrate me I decided to wait longer
to take the plunge.  i mostly run it in a direct session on a Ubuntu
virtual machine which is faster than Windows but  I do see a few seconds
lags with the enter key and I play a lot of solitaire during those pesky
automagical file backups every half hour.  I think that GnuCash has to do a
huge number of background computations, mostly single threaded, which are
likely caused by the compiler and not really needed every time the enter
key is pressed, even in Linux.  Oh, my data file is stored on a separate
NAS elsewhere on my LAN and I don't do anything special to try using a
local copy with manual backups to the NAS.  My Ethernet is mostly Gigahertz
but I recently found that  it has some 10 MHz bottlenecks which I have not
yet pinpointed and fixed.  Some parts are now well over 15 years old and
there may still be some short Cat 5 segments in the system. So there is
some room for improvement there.

My data file starts in September 2007 with bank accounts and gradually gets
more complete until it has almost everything after 2010. My data file is
7.2 Meg in compressed XML.  It ties up 1.7 GiB out of 5.5 GiB RAM in the
Ubuntu Virtual Box guest.  I forgot where to look for the number of
transactions.  All this in a Dell Precision 7510 Laptop with an Intel Core
I7-6820HQ processor @ 2.7 GHz CPU with 24 GiB RAM running Windows 7.

That is where I am at right now.

David Carlson

On Fri, Apr 26, 2019 at 11:45 PM David T. <sunfish62 at yahoo.com> wrote:

> David,
>
> I think a couple of additional points of reference from you would be
> helpful.
>
> What operating system are you experiencing this slow performance? I
> believe it’s WIndows, but I could be mistaken. The relative performance
> benchmarks seem to be OS-dependent. I am using MacOS, and while some
> reports do load slowly (the Asset Barchart comes to mind), most of my
> experience has been positive on this regard. I do not see delays on
> entry—certainly not on order of seconds.
>
> Similarly, what size is your data file? Mine goes back 13 years, and a
> quick survey of the file in SQL format shows 900 entries in accounts, a
> little over 27,000 transactions with 63,000 splits overall. As compressed
> XML, it’s just over 4MB. My data file is larger than Cricket’s proposed
> import, and I am not seeing problems on my setup. Of course, that all could
> change when I migrate to different machine and OS later this year...
>
> My point is that your experience, while real and a valid data point, is
> nonetheless your experience, and not necessarily the norm. My own
> (similarly singular) experience is starkly different from yours. I think
> the overall user experience is somewhere along the line between these two.
>
> FWIW, my import process from Quicken (way back when) took a while to work
> out, but I eventually was able to import the entire QIF file (account
> structure and all) directly into GnuCash. My method was to export the QIF,
> import into GnuCash, see the gaps and failures, go back to Quicken (where
> my familiarity allowed me to fix things quickly), change the Quicken data
> to address the gaps, and repeat until such point as the incoming data was
> clean enough to move forward.
>
> David T.
>
> > On Apr 27, 2019, at 3:20 AM, David Carlson <david.carlson.417 at gmail.com>
> wrote:
> >
> > Cricket,
> >
> > One more point...
> >
> > Strongly consider only moving one or two years data to GnuCash.  When
> your
> > file gets large you may prefer to have a new file for each year or
> whatever
> > to keep GnuCash from bogging down and running too slow.  It gets worse
> > quickly if you leave a lot of reports and register windows open, or if
> you
> > have a Hi-Res monitor, and having a powerful computer is not enough to
> > prevent that.
> >
> > In my case it takes over a minute to open my file and about 10 seconds
> > every time I press the Enter key.  the periodic saves take so long that I
> > can squeeze in about 1/2 a game of Solitaire.
> >
> > David Carlson
> >
> > On Fri, Apr 26, 2019 at 4:31 PM Cricket Onebit <
> cricketbeautiful at gmail.com>
> > wrote:
> >
> >> On Thu, 25 Apr 2019 at 17:29, Adrien Monteleone <
> >> adrien.monteleone at lusfiber.net> wrote:
> >>
> >>>> I'm preparing to convert 14,000 transactions from Quicken. It's a bit
> >>>> overwhelming, and I've put it off for years.
> >>>
> >>> Sounds like a daunting task. I don’t do such imports, but reading the
> >>> various threads over the years, this will be a doozy.
> >>>
> >>
> >> Yes, it's a daunting task, but I do't want to lose 10+ years of data.
> I'm
> >> using an old version of Quicken, and don't trust the new version to be
> able
> >> to read it. The old version might not work on newer Windows. I need to
> move
> >> it.
> >>
> >> I’d try finding some of those threads or asking about a general ‘best
> >>> practices’ or ‘workflow’ to accomplish this with the least amount of
> >>> headache.
> >>>
> >>
> >> There doesn't seem to be an easy way to find those threads. I might not
> be
> >> searching right. "site:lists.gnucash.org quicken" has posts from many
> >> years
> >> ago on the first page, but not any from 2019. I know there are some from
> >> 2019.
> >>
> >> Two gnucash.org pages need to be updated. They have broken links.
> >> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> >> https://lists.gnucash.org/search/?idxinfo=gnucash-user
> >>
> >> Yes, you can do this at will, over and over till you have your tree the
> way
> >>> you like it. (and then re-factor it again months later when you change
> >> your
> >>> mind)
> >>>
> >>
> >> Yay! for easy moving / promoting / demoting / etc child / parent /
> sibling
> >> / cousin accounts. Both your suggested methods make sense.
> >>
> >> One less reason to put off the migration.
> >>
> >> Also, for this particular example, I’d opt for tags for each kid
> instead of
> >>> accounts, see below about Cars.
> >>>
> >>
> >> I didn't think GNUCash supported tags. I plan to use #tagone in a text
> >> field. That's one of the things I need to do in Quicken. Search for each
> >> transaction using a label, and put it in a text field. Problem: The text
> >> fields are short. Maybe I'll create accounts/categories for each tag,
> and
> >> add a $0.01 line. That will at least preserve the info. Then in GC edit
> >> again to put the tags in a text field and undo the $0.01 split. (Not $0
> >> because I think Quicken ignores $0 lines.)
> >>
> >> I’ve not found one yet, though that may be in the documentation. Look
> into
> >>> Double-Line mode to give you an extra note field. Once discovered, I
> now
> >>> use it extensively.
> >>>
> >>
> >> I like that GNUCash makes it easy to switch between journal, register,
> and
> >> split views.
> >>
> >> The #1 piece of advice I could give to anyone who even thinks they might
> >>> want detail or multiple levels of categorization is to record as much
> >> info
> >>> as possible in each transaction. Record each split line from a receipt
> >> and
> >>> never combine items. Be descriptive for each split. That way, in the
> >>> future, should you decide to break something out into its own account,
> >> the
> >>> process will be much easier. But if you don’t have that info in the
> first
> >>> place...
> >>>
> >>
> >> I've annoyed many people with data. It runs in the family.
> >>
> >> "Nothing ruins an argument like a fact." -- RM Thomas, P.Eng. (aka Dad)
> >>
> >>
> >>> On the reporting end, if you really need to have parents with their own
> >>> transactions (instead of just as placeholders), set your options like
> >> this
> >>> on the P&L (Income Statement):
> >>>
> >>> Options > Display > Parent account balances > Account Balance
> >>> Options > Display > Parent account subtotals > Show Subtotals
> >>
> >> ...
> >>
> >> Parts` has one break-out sub-account with the remaining transactions in
> >>> itself. So that one shows its own balance, and the `Total Parts` line
> >>> combines it with its sub-account.
> >>>
> >>
> >> Yay! I wasn't looking forward to creating "misc" (grand) child accounts
> for
> >> every parent and moving the transactions. The GC report is similar to
> the
> >> Quicken report.
> >>
> >> Does it also do that for a Transaction, Group by (sub-)account report? I
> >> use that one a lot. There's probably something close enough.
> >>
> >> +++
> >>
> >> Enough stalling. Today's goal is to replace all the investment accounts
> >> with basic accounts. Apparently migrating investments isn't reliable. I
> >> won't lose any data because I haven't tracked investment details in
> Quicken
> >> in years. Also create appropriate income/loss accounts for the
> investment
> >> accounts so the year-end balances work. And, of course, lots of pre/post
> >> change reports to catch problems.
> >>
> >> Wish me luck!
> >> _______________________________________________
> >> gnucash-user mailing list
> >> gnucash-user at gnucash.org
> >> To update your subscription preferences or to unsubscribe:
> >> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> >> If you are using Nabble or Gmane, please see
> >> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> >> -----
> >> Please remember to CC this list on all your replies.
> >> You can do this by using Reply-To-List or Reply-All.
> >>
> >
> >
> > --
> > David Carlson
> > _______________________________________________
> > gnucash-user mailing list
> > gnucash-user at gnucash.org
> > To update your subscription preferences or to unsubscribe:
> > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > If you are using Nabble or Gmane, please see
> https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
> > -----
> > Please remember to CC this list on all your replies.
> > You can do this by using Reply-To-List or Reply-All.
>
>

-- 
David Carlson