generating reports takes minutes

[Derek Atkins]

Quoting Thomas John Vitolo <tjvitolo at bu.edu>:

> But so what?  It's true that a user could add data that was dated 
> prior to the
> "end date" of a report, but it wouldn't be hard to include a binary 
> indicator:
> data has changed since this report generated, or not.

But how would user B know when user A last ran gnucash?  We definitely
do NOT want to write out the data file every time you run gnucash.. And
as I said before, reports are stored per user -- user A has no idea what
reports user B has.

> For me at least, the charts are extremely useful, but not every time I use
> gnucash.  In fact, they're useful maybe 1 in 10 times.  The other 9 
> times all I
> want to do is add data, and it's foolish for me to have to wait a few seconds
> for my accounts to be loaded and then minutes for the charts to be 
> created, when
> all I want to do is manipulate the data in the accounts.

Then don't keep them open.  Change the report name, click on "Save Custom
Report", and then you can run the report by hand when you want to see it.

> After all, the charts don't automatically get re-generated every time 
> I change
> data in an account, so why should they be re-generated every time I open
> gnucash?  Conceptually, there's no reason to "believe" the charts are 
> accurate
> if the gnucash application is open, since the data may have changed and the
> charts not reloaded.  So, why should this be different when I open 
> the application?

That's a good question.   I'm afraid I don't have an answer for you
other than "that's the way it works".

> P.S. I do run gnucash locally.  As for the bottleneck time, either I wasn't
> clear with my question or my understanding isn't clear.  The "Accounts" tab
> loads in a few seconds, but the charts take minutes.  It seems to me that the
> requirements for generating a chart are (1) to aggregate the 
> appropriate subset
> of the "Accounts" data necessary for that particular chart, and then 
> (2) the GUI
> to actually draw the charts.  I wonder how much time is spent in (1) 
> and (2) for
> any given plot.  If most of the time is spent in (2), than my 
> proposal doesn't
> help anyway.  If, however, most of the time is spent in (1), which I 
> suspect to
> be the case as users generate more data, than my proposal of 
> "caching" the data
> would substantially improve the time required to open the application, at the
> cost of initially generating plots that aren't known to be correct 
> (which is the
> case at any other time when the application is open, since the data may have
> been changed by the user but the plots haven't yet been manually 
> reloaded by the
> user).

It depends on the reports.  Each report is implemented differently.
In GENERAL there are the following steps to run a report:

  1) Run a query to obtain a set of transactions
  2) iterate over the set of transactions and compute something.
  3) generate the output
  4) render the output (GtkHTML/GoG)

We don't really have a good profiling system in place so I dont think
you could easily measure how much time is spent in each of these
steps.   I'm fairly sure that step #4 is quick, the time would be
spent in #1,2,3.   how much time is spent in each step would probably
depend on:

  Your machine (CPU/RAM)
  How many accounts and transactions in your data file
  The algorithms implemented in the report itself.

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord at MIT.EDU                        PGP key available