optimizing reports

[Josh Sled]

Andrew Sackville-West <ajswest at mindspring.com> writes:
> I suppose this raises the larger issue of what is going on long-term
> with the reports. Is the amount of work necesary to fix the
> performance issues sufficient to warrant looking at the long term goal
> with reporting with an eye to just doing that change now? I know
> Derek wants to work e-guile into the mix and implement some kind of
> templating. Others have suggested binding in a whole 'nother language
> for reporting. I don't know the answers and lack the knowledge,
> experience and position to answer them.
>
> Should the reporting structure be re-worked altogether? now or later?
> Should the current structure be cleaned up in parallel to that effort
> to improve performance in the interim? Or should it be left as it is,
> just improved?

(What follows are excerpts from some notes I've been working on on the
subject.)


Reports have some problems:

- defined in scheme, hard to modify, weird errors, &c.
- emit HTML, no conventional templating
- bad html, no css
- name-based identity
- options are stored as scheme code
- strange to add new report types
- report options are inconsistent
- report options dialogs are not HIG-compliant

In particular, the storage of report options and saved/open reports as
evaluated lisp expressions tightly couples us to a particular technology for
the reports.

The rough form of a revised reporting infrastructure I'd like to see is:

- reports declared as data, rather than registered as code.
- separation of the "report generation" code from the "report rendering"
  code. something like:

     book
      v  ------------ generator, report-provided
report-model  (dict)
      v  ------------ renderer (template application)
report-output (html)
      v  ------------ HTML engine, GOG
   display
      v
{screen,file}

Where the report generator specifically emits only a data structure (language
neutral, dictionary-list-string-number-boolean, &c.) that is the input to the
rendering phase.  The separation supports separation of concerns, layering,
independent evolution and development/testing.


I think a report might be well-suited to be a "bundle" ... some structured,
self-contained collection of files.  It is a ${format} archive with a
known-location "manifest" file "report.def", which contains the basic report
definition.  By example:

foo.tar:
    - report.def
    - report.script
    - template.html
    - local.css

report.def:

    [gnucash-report-v2]

    name = Foo Report
    desc = The ISO-1234-26b "foo" report.
    
    id = 0a1b2c3d4e5f6a7b8c9d0e1f

    parent = assets-expense

    report-type = scm
    load-files = report.asl, helper.asl
    options-entry-point = foo-options
    generator-entry-point = foo-report
    renderer-entry-point = template_apply

    template-file = template.thtml

    name.fr = Foo Réport
    desc.fr = Lé réport du generallies foo...

A goal would be to have something like ~/.gnucash/reports.d/, where a user
could publish a new report (as that single archvie), and other users could
simply save it there and have it appear in their gnucash instance.


On the front-end, we should move to a more "normal" generation scripting
language (perl, python, ruby) and template-based rendering solution.  That
should be consumed by gecko, not gtkhtml, as it doesn't suck.

I could see a time where we are in transition, and have both "v1" (existing)
and "v2" (proposed here) reports co-implemented.


With respect to the Options, we should convert the options implementation
From scheme + closures to C + GObject/GInterface + signals.  The existing
saved/open reports are all basically of the form:

  (let ((optionDb (report-default-options report-name)))
     (set optionDb 'optionA "new-value")
     (set optionDb 'optionB "new-value")
     (create-report report-name optionDB))

As such, we strongly require a guile interpreter to parse/eval all this.  As
the options are moved into C, they'll need bindings to support at least
handling these existing files, but should then serialize back into a
non-guile-specific format.

I've started down this path on a private source tree.

-- 
...jsled
http://asynchronous.org/ - a=jsled; b=asynchronous.org; echo ${a}@${b}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
Url : http://lists.gnucash.org/pipermail/gnucash-devel/attachments/20071010/b6a712e5/attachment.bin