Reporting system and potentially Python

[John Ralls]

On Jul 7, 2011, at 9:48 PM, Tim M wrote:

> What I'm looking for is this:
> 
> Separate the act of actually gathering the data from the GnuCash database from transforming it for display, so that the output can be made much more interactive and easy to style and make 'pretty' :).  Use a well known programming language to aggregate the data to make it easier to create new reports which output different sets of data.
> 
> I've thought about this a little bit, and here is what I am thinking.  Let me know if this sounds un/reasonable or just plain incorrect.  (Note that I am basing this off of not being familiar yet with how the current scheme architecture actually manages pulling the data out of gnucash, so please correct me where I am wrong or point out the significant flaws)
> 
> 1. Create the 'new' reporting system alongside the existing one so that the reports do not suffer until the existing functionality can be fully replaced by the new system.  After all reports are replaced and working, the old scheme code could be pulled.
> 2. Create a set of libraries and/or use the existing gnucash libraries for querying information from the database.  Based on the discussion, I presume C or Objective C would be best and just avoid Python/Scheme altogether (I am not sure how the scheme system actually performs the data queries right now).  If all the necessary code is already in place, then this does not require any work.
> 3. Using these libraries, create the code for aggregating the data (also in C or Objective C) for each report.  A single prototype report would be OK to start.  Output the report data as structured XML.  The XML data should adhere to an XML schema.
> 4. For each report, create an XSLT file which will transform the data into HTML/XHTML for display.
> 5. A small amount of javascript would be needed to perform the actual XSLT transformation but it would be pretty small.
> 6. Style the page elements using CSS.  Also, I think the jqplot patch that has been made could be migrated in to the new reporting system, as those graphs look really nice.
> 
> 
> I think there are several benefits to this approach:
> 
> 1. Currently reports can only be exported as HTML.  By making the reporting code export an XML data structure, reports could be easily exported from GnuCash as XML (pre-transformation) or as HTML (post-transformation).
> 2. Anyone could write their own XSLT transformation file to display the data differently.
> 3. The actual XSLT, HTML, and CSS code would be very easy to read and modify by anyone familiar with it.  The backend reports would still be in C/Objective C for compiling the data and as such require some coding knowledge, but it would be much easier to start using than the existing scheme code.
> 4. The XSLT and CSS could be very easily modified and applied not only inside of GnuCash, but a user could also take the XSLT and CSS scripts, modify and apply them to the exported XML reports to give it their own look without touching the GnuCash code base at all.  For example they might want to style a report with a company logo or different colors or other graphical elements.
> 5a. In the future, it would be possible to provide multiple different XSLT and/or CSS styles for the reports so users could select the appearance of the reports. (If someone writes additional styles)
> 5b. In the future, it would be possible to allow users to import their own style sheets into GnuCash and apply them to the reports. (if someone writes the code to manage the import and style selection)
> 6. No reliance on Scheme or Python. Minimal Javascript, but that should be handled easily by webkit.

I like the idea of XML output with XSLT to transform it to HTML. That would be a great enhancement. 

Moving away from a heavy-weight general language like Scheme or Python is the right directin, but switching to C is unfortunately a step backward for custom reports. The current Scheme query interface makes it difficult to write a custom query; a C (or any other compiled language) query would make it impossible for anyone who isn't able to recompile Gnucash. Besides, C isn't really very expressive as a query language. Something SQL-like would be better. A query-by-example GUI would be best. I suppose that could be split off as a separate project, though, so that the Scheme query capability would remain  for custom reports while the standard reports have their queries written in C.

Regards,
John Ralls