Reporting system and potentially Python

[John Ralls]

On Jul 8, 2011, at 10:24 AM, Tim M wrote:

> 
> 
> On Fri, Jul 8, 2011 at 8:44 AM, John Ralls <jralls at ceridwen.us> wrote:
> 
> This is in general a good plan, but you need to know that there's a reason for the engine code to be between the storage backends and the GUI and reporting system: It's the part that knows how to manage currencies and lots and invoices and such. In database applications that's called the business logic layer. Now it's true that a lot of what is in our engine belongs in the  database layer (the backend), but there's also a lot that can't go there because databases don't know how to do it. Fixing that is another long-term project.
> 
> Reports, the GUI, and even the command-line are called the presentation layer. The presentation layer has to pass all of its requests through the business logic layer to ensure that all of the accounting rules are applied correctly; nothing but the business logic layer should talk to the database layer. That's the theory. The reality of Gnucash is that the engine API is so confusing and inflexible that the new DBI backend has tempted people to try querying the backend directly with SQL so that they can get the reports they want without having to spend the time to learn the engine's API.
> 
> Geert mentioned the QofQuery class. I'm concerned that it's both too limited in function and too low-level (it's used by the engine to query data structures, so it may not apply business logic to its results) to work as a direct interface for a report service. It's worth studying, though, because modifying it might be the shortest path to getting a workable query interface.
> 
> Regards,
> John Ralls
> 
> 
> John,
> 
> Thank you for the clarification.
> 
> I think we are on the same page.  Any sort of "web service" would act just as an interface to the outside (the Report system, or other third-party requests to the GnuCash web service) and then internally it would perform queries against the GnuCash DB via the engine code.  I never had any intention of querying the DB directly - I only intended to wrap the engine code (whether it be QofQuery or whatever else is appropriate).  The web service methods would only call the necessary engine code to query the data, and then wrap the results into the XML output and send it back to the client.
> 
> Are you saying that it may be a problem to implement a web service to query the database using the engine code to generate the reports this way?  Isn't this what the scheme reports do now?  Otherwise how do they access the data to generate the reports?  I presume they do not access the DB directly but use the engine code to do it.
> 
> As I see it, the web service should work more or less as follows:
> 1. A Report or other service issues a SOAP/REST request to the GnuCash web service.  The request must adhere to the GnuCash WSDL (anything not adhering to it will be ignored).
> 2. The GnuCash web service receives the request.
> 3. The web service queries the database using the GnuCash engine code.
> 4. The web service wraps the results from the engine code into an XML wrapper.
> 5. The web service sends the XML output back to the client.
> 6. The XSLT translation occurs on the client to display the data.

Tim,
You confused me with the line
>>> I think it could be implemented as just another interface to the data such as we have XML and DBI currently
Those are the database or storage layer interfaces that engine uses for persistence, not interfaces that presentation layer code should use.

My concern with WSDL is that each Scheme report is a function which calls engine functions. There's no generalized query language or query interface AFAICT. I don't know anything about building a WSDL, but if it's like other query languages (e.g. SQL) it seems likely to me that there will be a pretty bad impedance mismatch in some areas. 

Also, as you've laid out the flow, it seems a bit heavyweight for the internal report generator. Rather than having to format an http request and setting up an IP listener in Gnucash (and consequent (x)inetd setup), wouldn't a socket interface that both the Report subsystem and the web server connect to?  Simplifying the report side further, once we have the code to handle general query strings and return XML, that interface can pretty easily be provided both as an API (returning a DOM tree to save writing/parsing overhead) and a socket interface for a SOAP/REST server -- or anything else we dream up later -- to connect to.

Regards,
John Ralls