report in python, a first version

[John Ralls]

> On Nov 14, 2016, at 10:43 PM, Sébastien de Menten <sdementen at gmail.com> wrote:
> 
> 
> On Tue, Nov 15, 2016 at 5:37 AM, John Ralls <jralls at ceridwen.us <mailto:jralls at ceridwen.us>> wrote:
> 
>> On Nov 14, 2016, at 7:05 PM, Sébastien de Menten <sdementen at gmail.com <mailto:sdementen at gmail.com>> wrote:
>> 
>> On Nov 15, 2016 1:27 AM, "John Ralls" <jralls at ceridwen.us <mailto:jralls at ceridwen.us>> wrote:
>> >
>> >
>> > > On Nov 14, 2016, at 1:31 PM, Sébastien de Menten <sdementen at gmail.com <mailto:sdementen at gmail.com>> wrote:
>> > >
>> > > Hello,
>> > >
>> > > I have finished a first version of a mechanism to use python for reporting
>> > > in gnucash.
>> > > It uses a scheme script to start a python process that does the heavy
>> > > lifthing.
>> > >
>> > > Should you be interested, you can find documentation/instructions here:
>> > > http://gnucash-utilities.readthedocs.io/en/master/doc/doc.html#report-creation-linux-and-windows-python-3-5 <http://gnucash-utilities.readthedocs.io/en/master/doc/doc.html#report-creation-linux-and-windows-python-3-5>
>> > > as well as an example of the simplest report possible
>> > > http://gnucash-utilities.readthedocs.io/en/master/doc/doc.html#a-simple-report <http://gnucash-utilities.readthedocs.io/en/master/doc/doc.html#a-simple-report>
>> > >
>> > > Any feedback appreciated.
>> >
>> > Sébastien,
>> >
>> > Did you decide to not use the facility in src/optional/python to start a Python interpreter inside of GnuCash because you didn't know about it, because it was easier to do something in Scheme, or because that interpreter can't interact with the existing (Scheme) report facilities?
>> >
>> > Regards,
>> > John Ralls
>> >
>> 
>> I didn't (and still don't) know a lot about this embedded python interpreter. Any link pointing to doc by any chance ?
>> I wanted something that could leverage the "report options" framework (this is purely in scheme, correct ?), could be hot plugable on gnucash (no recompilation) and rather generic (the subprocess can be any specific python intepreter, usually one in a virtualenv,  but this could be adapted to launch any subprocess ... perl ? node.js ?).
>> I therefore hacked a way via scheme reusing logic found in the quote updater to spawn external processes.
>> If anything can be simplified/better designed, I am in.
>> 
>> 
> 
> Sébastien,
> 
> Sorry, wrong directory. The regular python bindings are in src/optional/python; the embedded interpreter is in src/python.
> I don't think there's any docs beyond whatever comments are in the module. History says it was a drive-by contribution 5+ years ago that Christian polished up a bit and committed.
> 
> Options, including report options but not preferences, are a horror-show that bounces back and forth between C and Scheme. The C bits are obviously swigged so that Scheme can see them. IIRC (I haven't looked at it in a bit over a year) Scheme passes callbacks to C rather than there being anything directly callable from C, so any Python code that wanted to get at it would have to duplicate the Scheme part of the functionality. There might be a way to get at the Scheme functions via Guile (whose internals are exposed both as C and Scheme), but it wouldn't be a trivial undertaking.
> 
> Regards,
> John Ralls
> 
> 
> That's what I "feared" (to dig up all the dirt) and that led me to the decision to keep Scheme as an "entry point".
> A path to improve (and generalise) the current solution I built could be :
> - have in config.user the ability to load a report by reading a json file
> (load-report-from-json (gnc-build-dotgnucash-path "my_report.json"))
> with my_report.json being something like
> 
> {
>   "_type": "report",
>   "title": "My simplest report",
>   "name": "piecash-simple-report",
>   "menu_tip": "This simple report ever",
>   "options_default_section": "general",
>   "options": [
>     {
>       "_type": "option-range",
>       "name": "my_number",
>       "section": "main",
>       "sort_tag": "a",
>       "documentation_string": "This is a number",
>       "default_value": 3
>     },
>     {
>       "_type": "option-date",
>       "name": "my_date",
>       "section": "main",
>       "sort_tag": "b",
>       "documentation_string": "This is a date",
>       "default_value": "today"
>     }
>   ],
>   "process_win" : "c:\path\to\chosen\python\interpreter\pythonw.exe",
>   "process_OSX" : "/path/to/chosen/python/interpreter/python",
>   "process_linux" : "/path/to/chosen/python/interpreter/python",
>   "process_arguments": []
> }
> 
> The Scheme function would then load (dynamically if possible or at start time if not possible) this file and have everything required to instantiate the options UI and start the process by calling 
> process_arch process_arguments "format of book" "URL of book" 
> like
> c:\path\to\chosen\python\interpreter\pythonw.exe XML c:\...\mybook.xml
> To pass the value of the option to the process, the Scheme would have to pipe a serialisation version of the option values to the stdin of the process (which will unserialise them). Best would be json (currently, I use a simple csv format for serialisation) like
> {
>   "my_number": 5.4,
>   "my_date":"2016-03-02"
> }
> The process would return the result to the Scheme script via pipeing to stdout the HTML string.
> 
> With this logic, we could plug/connect any process to build HTML report but also to build a csv/excel/openoffice report (which path would be returned as HTML) without waiting for the complete rewrite of the reporting mechanism of GnuCash.
> 
> However, I have tinkered a bit with Scheme and JSON but wasn't even able to install https://github.com/aconchillo/guile-json <https://github.com/aconchillo/guile-json> ... Some help needed !

Sébastien,

That's a bit more "external" than I had in mind. The plug/connect any process part is a serious security issue.

I don't understand the point of the json file. Users will want the report to display and then to be able to open the report options dialog to make changes like selecting a date range or modifying the accounts reported on, and expect the report to regenerate when they click "Apply". ISTM to accomplish that the report module needs to be in control and to pass to the Python report-builder the options parameters. The second thing that won't work is using PyCash: The user wants a report on what her book looks like in memory, not what it looked like the last time she saved, so the report will need to use the python bindings and connect to the current session.

It looks to me from the Makefile.am that building guile-json requires Guile-2.0, which you won't have if you're using the Guile in the GnuCash Windows installation. You could try to simply copy the four .scm files into share/guile/1.8/ and see if it works. It will only if the author didn't use any Guile-2-only features.

Regards,
John Ralls