# report in python, a first version

Sébastien de Menten sdementen at gmail.com
Tue Nov 15 12:24:05 EST 2016

On Tue, Nov 15, 2016 at 4:17 PM, John Ralls <jralls at ceridwen.us> wrote:

>
> On Nov 14, 2016, at 10:43 PM, Sébastien de Menten <sdementen at gmail.com>
> wrote:
>
>
> On Tue, Nov 15, 2016 at 5:37 AM, John Ralls <jralls at ceridwen.us> wrote:
>
>>
>> On Nov 14, 2016, at 7:05 PM, Sébastien de Menten <sdementen at gmail.com>
>> wrote:
>>
>> On Nov 15, 2016 1:27 AM, "John Ralls" <jralls at ceridwen.us> wrote:
>> >
>> >
>> > > On Nov 14, 2016, at 1:31 PM, Sébastien de Menten <sdementen at gmail.com>
>> wrote:
>> > >
>> > > Hello,
>> > >
>> > > I have finished a first version of a mechanism to use python for
>> reporting
>> > > in gnucash.
>> > > It uses a scheme script to start a python process that does the heavy
>> > > lifthing.
>> > >
>> > > Should you be interested, you can find documentation/instructions
>> here:
>> html#report-creation-linux-and-windows-python-3-5
>> > > as well as an example of the simplest report possible
>> html#a-simple-report
>> > >
>> > > Any feedback appreciated.
>> >
>> > Sébastien,
>> >
>> > Did you decide to not use the facility in src/optional/python to start
>> a Python interpreter inside of GnuCash because you didn't know about it,
>> because it was easier to do something in Scheme, or because that
>> interpreter can't interact with the existing (Scheme) report facilities?
>> >
>> > Regards,
>> > John Ralls
>> >
>>
>> interpreter. Any link pointing to doc by any chance ?
>> I wanted something that could leverage the "report options" framework
>> (this is purely in scheme, correct ?), could be hot plugable on gnucash (no
>> recompilation) and rather generic (the subprocess can be any specific
>> python intepreter, usually one in a virtualenv,  but this could be adapted
>> to launch any subprocess ... perl ? node.js ?).
>> I therefore hacked a way via scheme reusing logic found in the quote
>> updater to spawn external processes.
>> If anything can be simplified/better designed, I am in.
>>
>>
>> Sébastien,
>>
>> Sorry, wrong directory. The regular python bindings are in
>> src/optional/python; the embedded interpreter is in src/python.
>> I don't think there's any docs beyond whatever comments are in the
>> module. History says it was a drive-by contribution 5+ years ago that
>> Christian polished up a bit and committed.
>>
>> Options, including report options but not preferences, are a horror-show
>> that bounces back and forth between C and Scheme. The C bits are obviously
>> swigged so that Scheme can see them. IIRC (I haven't looked at it in a bit
>> over a year) Scheme passes callbacks to C rather than there being anything
>> directly callable from C, so any Python code that wanted to get at it would
>> have to duplicate the Scheme part of the functionality. There might be a
>> way to get at the Scheme functions via Guile (whose internals are exposed
>> both as C and Scheme), but it wouldn't be a trivial undertaking.
>>
>> Regards,
>> John Ralls
>>
>>
> That's what I "feared" (to dig up all the dirt) and that led me to the
> decision to keep Scheme as an "entry point".
> A path to improve (and generalise) the current solution I built could be :
> - have in config.user the ability to load a report by reading a json file
>
>
> with my_report.json being something like
>
> {
>   "_type": "report",
>   "title": "My simplest report",
>   "name": "piecash-simple-report",
>   "menu_tip": "This simple report ever",
>   "options_default_section": "general",
>   "options": [
>     {
>       "_type": "option-range",
>       "name": "my_number",      "section": "main",
>       "sort_tag": "a",
>       "documentation_string": "This is a number",
>       "default_value": 3
>     },
>     {
>       "_type": "option-date",      "name": "my_date",
>       "section": "main",
>       "sort_tag": "b",
>       "documentation_string": "This is a date",
>       "default_value": "today"
>     }
>   ],
>   "process_win" : "c:\path\to\chosen\python\interpreter\pythonw.exe",
>   "process_OSX" : "/path/to/chosen/python/interpreter/python",
>   "process_linux" : "/path/to/chosen/python/interpreter/python",
>   "process_arguments": []
> }
>
>
> The Scheme function would then load (dynamically if possible or at start
> time if not possible) this file and have everything required to instantiate
> the options UI and start the process by calling
>
> process_arch process_arguments "format of book" "URL of book"
>
> like
>
> c:\path\to\chosen\python\interpreter\pythonw.exe XML c:\...\mybook.xml
>
> To pass the value of the option to the process, the Scheme would have to
> pipe a serialisation version of the option values to the stdin of the
> process (which will unserialise them). Best would be json (currently, I use
> a simple csv format for serialisation) like
>
> {
>   "my_number": 5.4,
>   "my_date":"2016-03-02"
> }
>
> The process would return the result to the Scheme script via pipeing to
> stdout the HTML string.
>
> With this logic, we could plug/connect any process to build HTML report
> but also to build a csv/excel/openoffice report (which path would be
> returned as HTML) without waiting for the complete rewrite of the reporting
> mechanism of GnuCash.
>
> However, I have tinkered a bit with Scheme and JSON but wasn't even able
> to install https://github.com/aconchillo/guile-json ... Some help needed !
>
>
> Sébastien,
>
> That's a bit more "external" than I had in mind. The plug/connect any
> process part is a serious security issue.
>
> I don't understand the point of the json file. Users will want the report
> to display and then to be able to open the report options dialog to make
> changes like selecting a date range or modifying the accounts reported on,
> and expect the report to regenerate when they click "Apply". ISTM to
> accomplish that the report module needs to be in control and to pass to the
> Python report-builder the options parameters. The second thing that won't
> work is using PyCash: The user wants a report on what her book looks like
> in memory, not what it looked like the last time she saved, so the report
> will need to use the python bindings and connect to the current session.
>
> It looks to me from the Makefile.am that building guile-json requires
> Guile-2.0, which you won't have if you're using the Guile in the GnuCash
> Windows installation. You could try to simply copy the four .scm files into
> share/guile/1.8/ and see if it works. It will only if the author didn't use
> any Guile-2-only features.
>
> Regards,
> John Ralls
>
>
On the security risk, it is no different than running the same process
outside GnuCash (in a terminal for instance), or is it ?
The json file is there to be able to specify the options for a report in a
better way than writing Scheme in an scm file. Currently, for a given
python report (report_test.py), I generate the Scheme code wrapping this
python report (report_test.scm) with:
- the proper Scheme code for the options defined in report_test.py
- the proper Scheme code to call the python process (ie "python
report_test.py")
Instead of having to generate scheme code, it would be better to have a
json file per report (report_test.json) that could be used to instantiate
the report in GnuCash.

For piecash, there is no specific issue as the change in the book are
immediately serialised to the SQL backend. However, for the xml backend
(that piecash does not support anyway), the GnuCash file should be saved
before. But nothing forbids someone to use the official python bindings to
connect to the current session and do as you describe. However, how the
python bindings know to which session to connect if there are multiple
files opened at the same time ? and can they open a session in a separate
process while GnuCash has a lock on the file ?