reporting, again aka python for people (was: History Quotes)

[Sébastien de Menten]

On Sat, Nov 14, 2015 at 3:35 PM, Wm... <tcnw81 at tarrcity.demon.co.uk> wrote:

> Sat, 14 Nov 2015 09:25:14 <CAB2pxDszjd=
> 8paSWti5OjXKwZxK_99j8Mod6P9bHpX5P_29fJg at mail.gmail.com>
> Sébastien de Menten <sdementen at gmail.com> wrote...
>
> Wm:
> > P.S. Sebastien: I've been meaning to get around to writing to you
> > about piecash and pandas and reporting because I'm seeing lots of
> > potential there, but that is another thread.
>
> [same thread, change of Subject, if it is new or not depends on your
> threading model]
>
> P.S. Wm : I was just thinking on working a bit today on exactly this !
> Pandas would be a boon as base for some reporting/charting once we can get
> the account/transaction info combined/joined with the price info.
>
>
> ===
> Some thoughts:
>
> at the moment (I think) piecash can only talk to GnuCash's data if it is
> in SQL.  How much work would be involved in getting piecash's SQLAlchemy to
> read [1] XML?  Do you think the effort would be worth it?  My concern is
> that even if piecash or similar is the route to reporting freedom there
> will still be some people that will resist changing file format even if
> they are told "it is all sql inside!"
>

piecash is based on sqlalchemy to connect all tables together and take care
of the sql transactions (commit, rollback, etc). Adapting it to XML would
mean rewriting all the GnuCash logic to manage SQL transactions (which are
free with sqlalchemy) => not sure it fits the approach.
Would you see specific issues with going sqlite vs XML ? for most of users,
it is rather transparent (except if the user wants to process XML but then
piecash is not relevant). Going postgres or mysql is another story and is
more relevant for advanced users wanting to access their gnucash books from
multiple locations in a 'server' mode (vs a 'dropbox' mode).


at the moment I am playing with pandas and my GnuCash data using Conda and
> Ipython, are you using something else?  My thought is that if we want other
> people to try this out we need a relatively easy path for installation and
> use, ideally something that can be scripted.  Anaconda is looking like a
> good base to me, I'm just wondering if you have come across a better
> alternative in your exploration.
>

I am also using conda as it allows hassle free linux and windows python
installations (maybe mac but I have no mac).
As a side note, last week, I have started CI (continuous integration) on
AppVeyor to run the piecash test suite on windows machine (on linux, I was
already using Travis). It works fine and AppVeyor use virtualenv so it
should also be possible to use a stock python installation for piecash with
plain "pip install" command.

what is the sticking point on the account/transaction info from your point
> of view?  if it is with regard to 2.6.9 not saving the prices table that is
> temporary and can safely be disregarded by anyone with a time horizon
> longer than the end of next week.
>
What I meant in my previous email is that it would be good to be able to
take all splits (with their transactions and accounts information) and
value them with the commodity or currency prices to build a pandas
dataframe with most of the information needed to do reporting.


> overall we should bear in mind that just because you and I and some other
> people can talk SQL and see reporting freedom in that it is worth a piece
> of shit in a gutter if we can't put it together in a way that most people
> can use; in plain terms if we can't make it simple there will always be
> people asking for the ever decreasing number of people that understand
> scheme to write a report for them.  That way, in the long term, GnuCash
> dies [2]
>

The idea behind piecash is that, for most of the needs, there is no need to
talk SQL nor to understand all the tables logic. With just plain python and
a knowledge of GnuCash object model, one can already do reports, generate
documents (invoices), change the data (clean it, remove duplicates, import
data, ...).
As one can see in the example in
https://github.com/sdementen/piecash/blob/master/examples/ipython/pyscash_session.ipynb,
there is no SQL or Gnucash table knwoledge.

I think GnuCash is an excellent GUI to work with books and start actions
(reporting, invoice generation, import/export data). However, these actions
have to be written in C or guile which are two languages relatively
difficult for beginners to work with (vs python). The gnuCash python
interface is not a full replacement of guile and is also complex to work
with (SWIG) and not that pythonic.
piecash is an alternative to "script" these actions in python. piecash
could be even more useful if it was callable from the GnuCash GUI (ability
to use it for reports or ability to call piecash python script from the
GnuCash menu).

===
>
> if you're reading this and wondering what on earth we are talking about
> take a look at this
>
> https://github.com/quantopian/pyfolio
>
> that and loads more comes for free once pandas is used to leverage the
> GnuCash data you already have.
>
> ===
>
> [1] to avoid confusion I suggest we concentrate on read access for the
> moment.  I know piecash can write, you know piecash can write but, I think,
> discussing it distracts from the main issue.
>
indeed, writing is another story ;-)

>
> [2] hyperbole alert :)
>
> --
> Wm...
>
> _______________________________________________
> gnucash-user mailing list
> gnucash-user at gnucash.org
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -----
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>