GRAND MASTER PLAN

Dru andru at treshna.com
Sun Mar 7 19:25:27 CST 2004


>>Database low level abstraction.
>>
>>If we were to all use the same database abstraction layer then it will 
>>make supporting a wide range of databases a lot easier. Also single 
>>library to maintain and debug.  But everyone has their own code here.
>>I've looked at both libdwi and libgda a lot but both still lack certain 
>>minor features that are critical for my code base.  libgda has the low 
>>level stuff but it also has support for all these other non-sql 
>>databases and higher level stuff in there.  libdwi supports less 
>>databases than libgda but is a pretty clean api.
>>What if we ripped out the sql drivers,
>>
> 
> which SQL drivers?

i'm talking about all the providers for sql databases.

>> put a new clean api on it and 
>>told libgda to talk to that api instead of directly as it is now. Modify 
>>all our projects to talk to this new library/project?
>>
> 
> I don't get what you're trying to say here, which API would be used from
> libgda? which SQL drivers would be ripped?

I'm thinking about the libgda providers for sql databases. At the moment 
libgda has everything in it, i think it should be cut up into smaller 
more managable peices.
Heres an example directory layout as follows (assuming every directory 
generated a single library):
libgda_providers -> where you have all the providers.
libgda_dbabs -> common database abstraction layer to talk to providers.
libgda_tools -> All the useful tools you have for libgda and supporting 
code.
libgda_xml -> all the xml stuff
libgda_meagrant -> the object abstraction layer. (GdaDataModel)
libgda -> a library that contains all other libraries to maintain 
backwarsd compitability and the overall api.

My code could link to the libgda_dbabs and libgda_providers initially. 
Reports would properly link to all of libgda.

>>In regard to xml queries, a very useful feature but all my interfaces 
>>are build around sql as is linas. I belive it would be more useful to 
>>have a xml to sql and a sql to xml convertor or hide sql convertor away. 
>>so sql'92 commands can be convereted into native sql commands.  And 
>>proberly have that in the main libgda library.
>>
> 
> there is a fully implemented XML query management thing in libgda. It
> supports most SQL commands, and allows that SQL<->XML conversion. The
> only problem with it is that we haven't used it much, but it should work
> straight on.
> 
I think there is a lot of unforeen work in dealing with making different 
sql dbs compitable with each other.
> 
>>Data abstraction layer.
>>
>>This layer that deals with dynamic data sets. Doing the write backs and 
>>saves of data.  There is libgda, bonddb, gnueappserver, qof. Dont touch 
>>gnueappserver, i wrote it and it suxs. I dont think we can look at 
>>working together on this one until we sort out the low level abstraction 
>>db.  How far are you down the track for qof linas?
>>
> 
> bonddb uses native PostgreSQL libraries for psql access, IIRC. QOF, IIRC
> what Linas said, uses a XML backend. So, and sorry for saying it myself,
> the choice here seems to be libgda. I am talking just about the low
> level data access stuff, not about the extra features bonddb and qof
> provide.

bonddb use a bonddbwrapper library which uses postgrsql. I can replace 
the bonddbwrapper library with libgda.
Data abstraction layer i'm talking about is the bonddb DbObject, linas 
qof stuff, and the GdaDataModel in libgda. Libgda aims to provide the 
same functionality as bonddb so it makes it harder to intergrate unless 
we refine the apis more.  Low level stuff should be in libgda yes. but 
managing datasets is each has its own way of doing at the moment.

>>Sql parser.
>>
>>I find this is needed for a data abstraction layer. I'm not sure if you 
>>have any sql langauge parsing done in your code linas or if you need 
>>one, but bonddb and libgda are currently using the same code base here.
>>
> 
> right, with some recent improvements from Vivien and yourself.
> 
>>I'd suggest moving libsql into a seperate library/package to make 
>>maintaining it easier or moving it directly into a sql low level 
>>abstraction library.  We should keep the sql parser to the sql standards 
>>or aim for that.
>>
> 
> that sounds good to me, although it means more maintainance, but if
> everybody needs it that way, then it's ok with me.

I think a problem that will come up more than once is trying to maintain 
the same code in multiple code bases.  No one really wants to give up 
control of there code and thats part of the problem. There is more 
admistractive work maintaining in seperate package but less coding in 
maintaining the same code.
Two ways to fix this, someone provide some cunning tool to keep 2 cvs 
trees in sync.  Or I work only of the gnome cvs tree.
Anyway these problems are more political in nature. What if there are 
conflicting ideas on what a library should do, in libsql example what if 
you wanted to impliment feature x with conflicts with feature y i 
want/need for my code.  if the code sits in the gnome-db cvs you have 
the ultimate say in what goes goes in. A seperate package/library would 
give it indepedence from both our projects but then the issue becomes 
having the community to back it. That was why i was thinking a common 
project/organisation for dealing with common code or alligning the work.
It may not be an issue, Its not really an issue for linux kernel and 
they deal with dozens of different branchs/organisations.  But they have 
  someone whos responsible for choosing what goes in and what stays out.
Maybe everything could be packaged and distrubited as one project.


>>Reporting
>>
>>I think we definetly need a common ground here or not write our own.
>>Unforuntly i already wrote my own, papyrus (stealing ideas from gnomedb) 
>>when there was nothing providing reporting, now there are dozens of 
>>reporting solutions out there (half are java/html though).  Linas, i 
>>know how reports are done in gnucash and they are dynamic. A reporting 
>>system should generate pdfs,ps,xml,html etc. And use the database low 
>>level abstraction library. Can gnomedb people use papyrus or another 
>>reporting engine, what is gnomedbs requirements?
>>
> 
> gnome-db can use papyrus, yes, although for that, it would need to use
> libgda, to make things easier to use / integrate.

I can switch papyrus other to libgda pretty easily. Only take a few hours.
The design of papyrus is as follows:
papyrus_backend reads an xml file (vagually similar to your 
format),populates it with data resolving using libgda and does all the 
calcuations and sql statements.  It generates an xml document ready for 
rendering. Once this is done it is then passed to a output engine, 
papyrus_latex, payprus_html etc.  this then creates a pretty report 
based on the populated xml document.
We used packing for placement, not absolute positing. And there 
currently is no colour support (apart from in images of course)

> gnome-db's requirements are not clear, we wanted to provide a "powerful,
> generic reporting system", with the low level stuff in libgda (look at
> libgda/report for the code written so far) and a report viewer / editor
> widget in libgnomedb.

What sort of low level stuff in libgda?   I havn't touched or done any 
report viewer or designer. I used gnome pdf viewer for my report viewer.
Should papyrus be add as a seperate compoonent to the 
libgda/meargent/gnomedb set?  gnomedb would still have to add the 
ability to run reports etc from a gui, i see this as a meagreant bonobo 
feature. People have asked me to code this but i have refrained so far. 
Its a task that better suited for gnomedb.


>>After a quick look here is the different reporting libraries i found.
>>If you know of others that work well suggest them.
>>http://rlib.sicompos.com/
>>(havn't looked into in detail)
>>http://papyrus.treshna.com/
>>(lacks chart and editable content but is mature)
>>
>>Report Designer.
>>
>>None of have anything here (expect gnue).  Its needed. I was thinking 
>>orginally using openoffice or abiword to design reports but i really 
>>havn't started looking at this at any detail. Any suggestions on how we 
>>are going to achive this? Does this sound feasible?
>>
> 
> using abiword for reports seem, as Martin said, feasible, so I'd say
> yes.

There are a lot of complicates to a report writer. I dont want to burden 
abiword with a huge feature set.  I dont think we should aim above mail 
merge data fields of what current word processes have.


  > right, if we all use the same base technology, each app can perfectly
> concentrate on its specific features, so mergeant would be the DB admin
> tool, GNUCash will provide the accounting/financial libraries/
> components, bonddb/qof will provide the RAD thing, etc
> 
> Also, as some people have mentioned, it would make it really easier to
> have all that data available to Gnumeric, Abiword, etc, for reporting,
> mail merging, etc, etc

Use libgda with there gdadatamodel objects to make it avialible to 
abiword/gnumeric. I think this should be seperate to reporting. And have 
mergeant/or some other tool/ provide a gui for doing reporting. 
Providing something similar to crystal reports.

>>The Politics.
>>
>>Do we all fall under gnomes ?
>>
> 
> I am, although, of course, I've got nothing against making all of these
> cross-platform.
I was talking more of a leadership role.


More information about the gnucash-devel mailing list