Module system and source tree reorg

[Michael Fair]

On 25 Jul 2001 10:46:30 -0500, Bill Gribble wrote:
> General principles:
> 
> 1. Modules are the basic organizing unit for the code.  All the code
> needed for a module goes in one place, separate from the code for
> other modules.  In my source tree, this is in subdirectories of
> src/modules/ but eventually it's my goal that *all* of gnucash should
> be in modules, so maybe we should just put modules in top-level
> subdirs of src/
Only if they are broken down into there catagorical areas via
a sub-heirarchy (like the 2.4 lib/modules heirarchy.
The system should be able to intuitively (or programmaticly)
access different pieces of the system.  The more things are broken
down, the less overwhelmed the end user gets.  

I imagine seeing something like (this is of course incomplete):
gui/linux/gnome-version/
gui/linux/kde-version/
gui/linux/ncurses-version/
gui/win98/
gui/win2k/
gui/macos9/
gui/macosX/
backend/storage/xml
backend/storage/sql
backend/storage/qif
external/transactions/some-xaction-standard
external/transactions/some-other-xaction-standard
external/stocks/web/yahoo/
reports/accounts/receivable
reports/accounts/payable
reports/accounts/chartof

> 2. A module should be self-contained.  All the stuff needed to build
> the module, including C code, Scheme code, g-wrap specs, g-wrap helper
> files, tests, and documentation should all go in one place.  For example,
> the "engine" module looks like this:
> <layout description snipped>

The layout you recommended looked fine except for I think we should 
create somewhat of a standardized template for what a module directory
should look like.  This will allow people to take the "hello-world"
module and modify its structure in a way that is consistent with other
modules.  It's also easier to start with something rather than scratch,
and if we provide programmers a template to modify odds are higher that
they will take the time to correct the "wrong" information (like the
README and documentation)and replace with something more specific to
their needs.

For instance a good start might be:
/src/module-catagory/module-name/
/src/module-catagory/module-name/README
/src/module-catagory/module-name/configure
/src/module-catagory/module-name/Configure.in
/src/module-catagory/module-name/Makefile.in
/src/module-catagory/module-name/src/sourcefilesandothersubdirs[.c|.h.xxx]
/src/module-catagory/module-name/Documentation/docsandstuff.html

Hopefully, the Help system can be made to incorporate the documents
in the Documentation directory straight into the Help index.  Or
perhaps a separate "Help-Files" directory in the modules template
is in order while the documentation directory  is more code base
related.

There should also be some attention paid to error reporting.
Any errors generated by a module MUST identify the module name
as part of the error (maybe not to the UI, but in a log file
somewhere or something).

We also need to consider how versions get incorporated into the
structure.  It should be possible to have multiple versions of
the same module installed at the same time to help in transitions
and upgrades.  for instance, module a upgrades then module b takes
some time to change its interface to use the new module a structure.
It would be best if module b could continue to use the existing 
module version.  There should also be a flag as to either
A) What version all modules in this instance use, or B) whether
or not this module allows for different concurrent verions.

Just think "shared library upgrades" and you'll see the rats nest
I'm trying to foresee.

> 3. Modules should be separately compilable and distributable.  ATM my
> source tree depends on a top-level configure script, but I think that
> should possibly change (at least for non-core modules). It would be
> really nice to get a bunch of contributed "plugins" a la Gimp.  Right
> now it would be a little awkward to write a C-only plugin that did
> anything useful without linking against the engine library, but it
> could be done, and it's easy to write a Scheme-only plugin that does
> useful stuff without any gnucash libraries available at compile time.
> In the future we will probably want to have a gnucash-devel package
> that allows third parties to write and build C plugins outside the
> gnucash source tree.
I'm not sure why linking against the engine library is a problem,
it would seem to me that if the engine library wasn't available
you wouldn't have much of a GNUCash system at all.  Furthermore,
if you didn't need any of the engine functionality (though I find
it difficult to imagine any such module as the financial engine
is kind of the whole point of the project) then you wouldn't
have to link against and its availability wouldn't be an issue.


> I'm a little unclear about how to use the naming conventions; at the
> moment, each module registers itself with an arbitrary string for a
> name, and that's what clients ask for with module-load.  I've been
> using "/" as a path-component separator in names with the idea that a
> GUI would display available modules in a tree using these path
> components, but I sort of like a flat structure in the actual source
> tree.
Do not even consider a flat directory src structure.  Think of the /dev
directory on your system, and then imagine what our systems would look
like if the linux kernel was organized as a flat heirarchy.  If
anything, I think the src directory structure should exactly match
the module heirarchy.  Perhaps one layer of indirection to allow for
"pretty printed names" to map onto "uglynameddirectories".

The heirarchy creates an inheritant organization that a would-be coder
can pick up and go to the code he is most interested in very quickly.


> Also we want to break out the dependencies a little; for example, the
> QIF i/o gui probably shouldn't be part of an omnibus GUI module,
> because the QIF gui will depend on the non-gui parts of QIF i/o, and
> it's silly for the basic GUI to depend on QIF i/o being loaded.  That
> would argue for a separate gnome-gui-qif-io module.  Opinions?
I think the qif i/o should go into some kind of 
"external/backend interfaces" catagory.  Think about the guy who
wants to use GNUCash to directly manipulate his Quickbooks databases.
i.e. he runs Quickbooks on windows, GNUCash on Linux, and wants to
share the same backend file.  With the QIF I/O module that should
be possible (assuming a high enough quality module).


Other than that, I really can't say much in the way of the current
heirarchy structure for the code, I only know that more organization
is always better.  If after doing it for a while we realize there is
redundant separation, we can remove it.  It's a hell of a lot easier
to merge to separate things together by removing a separation than
it is to try and separate things out later down the road.

-- Michael --