Finishing the neverending QOF flamewar

[Chris Shoemaker]

FTR, I want to clarify some things:

  I am _glad_ Neil wants to work on QOF.  I think that an external
QOF, of a certain design, and at a certain time, is a desirable thing.

  Relatively speaking, I am _not_ bothered by Neil breaking logging.
I think it was an honest mistake due to some lack of testing, and it
was easy to fix.

  In fixing it, I improved the logging API, without affecting any
existing QOF program.  Not a single message of any type, anywhere, is
logged with my patch that wasn't logged before, nor vice-versa.  There
is simply no technical disadvantage to my change.  It is 100%
compatible with every existing usage with no change, and it makes a
useless function useful.

  If I'm not allowed to make _that_ kind of change, then how can I be
considered a QOF developer?  I have _no_ hope of fixing bugs that
actually _do_ change behavior (like the fact that qof-log silently
drops all FATAL log messages!)  I _am_ bothered by that.

  Why should I be satisfied with that?  Does it serve the interests of
GnuCash users?  QOF users?  I don't think so.

> > I completely agree that Neil deserves to be treated with respect.  I
> > even would say that that respect doesn't need to be "earned" but
> > should be afforded to anyone by default!  But, that needs to be
> > balanced against the benefit to the users of GnuCash.  Neil's recent
> > behavior has been to make rather invasive changes that were not
> > approved of by any other developers, and which broke functionality and
> > introduced subtle data corruption bugs.  Now, he's using the _ruse_ of
> > maintaining library compatibility to claim exclusive ownership of QOF.
> 
> Well, he is trying to assert maintainership over that piece of code, I didn't 
> feel he made any secrets about it.  Furthermore no one has offered to do it 
> in his place.

  I don't have _any_ problem with Neil being the maintainer of QOF.
None at all.  Using a false justification of incompatibility to revert
of peopele's changes is a claim to _exclusive_ maintainership.  I _do_
have a problem with that.

> > So, how would you balance Neil's deserving of respect in this case
> > with the user's deserving a GnuCash whose core is developed by more
> > than one person?  I struggle with this question, so if you have a
> > suggestion I'm interested.
> 
> Well, while it's no magic solution,  I'll try to bluntly list the real issues 
> the two of you have as I perceive them.
> 

> First issue: The GnuCash project didn't make a clear commitement
> that it would maintain QOF as an independent library, with it's own
> release schedule, causing a lack of trust by Neil and other users of
> QOF that their needs are goind to be given due consideration.
> Normally, we'd have to chose between:
> 
> 1-The GnuCash project maintains QOF as an independent library in the
> gnuCash repository (that means giving equal merit to the needs of
> other projects, maintaining a separate build system, accepting
> patches in a timely manner, giving commit access to people who would
> commit frequently to QOF even if they never even build GnuCash,
> etc.) Then we can expect everyone else to sync to our copy.

FTR, this offer has been made by Derek and declined.

> 2-Developpement occurs elsewhere, we are just one of the
> stakeholders, and we sync to "their" copy.

This is pretty unpopular.

> 3-We don't give a rat's ass about cooperation with other projects, we tell 
> those who want to use QOF good luck, take it, we never want to hear about you 
> again.
> 
> Normally those three things would be mutually exclusive.  Lucky for us, we 
> also seem to have:
> 
> 4-One crazy person commits to keeping two repositories in sync to allow 
> everyone to develop where it's natural for them to do, giving some of the 
> benefits of 1 and 2 at the same time.  That's a very hard job, and it 
> requires that the participants of both repositories at least agree that QOF 
> is to be considered a self contained, generic library.

... and agree on who has permission to make changes.  I honestly don't
see why Neil thinks it's more natural to develop QOF on his own rather
than in GnuCash's repo, but I don't have _any_ problem with his doing
so.  I just don't think that should mean that I can't change QOF anymore.

   BTW, there is another option.  I think that Neil has placed _way_
too much emphasis on packaging QOF - to the detriment of true
collaboration and sharing of QOF code.  An obvious option is to
continue devloping QOF where it has been developed until it's really
ready to stand on it's own.  And, until then, to sync the QOF code
into the trees of QOF programs.  I _guarantee_ this is less work than
option 4.  And the downside?  One redundant .so for any system that
happens to use two QOF programs?  

In reality even this isn't a downside, because I would expect that the
tree where QOF is developed would be constantly releasing with a QOF
version later than had been copied to the other programs.  And that's
_good_.  It means that QOF can continue to develop with changes that
aren't 100% backward API-compatible because no external programs will
use the newer version until they sync with the development tree.  At
that point, they can adjust for any changes in API.

   _OR_ If Neil really wants those other QOF programs to use external
libraries, he can always package libraries from the development tree.
The point is, QOF is not mature enough to stand on it's own.  It needs
to mature within a development community larger than 1, and
practically speaking, that means that development on QOF within
GnuCash _must_ continue.  

   Furthermore, because QOF is not mature enough to have a stable API,
we must be free to make changes that are not backward compatible.
That means that releasing a library of QOF is no different than
copying the source -- every release may require changes to the QOF
programs.  Any other QOF programs that use an external library should
lock to the one version they have been tested with.

   Personally, I think the external library thing is premature and
creates more problems than it's worth.

> That's a political and philosophical issue.
> 
> Second issue: Keeping QOF usable by other project requires additional work to 
> be performed, delaying 2.0 by some unknown amount of time.  That's an 
> inevitable consequence of the first issue if 1, 2 or 4 is chosen.  

There are different degrees of additional work required by the
different options.  We're certainly not taking the easiest and least
disruptive approach to sharing code with other QOF programs.

> A fork is an inevitable consequence or 3.

Forks are good.  Every commit is the merging of a (small) fork.  We're
all working on different copies of the source, so our developements
are always forks from the common source until we merge them.  Making a
copy of the source to work on is _good_.  Neil has choosen to keep a
copy in such a way that makes it difficult for him to merge his
changes with the common source that has been developing in GnuCash's
repo.  It's completely understandable why he wants to revert
improvements to lib/libqof.  Every change makes more work for him.
Forks are good, and merging is _better_.  When you revert changes
because merging is too hard, it's time to change your tools.

> Third issue: How to handle code quality while QOF undergoes this
> transition.  This is a trust, time and communication issue.  Neil
> may or may not have the necessary skills and experience to maintain
> and sync QOF on his own.  If he does, he's still just a human being,
> and keping large codebases maintained in two places will inevitably
> causes glitches, especially during API changes.  If he doesn't,
> there's still only two things that can be done: replace him or help
> him.

I think I can speak (with great caution) for all the GnuCash
developers in saying that we want to _help_ Neil, not replace him.
But it's very difficult to help him when his development is in
isolation.  Even the infrequent commits to SF CVS are too far removed
from the people who can help.  We _want_ to help - we want to help
_here_, not at SF.

> Fourth issue (that received surprisingly little attention), is exactly where 
> QOF starts and end.  Some features of QOF may not really belong in the core 
> QOF library, conversely some issues may be resolved by moving services into 
> QOF.  That's an architecture issue.

This is a HUGE issue.  It's the MAIN issue why I'm unconfortable with
GnuCash using an external QOF.  It seems that the decision boundary
for what moved to qof was something like "whatever code could be used
by multiple projects."  That's a poor way to build a library.  I think
more than 50% of what's in libqof doesn't belong.  I also thing that
the querying infrastructure and the serialization probably don't
belong together.

> Fift issue is the specific mechanics of cooperation.  How often do
> we sync changes, how large the syncs should be, how do we handle the
> build issues, release schedules, etc.  That's a process issue.  I
> tought a reasonably clear process was outlined, but it may be worth
> writing a step by step list.

I thought the process was: 

1) Gnucash developers can still develop QOF.

2) Neil's in charge of copying code from svn to where ever he releases
libraries from.

3) If we make non-backward compatible changes to QOF, Neil would have
to reflect that in his library version numbers.

One thing that I don't think was hammered out was should Gnucash 2.0
use external QOF.  Neil has asserted that it should.  I'm a bit
uncomfortable with this because of lack of testing.  The dilemma I see
is:

1) We shouldn't release 2.0 with external QOF unless we've tested it
extensively with the version we're depending on.

2) We can't test with external QOF because we're frequently making
changes to QOF that aren't backward-compatible.

The responsible options are either: 

a) Stop internal QOF development, depend on external QOF, test with
external QOF, release with external QOF.

or

b) Continue interal QOF development, build with only internal QOF,
test with internal QOF, release with internal QOF.

Since we have to build, test and release with internal QOF _anyway_ to
support all platforms, and since I don't think Gnucash developers
shoud stop QOF development, (b) seems like an obvious choice.

Note that this _doesn't_ preclude sharing QOF code with other
projects, even in the form of an external library.  It's just a
question of whether or not _GnuCash_ should depend on an external
library for QOF functionality.

> The two of you seem to have spoken or unspoken disagreements on all
> five, and probably most other developers have (or had) disagreement
> on at least one.  There should probably be a discussion of each one
> individually to reach a consensus, and settle with some sort of vote
> if there if one isn't reached.  We can then move back to real
> technical issues in a productive climate.

I think you can tell that most developers are steering clear for now.

It might be because they don't approve of my assertive tone.  It's
probably because they'd rather write code than argue.  So would I.
However, I'm concerned that our continued silence leads to
misunderstanding and a sort of "squatter's rights" about QOF.

Neil's words say that I'm free to develop QOF, but his actions are
that 100% compatible improvements to the API are reverted.

I think it would be appropriate to have some _public_ discussion on
this point.  I'll post a question in a new thread to attempt to leave
some of the baggage of this thread behind.

-chris