IRC discussion on i18n, xml/utf8, and 1.8->2.0 data migration issues

[Chris Shoemaker]

On Fri, Feb 03, 2006 at 11:28:43AM -0500, Derek Atkins wrote:
> Chris Shoemaker <c.shoemaker at cox.net> writes:
> 
> >   I have some comments/proposals that are related to the 1.8<->2.0
> > compatibility issue and slightly related to the encoding issues.  So,
> > I'll just jump into this thread.
> >
> >   GnuCash 1.8 is pretty forward-incompatible, in the sense that it's
> > not easy to keep the format backward-compatible.  Aside from
> > encodings, it just bombs on unrecognized xml elements.  :(
> 
> So was 1.6..  New-in-1.8 features would cause your data file to become
> unreadable in 1.6.
> 
> >   To solve thing for the future, I propose that we let 2.0 at least
> > _try_ to continue reading a file that contains unknown elements.  That
> > way, if we're smart about how we extend the format, it will be
> > backward compatible.
> 
> You can read them, but then what?  You would need to /remember/ them
> somehow so you could put them back into the data file.. . Otherwise
> you just corrupt the data going back.

There's no way I'd expect the old version to _preserve_ new data-types
added by newer versions.  But IMO, _ignoring_ new data-types and
remaining usable is better than just bombing.

> Also, what happens when an unrecognized tag really has some core
> meaning about the object.  For example, let's say that there weren't
> a void flag in transactions, but in the next version we implement
> transaction voiding -- if you go back in time you lose the semantic
> meaning of the flag, or WORSE, destroy that flag.

Sure, if you make non-backward compatible changes to the file format,
you need to reflect that in the format version.

> It's HARD to do this..  Unless we kept the data in the DOM tree and
> modified it in-place it would be REALLY HARD to not lose data when we
> go back and then forward again.

Of course.  That's WAY too hard.  But that's not what I was
suggesting.  Currently, it's _impossible_ to change the format
_at_all_ without breaking old versions.  I'm just suggesting we should
at least make it _possible_ to make backward compatible changes.

Budgets is a perfect example.  There's _no_ good reason why a
pre-budgets version of gnucash shouldn't be able to open a data-file
from a version of gnucash that supports budgets.  Of course it can't
preserve budget data, but it should at least work with the data it
knows about.

If we _ever_ want to make any backward-compatible changes, we should
fix this.

> >   Unfortunately, that doesn't help for 1.8.  For that, I propose a
> > "save-as" or "Export" option that will be _sure_ to save a file that
> > can be opened with 1.8.  For 1.8 compatibility that file would have to
> > drop new features (e.g. no budgets), and if there turns out to be an
> > encoding incompatibilty, it can also handle that.
> 
> Nah, I don't think it's an issue.  We didn't get a significant amount
> of crying from 1.6->1.8 about data files..  I don't think we'll get
> much on 1.8->2.0 either.

I tend to be pretty conservative about these things.  As a user, if I
know that upgrading the program means that I lose the ability to load
the data file in my older version, it _is_ an issue I think twice
about.  I have certainly avoided upgrading certain financial
applications for _exactly_ this reason.

I'm not saying we _always_ have to be backward compatible.  We should
be free to consciously break compatibility, but I think we should also
have the option of making backward-compatible changes.

> 
> >   Note: 2.0 datafiles are mostly 1.8-compatible, but this option would
> > have to _always_ work.
> 
> I just don't think this is a good idea for the above reasoning.

Well, the 1.8->2.0 is a bit of a separate issue, since the fix is
completely different.  I'm still not quite clear on whether the
encoding issue is solvable.  But personally, I'd feel a bit
irresponsible if we broke backward-compatibility for no other reason
than introducing budgets.  It just feels... rude.

-chris