IRC discussion on i18n, xml/utf8, and 1.8->2.0 data migration issues

[Derek Atkins]

Quoting Chris Shoemaker <c.shoemaker at cox.net>:

>> You can read them, but then what?  You would need to /remember/ them
>> somehow so you could put them back into the data file.. . Otherwise
>> you just corrupt the data going back.
>
> There's no way I'd expect the old version to _preserve_ new data-types
> added by newer versions.  But IMO, _ignoring_ new data-types and
> remaining usable is better than just bombing.

I completely disagree.  As soon as you save off the data file you've
now lost data.  I do agree that it shouldn't "bomb", but I do not believe
that it should read the data.  IMNSHO It should say "you need a newer
version of GnuCash to read this file" and just refuse to open it.

> Sure, if you make non-backward compatible changes to the file format,
> you need to reflect that in the format version.

ANY unrecognized option is a non-backwards compatible change to the file
format..  And losing that data is even worse.  Imagine if 1.6 did this.
Now a user of 1.8 who has all their business and SX data in the file
loads it into 1.6 which prompty ignores the business and SX data but
doesn't tell the user.  User now saves file and LOSES all their business
and SX data.  User now complains to us.

> Of course.  That's WAY too hard.  But that's not what I was
> suggesting.  Currently, it's _impossible_ to change the format
> _at_all_ without breaking old versions.  I'm just suggesting we should
> at least make it _possible_ to make backward compatible changes.

That's not completely true.  This is what the KVP frames are for.
If you put new data into the KVP frame then previous versions can
read it just fine..  But that makes it much harder on, say, DB schemas.

> Budgets is a perfect example.  There's _no_ good reason why a
> pre-budgets version of gnucash shouldn't be able to open a data-file
> from a version of gnucash that supports budgets.  Of course it can't
> preserve budget data, but it should at least work with the data it
> knows about.

I disagree.  See above.  If you DONT have budget data, or any 
new-in-new-version
features in your data file, then yes, it should be backwards compatible.
But as soon as you get a new-in-new-version feature the file should NOT
be readable by previous versions due to the data loss issue.

Solve the data loss issue and I'll change my opinion, but not before.

> If we _ever_ want to make any backward-compatible changes, we should
> fix this.

I don't see it as a problem.

> I tend to be pretty conservative about these things.  As a user, if I
> know that upgrading the program means that I lose the ability to load
> the data file in my older version, it _is_ an issue I think twice
> about.  I have certainly avoided upgrading certain financial
> applications for _exactly_ this reason.
>
> I'm not saying we _always_ have to be backward compatible.  We should
> be free to consciously break compatibility, but I think we should also
> have the option of making backward-compatible changes.

It's a trade off.  I agree that upgrading /by itself/ should not render
a data file unreadable...  And that /IS/ the case.   But I'm also worried
about data loss by a casual user..  To me, unintentional data loss is a
BIGGER concern than data incompatibilty.

> Well, the 1.8->2.0 is a bit of a separate issue, since the fix is
> completely different.  I'm still not quite clear on whether the
> encoding issue is solvable.  But personally, I'd feel a bit
> irresponsible if we broke backward-compatibility for no other reason
> than introducing budgets.  It just feels... rude.

We broke it for Business and SX in 1.8.  It was broken from 1.4->1.6
for other reasons that I don't recall...  But it's broken ONLY IF
YOU USE THE NEW FEATURE..  Just running the program doesn't break
compatibility..  (xml encoding issues asside).

Please stop thinking like a developer.  Think like a user, a dumb
user, a dumb user who knows NOTHING about computers or programming
or the issues of when data loss can occur.  They care more about data
integrity than compatibility.  Go ahead, ask on -user.  Ask this question
and see what response you get:

   Would you rather be able to always read a data file created in a new
   version of gnucash using an older version of gnucash, where saving that
   data file will lose data when you re-open it in the new version, or would
   you rather the older version of gnucash fail to load the data when it
   sees data it doesn't understand in order to prevent accidental data loss?

> -chris

-derek
-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord at MIT.EDU                        PGP key available