Fun with roundoff error

[Bill Gribble]

On Tue, Dec 05, 2000 at 10:26:36PM -0500, Randolph Fritz wrote:
> As far as I know, major US banks uniformly use 1/10,000 as the
> denominator, by the way.  I consulted an old friend who used to
> consult for the NYC banks on this some months back.

Maybe for their math, but certainly not for reporting or accumulating
your balance.  If you buy 1 share of stock at 9 7/8, you will have to
withdraw 9.88 to pay for it, not 9.875.  Gnucash isn't trying to
duplicate the internal processes of the bank (computing interest and
so on); it's just recording what happens, so the resolution of
transactions in gnucash should be what's recorded and reported by the
bank.

On the other hand, major brokerages often report fractional share
quantities and even dollar balances down to very small fractions.  Of
course you can't take .00001 shares of Fidelity Select home and put it
in your shoebox like you can USD .01, but it's just as real.  In those
cases, gnucash needs to accumulate balances to match the reports of
the institution.

Intermediate results, in the other hand, should be represented to
substantially better precision; if you buy 10 shares at 9 7/8, it
costs 98.75, not 10*(9.88) == 98.80.  Gnucash represents all
intermediate results as exact quantities, rounding/truncating only for
conversion to the denominator required in the financial context that
the result is used.

b.g.