Rethinking Numeric
Christian Stimming
christian at cstimming.de
Sat May 31 17:19:23 EDT 2014
Am Sonntag, 25. Mai 2014, 07:34:14 schrieb John Ralls:
> >> If we've reached the point where our int64 rational numbers do not fit
> >> our problem requirements anymore, I'd rather look for a different number
> >> representation that fits our application domain better. I'm thinking
> >> about replacing rational numbers by decimal floating point numbers. That
> >> is, a number is represented by m * 10^e with the mantissa m and exponent
> >> e as signed integers. This is different from our normal *binary*
> >> floating point in that we use the exponent with base 10. However, all
> >> common rules for floating point can be applied just as normal. By the
> >> way, maybe there is even a standard comparable to IEEE 754 available?
> >>
> >> Just another possible way to proceed for solving this problem...
> >
> > Is this http://en.wikipedia.org/wiki/Decimal_floating_point what you're
> > talking about? If so, it says that the IEEE spec is 854, and answers some
> > of my questions, but leaves out database support.
>
> A different spec: http://speleotrove.com/decimal/decarith.html
> And an implementation, which is used in CPython:
> http://www.bytereef.org/mpdecimal/index.html
Thanks for the pointers: Yes, that was exactly what I was talking about. Turns
out the 2008 version of IEEE 754 [1] now also has this included (as
"decimal64" etc), but the implementations in everyday compilers and/or
hardware come along rather slowly. There are well-established library
implementations available, though, such as the one on speleotrove you
mentioned, called "decNumber", but others just as well. If we want to, we can
very well include a library such as that one into gnucash and start using
that.
But back to your initial question: You said we occasionally "encounter
overflow errors". I don't understand (yet) what the actual problem is. With
our current rational numbers and int64_t numerator we have approx. 19 decimal
digits of precision (see [2] for the digits of a 64 bit signed integer), if I
consider the numerator as fully used.
Are 19 significant decimal digits not enough? Are there thinkable cases when
they are not enough? I tend to think the problem is rather found in our
rational number's rounding, which is not the suitable rounding method for our
financial application domain. If this is the problem, a different data type
that does the rounding always according to decimal numbers, and not according
to (in normal float/double calculations) binary floating point numbers, or (in
gnc_numeric) according to rational numbers with some potentially unknown
denominator.
If this is indeed the problem, switching to a data type with strict decimal
number behaviour might be the solution. And the IEEE 754-2008 decimal64 type
might be one of the possible implementations, available in one of the
mentioned libraries. For the record, decimal64 has 16 digits precision [1],
i.e. it won't give us more digits in its 8 bytes compared to our 16 bytes so
far. Maybe we want decimal128, which has 34 digits precision [1]. My gut
feeling says the digits are not the problem and 16 digits are sufficient, but
the rounding behaviour is indeed the problem.
As for database implementations: The speleotrove site [3] says something about
some data bases that directly have a DECFLOAT type (such as ABAP) but
apparently this is not the case for the databases we're looking at. Hence the
storage would have to be done manually, maybe in two integers (significant and
exponent), or in a string, but both would require further calculations before
they can be used in a query directly.
Maybe not yet an easy solution available? But what again was the core of the
problem?
Regards,
Christian
[1] http://en.wikipedia.org/wiki/IEEE_754-2008
[2] http://en.wikipedia.org/wiki/Integer_(computer_science)
[3] http://speleotrove.com/decimal/
More information about the gnucash-devel
mailing list