gnc_numeric: fractional displays and rounding

[Charles Day]

On Thu, Jul 31, 2008 at 10:01 AM, Derek Atkins <warlord at mit.edu> wrote:

> Quoting Charles Day <cedayiv at gmail.com>:
>
>  Working this way would seem to make little sense. The first three cases
>> print identically, and the last doesn't work for non-decimal values.
>> Here's
>> how I'm guessing it was intended to work:
>>
>> round force_fit decimal  non-decimal
>> ===== ========= ======= =============
>>    0         0   0.999       0 + 1/3
>>    1         0   1.000       0 + 1/3
>>    0         1   0.999         0.999
>>    1         1   1.000         1.000
>>
>> Is this correct? If so, I can readily provide a patch.
>>
>
> I would expect given your inputs that the last two rows in the far
> right column would both be 0.333


Sorry, yes, you're right. Here's a corrected second table:

 round force_fit decimal  non-decimal
 ===== ========= ======= =============
     0         0   0.999       0 + 1/3
     1         0   1.000       0 + 1/3
     0         1   0.999         0.333 *a better example might be 2/3
(prints as 0.666)
     1         1   1.000         0.333 *a better example might be 2/3
(prints as 0.667)


>
>  Cheers,
>> Charles
>>
>
> -derek
>
> --
>      Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
>      Member, MIT Student Information Processing Board  (SIPB)
>      URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
>      warlord at MIT.EDU                        PGP key available
>
>