Rounding in the price db.

[John Ralls]

> On Sep 1, 2015, at 6:29 PM, David Carlson <david.carlson.417 at gmail.com> wrote:
> 
> On 9/1/2015 5:13 PM, John Ralls wrote:
>>> On Aug 28, 2015, at 8:17 PM, John Ralls <jralls at ceridwen.us> wrote:
>>> 
>>> 
>>>> On Aug 28, 2015, at 6:43 AM, Geert Janssens <geert.gnucash at kobaltwit.be> wrote:
>>>> 
>>>> On Friday 28 August 2015 08:55:53 John Ralls wrote:
>>>>> I’ve pushed a feature branch, single-price, to my Github repo
>>>>> (https://github.com/jralls/gnucash <https://github.com/jralls/gnucash>) which covers most of what we’ve
>>>>> discussed here. I’m still wrestling with the math of how to sensibly
>>>>> handle rounding itself, so what’s there still uses the hard-coded
>>>>> 10^6 denom. The branch is from maint because I’d like to put these
>>>>> changes in 2.6.8.
>>>>> 
>>>>> The actual changes are explained in the commit notes. In my limited
>>>>> testing it appears to work and to provide stability when doing
>>>>> multiple transactions with the same exchange rate. Please test some
>>>>> more; I’m sure I didn’t think of every possible variation.
>>>>> 
>>>> Hi John,
>>>> 
>>>> Thanks for your work on this.
>>>> 
>>>> From my first tests and reading the code I have the following observations:
>>>> 
>>>> - Let me start with a nit-pick: while reading through the commits I got confused by the return values of check_account and check_edit. They return TRUE if the check fails (that is when anything is not ok to continue with a transfer). From a distance that seems backwards. I usually expect a check function to return TRUE if all checks pass correctly.
>>>> 
>>>> - Next I created a vendor bill, posted it and then paid it in a foreign currency. This also pops up the transfer dialog. I entered a price and continued. This added the price to the db (as expected). Next I remove the payment transaction (from the bank account in the foreign currency) and issue a new payment via the payment dialog, again in the same foreign currency. This time however, I enter a to amount directly (ensuring it would result in a completely different price). Check the price db and note the existing price hasn't changed.
>>>> 
>>>> - The currencies I was playing with are € (from currency) and HKD (to currency). Before your changes my price db listed a HKD security EUR currency. On your branch the code now adds a EUR security in HKD currency. That change is fine in itself. I prefer your normalized way to store (currency) exchange rates in the db. The issue with this however is that F::Q won't return an exchange rate for the new price, while it did (and still does) for the old one.
>>>> 
>>>>> As for the math, here’s the conundrum: I proposed earlier to base the
>>>>> rounding on what would make a 1 scu change in the “to” commodity. The
>>>>> problems with that idea are that it depends entirely on the amount in
>>>>> the “from” commodity and that prices are often quoted in fractions of
>>>>> a scu. For example, the Wall Street Journal website quotes the Yen at
>>>>> 120.98 to the USD. The Yen’s scu is 1, and the change in the rate to
>>>>> make a 1¥ change in the value is different if the USD amount is $10
>>>>> from what it would be if the amount was $1000. Carry that to its
>>>>> illogical conclusion and we need infinite precision, and that’s
>>>>> ignoring the fact that we need infinite precision to exactly
>>>>> represent a lot of rational fractions, but since all the real money
>>>>> systems use decimal math nowadays that’s not really germane.
>>>>> 
>>>>> So I have a new proposal: If the commodities are both currencies,
>>>>> store exchange rates in the direction where the rate > 1, set the
>>>>> denominator to 1000, and round-half-up. The price retrieval code
>>>>> already checks in both directions. If only one of the commodities is
>>>>> a currency then it’s a price and we store it in the currency with the
>>>>> denominator = the currency’s scu * 10000.
>>>>> 
>>>> See above: the check in both directions is not reliable/borked.
>>>> 
>>>>> That leaves commodity-commodity prices. The most common example in
>>>>> modern life is stock-for-stock exchanges resulting from mergers or
>>>>> spin-offs. These tend to be one-offs, so no rounding required. Barter
>>>>> exchange, where one exchanges one commodity for another (e.g. two
>>>>> bushels of corn for a cow), is similarly fractional rather than
>>>>> decimal, so again not rounding is appropriate. The third case is the
>>>>> problem: Bitcoin and similar pseudo-currencies. For maint I think
>>>>> we’re going to have to leave those prices unrounded as well, but
>>>>> perhaps for master we should consider creating a separate commodity
>>>>> category so that users can create commodities that GnuCash treats as
>>>>> currencies.
>>>>> 
>>>> That looks like a very sensible proposal to me.
>>> Geert,
>>> 
>>> Thanks for testing. I agree that the check_foo() semantics are clumsy. I did it that way to avoid negating the return value in the if conditional, but in retrospect that would be clearer, so I’ll flip it.
>>> 
>>> Roger that the checks aren’t reliably bidirectional. I’ll dig into that. I hadn’t yet changed anything with regards to which direction prices are recorded, at least not on purpose, so I’ll have to track that down too.
>>> 
>>> I coded up the price-rounding algorithm on the flight back today and played with it a little. I think it may need some adjustment.
>> I’ve pushed more changes to single-price which I think address Geert’s comments and some tweaks to maximize significant digit preservation while keeping denominators <= 10E6 in most cases. Please test some more!
>> 
>> Regards,
>> John Ralls
>> _______________________________________________
>> gnucash-devel mailing list
>> gnucash-devel at gnucash.org
>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
> 
> I kind of lost track of how it is supposed to work for the user.  If I
> am slow to enter a buy or sell for a certain date, will GnuCash
> overwrite a price that was downloaded earlier for the same security and
> date or possibly manually entered earlier for that date?  Sometimes I
> create a buy or sell by duplicating a transaction that has the wrong
> information then do several transaction saves as I am changing to the
> correct accounts, amounts and date. 

David,

Quotes from F::Q are supposed to overwrite user entries for the day, but although F::Q is quite capable of retrieving historical quotes, GnuCash doesn’t use that facility. So if you create a transaction with a date_posted in the past and there isn’t already an F::Q quote for that day then the price generated from that transaction won’t get overwritten later on. On the other hand, if you do have an F::Q quote already on that day then the user price won’t get written to the price db.

Umm, are you able to build from git? This code is currently on a feature branch so it isn’t going to get into the nightlies for you to test with otherwise. I think I understand the first test-case well enough to run tests myself, but I need more detail on the second (the duplicate-and-modify) one to test it.

Regards,
John Ralls