Rounding in the price db.

[John Ralls]

> On Aug 12, 2015, at 9:23 AM, Geert Janssens <geert.gnucash at kobaltwit.be> wrote:
> 
> On Tuesday 11 August 2015 23:12:58 Mike Alexander wrote:
>> --On August 11, 2015 at 9:20:43 AM +0100 John Ralls
>> 
>> <jralls at ceridwen.us> wrote:
>>> Hmm. That’s doable in master with careful rounding control. I’m
>>> pretty sure it would produce overflows in maint because the old
>>> int128 math is really int128-int64 and it barfs if any intermediate
>>> result won’t fit in int64_t. On the other hand, the current
>>> implementation with no rounding is likely to produce overflows if
>>> one
>>> tries to do a large transaction in Dinars->Dobe (or Bitcoin to a lot
>>> of minor currencies, for that matter). ISTM arbitrarily setting a
>>> 10^-10 rounding isn’t quite right, though. If we’re going to do
>>> something like that, and see the next para, it would be more correct
>>> to round to the number of digits which make a 1-scu change in the
>>> smaller currency or perhaps one more digit than that. 10 digits
>>> might
>>> not be enough for converting STD to BTC with its 10^-6 scu.
>>> 
>>> The whole issue of restricting denominators to powers of 10, implied
>>> by “significant digits” and “decimal places" is also worthy of
>>> discussion. Both terms are meaningless with rational numbers unless
>>> one imposes that constraint. Once one does, though, significant
>>> digits is no more difficult to implement than decimal places: One
>>> just rounds the numerator to x digits after doing the denominator
>>> conversion, adjusting the denominator to account for any reduction
>>> of
>>> the digits in the numerator.
>>> 
>>> None of which does anything to help deal with the small moves in
>>> prices due to rounding in the amount and value to their respective
>>> SCUs getting recorded in the pricedb. Mike’s recommendation to
>>> capture the user input works in the case where the user inputs (or
>>> retrieves from F::Q) a price, but not if she uses the
>>> other-account-amount entry.
>> 
>> This is what I meant.  If she enters two amounts then there's no
>> choice but to compute a price.
>> 
>>> IIRC I implemented the price db storage
>>> as after-rounding-to-scu so that there’d be only one place in the
>>> code where it was written out. That can be changed easily enough and
>>> might reduce some of the complaints about the presented price
>>> jumping
>>> around when entering a bunch of foreign currency transactions.
>>> *That*
>>> change could go in maint. I clearly hadn’t sufficiently thought
>>> through rounding when I started this thread. It seems sufficiently
>>> complicated that any implementation should go into master instead.
>> 
>> Yes, thinking of it as a rational number problem instead of a decimal
>> number problem probably helps.
>> 
>>> We seem to agree that there need be only one price per day in the
>>> price db, but no one’s said anything about what price that should
>>> be. I’m inclined towards any new price replaces the existing one
>>> unless they’re the same. What about source? Should a F::Q price
>>> take precedence over a transfer dialog price or vice-versa?
>> 
>> My first guess is to take the last price entered and prefer F::Q price
>> to a calculated price.  I keep daily F::Q prices back for quite a
>> while and when I enter historical transactions it's sometimes
>> annoying when it adds a new price for that day.  I usually go back
>> and delete the transaction price which is often different from the
>> F::Q price due to banks adjusting the rate to their advantage.  I
>> really wouldn't want it to delete the F::Q price for that day. 
>> Furthermore if I use F::Q to get quotes new F::Q prices should
>> replace transaction prices for that day, if any.
>> 
>> It's complicated.  I've always said that I hate dealing with numbers
>> and I certainly don't pretend to be an expert on this.
>> 
>>      Mike
> 
> Exchange rates and prices is not my area at all. I never really used it. 
> So I'm trying my best to imagine how the prices are being used.
> 
> Having said that, here are my thoughts. Feel free to dismiss them if I'm 
> talking nonsense.
> 
> I'm not really sure about the price entered vs F::Q price. I would 
> imagine that if you are tracking stock, you would be interested in the 
> exact real price you bought/sold it, which is not necessarily exactly 
> the price you get from F::Q. Wouldn't that mean that the price entered 
> should be preferred over F::Q prices at least in some cases ?
> 
> That seems the use case for which the transfer dialog is currently 
> coded. It falls apart when a user is entering transfers by amount 
> instead of by price.
> 
> And perhaps you might even want multiple prices for one day if you 
> happened to buy/sell at a big price difference intra-day. Perhaps that 
> won't happen often, but it would be bad if it couldn't be modeled in 
> gnucash.
> 
> All of this assumes of course that the gnucash reports know which price 
> to use given multiple choices.
> 
> Half-baked idea: suppose a user enters a new transaction and uses the 
> amount fields in the tranfer dialog. Instead of unconditionally creating 
> a new price based on this, we could go over the existing prices for that 
> day and calculate if any of them would give the same amount results 
> after rounding. If so, we can keep that price. I have no idea if this 
> makes sense at all yet.
> 
> Another idea is to ask the user if a price already exists that would 
> yield the same result within a preset margin (say 5%).
> 
> As Mike says, it's complicated. I agree.

Indeed.

It might help to consider what the price db is used for: Aside from providing a default rate in the transfer dialog, it’s used for pricing assets in the Accounts page and the summary bar — only the latest entry is used there. It’s used for calculating values in reports, where the reports can use a weighted average, nearest in time, most recent, or average cost. I need to look to see if the average cost actually looks at the buy transactions or if it does something similar to nearest in time and at weighted average to see what it’s averaging and how it’s weighting the averaged values. Depending on what those two are doing might drive whether multiple F::Q entries per day make sense, assuming that the algorithms make sense.

Because we don’t record times posted_date there’s way to assign price db values to individual transactions on a single day. In any case every transaction has a price associated with it (the ratio of amount and value) completely independent of the price db. Modeling multiple buys and/or sells in a day is handled by that, not by the price db. Since there’s no timestamp it doesn’t make sense to record more than one transfer dialog price per day in the price db even if we do capture multiple F::Q entries. 

If you’re tracking a stock, you’re interested in how much you actually paid vs. its current quote. The first is in the transaction itself, and because of the lack of a timestamp can’t be associated with any price in the price db except in the case of open-ended mutual funds that are repriced daily after the market closes. In any case you might have bought or sold only part of a position and the price you got in that transaction wouldn’t necessarily apply to the rest of the position.

I like the idea of setting a tolerance for overwrites from using the value line instead of the price line, but I’d be inclined to use the delta that results in a 1 scu change in the amount rather than a fixed tolerance.

Regards,
John Ralls