[GNC-dev] [GNC] some of my reconciliations in 3.9 are completely bonkers

Thu Apr 2 01:17:23 EDT 2020

I’m glad it works. The next question is could this calculation be the basis for calculating balance-on-a-date for the purpose of calculating interest on loan payments that include irregular, extra principal payments? I’m just about done paying off loans, but I pay ahead on my mortgage, and my credit union calculates interest on days since last payment (which changes in at least 11 out of 12 months anyway).
--
Dave Reiser
dbreiser at icloud.com


> On Apr 2, 2020, at 1:10 AM, Christopher Lam <christopher.lck at gmail.com> wrote:
> 
> Thank you for helping troubleshoot this issue.
> 
> The dilemma is whether to keep this change which exposes invalid reconciliation statement dates, or revert to previous behaviour. The only UI available to fix these dates is to unreconcile the old splits and rereconcile old statements (or use latest statement).
> 
> On Thu, 2 Apr 2020 at 05:06, David Reiser <dbreiser at icloud.com <mailto:dbreiser at icloud.com>> wrote:
> Correct. I edited the raw .xml file. Seems to be fixed.
> --
> Dave Reiser
> dbreiser at icloud.com <mailto:dbreiser at icloud.com>
> 
> 
> 
> 
> 
>> On Apr 2, 2020, at 1:04 AM, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>> 
>> My suspicion is that your reconciliation for June 2006 used a bogus date. Try expand the date filter to End-Date = 31/12/9999.
>> 
>> If I'm right then you could unreconcile the June 2006 splits and re-reconcile the 30 June 2006 statement?
>> 
>> On Thu, 2 Apr 2020 at 01:49, David Reiser <dbreiser at icloud.com <mailto:dbreiser at icloud.com>> wrote:
>> The reconcile report is missing all the transactions from June 2006 (06/06…).
>> --
>> Dave Reiser
>> dbreiser at icloud.com <mailto:dbreiser at icloud.com>
>> 
>> 
>> 
>> 
>> 
>>> On Apr 1, 2020, at 11:47 AM, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>>> 
>>> Would you mind trying the Reconciliation Report. We're confirming that *all* reconciled_dates column are reasonable. Please feel free to send the resulting report privately.
>>> 
>>> Account = Savings
>>> Start Date = 1/1/1970
>>> End Date = today
>>> 
>>> 
>>> On Wed, 1 Apr 2020 at 15:23, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>>> (snipped screenshot showing a very regular savings account with monthly reconciled interest splits until 31.1.2020 and monthly cleared interest splits afterwards).
>>> 
>>> Apologies I cannot understand how/why the starting balance calculator would not show the appropriate amount. I assume you're not including subaccounts in your reconciliation. The 3.9 starting balance is calculated at https://github.com/Gnucash/gnucash/blob/90d3e6c6721ffb3f7e53adfd8bbd2f1b6933cb3d/libgnucash/engine/Account.cpp#L3402 <https://github.com/Gnucash/gnucash/blob/90d3e6c6721ffb3f7e53adfd8bbd2f1b6933cb3d/libgnucash/engine/Account.cpp#L3402> --
>>> 
>>> Scan all (savings) account splits, if split's reconciled_status is 'y' and the reconciled_date <= statement_date then accumulate its value. Note the reconciled_date will be the statement_date for previous months' reconciliations, therefore all splits from previous reconciliations should be counted without fail. If you could create a dev environment I could offer a repository with appropriate logging. Unfortunately I do not know how to package a .dmg.
>>> 
>>> Alternatively if you can install a custom report I can create a .scm file to log a similar reconciled_balance accumulator.
>>> 
>>> C
>>>> On Apr 1, 2020, at 10:40 AM, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>>>> 
>>>> Specifically interested in the register screenshot from 1-Jan-2020 onwards -- wish to verify balances/reconcile-status etc. Thank you. Regards
>>>> Care to share a screenshot of your savings account register privately? Earlier splits are not needed. Thanks.
>>>> 
>>>> On Wed, 1 Apr 2020, 10:06 pm David Reiser, <dbreiser at icloud.com <mailto:dbreiser at icloud.com>> wrote:
>>>> I have no problem with (A), except maybe if you’re using the gnucash posting date. A reconciliation resolves transactions based on the bank’s posting date, not gnucash’s. If I don’t enter March 31, 2020 interest until April 1, I’m still going to enter it in my account as having occurred on 3/31. But even that distinction can’t be causing what I’m seeing.
>>>> 
>>>> In my simplest case, my savings account was last reconciled based on the interest paid 1/31/2020. On 2/29 I received $0.96 in interest and on 3/31 I received $1.03 in interest. There were no other transactions during that time. I have no idea when I entered the February interest, but I did enter the March interest on 3/31/2020 (at least it was still 3/31 on the U.S. east coast).
>>>> 
>>>> When I click Reconcile, in the Reconcile Information dialog, I get:
>>>> Statement date 02/29/2020
>>>> Starting Balance $3024.39 (this amount was reconciled in gnucash 3.7)
>>>> ending Balance $3025.36  (reasonable, since 3024.39 + 0.96 = 3025.35)
>>>> 
>>>> Now if I click OK to get to the transaction check-off dialog, I get:
>>>> 
>>>> Starting balance: 375.15  (What???)
>>>> Ending Balance: 3025.36 (well, it remembered what I entered)
>>>> Reconciled balance: 376.11 (well, it is 375.15 + 0.96 …)
>>>> 
>>>> Difference: 2649.24 (the math is locally consistent, but the starting balance doesn’t represent anything in my register or what gnucash showed me on the prior screen)
>>>> 
>>>>> On Apr 1, 2020, at 1:19 AM, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>>>>> 
>>>>> This is due to https://bugs.gnucash.org/show_bug.cgi?id=797640 <https://bugs.gnucash.org/show_bug.cgi?id=797640>
>>>>> 
>>>>> This change modifies the reconciliation starting balance calculator from Account->reconciled_balance to account->reconciled_balance on statement date.
>>>>> 
>>>>> The reasoning for this change is with the observation:
>>>>> 
>>>>> (A) if reconciliation is performed from a statement dated 31/01/2019, the starting balance calculator should ignore transactions posted later than 31/01/2019.
>>>>> 
>>>>> (B) subsequent work https://github.com/Gnucash/gnucash/pull/667 <https://github.com/Gnucash/gnucash/pull/667> aims to store past reconciliation ending balances and statement date. As a result we can re-reconcile any past statement.
>>>>> 
>>>>> (C) subsequent work will allow a sanity-check type report (illustrated in above PR)  which will compare account reconciled balances at previous statement dates, and highlight any discrepancy.
>>>>> 
>>>>> There have been previous feature requests to store and retrieve reconciled balances and reconciliation dates, and I believe it's possible and reasonable.
>>>>> 
>>>>> Having explained the rationale, the reasoning (A) may be incorrect -- please file in bug 797640 how/why starting_balance should include transactions posted after statement date. If reasoning (A) is invalid then we will need to revert the change, which means (B) and (C) above cannot happen.
>>>>> 
>>>>> On Wed, 1 Apr 2020, 11:48 am David Reiser via gnucash-user, <gnucash-user at gnucash.org <mailto:gnucash-user at gnucash.org>> wrote:
>>>>> My checking account reconciled fine.
>>>>> Then I went to two savings accounts and got completely off-the-wall results. When I first clicked the reconcile button, the dialog that comes up has the correct starting balance and the correct proposed to-be-reconciled balance. When I click OK to get to the transaction check-off dialog, gnucash has completely changed the starting balance to a much smaller number (and one that has never appeared in the account ledger) and thus presents an out of balance total that cannot be reconciled with the transactions that are in the account.
>> 
>> 
>> On Thu, 2 Apr 2020 at 01:49, David Reiser <dbreiser at icloud.com <mailto:dbreiser at icloud.com>> wrote:
>> The reconcile report is missing all the transactions from June 2006 (06/06…).
>> --
>> Dave Reiser
>> dbreiser at icloud.com <mailto:dbreiser at icloud.com>
>> 
>> 
>> 
>> 
>> 
>>> On Apr 1, 2020, at 11:47 AM, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>>> 
>>> Would you mind trying the Reconciliation Report. We're confirming that *all* reconciled_dates column are reasonable. Please feel free to send the resulting report privately.
>>> 
>>> Account = Savings
>>> Start Date = 1/1/1970
>>> End Date = today
>>> 
>>> 
>>> On Wed, 1 Apr 2020 at 15:23, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>>> (snipped screenshot showing a very regular savings account with monthly reconciled interest splits until 31.1.2020 and monthly cleared interest splits afterwards).
>>> 
>>> Apologies I cannot understand how/why the starting balance calculator would not show the appropriate amount. I assume you're not including subaccounts in your reconciliation. The 3.9 starting balance is calculated at https://github.com/Gnucash/gnucash/blob/90d3e6c6721ffb3f7e53adfd8bbd2f1b6933cb3d/libgnucash/engine/Account.cpp#L3402 <https://github.com/Gnucash/gnucash/blob/90d3e6c6721ffb3f7e53adfd8bbd2f1b6933cb3d/libgnucash/engine/Account.cpp#L3402> --
>>> 
>>> Scan all (savings) account splits, if split's reconciled_status is 'y' and the reconciled_date <= statement_date then accumulate its value. Note the reconciled_date will be the statement_date for previous months' reconciliations, therefore all splits from previous reconciliations should be counted without fail. If you could create a dev environment I could offer a repository with appropriate logging. Unfortunately I do not know how to package a .dmg.
>>> 
>>> Alternatively if you can install a custom report I can create a .scm file to log a similar reconciled_balance accumulator.
>>> 
>>> C
>>>> On Apr 1, 2020, at 10:40 AM, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>>>> 
>>>> Specifically interested in the register screenshot from 1-Jan-2020 onwards -- wish to verify balances/reconcile-status etc. Thank you. Regards
>>>> Care to share a screenshot of your savings account register privately? Earlier splits are not needed. Thanks.
>>>> 
>>>> On Wed, 1 Apr 2020, 10:06 pm David Reiser, <dbreiser at icloud.com <mailto:dbreiser at icloud.com>> wrote:
>>>> I have no problem with (A), except maybe if you’re using the gnucash posting date. A reconciliation resolves transactions based on the bank’s posting date, not gnucash’s. If I don’t enter March 31, 2020 interest until April 1, I’m still going to enter it in my account as having occurred on 3/31. But even that distinction can’t be causing what I’m seeing.
>>>> 
>>>> In my simplest case, my savings account was last reconciled based on the interest paid 1/31/2020. On 2/29 I received $0.96 in interest and on 3/31 I received $1.03 in interest. There were no other transactions during that time. I have no idea when I entered the February interest, but I did enter the March interest on 3/31/2020 (at least it was still 3/31 on the U.S. east coast).
>>>> 
>>>> When I click Reconcile, in the Reconcile Information dialog, I get:
>>>> Statement date 02/29/2020
>>>> Starting Balance $3024.39 (this amount was reconciled in gnucash 3.7)
>>>> ending Balance $3025.36  (reasonable, since 3024.39 + 0.96 = 3025.35)
>>>> 
>>>> Now if I click OK to get to the transaction check-off dialog, I get:
>>>> 
>>>> Starting balance: 375.15  (What???)
>>>> Ending Balance: 3025.36 (well, it remembered what I entered)
>>>> Reconciled balance: 376.11 (well, it is 375.15 + 0.96 …)
>>>> 
>>>> Difference: 2649.24 (the math is locally consistent, but the starting balance doesn’t represent anything in my register or what gnucash showed me on the prior screen)
>>>> 
>>>>> On Apr 1, 2020, at 1:19 AM, Christopher Lam <christopher.lck at gmail.com <mailto:christopher.lck at gmail.com>> wrote:
>>>>> 
>>>>> This is due to https://bugs.gnucash.org/show_bug.cgi?id=797640 <https://bugs.gnucash.org/show_bug.cgi?id=797640>
>>>>> 
>>>>> This change modifies the reconciliation starting balance calculator from Account->reconciled_balance to account->reconciled_balance on statement date.
>>>>> 
>>>>> The reasoning for this change is with the observation:
>>>>> 
>>>>> (A) if reconciliation is performed from a statement dated 31/01/2019, the starting balance calculator should ignore transactions posted later than 31/01/2019.
>>>>> 
>>>>> (B) subsequent work https://github.com/Gnucash/gnucash/pull/667 <https://github.com/Gnucash/gnucash/pull/667> aims to store past reconciliation ending balances and statement date. As a result we can re-reconcile any past statement.
>>>>> 
>>>>> (C) subsequent work will allow a sanity-check type report (illustrated in above PR)  which will compare account reconciled balances at previous statement dates, and highlight any discrepancy.
>>>>> 
>>>>> There have been previous feature requests to store and retrieve reconciled balances and reconciliation dates, and I believe it's possible and reasonable.
>>>>> 
>>>>> Having explained the rationale, the reasoning (A) may be incorrect -- please file in bug 797640 how/why starting_balance should include transactions posted after statement date. If reasoning (A) is invalid then we will need to revert the change, which means (B) and (C) above cannot happen.
>>>>> 
>>>>> On Wed, 1 Apr 2020, 11:48 am David Reiser via gnucash-user, <gnucash-user at gnucash.org <mailto:gnucash-user at gnucash.org>> wrote:
>>>>> My checking account reconciled fine.
>>>>> Then I went to two savings accounts and got completely off-the-wall results. When I first clicked the reconcile button, the dialog that comes up has the correct starting balance and the correct proposed to-be-reconciled balance. When I click OK to get to the transaction check-off dialog, gnucash has completely changed the starting balance to a much smaller number (and one that has never appeared in the account ledger) and thus presents an out of balance total that cannot be reconciled with the transactions that are in the account.
>> 
>