[GNC-dev] Bug 797463 - CSV Import of transactions into a new file hangs

Christian Gruber christian.gruber at posteo.de
Sun Nov 10 15:07:04 EST 2019


Am 08.11.19 um 23:08 schrieb John Ralls:
>
>> On Nov 8, 2019, at 1:58 PM, Christian Gruber <christian.gruber at posteo.de> wrote:
>>
>> Am 08.11.19 um 04:39 schrieb John Ralls:
>>> Christian,
>>>
>>> It's not that it's not prepared for Bayesian matching, it's that older versions of GnuCash stored the Bayesian match tokens hierarchically. Aaron Laws (lmat) changed it to a flatter structure with somewhat better memory locality for faster access. imap_convert_bayes_to_flat should run once to convert the data and set the feature, after which check_import_map_data will see the flag and return. A file created with 3.x and Baysian maps would already have the feature set.
>>>
>>>
>>> With that background, to your questions:
>>>
>>> Why does it take so long? Because it traverses the entire tree of accounts, every time. The test book has 1127 accounts. Add to that that there are some things inside the loop that shouldn't be and that convert_imap_account_bayes_to_flat doesn't use some obvious short circuits and you get taking a long time.
>>>
>>> Why does it run twice? Because there aren't any accounts with import-map-bayes slots, so it does no conversions so it doesn't set the feature.
>> Why isn't the feature set in any case after conversion is done, whether there was any slot to convert or not?
>>
>> I would expect that, if the hierarchical structure should not be used anymore. I wouldn't only set the feature, if there has been any error during conversion. But it's not an error, if convert_imap_account_bayes_to_flat() returns false.
> The feature isn't set because that would prevent using the file with an older version of GnuCash for no good reason: There aren't any import-map-bayes tags in the new format so there's nothing for the older version to mess up.
>
> Regards,
> John Ralls
>
Sounds meaningful, but it's not obvious. The conversion process is not 
much transparent to the user. The user is not informed about conversion 
and additionally the conditions, when the GnuCash file is converted and 
when not, are not quite intuitive, I guess. And the second question for 
me is, if this is really a relevant use case to keep the GnuCash file 
unconverted if possible, when it is used with a newer GnuCash version. 
Maybe it's relevant, if several people, who use different GnuCash 
versions work on the same GnuCash file. But what happens, when one with 
a newer GnuCash version uses the bayesian import matcher for the first 
time, which silently converts the GnuCash file? Will it still be usable 
with an older GnuCash version?

But actually this is another discussion, which is not important for the 
bug ticket. What's more important, I created a new fresh GnuCash file 
with the SKR04 accounts template using my GnuCash version 3.7 and if I 
checked this correctly, even with this GnuCash version the feature 
GNC_FEATURE_GUID_FLAT_BAYESIAN is not set in the GnuCash file. Can you 
confirm this? If yes, then the bug report is relevant not only for 
migration from older to current GnuCash versions but even for current 
GnuCash versions in general.

Regards,
Christian




More information about the gnucash-devel mailing list