[GNC] Large GnuCash file very slow to write to

[Tim Hume]

Hi people,

I guess the underlying issue is probably Piecash - or possibly the way it interacts with SQLite. Though GnuCash itself is pretty slow at exit time too (maybe 30 seconds to a minute) - but not in the same league of slowness as my Python program using Piecash.

I was simply hoping that maybe someone else had experienced this and might have some tips - I realise this mailing list is for GnuCash, but given Piecash is used for manipulating GnuCash files, the thought occurred that others on the list may have some tips. The MySQL/PostgreSQL suggestion may be a good one. But before I go down this track I'll a browse though the Piecash code.

I'll report back if I find anything or have success in speeding things up.

Cheers,

Tim.

Sent with [ProtonMail](https://protonmail.com/) secure email.

------- Original Message -------
On Sunday, May 8th, 2022 at 15:30, john <jralls at ceridwen.us> wrote:

> Is this about your python program being slow or about GnuCash being slow? Note that Piecash isn't GnuCash and we don't provide any support for it, nor obviously can we provide any support for your program.
>
> If it's a GnuCash problem, please provide more details about exactly what you're doing and what step is taking 20 minutes.
>
> Regards,
> John Ralls
>
>> On May 8, 2022, at 4:50 AM, Tim Hume via gnucash-user <gnucash-user at gnucash.org> wrote:
>>
>> It is an SSD drive (I'm using a Mac Mini about one year old - with the M1 chip).
>>
>> The slow part seems to be saving the database - inserting the transactions is fast enough, the save operation is what takes so long. It may be 20 minutes - I haven't timed it. Even closing GnuCash is slow - perhaps 30 seconds to a minute.
>>
>> Sent with ProtonMail secure email.
>> ------- Original Message -------
>> On Sunday, May 8th, 2022 at 11:40, Kenneth Schneider <kschneider at bout-tyme.net> wrote:
>>
>>> I don’t believe the file is getting too large. If you haven’t looked into using an SSD drive at least for the database I highly recommend it. Not only will you amazed at how fast data will be written to the file but also how fast you will be able to generate reports.
>>>
>>> Ken Schneider
>>>
>>>> On May 8, 2022, at 7:24 AM, Tim Hume via gnucash-user gnucash-user at gnucash.org wrote:
>>>>
>>>> Hi,
>>>>
>>>> My GnuCash file (SQLite format) has become rather large - 110 MBytes. Every month or two I import a lot more transactions in CSV format. I use piecash in a Python script I wrote, because the importing of CSV data built into GnuCash crashes when I import too many transactions.
>>>>
>>>> I might be importing perhaps 4000 or 5000 transactions. Saving them into my GnuCash file takes many many minutes. I know my file is getting too large, but it surely shouldn't take 10+ minutes to do this operation? Whilst I do it, I can see Python is taking 100% of a CPU core.
>>>>
>>>> Come the end of the financial year (30 June) I'm going to start a new file. But this will quickly grow in size, and no doubt I'll run into this problem again. Is there anything I can do to speed this up, or do I just have to live with it?
>>>>
>>>> Cheers,
>>>>
>>>> Tim.
>>>>
>>>> Sent with ProtonMail secure email.
>>>> _______________________________________________
>>>> gnucash-user mailing list
>>>> gnucash-user at gnucash.org
>>>> To update your subscription preferences or to unsubscribe:
>>>> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>>>> If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
>>>> -----
>>>> Please remember to CC this list on all your replies.
>>>> You can do this by using Reply-To-List or Reply-All.
>>>
>>> _______________________________________________
>>> gnucash-user mailing list
>>> gnucash-user at gnucash.org
>>> To update your subscription preferences or to unsubscribe:
>>> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>>> If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
>>> -----
>>> Please remember to CC this list on all your replies.
>>> You can do this by using Reply-To-List or Reply-All.
>>
>> _______________________________________________
>> gnucash-user mailing list
>> gnucash-user at gnucash.org
>> To update your subscription preferences or to unsubscribe:
>> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>> If you are using Nabble or Gmane, please seehttps://wiki.gnucash.org/wiki/Mailing_Listsfor more information.
>> -----
>> Please remember to CC this list on all your replies.
>> You can do this by using Reply-To-List or Reply-All.