Price Editor/Database lag

[David]

I just discovered my swap was all turned off.  It might also explain very un-VM messages like "Your computer does not have enough free memory to automatically analyze the problem and send a report to the developers."  Now I have a huge, dedicated drive SSD swap (which one article said was a bad idea, but maybe any churn wear will be more distributed on a huge (150G) swap).  Let me see if that helps!

Of course, I am also moving my price source to Yahoo_JSON.

 

 

 

-----Original Message-----
From: John Ralls <jralls at ceridwen.us>
To: David <dgpickett at aol.com>
Cc: gnucash-user <gnucash-user at gnucash.org>
Sent: Sun, Apr 1, 2018 11:09 pm
Subject: Re: Price Editor/Database lag


Entirely possible. I’m not too familiar with the GtkTreeList implementation.


Regards,
John Ralls




On Apr 1, 2018, at 6:59 PM, David <dgpickett at aol.com> wrote:


John,

OK, that makes sense, except the lag can occur pretty randomly as I select lists of symbols, select symbols from that list, ask for an add window, or work in the add window with date, price.  (I select the most recent month end nav so it is cloned with the add, just needs a new date and price.)  Maybe it is going into thrashing?


 Thanks,

David

 

 

-----Original Message-----
From: John Ralls <jralls at ceridwen.us>
To: David <dgpickett at aol.com>
Cc: Gnucash Users <gnucash-user at gnucash.org>
Sent: Sun, Apr 1, 2018 3:48 pm
Subject: Re: Price Editor/Database lag

Please remember to copy the list on all replies.

The data file is read exactly once, when it's loaded. After that everything is stored in GnuCash objects. That's what "all of the data is always in memory" means. We store data as either XML or in a SQL database, so mmap() won't do anything for us.

The likely problem is that the Price Editor is based on a GtkTreeView, and its model needs to be loaded every time you open the Price Editor. We could indeed get a pretty big performance enhancement by hiding the Price Editor on close instead of destroying it.

Yes, the price db could be kept apart from book data, but it isn't. That would be a pretty big design change, and I think better left until we're ready to change to query-as-needed database use as that will bring about a lot of incompatible storage changes.

Regards,
John Ralls

> On Apr 1, 2018, at 8:40 AM, David <dgpickett at aol.com> wrote:
> 
> That is an OK excuse for one delay, not delay after delay not in sync with any part of the edit process.
> 
> It is a bug if it reloads the database, rather than keeping an in memory and in file model that can be updated, and an in file model that only gets updated at save time.  Also, it should mmap64() the file, so it is cached in RAM, not read from scratch.  An unsorted flat file pretending to be a database would outperform this.
> 
> The price history might be kept outside the books files, as it is a cache of public data, and can be shared with all if they use the same key symbols, updated once for all sets of books!
> 
> 
> 
> -----Original Message-----
> From: John Ralls <jralls at ceridwen.us>
> To: DGPickett <dgpickett at aol.com>
> Cc: gnucash-user <gnucash-user at gnucash.org>
> Sent: Sun, Apr 1, 2018 9:39 am
> Subject: Re: Price Editor/Database lag
> 
> 
> 
> > On Apr 1, 2018, at 5:51 AM, DGPickett via gnucash-user <gnucash-user at gnucash.org> wrote:
> > 
> > Since I was collecting prices daily, I guess my price database is pretty
> > large, and my family finances are in there for about 5 years. I have xml
> > database, and compressed, but I checked my save intervals, even increased
> > them, but no help. My PC has 8 GB ram, and a very large solid state swap
> > drive, so anyy database should stay in memory.
> 
> GnuCash isn’t yet a database application, so all of the data is always in memory regardless of the backend. If the delay you experience is in opening the price db dialog then it’s probably due to loading the dialog’s tree model from the price db.
> 
> Regards,
> John Ralls
>