[GNC] Gnucash logs

[Chris Good]

From: Geert Janssens <geert.gnucash at kobaltwit.be> 
Sent: Thursday, 30 April 2020 7:24 PM
To: gnucash-user at gnucash.org
Cc: Chris Good <goodchris96 at gmail.com>; 'David T.' <sunfish62 at yahoo.com>; 'David H' <hellvee at gmail.com>
Subject: Re: [GNC] Gnucash logs

 

Op donderdag 30 april 2020 06:04:10 CEST schreef Chris Good:

> Hi Geert,

> 

> 

> 

> I think options 2-5 are too disruptive and more work than the benefits

> deserve.

> 

> 

> 

> Re option 1,

> 

> I’d prefer to put logs and backups in separate folders. Maybe (going with

> your name) book.dir/backup and book.dir/log. Many people seem to be

> confused about what all these long filenames are when mixed together.

> 

I see I didn't mention this, but that making separate subdirectories for those was also intended in my suggestion.

> 

> 

> Also putting the metadata (book.gcm) in book.dir would make if hard to

> notice, so I think it should be in the folder holding book.gnucash (or

> maybe a separate ‘metadata’ subfolder as some people think the metadata

> file is their book).

 

I'm not decided on whether this is a good idea yet. I was throwing it in for further consideration. Note that if we do switch to a model with a data file and associated directory for other stuff, the metadata file could just be called that - metadata.gcm. It's already in the directory with the same name as the book. No need to keep that file itself named as the book is.

 

For that matter even the log and backups files could be reduced in file name length to only the timestamp. The same applies, the base directory (book.dir) uniquely identifies which book they belong to.

 

> 

> Not having all the metadata files for multiple books in the same folder

> means it is more fiddly to share metadata between multiple books by giving

> the books the same name (books in different directories) but there is

> probably nobody doing that anyway (as it is easy to get confused about

> which book is open).

> 

Well, actually you can't. GnuCash has guards in place that make each file name in the current metadata directory unique. The filename is just a first indication. The actual match between your book and the metadata file is done by comparing the root account's GUID in your book with the GUID stored in the metadata file.

So if you would have two books named "book.gnucash", you will end up with a "books.gcm" and a "books-1.gcm" metadata file. As they are created for diferent books, they will both store a unique GUID. When opening either book, gnucash will check both metadata files for the one that has your book's GUID stored in it.

 

And yes, you could thus circumvent that safety measure by manipulating your books to have the same root account GUID. But that's not what the system was designed for so we can ignore that hack in this discussion :)

 

Regards,

 

Geert

> 

> 

> Regards,

> 

> Chris Good

> 

> From: Geert Janssens <geert.gnucash at kobaltwit.be <mailto:geert.gnucash at kobaltwit.be> >

> Sent: Wednesday, 29 April 2020 7:57 PM

> To: gnucash-user at gnucash.org <mailto:gnucash-user at gnucash.org> 

> Cc: Chris Good <goodchris96 at gmail.com <mailto:goodchris96 at gmail.com> >; 'David T.' <sunfish62 at yahoo.com <mailto:sunfish62 at yahoo.com> >;

> 'David H' <hellvee at gmail.com <mailto:hellvee at gmail.com> > Subject: Re: [GNC] Gnucash logs

> 

> Hi,

> 

> For the file based books (xml and sqlite) there are several approaches:

> 

> 1. Have a book.gnucash and a book.dir side by side, per book you create.

> 

> book.gnucash is your actual data file.

> 

> book.dir is a directory containing extra files related to this data file.

> These include log files, backup files and can potentially be extended to

> hold attachments and possibly book metadata

> 

> Advantages:

> 

> - all data related to one book is stored together in one place. No

> intermingling of (meta)data with other books. Makes it easy to move a book

> to somewhere else

> 

> - as the main book is outside of the the directory it's easy to spot your

> book to open it.

> 

> Disadvantage: per book you will have two items: a file and a directory. At

> the file management level you always have to treat them as one (move them

> together, back them up together,...)

> 

> 2. Have a directory book.gnucash and store all data in there in a structured

> form (logs in their own subdirectory,...). To save a book, you enter the

> book's name and gnucash will create that directory for you. Inside the

> directory your book will now have a generic name (as the directory already

> indicates the unique name) as would log files and backup files. To open a

> book, you select the directory in gnucash and it will open the actual book

> inside of it for you. Backup management could be integrated in gnucash.

> 

> Advantages:

> 

> - like option 1, all data related to a single book is encapsulated closely

> together.

> 

> - only one single element on the file system level to remember: the book.dir

> is your book. Makes it slightly easier to move things around, make backups

> and so on.

> 

> Disadvantage: double-clicking a file to open it in gnucash will no longer

> work. The file manager will just open the directory for you and the data in

> there is now private to gnucash. You could argue that you can make a

> double-click on the book file inside the directory would then open gnucash

> for you. It could at the expense of vastly more complicated load code. It

> now has to check for both a file or directory being opened and do the right

> thing in both cases.

> 

> 3 Same as 2, except this time the directory will be zipped.

> 

> Advantages:

> 

> - like option 1, all data related to a single book is encapsulated closely

> together.

> 

> - only one single element on the file system level to remember: the book.dir

> is your book. Makes it slightly easier to move things around, make backups

> and so on.

> 

> - double-click would work again

> 

> Disadvantages:

> 

> - The single file could become rather huge. Particularly if you also choose

> to include the attachments in there

> 

> - Manual backup and log management becomes harder (one has to know how to

> unzip the gnucash file, akin to how libreoffice documents are zipped up

> directories)

> 

> - Embedding backups directly in the data file feels wrong.

> 

> 4. The approach where one single global log directory is created (like in

> GNC_DATA_DIR) and one single global backup directory. All log and backup

> files will be written to these global directories.

> 

> Advantages:

> 

> - the directory containing your book(s) will be fully uncluttered. It will

> effectively only contain your book(s). No more backups or logs. So easy on

> the eye and easy to understand. As you don't need backups often, you can

> find them when needed.

> 

> Disadvantages:

> 

> - your book and its backups get separated. So if you need to manage your

> backups you have to search for them in another directory

> 

> - if you have multiple books the logs/backups of all books get lumped

> together. For backup or cleanup purposes this is more messy. You have to

> take care to only select the backups/logs for the books you are interested

> in.

> 

> - a real risk of file name clashes. If you have two same-named books in

> separate directories, their logs and backups will overlap or even conflict.

> There are technical solutions for this (we use one for the metadata files)

> but it's cludgy.

> 

> 5. Just drop the whole multiple backup/log files idea and move to a model

> similar to libreoffice. That is, while a document is open, it keeps one

> copy of your current state somewhere written to disk. In case the program

> crashes it will offer to restore to the last known good state. Other than

> that rely on the user to have a decent backup solution to revert back to in

> case of disk failure.

> 

> Advantages:

> 

> - clean directory, will only hold your actual data files

> 

> - guard against data loss in case of a program crash

> 

> Disadvantages:

> 

> - won't allow to go back to older known save states. Users will have to

> implement their own backup strategy for that.

> 

> Personally I don't think 2, 3 and 4 are good candidates. They all have

> significant drawbacks that will confuse new and less experienced users. 5

> is controversial. I have added it anyway to test common opinion on that

> idea. 1 is the most conservative in terms of number of changes to make and

> in my personal opinion a good trade-off between value and complexity.

> 

> Note none of this even considers mysql of postgresql based books. Granted

> they don't have the same clutter problems wrt to backups. But they do also

> write log files (for those who don't know, you will find them in

> GNC_DATA_DIR/translog).

> 

> Regards,

> 

> Geert

> 

> Op woensdag 29 april 2020 08:54:22 CEST schreef Chris Good:

> > -----Original Message-----

> > 

> > From: David T. <sunfish62 at yahoo.com  <mailto:sunfish62 at yahoo.com%20%3cmailto:sunfish62 at yahoo.com> <mailto:sunfish62 at yahoo.com> >

> > 

> > Sent: Wednesday, 29 April 2020 4:11 PM

> > 

> > To: Chris Good <goodchris96 at gmail.com  <mailto:goodchris96 at gmail.com%20%3cmailto:goodchris96 at gmail.com> <mailto:goodchris96 at gmail.com> >;

> > 'David H' <hellvee at gmail.com  <mailto:hellvee at gmail.com%20%3cmailto:hellvee at gmail.com> <mailto:hellvee at gmail.com> >

> > 

> > Cc: 'Gnucash Users' <gnucash-user at gnucash.org

> > <mailto:gnucash-user at gnucash.org> >

> > 

> > Subject: Re: [GNC] Gnucash logs

> > 

> > If the setting were per book and user-configurable, then a user could

> > choose

> > 

> > to set each data file to use a different set of folders, by changing the

> > 

> > file default from "logs" to "logs-file1"

> > 

> > 

> > 

> > Does the Attachments functionality provide a blueprint for how this could

> > 

> > work for backups and logs? It seems to me that there is a setting

> > somewhere

> > 

> > that allows the user to point to a particular folder for external file

> > 

> > association (although my attempts at locating that setting have failed).

> > 

> > But your point about the logs and backups having different filenames is

> > apt

> > 

> > as well.

> > 

> > David T.

> > 

> > On 4/29/2020 8:55 AM, Chris Good wrote:

> > > From: David H <hellvee at gmail.com  <mailto:hellvee at gmail.com%20%3cmailto:hellvee at gmail.com> <mailto:hellvee at gmail.com> >

> > > 

> > > Sent: Wednesday, 29 April 2020 12:49 PM

> > > 

> > > To: Chris Good <goodchris96 at gmail.com  <mailto:goodchris96 at gmail.com%20%3cmailto:goodchris96 at gmail.com> <mailto:goodchris96 at gmail.com> >

> > > 

> > > Cc: Gnucash Users <gnucash-user at gnucash.org

> > > <mailto:gnucash-user at gnucash.org> >

> > > 

> > > Subject: Re: [GNC] Gnucash logs

> > > 

> > > Chris,

> > > 

> > > How would you see this working with my 2 data files that I have in the

> > > 

> > > same gnucash folder ??? Would the subdirectory(ies) include the name of

> > > 

> > > the data file to clearly identify them or would all log/backup files be

> > > 

> > > lumped in together in a single subdirectory ? When I save my gnucash

> > > 

> > > file under another name using save as would it create the required

> > > 

> > > subdirectories for the new data file if they are separate subdirectories

> > > 

> > > ?

> > > My own situation is that I've set the log files to be deleted after 30

> > > 

> > > days which seems to work on Mac Catalina. My data files are in a

> > > 

> > > dedicated folder in my Documents folder and I must confess that I have

> > > 

> > > NEVER had a need to go looking for backup files or log files since

> > > 

> > > starting with Gnucash in 2010.

> > > 

> > > Cheers David Halverson.

> > > 

> > > On Wed, 29 Apr 2020 at 11:48, Chris Good <goodchris96 at gmail.com

> > > 

> > > <mailto:goodchris96 at gmail.com> > wrote:

> > > 

> > > -------- Original Message --------

> > > 

> > > From: Michael or Penny Novack <stepbystepfarm at comcast.net

> > > 

> > > <mailto:stepbystepfarm at comcast.net> >

> > > 

> > > Sent: Tue Apr 28 22:11:04 GMT+05:30 2020

> > > 

> > > To: "D." <sunfish62 at yahoo.com 

> > > <mailto:sunfish62 at yahoo.com%20%3cmailto:sunfish62 at yahoo.com>

> > > <mailto:sunfish62 at yahoo.com> >

> > > 

> > > Cc: Gnucash Users <gnucash-user at gnucash.org

> > > 

> > > <mailto:gnucash-user at gnucash.org> >

> > > 

> > > Subject: Re: [GNC] Gnucash logs

> > > 

> > > On 4/27/2020 10:50 PM, D. wrote:

> > >> Michael,

> > >> 

> > >> I take your point; we users will often create duplicated file names

> > >> 

> > >> for

> > > 

> > > different content. I believe that many users only keep one set of

> > > 

> > > books. I don't have any statistics on general GnuCash usage to be able

> > > 

> > > to say whether more users have one file or many.

> > > 

> > >> However, given that operating systems prevent identical file names to

> > >> 

> > >> coexist in a folder, it would seem to me that there is a simple

> > >> 

> > >> remedy to overlapping log files: per book log folder settings. In

> > >> 

> > >> your case, business1financials/ledger.gnucash would use

> > >> 

> > >> business1financials/logs and organization1financials/ledger.gnucash

> > >> 

> > >> would use organization1financials/logs

> > >> 

> > >> 

> > >> 

> > >> Problem solved.

> > > 

> > > Yes, but that is solving the problem by a different method (not

> > > 

> > > writing the log files to a dedicated directory for all log files but

> > > 

> > > writing them into a subdirectory of the directory containing the data

> > > 

> > > file). Would be easy for gnucash when starting to check for the

> > > existence

> > > 

> > > of this subdirectory, if not there, create it. Much as it checks for the

> > > 

> > > existence of a lock file.

> > > 

> > > 

> > > 

> > > Had THAT been suggested you would not have seen an objection from me.

> > > 

> > > But about "almost all users have only one set of books" I have to laugh.

> > > 

> > > If all those rare situations never existed, designing/writing software

> > > 

> > > would be a snap. Software solution have to ALWAYS work. A commonly

> > > 

> > > quoted rule of rule of thumb, 80% of the design/write time will be

> > > 

> > > handling just 20% of the cases, but in my experience, 50% of the time

> > > 

> > > will be handling those that are 1% or less.

> > > 

> > > Michael

> > > 

> > > Hi,

> > > 

> > > I like the idea of putting logs in a 'log' subdirectory of the data

> > > 

> > > file folder and also putting the backups in a 'backup' subdirectory of

> > > 

> > > the data file folder. GnuCash would create these on startup if needed.

> > > 

> > > If I could get general approval in principle, I could even start work on

> > > 

> > > it.

> > > 

> > > Regards, Chris Good

> > > 

> > > _______________________________________________

> > > 

> > > 

> > > Hi David H,

> > > 

> > > 

> > > The backup filenames and log filenames already include the book name and

> > > I

> > > 

> > > see no reason to change that.

> > > 

> > > People could still have multiple books in the same directory, although I

> > > 

> > > suggest best practice is to put each book’s data file in a separate

> > > 

> > > directory.

> > > 

> > > Regards, Chris Good

> > 

> > Hi David T, 

> > 

> > Adding new options and a directory selection dialog seems like a lot of

> > 

> > extra work, extra support complication and very little gain over putting

> > 

> > each book data in its own directory, with backups and logs in

> > 

> > subdirectories. Some people have enough trouble finding their data files,

> > 

> > without enabling them to put their backups and logs somewhere unrelated to

> > 

> > the data file.

> > 

> > Someone else seemed to suggest it would be a good idea to put logs from

> > 

> > different books together in a single directory. I cannot think of a good

> > 

> > reason for that and think that will just cause confusion.

> > 

> > My 0.2c.

> > 

> > Regards, Chris Good

> > 

 

Does anyone else have some input?

If this is going to be ready for GnuCash 4.0 due 20 June 2020, I need to get started ASAP. Otherwise, it 

won’t be available till 5.0 which is not listed in https://wiki.gnucash.org/wiki/Release_Schedule

but 4.11 is planned for 2022-12-18.

 

Regards, Chris Good