[GNC] File Format Documentation (Bug 777893)

Stephen M. Butler kg7je at arrl.net
Thu Aug 16 12:20:01 EDT 2018


I am not familiar with SQLite.  However, general database theory implies
that the log files would be useful if you lost the database and had to
revert to an earlier backup.  Then you could replay the logs and have
the database roll forward to the last transaction recorded in the log
files.  Hence, those who practice the art (sometimes science) of
database administration have the log files on a different physical set
of drives than where the database files reside.  Hopefully, you lose
only one and not both.

So, I wouldn't call them useless but they are there only for that
perfect storm as a last ditch strategy to recover lost transactions.  As
a DBA I tried to keep three full backups and the corresponding log files
to recover from the earliest said backup.  That way if the unthinkable
happened I'd still have a fighting chance of getting all the
transactions back (or at least up to the last log file backup -- done
every time the log file switched throughout the day).

But, I doubt an individual would go to those lengths to ensure
recoverability!

On 08/15/2018 07:18 PM, Adrien Monteleone wrote:
> Roger that. Curious though if this issue should be addressed in documentation, or bugzilla. It’s probably a bug if the format can’t make use of them. They likely should not be generated at all.
>
> Regards,
> Adrien
>
>> On Aug 15, 2018, at 7:22 PM, David T. <sunfish62 at yahoo.com> wrote:
>>
>>
>>
>>> On Aug 15, 2018, at 1:35 PM, Adrien Monteleone <adrien.monteleone at lusfiber.net> wrote:
>>>
>>> Looks pretty clear.
>>>
>>> Though I’m noticing sqlite does generate log files too. Note, these are transaction replay logs from what I understand in case of a crash. As I believe John Ralls noted, any of these past the last successful save are useless. I’m not sure if this is what you are referring to by ‘Uses log files.’  I don’t know if sqlite can even utilize them at all. I’ve never had to try. You might want to get clarification on that point.
>> I am no expert at the SQL back ends, but I was given to understand that the log files are generated regardless of the back end being used—but that for the SQL back ends, they aren’t useful, since changes are written to the file immediately.
>>
>>> Also, I’d get clarification about the state of GnuCash sqlite on Windows. It may or may not be packaged with GnuCash by default and may or may not need some additional software. (if so, you might need a footnote) On Mac & Linux, it’s just a file format choice without any other effort.
>>>
>>> Regards,
>>> Adrien 
>>>
>>>> On Aug 15, 2018, at 12:49 PM, David T. <sunfish62 at yahoo.com> wrote:
>>>>
>>>> Here is a newer version of the table:
>>>>
>>>> Storage Comparison Table
>>>> 	XML	SQLite	MySQL	PostgreSQL
>>>> Installation	Default	Default	libdbi	libdbi
>>>> File extension	gnucash	gnucash	N/A	N/A
>>>> Additional software	None	None	MySQL	PostgreSQL
>>>> Additional expertise	None	None	DBMS	DBMS 
>>>> Compression	Y	N	N	N
>>>> Save on command	Y	N	N	N
>>>> Save on commit	N	Y	Y	Y
>>>> Uses log files	Y	N	N	N
>>>> Multi-user	N	N	N	N
>>>>
>>>> How does that seem?
>>>>
>>>>> On Aug 15, 2018, at 10:34 AM, Adrien Monteleone <adrien.monteleone at lusfiber.net> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> On Aug 15, 2018, at 12:11 PM, David T. <sunfish62 at yahoo.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Aug 15, 2018, at 10:02 AM, Adrien Monteleone <adrien.monteleone at lusfiber.net> wrote:
>>>>>>>
>>>>>>> But it’s not a ‘plain file’ as it is XML formatted. Someone expecting plain text and trying to view it is going to be met with tag soup they’ve never seen before and might very well not know how to read it.
>>>>>> Not to mention that it’s compressed.
>>>>> True, forgot about that. Certainly, they’ll see gibberish mostly.
>>>>>
>>>>>>> It also carries an .xml extension. So specifying the format is very specific and informative, even for users who aren’t familiar with XML. They’ll see in their file manager the extension, and/or the OS’s interpretation of the file type itself. (in this case both XML)
>>>>>> However, the file extension used is “gnucash” and not “xml”
>>>>> Facepalm. I forgot about that. (I honestly rarely even look at the location where it’s stored anyway) I’d suspect unless Win10 uses the file descriptor for file type instead of the extension as was the practice through at least Win7, then no, those users won’t see XML anywhere. (if the descriptor is set as XML that is)
>>>>>
>>>>> So I just checked on both MacOS and Ubuntu, MacOS reports the ‘Kind’ as ‘Gnucash Document’ regardless if sqlite or xml, and at least with xml, Ubuntu reports the file type as ’spreadsheet’. (yes, it’s registered to open with GnuCash, but this was built from source, so perhaps the file type was not registered properly, repo versions may vary)
>>>>>
>>>>> So I guess on that point I was way off.
>>>>>
>>>>>
>>>>>> Perhaps the save process needs to be refactored to identify clearly and separately the name of the data file AND its format?
>>>>> Since .gnucash is not really proprietary or somehow a special format from XML then I agree, the extension should be .xml.
>>>>>
>>>>> Combine this with the fact that the sqlite version of the file ALSO uses the .gnucash extension can make for some confusion. At a glance, you can’t tell what the format is. You can’t even tell until you try to open it with something other than GnuCash. (or you notice that GnuCash doesn’t offer a Save option) The only reason I know which is which is I had to use filename.xml.gnucash to tell them apart. That’s a usability bug in my opinion. I don’t know how hard that is to change, but I’d support the move.
>>>>>
>>>>> On that note, the documentation somewhere (I suppose in the ‘file > save/save as’ section) should document that the extension is currently ‘.gnucash’. A new user shouldn’t have to go to a wiki or website FAQ after reading the documentation for something this basic.
>>>>>
>>>>> Would it be out of order to include in your table that both use this extension? If you expand the table to show MySQL and Postgres, I suppose that row would have some other note since their data stores are very different than single files. (though in this case they might store it that way, I haven’t used either to know)
>>>>>
>>>>> Regards,
>>>>> Adrien
>>>>>
>>>>>
>>>>>>> Knowing this might very well help them find their file if they know the format they are looking for.
>>>>>>>
>>>>>>> But I do agree, the documentation should cover where files are stored. Ideally, this should be made part of the Help or Guide in the Getting Started section. It is certainly a common enough issue on the list.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Adrien
>>>>>>>
>>>>>>>> On Aug 15, 2018, at 10:24 AM, Christoph R <subscriptions+listen at rohland.net> wrote:
>>>>>>>>
>>>>>>>> Hi David,
>>>>>>>>
>>>>>>>>> The default file storage format is XML
>>>>>>>> I would not call this “XML" but "plain file”. From a user perspective it is not important in which internal format it is stored. But it makes a big difference if it is a simple file created by Gnucash or if Gnucash needs to connect to a DBMS.
>>>>>>>>
>>>>>>>> And one of the biggest confusion for users on the mailing list is the question: “Where is my data?”. Pointing out that all your accounts and transactions are in a simple file might reduce that problem.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Christoph
>>>>>>>>
>>>>>>
>>>>>

-- 
Stephen M Butler, PMP, PSM
Stephen.M.Butler51 at gmail.com
kg7je at arrl.net
253-350-0166
-------------------------------------------
GnuPG Fingerprint:  8A25 9726 D439 758D D846 E5D4 282A 5477 0385 81D8



More information about the gnucash-user mailing list