Gnucash 2.6: problem opening file

[John Ralls]

On Jan 18, 2014, at 1:26 AM, Geert Janssens <janssens-geert at telenet.be> wrote:

> On Friday 17 January 2014 14:29:39 Cristian Marchi wrote:
>> Il 17/01/2014 13.11, Geert Janssens ha scritto:
>>> On Friday 17 January 2014 12:51:00 Cristian Marchi wrote:
>>>> Il 17/01/2014 1.17, John Ralls ha scritto:
>>>>> Thanks to the trace file provided in Bug 721687, I was able to
>>>>> 
>>>>> quickly track this down to a call to
>>>>> 
>>>>> g_win32_locale_filename_from_utf8. Removing that call fixed the
>>>>> 
>>>>> problem for me on a US edition of Win7 Home Pro, but I’m
>>>>> concerned
>>>>> 
>>>>> about folks with non-English Windows editions, particularly XP.
>>>>> I’d
>>>>> 
>>>>> appreciate it if everyone with such installations can test
>>>>> tomorrow
>>>>> 
>>>>> morning’s (in the Eastern US time zone) nightly build. It will
>>>>> be
>>>>> 
>>>>> at http://code.gnucash.org/builds/win32/trunk/ and will have a
>>>>> 
>>>>> filename like gnucash-2.6.0-2014-01-17-git-xxxxxxx+-setup.exe
>>>>> with
>>>>> 
>>>>> some hex gibberish instead of the row of ‘x’s.
>>>>> 
>>>>> 
>>>>> 
>>>>> For those interested, the change is r23700.
>>>>> 
>>>>> 
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> John Ralls
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> 
>>>>> gnucash-user mailing list
>>>>> 
>>>>> gnucash-user at gnucash.org
>>>>> 
>>>>> https://lists.gnucash.org/mailman/listinfo/gnucash-user
>>>>> 
>>>>> -----
>>>>> 
>>>>> Please remember to CC this list on all your replies.
>>>>> 
>>>>> You can do this by using Reply-To-List or Reply-All.
>>>> 
>>>> I've just tested the nightly build under XP with Italian locale
>>>> and it
>>>> 
>>>> works. Thanks for your work John!
>>>> 
>>>> 
>>>> 
>>>> Regards
>>>> 
>>>> Cristian
>>> 
>>> Cristian, thanks for your test and feedback.
>>> 
>>> I'm not sure though that our European languages are "foreign" enough
>>> to trigger the bug.
>>> 
>>> To be sure we'd need a tester for languages such as Russian,
>>> Chinese,
>>> or similar. Those that have a different code page on Windows than
>>> the
>>> Western code page.
>>> 
>>> Geert
>> 
>> I've personally tested that the bug was affecting me [1]. I don't know
>> if Italian accented characters are only a subcase, but with the
>> official 2.6.0 I was effected by this bug and with the latest nightly
>> build the problem is solved.
>> 
>> Regards
>> Cristian
>> 
>> [1]
>> http://gnucash.1415818.n4.nabble.com/Gnucash-2-6-problem-opening-file-> td4666831.html#a4667009
>> 
>> 
> Ahem, I clearly have too much different things to keep track of at the moment...
> 
> So your test is a good confirmation. Great !
> 
> More in general it was something I read on the glib website some time ago that made me 
> believe that what Mike did was indeed necessary for some locales. And at the time it made me 
> believe we had some more work ahead of us to find all situations where we were using strings 
> passed in from the operating system and treat them in the same way.
> 
> I have just read that page [1] again and it looks like I misunderstood. We should apparently 
> only convert file system strings if we want to show them to the user. So it looks like John's fix is 
> sufficient.
> 

No, we shouldn't convert file system strings at all, at least in that direction. GLib takes care of that for us.
The manual page you're referring to is talking about reading file names from a Unix OS, almost all modern versions
of which do indeed store filenames in UTF8. Windows uses UTF16 for encoding filenames and the system code page for
displaying them to the user. GLib knows this and does the conversion before calling fopen().

The only time we'd need to convert a file system string would be to pass it back to a native Windows function,
and in that case we'd normally convert *from* UTF8 to UTF16 for almost everything; we'd only convert it to the
locale codepage to display it in a non-MSYS command window. 

For those curious about why I know this, I rewrote the Gramps localization code for the new Gtk3 version a year ago.
Gramps uses Gtk only for the GUI, relying on Python for everything else, and Python doesn't do any transcoding automatically.
I got to learn more than I ever wanted to know about Windows codepages.

Regards,
John Ralls