Text encoding issue when importing Japanese OFX files

[z33b0b at gmail.com]

Hello dev folks --

Running GnuCash 2.3.7 on Ubuntu 9.04.

I might be able to serve as a fun test case, as I have accounts in Japan.  I
can download data in OFX format with the following header:


OFXHEADER:100
DATA:OFXSGML
VERSION:102
SECURITY:NONE
ENCODING:UTF-8
CHARSET:CSUNICODE
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:NONE


The transaction list includes Japanese text.  I assume the encoding is as
marked, as GVim and other text editors (and even CLI less) seem to display
it just fine.  Attempting to import into GnuCash, however, renders all
Japanese text into ASCII gibberish, a phenomenon well-known among
Japanese-savvy circles as "moji-bake" (that's pronounced "bah keh", not
"bake").  I've attached a screenshot in .png format for reference.

I cannot tell if the issue is simply a font problem, or if GnuCash isn't
recognizing the UTF-8 text.  For that matter, I'm not even sure where to
specify fonts.  I tried altering the fonts specified in the Default CSS
setting, but to no apparent avail.  For what it's worth, most font display
problems will show unrecognized glyphs as empty squares rather than random
ASCII, which makes me think that GnuCash isn't respecting the UTF-8 encoding
marked in the header, and is instead trying to read the file as ASCII.  My
system locale is set to use UTF-8 by default:


eirikr at Boreas:~$ echo $LANG
en_US.UTF-8
eirikr at Boreas:~$


Meanwhile, I can enter Japanese text directly into GnuCash just fine, but
I'd really rather not have to manually enter all my Japanese-language
transactions when I can just download them instead.  Any ideas what's going
wrong with the import, and better yet, how to fix it?

Cheers,

-- Erik Anderson
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot-Generic import transaction matcher.png
Type: image/png
Size: 52138 bytes
Desc: not available
URL: <http://lists.gnucash.org/pipermail/gnucash-devel/attachments/20091026/f26d13f3/attachment-0001.png>