Text encoding issue when importing Japanese OFX files

[z33b0b at gmail.com]

Actually, I just found that I was wrong earlier -- I can type in Japanese
when entering the name of an account on account creation, but I cannot type
in Japanese in any transaction entry.  I'm using Gnome with SCIM, with the
Anthy conversion engine for Japanese.

Cheers,

-- Erik


2009/10/27 Christian Stimming <stimming at tuhh.de>

> Dear Erik,
>
> thanks for the nice explanation of your OFX import with utf-8 characters.
> Some
> questions:
>
> -  Which menu item did you use for the OFX import in gnucash - "File ->
> Import -> Import OFX", or any of the Online Banking items?
> - Which libofx version is installed on your system?
> - Can you provide a (small) example file with at least one line of Japanese
> characters which is currently not imported correctly?
>
> Regards,
>
> Christian
>
> Am Dienstag, 27. Oktober 2009 07:09 schrieb z33b0b at gmail.com:
> > Hello dev folks --
> >
> > Running GnuCash 2.3.7 on Ubuntu 9.04.
> >
> > I might be able to serve as a fun test case, as I have accounts in Japan.
> > I can download data in OFX format with the following header:
> >
> >
> > OFXHEADER:100
> > DATA:OFXSGML
> > VERSION:102
> > SECURITY:NONE
> > ENCODING:UTF-8
> > CHARSET:CSUNICODE
> > COMPRESSION:NONE
> > OLDFILEUID:NONE
> > NEWFILEUID:NONE
> >
> >
> > The transaction list includes Japanese text.  I assume the encoding is as
> > marked, as GVim and other text editors (and even CLI less) seem to
> display
> > it just fine.  Attempting to import into GnuCash, however, renders all
> > Japanese text into ASCII gibberish, a phenomenon well-known among
> > Japanese-savvy circles as "moji-bake" (that's pronounced "bah keh", not
> > "bake").  I've attached a screenshot in .png format for reference.
> >
> > I cannot tell if the issue is simply a font problem, or if GnuCash isn't
> > recognizing the UTF-8 text.  For that matter, I'm not even sure where to
> > specify fonts.  I tried altering the fonts specified in the Default CSS
> > setting, but to no apparent avail.  For what it's worth, most font
> display
> > problems will show unrecognized glyphs as empty squares rather than
> random
> > ASCII, which makes me think that GnuCash isn't respecting the UTF-8
> > encoding marked in the header, and is instead trying to read the file as
> > ASCII.  My system locale is set to use UTF-8 by default:
> >
> >
> > eirikr at Boreas:~$ echo $LANG
> > en_US.UTF-8
> > eirikr at Boreas:~$
> >
> >
> > Meanwhile, I can enter Japanese text directly into GnuCash just fine, but
> > I'd really rather not have to manually enter all my Japanese-language
> > transactions when I can just download them instead.  Any ideas what's
> going
> > wrong with the import, and better yet, how to fix it?
> >
> > Cheers,
> >
> > -- Erik Anderson
>