[GNC] GnuCash 5.5 (macOS) can't get quotes, because it doesn't see JSON::Parse
john
jralls at ceridwen.us
Sun Dec 24 13:25:25 EST 2023
> On Dec 23, 2023, at 20:03, Jim DeLaHunt <list+gnucash at jdlh.com> wrote:
>
>
> On 2023-12-23 19:19, Jim DeLaHunt wrote:
>>
>> On 2023-12-23 15:38, John Ralls wrote:
>>>
>>>> On Dec 23, 2023, at 12:29 PM, Jim DeLaHunt <list+gnucash at jdlh.com> wrote:
>>>>
>>>> Hi, folks:
>>>>
>>>> I recently upgraded to GnuCash 5.5-1 on macOS, ...retrieving quotes via AlphaVantage for ... stopped working.
>>>> ...
>>>> But the gnucash-fq-update utility thinks everything is fine:
>>>>
>>>> % PATH='/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin' sudo /Applications/Gnucash.app/Contents/Resources/bin/gnc-fq-update
>>>> Password:
>>>>
>>>> Starting with /...[elided].../ this dialog will disappear.
>>>> Reading '/Users/jdlh/.cpan/Metadata'
>>>> Database was generated on Fri, 22 Dec 2023 23:54:07 GMT
>>>> Test2 is up to date (1.302198).
>>>> Finance::Quote is up to date (1.58).
>>>> JSON::Parse is up to date (0.62).
>>>>
>>>> And CPAN seems to think that JSON::Parse is present:
>>>>
>>>> % PATH='/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin' sudo cpan
>>>> Password:
>>>> Loading internal logger. Log::Log4perl recommended for better logging
>>>>
>>>> Starting with /...[elided].../ this dialog will disappear.
>>>> Terminal does not support /...[elided]...
>>>> /cpan shell -- CPAN exploration and modules installation (v2.36)
>>>> Enter 'h' for help.
>>>>
>>>> cpan[1]> m JSON::Parse
>>>> Reading '/Users/jdlh/.cpan/Metadata'
>>>> Database was generated on Fri, 22 Dec 2023 23:54:07 GMT
>>>> Module id = JSON::Parse
>>>> CPAN_USERID BKB (Ben Bullock <bkb at cpan.org>)
>>>> CPAN_VERSION 0.62
>>>> CPAN_FILE B/BK/BKB/JSON-Parse-0.62.tar.gz
>>>> UPLOAD_DATE 2022-07-15
>>>> MANPAGE JSON::Parse - Parse JSON
>>>> INST_FILE /Library/Perl/5.30/darwin-thread-multi-2level/JSON/Parse.pm
>>>> INST_VERSION 0.62
>>>>
>>>>
>>>> cpan[2]> install JSON::Parse
>>>> JSON::Parse is up to date (0.62).
>>>>
>>>> ...For what it's worth, MacPorts has perl 5.34.3 installed, and the perl 5.34 edition of JSON::Parse as well. The system has perl 5.30.3 installed. I do not know if this difference is significant.
>>>>
>>>> ...Thank you in advance for your help. And many thanks to the GnuCash developers for all the improvements in the 5.x versions!
>>> ...You can force using that system perl on the command line by using the path instead of just `perl`, e.g.
>>> sudo /usr/bin/perl /Applications/Gnucash.app/Contents/Resources/bin/gnc-fq-update
>>>
>>> will force using the system perl and its builtin @INC as long as $PERL5LIB isn't set in the environment.
>>>
>>> Note that if your mac is an Apple Silicon one that you need to specify arch x86_64, as in
>>> sudo arch -arch x86_64 /usr/bin/perl /Applications/Gnucash.app/Contents/Resources/bin/gnc-fq-update
>>>
>>> (all on one line) because otherwise perl will build and install F::Q and friends for Apple Silicon....
>>
>> My computer does indeed use Apple Silicon. Specifying the path to perl had no effect:
>>
>> % sudo arch -arch x86_64 /usr/bin/perl /Applications/Gnucash.app/Contents/Resources/bin/gnc-fq-update
>>
>> Starting with /...[elided].../ this dialog will disappear.
>> Reading '/Users/jdlh/.cpan/Metadata'
>> Database was generated on Fri, 22 Dec 2023 23:54:07 GMT
>> Test2 is up to date (1.302198).
>> Finance::Quote is up to date (1.58).
>> JSON::Parse is up to date (0.62).
>> %
>>
>> Maybe my installation somehow installed JSON::Parse for Apple Silicon, and that is why the GnuCash application cannot find it.
>>
>> I would try to uninstall JSON::Parse and Finance::Quote, and let cpan or gnc-fq-update reinstall them, but a brief web search seems to say that CPAN has no command to uninstall modules.
>>
>> I do think I have found where the actual files might be:
>>
>> % ls -ld /Library/Perl/5.30/{,darwin-thread-multi-2level/auto/}{JSON/Parse,Finance/Quote}
>> ls: /Library/Perl/5.30/JSON/Parse: No such file or directory
>> drwxr-xr-x 59 root wheel 1888 21 Dec 00:32 /Library/Perl/5.30/Finance/Quote
>> drwxr-xr-x 3 root wheel 96 10 May 2022 /Library/Perl/5.30/darwin-thread-multi-2level/auto/Finance/Quote
>> drwxr-xr-x 4 root wheel 128 21 Dec 00:32 /Library/Perl/5.30/darwin-thread-multi-2level/auto/JSON/Parse
>>
>> Is it significant that Finance::Quote has an entry in /Library/Perl/5.30/, but JSON::Parse does not?
>>
> I appear to have fixed it. I suspect that John's comment about
>
>> ...you need to specify arch x86_64...because otherwise perl will build and install F::Q and friends for Apple Silicon. GnuCash is built for Intel only because WebKitGtk doesn't work when built for Apple Silicon and GnuCash will run an x86_64 perl under Rosetta2; if a needed module isn't available for Intel then F::Q will fail.
> I followed a hypothesis that some of the Perl modules in my system location were built for Apple Silicon, and that the GnuCash application (being built for Intel only) could not find them.
>
> I could run /usr/bin/cpan to look at modules in the system location. See transcript above. It includes:
>
> cpan[1]> m JSON::Parse
> /...[elided].../
> Module id = JSON::Parse
> / ...[elided].../
> INST_FILE /Library/Perl/5.30/darwin-thread-multi-2level/JSON/Parse.pm
> INST_VERSION 0.62
>
> I found files at that path, JSON/Parse.pm and JSON/Parse.pod. I deleted them:
>
> % sudo rm /Library/Perl/5.30/darwin-thread-multi-2level/JSON/Parse*
>
> Now CPAN considered the file uninstalled. So I had GnuCash do gnc-fq-update:
>
> % sudo arch -arch x86_64 /usr/bin/perl /Applications/Gnucash.app/Contents/Resources/bin/gnc-fq-update
> Reading '/Users/jdlh/.cpan/Metadata'
> Database was generated on Fri, 22 Dec 2023 23:54:07 GMT
> Test2 is up to date (1.302198).
> Finance::Quote is up to date (1.58).
> Running install for module 'JSON::Parse'
> /...[hundreds of lines elided].../
> Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
> Installing /Library/Perl/5.30/darwin-thread-multi-2level/auto/JSON/Parse/Parse.bundle
> Installing /Library/Perl/5.30/darwin-thread-multi-2level/JSON/Parse.pod
> Installing /Library/Perl/5.30/darwin-thread-multi-2level/JSON/Parse.pm
> Appending installation info to /Library/Perl/Updates/5.30.3/darwin-thread-multi-2level/perllocal.pod
> BKB/JSON-Parse-0.62.tar.gz
> /usr/bin/make install -- OK
> %
>
> I asked GnuCash-cli what it thought of my Finance::Quote installation now.
>
> % PATH='/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin' /Applications/Gnucash.app/Contents/MacOS/gnucash-cli --quotes info
> Application Path /Applications/Gnucash.app/Contents/MacOS/gnucash-cli
> Finance::Quote check returned error Can't load '/Library/Perl/5.30/darwin-thread-multi-2level/auto/DateTime/DateTime.bundle' for module DateTime: dlopen(/Library/Perl/5.30/darwin-thread-multi-2level/auto/DateTime/DateTime.bundle, 0x0001): tried: '/Library/Perl/5.30/darwin-thread-multi-2level/auto/DateTime/DateTime.bundle' (mach-o file, but is an incompatible architecture (have (arm64), need (x86_64))) at /System/Library/Perl/5.30/XSLoader.pm line 96.
>
> Apparently my DateTime module was also unsatisfactory. The message "incompatible architecture (have (arm64), need (x86_64))" seems to point to what John was saying about installing Perl modules for x86 not Apple Silicon.
>
> CPAN told me where to find the DateTime module:
>
> cpan[1]> i DateTime
> /...[elided].../
> Module id = DateTime
> / ...[elided].../
> INST_FILE /Library/Perl/5.30/darwin-thread-multi-2level/DateTime.pm
> INST_VERSION 1.58
>
> That module seemed to be a single file. I deleted that file:
>
> % sudo rm /Library/Perl/5.30/darwin-thread-multi-2level/DateTime.pm
>
> Then I used CPAN, guided to the x86 architecture, to install the DateTime module again,
>
> % sudo arch -arch x86_64 /usr/bin/cpan
> /...[elided].../
>
> cpan[1]> install DateTime
> Reading '/Users/jdlh/.cpan/Metadata'
> Database was generated on Fri, 22 Dec 2023 23:54:07 GMT
> Running install for module 'DateTime'
> /...[elided]...
> /Installing /Library/Perl/5.30/darwin-thread-multi-2level/auto/DateTime/DateTime.bundle
> Installing /Library/Perl/5.30/darwin-thread-multi-2level/DateTime.pm
> Installing /Library/Perl/5.30/darwin-thread-multi-2level/DateTime/Infinite.pm
> /...[elided]...
> / DROLSKY/DateTime-1.65.tar.gz
> /usr/bin/make install -- OK
>
> Again, I asked GnuCash-cli what it thought of my Finance::Quote installation.
>
> % PATH='/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin' /Applications/Gnucash.app/Contents/MacOS/gnucash-cli --quotes info
> Application Path /Applications/Gnucash.app/Contents/MacOS/gnucash-cli
> Found Finance::Quote version 1.58.
> Finance::Quote sources:
> aex alphavantage amfiindia asegr asx
> /...[elided]...
> /xetra yahoo_json yahooweb za
>
> Then I launched the GnuCash application. Now, the "Get Quotes" button in the Price Database dialogue was active, and I could retrieve quotes successfully.
>
> Yay!
>
> Conclusions I tentatively draw from this experience:
>
> 1. GnuCash users on macOS with Apple Silicon must be careful to run
> gnc-fq-update with the arch -arch x86_64 qualifier.
> 2. gnc-fq-update fails to report at least some problems with Perl
> modules compiled for Apple Silicon, where the GnuCash application
> rejects the configuration.
> 3. When the GnuCash application rejects the configuration due to Perl
> modules compiled for Apple Silicon, neither the diagnostics printed
> by gnucash-cli nor the telemetry in the --debug log written by the
> GnuCash application mentions that Perl modules compiled for the
> wrong architecture are the problem.
> 4. It would be nice if the GnuCash + Finance::Quote system did not
> suffer from this fragility. But I understand that the root cause of
> the fragility is that WebKitGtk doesn't work when built for Apple
> Silicon, and that is not trivial to overcome.
> 5. The Online Quotes wiki page should get some information about the
> need to get Perl modules compiled for x86 architecture, but I don't
> think I know enough to write that information correctly.
>
> Does that sound right?
>
> Again, thank you for your help, John.
Jim,
JSON/Parse.pm and DateTime.pm are the perl parts of those modules. While removing them was sufficient to get cpan to reinstall them, the parts that were actually causing the problem are /Library/Perl/5.30/darwin-thread-multi-2level/auto/DateTime/DateTime.bundle and JSON/Parse.bundle, the compiled C resources that the perl parts depend on, that need to be recompiled to the right architecture. You can learn the architectures of all of the compiled modules in /Library/Perl with
find /Library/Perl -name "*.bundle" -exec file {} \;
GnuCash's perl scripts, finance-quote-wrapper and gnc-fq-update, are of necessity quite generic: They have to run on both Unix and Microsoft operating systems. Moreover perl itself doesn't report many errors in machine-interpretable form. It would be difficult to get the information that modules installed for the wrong architecture into a tracefile.
As for the wiki's Online Quotes page, are the year-old directions to prefix gnc-fq-update with arch -arch x86_64 on Apple Silicon macs not clear enough? What more should it say? Perhaps something about diagnosing a failed installation and the steps needed to repair it?
Regards,
John Ralls
More information about the gnucash-user
mailing list