[GNC] ASX quotes not working
Geoff
cleanoutmyshed at gmail.com
Fri Oct 16 23:40:58 EDT 2020
Hi John
Thanks for your patience and detailed feedback, it has been very helpful.
Firstly, I installed strawberry-perl-5.18.2.2-32bit and was able to
reproduce the original error you reported on line 138. This was due to
sloppy coding on my part. Later versions of perl are more forgiving,
however that line should in fact be:
$info{$stock,$label} = $decoded_json->{$label_map{$label}};
(The explanation being that $decoded_json is a *reference* to a hash,
and the correct way to dereference it and access its values is with the
"->" notation, not "%$".)
Secondly, the error you encountered with stock "14D" (in your email
below) is not due to the stock code starting with a number, but a lack
of defensive coding on my part. For this stock, the ASX did not return
the "year_range" data element, and the message you saw (which is only a
warning) says that the element has an undefined value. I have attached
an updated version of ASX.pm with a fix for this:
C:\Program Files (x86)\gnucash\bin>perl gnc-fq-dump asx 14d
Finance::Quote fields Gnucash uses:
symbol: 14d <=== required
date: 10/16/2020 <=== recommended
currency: AUD <=== required
last: 0.135 <=\
nav: <=== one of these
price: 0.135 <=/
timezone: <=== optional
Thirdly, I can successfully retrieve prices in GnuCash itself - see
attached screenshot. If you don't mind, can you review your
configuration against that, and try again with the updated ASX.pm module?
Fourthly, the Finance::Quote developers have contacted me and I hope to
be able to submit this as a fix that will go into the distribution.
After some more rigourous testing obviously!
Regards
Geoff
=====
On 16/10/2020 3:29 pm, John Bonnett wrote:
> Hi Geoff,
>
> I am still not getting all my stocks retrieved inside GnuCash but I may
> have made some progress.
>
> I checked the stocks not working using gnc-fq-dump and they all
> retrieved except for one I still had that had been de-listed, so that is
> as expected, but does not explain why they don't work inside GnuCash.
>
> I retrieved them all in one call to gnc-fq-dump and I noticed there was
> an error message at the start. After a little more digging I found just
> one stock was causing that problem as follows:
>
> PS C:\Program Files (x86)\gnucash\bin> perl gnc-fq-dump ASX 14D
> Use of uninitialized value in transliteration (tr///) at
> C:/strawberry/perl/site/lib/Finance/Quote/ASX.pm line 141.
> Finance::Quote fields Gnucash uses:
> symbol: 14D <=== required
> date: 10/16/2020 <=== recommended
> currency: AUD <=== required
> last: 0.145 <=\
> nav: <=== one of these
> price: 0.145 <=/
> timezone: <=== optional
> PS C:\Program Files (x86)\gnucash\bin>
>
> It may be that having a stock symbol starting with a number is causing
> the problem, but it still retrieves OK. It did occur to me that,
> depending on the order that GnuCash does the retrieving internally, that
> error may make the program think there are no valid values after that
> message and explain why some of the stocks are missing in the results.
>
> Best regards,
> John Bonnett
>
> On 16/10/2020 9:05 am, John Bonnett wrote:
>> Hi Geoff,
>>
>> I seem to have partly solved my problem. Following up on a comment
>> from Bill and your interest in the version of Perl I was running, I
>> chased up a recent version of Strawberry Perl.
>>
>> I got v5.32.0 64bit and installed that. It still thought I had the
>> older 32bit installed so I tried uninstalling that. This broke the
>> 64bit install, so I repaired it and then ran the "Install Online Price
>> Retrieval for GnuCash" option on the Windows GnuCash menu. This got
>> the standard installation back. I then followed your original
>> instructions about replacing ASX.pm and ran your check as follows:
>>
>> PS C:\WINDOWS\system32> cd "C:\Program Files (x86)\gnucash\bin"
>> PS C:\Program Files (x86)\gnucash\bin> perl gnc-fq-dump ASX BHP
>> Finance::Quote fields Gnucash uses:
>> symbol: BHP <=== required
>> date: 10/15/2020 <=== recommended
>> currency: AUD <=== required
>> last: 36.77 <=\
>> nav: <=== one of these
>> price: 36.77 <=/
>> timezone: <=== optional
>> PS C:\Program Files (x86)\gnucash\bin>
>>
>> I then tried running "Get Quotes" from inside GnuCash and it seems to
>> have partly worked. Some of my ASX stocks are still not retrieving but
>> others are. I will investigate using gnc-fq-dump and report back.
>>
>> Thanks for your help,
>>
>> John Bonnett
>>
>
-------------- next part --------------
#!/usr/bin/perl -w
#
# Copyright (C) 1998, Dj Padzensky <djpadz at padz.net>
# Copyright (C) 1998, 1999 Linas Vepstas <linas at linas.org>
# Copyright (C) 2000, Yannick LE NY <y-le-ny at ifrance.com>
# Copyright (C) 2000, Brent Neal <brentn at users.sourceforge.net>
# Copyright (C) 2001, Leigh Wedding <leigh.wedding at telstra.com>
# Copyright (C) 2000-2004, Paul Fenwick <pjf at cpan.org>
# Copyright (C) 2014, Chris Good <chris.good@@ozemail.com.au>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
# 02111-1307, USA
#
#
# This code derived from Padzensky's work on package Finance::YahooQuote,
# but extends its capabilites to encompas a greater number of data sources.
#
# This code was developed as part of GnuCash <http://www.gnucash.org/>
require 5.005;
use strict;
use warnings;
package Finance::Quote::ASX;
use HTTP::Request::Common;
use LWP::UserAgent;
use HTML::TableExtract;
use Encode;
use JSON;
use vars qw/$ASX_URL @ASX_SEC_CODES/;
our $VERSION = '1.49'; # VERSION
$ASX_URL = 'https://www.asx.com.au/asx/1/share/';
# These are the ASX codes starting with X that are securities not indexes
# and thus need currency AUD returned
# See http://www.asx.com.au/asx/research/listedCompanies.do
@ASX_SEC_CODES = (qw/XAM XIP XRO XPD XPE XF1 XRF XST XTD XTE XTV/);
sub methods {return (australia => \&asx,asx => \&asx)}
{
my @labels = qw/last net p_change bid offer high low volume
price method exchange/;
sub labels { return (australia => \@labels,
asx => \@labels); }
}
# Australian Stock Exchange (ASX)
# The ASX provides free delayed quotes through their webpage.
#
# Maintainer of this section is Paul Fenwick <pjf at cpan.org>
# 5-May-2001 Updated by Leigh Wedding <leigh.wedding at telstra.com>
# 24-Feb-2014 Updated by Chris Good <chris.good@@ozemail.com.au>
sub asx {
my $quoter = shift;
my @all_stocks = @_;
return unless @all_stocks;
my @stocks;
my %info;
my $ua = $quoter->user_agent;
# Map the Finance::Quote labels (left) to the corresponding ASX labels (right)
my %label_map = (
'bid' => 'bid_price',
'cap' => 'market_cap',
'close' => 'previous_close_price',
'date' => 'last_trade_date',
'div_yield' => 'annual_dividend_yield',
'eps' => 'eps',
'high' => 'day_high_price',
'last' => 'last_price',
'low' => 'day_low_price',
'net' => 'change_price',
'offer' => 'offer_price',
'open' => 'open_price',
'p_change' => 'change_in_percent',
'pe' => 'pe',
'volume' => 'volume',
'year_range' => 'year_change_price',
);
# ASX webpage only handles 1 quote requests at a time
while (@stocks = splice(@all_stocks, 0, 1)) {
sleep 1 if $#all_stocks > 0; # Don't hammer the ASX web site and get blocked!
my $response = $ua->request(GET $ASX_URL.join("%20", at stocks));
unless ($response->is_success) {
foreach my $stock (@stocks, @all_stocks) {
$info{$stock,"success"} = 0;
$info{$stock,"errormsg"} = "HTTP session failed";
}
return wantarray() ? %info : \%info;
}
my $json = $response->content;
my $decoded_json = eval{decode_json( $json )};
if($@) {
foreach my $stock (@stocks, @all_stocks) {
$info{$stock,"success"} = 0;
$info{$stock,"errormsg"} = "Failed to parse JSON data, error '$@'.";
}
return wantarray() ? %info : \%info;
}
# Pack the resulting data into our structure.
my $stock = $stocks[0];
# Delete spaces and '*' which sometimes appears after the code.
# Also delete high bit characters.
$stock =~ tr/* \200-\377//d;
# Delete any whitespace characters
$stock =~ s/\s//g;
$info{$stock,'symbol'} = $stock;
foreach my $label (keys %label_map) {
if ((exists $decoded_json->{$label_map{$label}}) &&
(defined $decoded_json->{$label_map{$label}})) {
$info{$stock,$label} = $decoded_json->{$label_map{$label}};
# Again, get rid of nasty high-bit characters.
$info{$stock,$label} =~ tr/ \200-\377//d
unless ($label eq "name");
}
else {
$info{$stock,$label} = '';
}
}
# get rid of trailing whitespace after 'last'
$info{$stock,'last'} =~ s/\s//g;
# If that stock does not exist, it will have a empty
# string for all the fields. The "last" price should
# always be defined (even if zero), if we see an empty
# string here then we know we've found a bogus stock.
if ($info{$stock,'last'} eq '') {
$info{$stock,'success'} = 0;
$info{$stock,'errormsg'}="Stock does not exist on ASX.";
next;
}
# Drop commas from volume.
$info{$stock,"volume"} =~ tr/,//d;
# The ASX returns zeros for a number of things if there
# has been no trading. This not only looks silly, but
# can break things later. "correct" zero'd data.
foreach my $label (qw/open high low/) {
if ($info{$stock,$label} == 0) {
$info{$stock,$label} = $info{$stock,"last"};
}
}
# Remove trailing percentage sign from p_change
$info{$stock,"p_change"} =~ tr/%//d;
## This logic is flawed because the hard coded list of "X" companies is out of date
## # Australian indexes all begin with X, so don't tag them
## # as having currency info.
##
## $info{$stock, "currency"} = "AUD" unless ($stock =~ /^X/);
##
## # There are some companies starting with X, so DO tag
## # them with currency AUD
##
## if ( grep( /^$stock$/, @ASX_SEC_CODES ) ) {
## $info{$stock, "currency"} = "AUD";
## }
$info{$stock, "currency"} = "AUD";
# Convert Date from 'CCYY-MM-DDThh:mm:ss+tztz' to Finance::Quote's MM/DD/CCYY format
$info{$stock, "date"} = substr($info{$stock, "date"},5,2) . '/' .
substr($info{$stock, "date"},8,2) . '/' .
substr($info{$stock, "date"},0,4);
$info{$stock, "method"} = "asx";
$info{$stock, "exchange"} = "Australian Stock Exchange";
$info{$stock, "price"} = $info{$stock,"last"};
$info{$stock, "success"} = 1;
}
# All done.
return %info if wantarray;
return \%info;
}
1;
=head1 NAME
Finance::Quote::ASX - Obtain quotes from the Australian Stock Exchange.
=head1 SYNOPSIS
use Finance::Quote;
$q = Finance::Quote->new;
%stockinfo = $q->fetch("asx","BHP"); # Only query ASX.
%stockinfo = $q->fetch("australia","BHP"); # Failover to other sources OK.
=head1 DESCRIPTION
This module obtains information from the Australian Stock Exchange
http://www.asx.com.au/. All Australian stocks and indicies are
available. Indexes start with the letter 'X'. For example, the
All Ordinaries is "XAO".
This module is loaded by default on a Finance::Quote object. It's
also possible to load it explicity by placing "ASX" in the argument
list to Finance::Quote->new().
This module provides both the "asx" and "australia" fetch methods.
Please use the "australia" fetch method if you wish to have failover
with other sources for Australian stocks (such as Yahoo). Using
the "asx" method will guarantee that your information only comes
from the Australian Stock Exchange.
Information returned by this module is governed by the Australian
Stock Exchange's terms and conditions.
=head1 LABELS RETURNED
The following labels may be returned by Finance::Quote::ASX:
bid, offer, open, high, low, last, net, p_change, volume,
and price.
=head1 SEE ALSO
Australian Stock Exchange, http://www.asx.com.au/
Finance::Quote::Yahoo::Australia.
=cut
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gnc_asx_prices.jpg
Type: image/jpeg
Size: 120711 bytes
Desc: not available
URL: <http://lists.gnucash.org/pipermail/gnucash-user/attachments/20201017/217462c0/attachment-0001.jpg>
More information about the gnucash-user
mailing list