Historical price quotes
Jon Hamkins
hamkins at alumni.caltech.edu
Sun Jul 3 02:11:24 EDT 2005
David Hampton wrote:
> On Sat, 2005-07-02 at 17:10 -0700, Jon Hamkins wrote:
>
>>I am interested in getting historical quotes.
> They're stored between <price> and </price> markers. The problem
> would be generating unique ID numbers for each price. You could do
> it, but of course I don't recommend hand editing the data file.
>
> If you want to pull a lot of quotes for the same time range, you might
> try rewriting the perl finance-quote-helper script to pull from F::QH
> instead of F::Q.
I took your suggestion and rewrote finance-quote-helper to use
Finance::QuoteHist instead of Finance::Quote. The attached file is a
drop-in replacement for the original, and it produces all the historical
output I was looking for. I've even carefully schemified the output so
that gnucash would be happy reading its output. For example, when set
to retrieve the last two weeks, you can get this kind of output:
$ echo '(yahoo "IBM")' | ./finance-quotehist-helper
(("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-20 00:00:00")
(last . 76.5500) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-21 00:00:00") (last
. 76.4100) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-22 00:00:00") (last
. 77.2300) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-23 00:00:00") (last
. 75.4100) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-24 00:00:00") (last
. 74.0100) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-27 00:00:00") (last
. 73.8800) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-28 00:00:00") (last
. 75.3000) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-29 00:00:00") (last
. 74.7300) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-06-30 00:00:00") (last
. 74.2000) (currency . "USD"))
("IBM" (symbol . "IBM") (gnc:time-no-zone . "2005-07-01 00:00:00") (last
. 74.6700) (currency . "USD")))
Unfortunately, gnucash hangs when I click on "Get Quotes", even when I
set the script to return a single day's prices. I get a endless
spinning hourglass. I imagine a fairly small change to price-quotes.scm
would get this to work.
----Jon
-------------- next part --------------
#!/usr/bin/perl -w
######################################################################
### finance-quotehist-helper - present scheme interface to Finance::QuoteHist
### Copyright 2001 Rob Browning <rlb at cs.utexas.edu>
### Revised: 07/02/05 by Jon Hamkins <hamkins at alumni.caltech.edu>, to
### get historical quotes using Finance::QuoteHist instead of
### (only) the latest quote using Finance::Quote. Note,
### Finance::QuoteHist does not support currency conversion,
### so you may want to keep the original finance-quote-helper
### around if you need to fetch currency conversions.
###
### This program is free software; you can redistribute it and/or
### modify it under the terms of the GNU General Public License as
### published by the Free Software Foundation; either version 2 of
### the License, or (at your option) any later version.
###
### This program is distributed in the hope that it will be useful,
### but WITHOUT ANY WARRANTY; without even the implied warranty of
### MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
### GNU General Public License for more details.
###
### You should have received a copy of the GNU General Public License
### along with this program# if not, contact:
###
### Free Software Foundation Voice: +1-617-542-5942
### 59 Temple Place - Suite 330 Fax: +1-617-542-2652
### Boston, MA 02111-1307, USA gnu at gnu.org
######################################################################
use lib '/usr/lib/perl5/5.8.5/i386-linux-thread-multi';
use strict;
use English;
use FileHandle;
# Input: (on standard input - one entry per line and one line per
# entry, and double quotes must only be delimiters, not string
# content -- remember, we don't have a real scheme parser on the perl
# side :>).
# (<method-name> symbol symbol symbol ...)
# where <method-name> indicates the desired Finance::Quote method.
# The currently recognized subset is yahoo, yahoo_europe,
# fidelity_direct, troweprice_direct, vanguard, asx, tiaacref,
# and currency.
# Since gnucash is sending the syntax above, we won't change it.
# However, Finance::QuoteHist supplies it's own lineup of websites
# (unless you want to use a particular one, in which case you should use
# a module specific to it, such as Finance::QuoteHist::Yahoo), so
# method-name is just ignored in this script.
# For currency quotes, the symbols alternate between the 'from'
# and 'to' currencies.
# For example:
#
# (yahoo "IBM" "LNUX")
# (fidelity_direct "FBIOX" "FSELX")
# (currency "USD" "AUD")
# Output (on standard output, one output form per input line):
# Schemified version of finance-quote's output, basically an alist of
# alists, as in the example below. Right now, only the fields that
# this script knows about (and knows how to convert to scheme) are
# returned, so the conversion function will have to be updated
# whenever Finance::Quote changes. Currently you'll get symbol,
# gnc:time-no-zone, and currency, and either last, nav, or price.
# Fields with gnc: prefixes are non-Finance::Quote fields.
# gnc:time-no-zone is returned as a string of the form "YYYY-MM-DD
# HH:MM:SS", basically the unmolested (and underspecified) output of
# the quote source. It's up to you to know what it's proper timezone
# really is. i.e. if you know the time was in America/Chicago, you'll
# need to convert it to that.
# For example:
# $ echo '(yahoo "CSCO" "JDSU" "^IXIC")' | ./finance-quote-helper
# (("CSCO" (symbol . "CSCO")
# (gnc:time-no-zone . "2001-03-13 19:27:00")
# (last . 20.375)
# (currency . "USD"))
# ("JDSU" (symbol . "JDSU")
# (gnc:time-no-zone . "2001-03-13 19:27:00")
# (last . 23.5625)
# (currency . "USD"))
# ("^IXIC" (symbol . ^IXIC)
# (gnc:time-no-zone . 2002-12-04 17:16:00)
# (last . 1430.35)
# (currency . failed-conversion)))
# On error, the overall result may be #f, or on individual errors, the
# list sub-item for a given symbol may be #f, like this:
# $ echo '(yahoo "CSCO" "JDSU")' | ./finance-quote-helper
# (#f
# ("JDSU" (symbol . "JDSU")
# (gnc:time-no-zone . "2001-03-13 19:27:00")
# (last . 23.5625)
# (currency . "USD")))
# further, errors may be stored with each quote as indicated in
# Finance::Quote, and whenever the conversion to scheme data fails,
# the field will have the value 'failed-conversion, and accordingly
# this symbol will never be a legitimate conversion.
# Exit status
#
# 0 - success
# non-zero - failure
# The methods we know about. For now we assume they all have the same
# signature so this works OK.
sub check_modules {
my @modules = qw(Date::Manip Finance::QuoteHist LWP);
my @missing;
foreach my $mod (@modules) {
if (eval "require $mod") {
$mod->import();
}
else {
push (@missing, $mod);
}
}
return unless @missing;
print STDERR "\n";
print STDERR "You need to install the following Perl modules:\n";
foreach my $mod (@missing) {
print STDERR " ".$mod."\n";
}
print STDERR "\n";
print STDERR "Run 'update-finance-quote' as root to install them.\n";
print "missing-lib";
exit 1;
}
sub schemify_string {
my($str) = @_;
if(!$str) { return "failed-conversion"; }
# FIXME: Is this safe? Can we just double all backslashes and backslash
# escape all double quotes and get the right answer?
# double all backslashes.
my $bs = "\\";
$str =~ s/$bs$bs/$bs$bs/gmo;
# escape all double quotes.
# Have to do this because the perl-mode parser freaks out otherwise.
my $dq = '"';
$str =~ s/$dq/$bs$dq/gmo;
return '"' . $str . '"';
}
sub schemify_boolean {
my($bool) = @_;
if($bool) {
return "#t";
} else {
return "#f";
}
}
sub schemify_num {
my($numstr) = @_;
# This is for normal numbers, not the funny ones like "2.346B".
# For now we don't need to do anything.
if(!$numstr) { return "failed-conversion"; }
if($numstr =~ /^\s*(\d+(\.\d+)?)$/o) {
return $1;
} else {
return "failed-conversion";
}
}
sub get_quote_time {
# return the date.
my ($datestr) = @_;
if(!$datestr) {
return undef;
}
my $parsestr = ParseDateString($datestr);
my $result = UnixDate($parsestr, "\"%Y-%m-%d %H:%M:%S\"");
if($result !~ /^\"\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\"$/) {
$result = "failed-conversion";
}
return $result;
}
# Schemify a row of Finance::QuoteHist data
sub schemify_row {
my($symbol, $date, $open, $high, $low, $close, $volume) = @_;
my $scmname = schemify_string($symbol);
my $quotedata = "";
my $field;
my $data;
$field = 'symbol';
$data = schemify_string($symbol);
$quotedata .= "($field . $data)";
$field = 'gnc:time-no-zone';
$data = get_quote_time($date);
$quotedata .= " ($field . $data)" if $data;
$field = 'last';
if ($close) { $data = schemify_num($close); }
elsif ($open) { $data = schemify_num($open); }
elsif ($high) { $data = schemify_num($high); }
elsif ($low) { $data = schemify_num($low); }
else { return schemify_boolean(0); }
$quotedata .= " ($field . $data)";
$quotedata .= ' (currency . "USD")';
# Note, Finance::QuoteHist does not return currency type-- I hope this
# isn't a problem.
return "($scmname $quotedata)";
}
sub parse_input_line {
# FIXME: we need to rewrite parsing to handle commands modularly.
# Right now all we do is hard-code "fetch".
my($input) = @_;
# Have to do this because the perl-mode parser freaks out otherwise.
my $dq = '"';
my @symbols;
# Make sure we have an opening ( preceeded only by whitespace.
# and followed by a one word method name composed of [a-z_]+.
# Also allow the '.' and '^' characters for stock indices.
# Kill off the whitespace if we do and grab the command.
if($input !~ s/^\s*\(\s*([\.\^a-z_]+)\s+//o) { return 0; }
my $quote_method_name = $1;
# Make sure we have an ending ) followed only by whitespace
# and kill it off if we do...
if($input !~ s/\s*\)\s*$//o) { return 0; }
while($input) {
# Items should look like "RHAT"
# Grab RHAT and delete "RHAT"\s*
if($input !~ s/^$dq([^$dq]+)$dq\s*//o) { return 0; }
my $symbol = $1;
push @symbols, $symbol;
}
my @result = ($quote_method_name, \@symbols);
return \@result;
}
#---------------------------------------------------------------------------
# Runtime.
# Check for and load non-standard modules
check_modules ();
# Define range of dates for which to return quotes. The dates are
# processed by Date::Manip, so it's pretty flexible. E.g., '01/27/2003'
# (but be careful of the MM-DD-YYYY/DD-MM-YYYY convention on your
# system), 'last thursday in January 2003', etc., are all allowed
#my $date_start = '24 months ago';
my $date_start = '2 weeks ago';
my $date_end = 'today';
# Create stock quote objects.
# $quoter is used for currency conversion only, since Finance::QuoteHist
# doesn't do currency conversions (only most recent conversion will be
# fetched). $histquoter is for historical quotes.
my $histquoter;
my $prgnam = "finance-quote-helper";
my $row;
my $resultstr = '(';
while(<>) {
my $result = parse_input_line($_);
if(!$result) {
print STDERR "$prgnam: bad input line ($_)\n";
exit 1;
}
my($quote_method_name, $symbols) = @$result;
$histquoter = Finance::QuoteHist->new(
symbols => [@$symbols],
start_date => $date_start,
end_date => $date_end,
);
if(!$histquoter->quotes()) {
print "#f\n";
exit 1;
}
foreach $row ($histquoter->quotes()) {
#my($symbol, $date, $open, $high, $low, $close, $volume) = @$row;
#print "symbol = $symbol\n";
#print "date = $date\n";
#print "open = $open\n";
#print "high = $high\n";
#print "low = $low\n";
#print "close = $close\n";
#print "volume = $volume\n";
$resultstr .= schemify_row(@$row) . "\n";
}
}
# Don't forget the closing perenthesis
chop $resultstr;
print $resultstr . ")\n";
STDOUT->flush();
exit 0;
__END__
More information about the gnucash-user
mailing list