[GNC] Finance::Quote enters previous day's price as today's

Sun Jun 28 13:17:39 EDT 2020

Colin,

For some reason the original email you sent through to the list in reply didn't make it to my email inbox, I just happened to see it referenced in a different email.  For your benefit and everyone else, where is the powershell script I am using to retrieve quotes from Alphavantage for all of my mutual funds.  In the below, where you see <KEYGOESHERE> this is where your alphavantage key goes.  This could probably easily be adapted to a bash or python script easily as well.  It's not doing anything too crazy.  I've also found the alpha vantage website good for documenting the API, so plenty of other options are available as well.

function Process-QuoteFile {
    param( $infile, $outfile, $symbolname, $skip)

    $ins = New-Object System.IO.StreamReader ($infile)
    $outs = New-Object System.IO.StreamWriter ($outfile, $true)
    try {
        # Skip the first N lines, but allow for fewer than N, as well
        for( $s = 1; $s -le $skip -and !$ins.EndOfStream; $s++ ) {
            $ins.ReadLine()
        }
        while( !$ins.EndOfStream ) {
            $outs.WriteLine( "$symbolname," + $ins.ReadLine() )
        }
    }
    finally {
        $outs.Close()
        $ins.Close()
    }

}

$Quotes = @("BSPIX","CAIBX","CWGIX","AEPGX","FKINX","AGTHX","ANWPX","NEWFX", "SMCWX","VWUAX","VTSAX", "FLDAX", "VTIVX", "VTI", "VBTLX", "VTABX", "VSMGX", "VFIFX")
$Namespaces = @("MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "MUTF", "FUND", "MUTF", "MUTF", "MUTF", "MUTF")
$dir = "C:\Users\mdforbis\Documents\Finance\Alphavantage\"
$skip = 0

echo "Cleaning up..."
Remove-Item -path "$dir*" -Filter *.csv
ForEach ($symbolname in $Quotes)
{
$a = "$dir$symbolname"
$combined = $dir + "CombinedQuotes.csv"

echo "Retrieving $symbolname ..."
wget -Outfile "$a.csv" "https://www.alphavantage.co/query?function=TIME_SERIES_DAILY&symbol=$symbolname&apikey=<KEYGOESHERE>&datatype=csv"
echo "Waiting 15 seconds..."
Start-Sleep -Seconds 15
echo "Processing $symbolname into combined file..."
$curnamespace = $Namespaces[$Quotes.IndexOf($symbolname)]
Process-QuoteFile -infile "$a.csv" -outfile $combined -symbolname "$curnamespace,$symbolname" -skip $skip
$skip = 1
}

Thanks,
Matt

On Wednesday, June 24, 2020, 9:46:33 AM CDT, David Reiser via gnucash-user <gnucash-user at gnucash.org> wrote: 

> On Jun 24, 2020, at 3:00 AM, Colin Arndt <lumicanis at gmail.com> wrote:
> 
> Interesting. It still sounds like a bug report is in order if it’s storing an incorrect price after normal user interaction with the feature. The bug is that it’s storing a new price for the day before there is a new price for the day, and only correcting it after it gets the day’s price. I can adjust my behavior to avoid this, but it will affect other people in the future. Does that sound reasonable?
> 

It’s more like a policy dispute between you and the likes of yahoo finance. Yahoo has decided that they will account for any overnight corrections to funds' prior day closing by coughing up a pre-market quote with today’s date and yesterday’s adjusted closing price. Most pre-market quotes will be exactly the same as yesterday’s early evening reported closing price, but not always. 

If the data broker hands finance-quote a valid date and a valid price, there’s no way gnucash can determine that it should ignore the data record.

It looks like the part of alphavantage that finance-quote uses comes closer to what you want than yahoo does, but then it wasn’t possible to get a quote of Monday’s close of VTSAX from alphavantage+finance quote (at least not on Tuesday—I don’t know about Monday night). Perhaps alphavantage’s policy is that any overnight corrections block any outgoing price quote for the corrected day for that fund, at least as far as “yesterday’s closing price” is concerned.

> Thanks for your help everyone.
> 
> Matt, I’m on Linux not Windows, but I’d love to have your script as a starting point in case I go the import route. Thanks for the offer.
> 
> On Wed, Jun 24, 2020 at 12:40 AM David Reiser <dbreiser at icloud.com <mailto:dbreiser at icloud.com>> wrote:
> 
> > On Jun 23, 2020, at 4:10 PM, David Carlson <david.carlson.417 at gmail.com <mailto:david.carlson.417 at gmail.com>> wrote:
> > 
> > I am not sure that this is your problem or not, but I have found that I
> > need to wait several hours after the markets have closed before downloading
> > mutual fund prices.  It seems to take hours for some of them to get posted
> > at Yahoo.com. One way to check would be to see what Yahoo.com is reporting
> > early after hours vs later.
> > 
> > GnuCash only keeps one downloaded price per day, and repeated prices are
> > discarded.  If you see that is what is happening, you might file a RFE to
> > keep the last price reported instead of the first.
> > 
> > 
> On my test today, gnucash did save the most recent downloaded price for today, not the first.
> 
> Fund prices can get weird, depending on data feed and intermediaries. Yahoo as JSON was reporting an 8am price for today that was slightly different from yesterday’s close for all three of these Vanguard funds. Alphavantage, as of 6:30pm today (Tuesday) was still reporting Friday’s price for these funds.
> 
> Yahoo got today’s closing NAV for VTIAX and VWELX at 5:39 pm (EDT). The VTSAX price didn’t update until 6:00pm.
> 
> 
> 
> > 
> > .
> > 
> > On Tue, Jun 23, 2020 at 2:19 PM Colin <lumicanis at gmail.com <mailto:lumicanis at gmail.com>> wrote:
> > 
> >> First off, I just installed GnuCash a couple weeks ago, after trying a few
> >> other options and finding they simply weren't mature enough or
> >> feature-complete enough for what I need. I can't say thank you enough to
> >> everyone who's developed this program, everyone who continues to develop it
> >> and everyone who provides support here. It's an incredible accomplishment!
> >> 
> >> I have an issue with Finance::Quote. It brings in prices for my three
> >> Vanguard funds when I run the following command, and I've been planning on
> >> setting up a cron job for it:
> >> gnucash --add-price-quotes path/to/my/gnucash/file
> >> However, for one fund (VTSAX) it enters the prior day's price again as
> >> today's. This requires me to look it up and fix it manually. Its settings
> >> under Security editor are the same as the other two (VTIAX and VWELX).
> >> They're set to get online quotes from a single source, Alphavantage.
> >> Can anyone provide guidance on what may be going on here?
> >> Second question:
> >> I have a Raspberry Pi and it would make sense to run the cron job there
> >> since it's always up. I also use Syncthing for syncing files between
> >> devices. However, the latest version in their repo's is GnuCash 3.4 and
> >> Finance::Quote 1.47. Would this introduce problems to have the Pi update my
> >> GnuCash file with price quotes?
> >> 
> >> Thank you,
> >> Colin
> 
> 
> --
> Dave Reiser
> dbreiser at icloud.com <mailto:dbreiser at icloud.com>
> 

--
Dave Reiser
dbreiser at icloud.com

_______________________________________________
gnucash-user mailing list
gnucash-user at gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.