[GNC-dev] Robust error handling and respecting the Apha Vantage API limits in gnc-fq-helper.

[Edward d'Auvergne]

On Tue, 11 Feb 2020 at 09:59, Edward d'Auvergne <true.bugman at gmail.com> wrote:
> On Mon, 10 Feb 2020 at 22:28, Mike Alexander <mta at umich.edu> wrote:
> > By the way, I didn't mean to suggest that we should encourage all users of GnuCash to patch F::Q. I still remain hopeful that the maintainers will merge my pull request that fixes this bug. In the meantime switching to the quote API might be a good idea even if it does have issues. The code would have to be smart enough to work around the problems. For example if the quote API fails for a given currency it could try the currency API. If the returned value appears to have lost all precision then it could try the reverse quote and compute the reciprocal itself.
>
> I finally found your pull request:
> https://github.com/finance-quote/finance-quote/pull/130.  Your
> changes, that were rejected, allow my previous script [1] to
> successfully fetch the quotes in serial [2].  I'll try to create a
> pure perl script to demonstrate that your bug fix is a proper bug fix,
> and that they should reopen your pull request, merge it, and then
> release F::Q v1.50!  If that works, then there wouldn't be any changes
> required on the GnuCash side.  Well, apart from the error handling to
> prevent GnuCash from segfaulting.

I've now created a script to demonstrate your fix in the pull request
[1].  On the master branch of F::Q, this shows the failure [2].  After
cherry-picking your commit 877c87bd3ea1cbd403c59dfcd4a58b97c14d232a,
most runs pass [3].  This does not include the "USD" "XAU" pair which
needs to be inverted.  Maybe the inversion should be directly
performed in F::Q rather than GnuCash?  Anyway, I'll post this to the
pull request and ask for it to be reopened.

Although most runs pass, I have noticed that some do not [4].  Have
you seen that behaviour or know what might be happening?

Regards,

Edward


[1]  Script to demonstrate the F::Q currency fetching failures:

"""
#!/usr/bin/env perl

use Finance::Quote;
use Time::HiRes qw(time);
print("Using Perl $^V\n");
print("Using Finance::Quote $Finance::Quote::VERSION\n\n");

# The currencies to fetch.
my @symbols = (
    "AUD",
    "BRL",
    "CAD",
    "CHF",
    "CNY",
    "CZK",
    "DKK",
    "EUR",
    "GBP",
    "HKD",
    "HRK",
    "ILS",
    "JPY",
    "KRW",
    "MYR",
    "NOK",
    "NZD",
    "RUB",
    "SGD",
    "TWD",
    "XAU"
);

# Fetch the currencies.
my $begin = time();
my $quoter = Finance::Quote->new();
foreach (@symbols) {
    $price = $quoter->currency("USD", $_);
    if (!defined $price) {
        $price = "Failed.";
    }
    print("USD$_: $price\n");
}
my $end = time();
printf("Fetching time %0.02f s\n", $end-$begin);
"""


[2] Failure of F::Q at bcfc947ee0173e059573236c2cf4a9bcb9a09b65:

"""
Using Perl v5.30.1
Using Finance::Quote 1.50

USDAUD: 1.4909
USDBRL: 4.325
USDCAD: 1.3294
USDCHF: 0.9778
USDCNY: 6.9745
USDCZK: Failed.
USDDKK: Failed.
USDEUR: Failed.
USDGBP: Failed.
USDHKD: Failed.
USDHRK: Failed.
USDILS: Failed.
USDJPY: Failed.
USDKRW: Failed.
USDMYR: Failed.
USDNOK: Failed.
USDNZD: Failed.
USDRUB: Failed.
USDSGD: Failed.
USDTWD: Failed.
USDXAU: Failed.
Fetching time 12.44 s
"""

[3] Passing run after cherry-picking 877c87bd3ea1cbd403c59dfcd4a58b97c14d232a:

"""
Using Perl v5.30.1
Using Finance::Quote 1.50

USDAUD: 1.4907
USDBRL: 4.3242
USDCAD: 1.3293
USDCHF: 0.9781
USDCNY: 6.9745
USDCZK: 22.833
USDDKK: 6.8465
USDEUR: 0.9162
USDGBP: 0.7734
USDHKD: 7.7646
USDHRK: 6.8309
USDILS: 3.4163
USDJPY: 109.85
USDKRW: 1183.9
USDMYR: 4.134
USDNOK: 9.2539
USDNZD: 1.5643
USDRUB: 63.644
USDSGD: 1.3872
USDTWD: 30.034
USDXAU: Failed.
Fetching time 219.69 s
""'

[4] Failing run after cherry-picking 877c87bd3ea1cbd403c59dfcd4a58b97c14d232a:

"""
Using Perl v5.30.1
Using Finance::Quote 1.50

USDAUD: 1.4907
USDBRL: 4.3229
USDCAD: 1.3293
USDCHF: 0.9778
USDCNY: 6.9745
USDCZK: 22.836
USDDKK: 6.8453
USDEUR: 0.9161
USDGBP: 0.7735
USDHKD: 7.7646
USDHRK: 6.8309
USDILS: Failed.
USDJPY: 109.84
USDKRW: 1184
USDMYR: 4.134
USDNOK: 9.2512
USDNZD: 1.5639
USDRUB: 63.716
USDSGD: 1.3872
USDTWD: 30.03
USDXAU: Failed.
Fetching time 167.91 s
"""