[GNC] Strange prices update issue (possibly with TSP)

[Kalpesh Patel]

I was reading up on the Hacker's guide at https://github.com/finance-quote/finance-quote/blob/master/Documentation/Hackers-Guide to see if that facility was available or not (and to write my own module) in the TSP.pm module. Appreciate confirming it and now debugging will become much more easier. I'll follow up with what I find.

Thank you, to both, for all these valuable pointers.


-----Original Message-----
From: Bruce Schuck <bschuck at asgard-systems.com> 
Sent: Friday, August 11, 2023 9:40 PM
To: gnucash-user at gnucash.org
Cc: Kalpesh Patel <kalpesh.patel at usa.net>; john <jralls at ceridwen.us>
Subject: Re: [GNC] Strange prices update issue (possibly with TSP)

On Aug 10, 2023, at 12:13, Kalpesh Patel <kalpesh.patel at usa.net <mailto:kalpesh.patel at usa.net> > wrote:

> The difficult part in debugging this is that when I do query three 
> times from the command line, it succeeds without any issue for me as 
> well, but when I update prices from GNC, it fails two times and then 
> succeeds third time consistently. So this is not an issue with 
> throttling from TSP source which you validated as well. Is there more 
> in terms of debugging this by any chance? Can I use STDERR and STDOUT 
> from Perl to emanate messages to console but not interfere with what 
> JSON comes back or butcher the response back to gnc-fq-helper so it 
> doesn't bomb to run a full cycles from GNC to test? Any help in debugging is appreciated.

Kalpesh,

Many, but not all, of the F::Q modules make use of the Smart::Comments module. TSP.pm is one of them. The environment variable DEBUG will need to be set, and with no changes to the Perl code you should see helpful information printed to STDERR. In the code, these are lines that begin with "###" (note, just 3 pound signs) in those modules using Smart::Comments.

A quick look at the TSP module and I see two where the "tsp" method will return *without* setting "success" and "errormsg" for any of the symbols passed to it. This will happen if the http get call fails, or whatever is downloaded is not 2 or more lines.

     my $reply = $ua->get($url, @HEADERS);
     ### [<now>] url  : $url
     ### [<now>] reply: $reply

     return unless ($reply->is_success);

     my @line = split(/\n/, $reply->content);

     return unless (@line > 1);

In the code snippet, there are two examples Smart::Comments use which could be useful in your case. With DEBUG set the code would print the url to STDERR, and entire $reply object (status, headers, cookies, body).

Whether or not this helps you, the omission of not returning the proper failure data is an issue.

Hope this helps.

Bruce S