Debian Codegraph @

Documentation/Hackers-Guide

raw
1 Finance::Quote Hackers Guide
2 Original Author: Paul Fenwick <[email protected]>, May 2000
3 # vi: set ts=4 sw=4 noai ic showmode showmatch:
4 #
5 # This file will be deprecated - Please see CONTRIBUTING.md
6 # in the repository root directory.
7
8 $Version$
9
10 0. Table of Contents
11 ====================
12
13 1. Introduction
14 2. Setup a Development Environment
15 3. How to write a Finance::Quote module
16 3.1. The package name
17 3.2. The methodinfo() subroutine
18 3.3. The methods subroutine
19 3.4. The functions specified by methods()
20 3.5. The parameters() subroutine - Modules requiring API Keys
21 3.6. Currency
22 3.7. Development & Debugging Code
23 3.8. Dates
24 3.10. Things to avoid
25 3.11. Using your new module
26 4. How to write a Finance::Quote::CurrencyRates module
27 4.1. The package name
28 4.2. The new() constructor
29 4.3. The multipliers() function
30 5. How to contribute your module to the world
31 6. How to find out more
32 7. How to join the mailing lists
33
34
35 1. Introduction
36 ===============
37
38 This hacker's guide is primarily a tutorial on how to build your own
39 Finance::Quote pluggable module. After reading this guide, you should
40 be able to write your own module to provide extra methods and functionality
41 to the Finance::Quote library.
42
43 This guide assumes that you are familiar with perl.
44
45
46 2. Setup a Development Environment
47 ==================================
48
49 To develop and test modules, you need a clone of the Finance-Quote git repo, a
50 recent version of perl, and the Dist::Zilla module and its dependencies. The
51 following steps provide a recipe for setup.
52
53 A. Clone the git repo
54
55 $ git clone https://github.com/finance-quote/finance-quote.git
56
57 B. (Optional) Install latest stable copy of perl in ~/perl5
58
59 $ curl -L https://install.perlbrew.pl | bash
60 $ echo "source ~/perl5/perlbrew/etc/bashrc" >> ~/.bash_profile # or the equivalent for your shell
61 $ perlbrew --notest install perl-5.28.1 # --notest is risky, but significant speeds installation
62 $ perlbrew switch perl-5.28.1
63 $ perlbrew install-cpanm
64
65 C. Install Dist::Zilla
66
67 $ cpanm install Dist::Zilla
68
69 # inside the finance-quote directory
70 # remove --missing to install missing & upgrade existing required package
71
72 $ dzil authordeps --missing | cpanm --notest
73 $ dzil listdeps --missing | cpanm --notest
74
75 $ cpanm install Data::Dumper::Perltidy Smart::Comments
76
77 D. Test out finance-quote
78
79 # Light-weight test mode that skips all online tests
80
81 $ dzil test
82
83 # To run all the skipped tests, you need three environment variables:
84 # TEST_AUTHOR - when set, tests are run to check for required modules
85 # ONLINE_TEST - when set, online tests are executed
86 # ALPHAVANTAGE_API_KEY - free API key available from https://www.alphavantage.co
87 #
88 # Using bash syntax to set environment variables
89
90 $ cpanm Test::Pod Test::Pod::Coverage Test::Kwalitee Test::Perl::Critic
91
92 # This runs a few code checks, including best practices as noted
93 # in "Perl Best Practices".
94
95 $ TEST_AUTHOR=1 dzil test
96
97 # Be aware of the limit of API keys. Running many tests may consume
98 # the maximum number of allowed queries.
99
100 $ TEST_AUTHOR=1 ONLINE_TEST=1 ALPHAVANTAGE_API_KEY=<YOUR API KEY> dzil test
101
102 # To do an online test for one module
103
104 $ ONLINE_TEST=1 dzil run prove -lv t/financeapi.t
105
106
107 # Use prove to during test development - it is fast
108
109 $ prove -lv t/fq-class-methods.t
110
111 # Use perlcritic on your module
112
113 $ perlcritic lib/Finance/Quote/YourModule.pm
114
115 3. How to write a Finance::Quote module
116 =======================================
117
118 Finding a source of information, and writing code to parse and interpret
119 that information is a difficult task. As such, we've aimed to make
120 writing a Finance::Quote module as easy as possible. There are only
121 a few simple rules you need to follow:
122
123 3.1. The package name
124 ---------------------
125 Finance::Quote expects that its loadable modules will be in the
126 Finance::Quote namespace somewhere. Hence, if you were writing
127 a module called "DodgyBank" that returned information on DodgyBank's
128 managed funds, a reasonable name for that module would be
129 Finance::Quote::DodgyBank.
130
131 3.2. The methodinfo() subroutine
132 --------------------------------
133 Your module must have a subroutine named methodinfo(). Future versions
134 of GnuCash may use this to dynamically determine the data sources
135 available. The subroutine returns a nested data structure of the
136 method(s) defined in the module, along with additional information.
137
138 In this example, the main method is "foobar". But this method can be used
139 for failover methods "usa", "nyse", or "nasdaq".
140
141 sub methodinfo {
142 return (
143 foobar => $METHODHASH,
144 usa => $METHODHASH,
145 nyse => $METHODHASH,
146 nasdaq => $METHODHASH,
147 );
148 }
149
150 The information for the method in contained in another data structure.
151 In our example, "$METHODHASH". This data structure will contain a brief name,
152 any features such as needing a API key, and the labels (data fields) the
153 method will returns.
154
155 our $DISPLAY = 'FooBar';
156 our $FEATURES = {'API_KEY' => 'registered user API key'};
157 our @LABELS = qw/date isodate open high low close volume last net p_change/;
158 our $METHODHASH = {subroutine => \&foobar,
159 display => $DISPLAY,
160 labels => \@LABELS,
161 features => $FEATURES};
162
163 It is uncommon, but some modules may contain 2 or more different methods.
164 Not to be confused with failover methods.
165 In this example, a module to retrieve data from "Dodgy Bank, USA" may
166 have a method defined for funds and another for bonds. Both require an
167 API key.
168
169 our $FUNDSDISPLAY = 'Dodgy Bank Funds';
170 our $FEATURES = {'API_KEY' => 'registered user API key'};
171 our @LABELS = qw/date isodate open high low close volume last net p_change/;
172 our $FUNDSHASH = {subroutine => \&funds,
173 display => $FUNDSDISPLAY,
174 labels => \@LABELS,
175 features => $FEATURES};
176
177 our $BONDSDISPLAY = 'Dodgy Bank Bonds';
178 our $BONDSHASH = {subroutine => \&bonds,
179 display => $BONDSDISPLAY,
180 labels => \@LABELS,
181 features => $FEATURES};
182
183 sub methodinfo {
184 return (
185 dodgyfunds => $FUNDSHASH,
186 dodgybonds => $BONDSHASH,
187 );
188 }
189
190 3.3. The methods() subroutine
191 -----------------------------
192 Your module must have a subroutine named methods(). This function will
193 be called by the Finance::Quote harness when it loads your module, and
194 is used to determine which methods your module provides. The methods()
195 function must return a hash of method names and subroutine references.
196 For example, if you had written a module which provides access to
197 DodgyBank's managed funds, you might have the following:
198
199 package Finance::Quote::DodgyBank;
200 .
201 .
202 sub methods {
203 my %m = methodinfo(); return map {$_ => $m{$_}{subroutine} } keys %m;
204 }
205
206 This would indicate that your package provides methods for
207 "dodgyfunds" and "dodgybonds", and that the subroutines
208 "funds" and "bonds" should be called to access that information.
209
210 The following method names should be used for the following information
211 sources:
212
213 Method-Name Source
214 ---------------------------------------------------------
215 australia Australian Stocks
216 canada Canadian Stocks
217 europe European Stocks
218 fidelity Fidelity Investments
219 nasdaq NASDAQ
220 nyse New York Stock Exchange
221 tiaacref TIAA-CREF
222 troweprice T. Rowe. Price
223 usa USA Stocks
224
225 Method names should be lower-case, consist of alphanumeric characters
226 (including underscore) only, and always begin with a letter. This is
227 not enforced, but future versions of the Finance::Quote framework may
228 rely upon it.
229
230 It's strongly recommended that you also provide a unique name for your
231 method, in case you (or others) wish to call that method exclusively
232 in code. Hence if you had written a module to fetch information from
233 the NYSE from Yohoo!, you might implement the following methods
234 function:
235
236 sub methods { return ( nyse => \&yohoo,
237 yohoo => \&yohoo ); }
238
239 This means that people who only want to use your function can use
240 $quoter->fetch('yohoo',@stocks), but those who don't care where
241 their NYSE stock information is fetched from can use
242 $quoter->fetch('nyse',@stocks). The first form allows you to know exactly
243 where the information is coming from. In the second, failover methods mean
244 that many different functions could be used to fetch the stock information,
245 not just the one you have defined.
246
247 3.4 The functions specified by methods()
248 ----------------------------------------
249 The functions referred to by methods() will be passed a Finance::Quote
250 object when called, and a list of zero or more symbol names. The
251 Finance::Quote object provides the following ready-to-use methods:
252
253 user_agent(); # Provides a ready-to-use LWP::UserAgent
254
255 parse_csv(); # Parses a list of comma-separated values
256 # and returns an array.
257
258 The user_agent() method should be used if possible to fetch the information,
259 as it should be already configured to use the timeout, proxy, and other
260 settings requested by the calling program.
261
262 Your function should return a two-dimensional hash as specified in the
263 Finance::Quote man-page. Eg:
264
265 $hash{$symbol,'last'} = $last_price;
266 $hash{$symbol,'name'} = $stock_name;
267 # etc etc.
268
269 When returning your hash, you should check the context that your
270 function was called in. If it was called in a scalar context, then
271 you should return a hashref instead. This can be easily done
272 with the following:
273
274 return wantarray() ? %hash : \%hash;
275
276 It is ESSENTIAL that your hash contain a true value for {$symbol,'success'}
277 for information that has been successfully obtained. If the information
278 was not obtained for any reason, then {$symbol,'success'} should
279 be set to a false value (preferably 0), and a human-readable error
280 message placed in {$symbol,'errormsg'}. The following code snippet
281 demonstrates this:
282
283 sub funds {
284
285 my $quoter = shift; # The Finance::Quote object.
286 my @stocks = @_;
287 my %info;
288
289 my $DODGY_URL = "http://dodgybank.xxx/funds.csv?";
290
291 my $ua = $quoter->user_agent; # This gives us a user-agent
292 # with timeouts, proxies,
293 # etc already configured.
294
295 my $response = $ua->request(GET $DODGY_URL);
296 unless ($response->is_success) {
297 foreach my $stock (@stocks) {
298 $info{$stock,"success"} = 0;
299 $info{$stock,"errormsg"} = "HTTP failure";
300 }
301 return wantarray ? %info : \%info;
302 }
303
304 # Do stuff with the information returned....
305
306 }
307
308 It is valid to use "return" with no arguments if all stock lookups failed,
309 however this does not provide any information as to WHY the lookups
310 failed. If at all possible, the errormsg labels should be set.
311
312 It is also very very strongly recommended that you place your module's
313 name in the {$stock,"source"} field. This allows others to check where
314 information was obtained, and to use it appropriately.
315
316 3.5 The parameters() subroutine - Modules requiring API Keys
317 ------------------------------------------------------------
318 Some data sources require an API Key (sometimes called a token) in
319 order to access the securities or exchange data. AlphaVantage.pm
320 is one example.
321
322 In order to assist programs using Finance::Quote to identify those
323 modules that require an API Key please include this function if
324 your module is using a data source that requires a key/token.
325
326 sub parameters {
327 return ('API_KEY');
328 }
329
330 3.6 Currency
331 -------------
332 Finance::Quote has support for multiple currencies and for currency
333 conversion. As long as you provide a little bit of information about
334 the information you are returning, the Finance::Quote framework can
335 do all the hard stuff for you.
336
337 If you are returning information on a stock in a particular currency,
338 then you can enter the ISO currency code into the "currency" field
339 associated with the stock. Eg:
340
341 $info{$stock,"currency"} = "AUD"; # Australian Dollars
342
343 If the information you are returning does not have a currency
344 (because it's an index like the Dow Jones Industrial or the
345 All Oridinaries, or because you're returning percentages) then
346 you should not set the currency field for that stock. Finance::Quote
347 knows not to attempt currency conversion for stocks without
348 a currency field.
349
350 If you do have a currency field, then by default Finance::Quote will
351 arrange for the automatic conversion of a number of fields. By
352 default, these fields are last, high, low, net, bid, ask, close, open,
353 day_range, year_range, eps, div, cap, nav and price. Of course,
354 there may be some cases where this set is not appropriate, or where there
355 are extra fields that should be converted. This can be indicated
356 by writing a function called "currency_fields()" in your module,
357 that returns a list of fields that can undergo currency conversion.
358 Eg:
359
360 sub currency_fields {
361 return qw/high low price bid/;
362 }
363
364 currency_fields() will be passed a Finance::Quote object as its
365 first argument, and a method called default_currency_fields()
366 is available through this object. This is useful if you want
367 to use the defaults, but also add some of your own:
368
369 sub currency_fields {
370 my $quoter = shift;
371 return ($quoter->default_currency_fields, "commission");
372 }
373
374 In the example above, the default fields would be available for currency
375 conversion, but the "commission" field would also be converted.
376
377 3.7 Development & Debugging Code
378 --------------------------------
379 There are multiple ways to include extra code in Perl modules to facilitate
380 development and debugging. For Finance::Quote modules, please use the following
381 strategy that uses a combination of a DEBUG constant and the Smart::Comments
382 module. If DEBUG is in the environment and set to a value that Perl evaluates
383 to true, then Smart::Comments is active and code blocks wrapped with if(DEBUG)
384 are also active. Otherwise, all the debugging code disappears during
385 compilation.
386
387 use strict; # use strict & warnings FIRST
388 use warnings;
389
390 use constant DEBUG => $ENV{DEBUG}; # create a constant from the environment
391
392 use if DEBUG, 'Smart::Comments'; # conditionally use Smart::Comments
393 use if DEBUG, 'Another::Module'; # example module only needed for debugging
394
395 use LWP::UserAgent; # and finally production module dependencies
396
397 my %h = ('a' => 1); # just an example variable
398
399 # These first two lines are always a comment. The third line
400 # prints a debugging string if DEBUG is in the environment.
401 ### [<now>] h: \%h
402
403 # For more complicated debugging, the following if-block
404 # will disappear during compilation because of constant folding.
405 if (DEBUG) {
406 print "More complex debugging code\n";
407 }
408
409 3.8 Dates
410 ----------
411 Do not parse dates directly in your module. Instead you should use
412 the function $q->store_date(), which handles a variety of date
413 formats. In its simplest form, you simply tell the function what
414 format the date is in and it handles all the parsing. The code should
415 look similar to this:
416
417 $quoter->store_date(\%info, $stock, {eurodate => @$row[1]});
418
419 If the web site doesn't have a data available, somply call the
420 function this way:
421
422 $quoter->store_date(\%info, $stock, {today => 1});
423
424 See the documentation in Quote.pm for more information.
425
426 3.9 Please don't "die"
427 ----------------------
428
429 Finance::Quote should always return data, even if no data was retrieved
430 for any of the securities. Instead, set "success" to 0 and an error
431 message ("errormsg") for all of the requested securities.
432
433
434 3.10 Things to avoid
435 --------------------
436 Some sources of information will provide more stock information than
437 requested. Some code may rely upon your code only returning information
438 about the stocks that the caller requested. As such, you should
439 never return information about stocks that were not requested, even
440 if you fetch and/or process that information.
441
442 3.11 Editor Settings
443 --------------------
444
445 As an OpenSource project, there have been many contributors to Finance::Quote.
446 Which means a variety of text editors and coding styles can be found.
447 We ask that if a module is being modified, that contributors follow
448 the formatting and indenting style of the original author.
449
450 One way to satisfy this is to take note of things like vi/vim modeline
451 commands or Emacs file variables that previous authors may have included.
452
453 For example I primarily use vim and include the following modeline near
454 the top of most scripts and code.
455
456 # vi: set ts=4 sw=4 noai ic expandtab showmatch showmode:
457
458 The equivalent for Emacs may be:
459
460 # -*- tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4; case-fold-search: nil; -*-
461
462
463 3.12 Using your new module
464 --------------------------
465 Using your new module is easy. Normally when using Finance::Quote you'd
466 do something like the following:
467
468 use Finance::Quote;
469 my $quoter = Finance::Quote->new();
470
471 To use your new module, simply specify the module name (without
472 the Finance::Quote prefix) in the new function. Hence:
473
474 use Finance::Quote;
475 my $quoter = Finance::Quote->new("DodgyBank");
476
477 The DodgyBank methods will now be available:
478
479 my %loaninfo = $quoter->fetch("dodgyloans","car","boat","house");
480 my %fundinfo = $quoter->fetch("dodgyfunds","lotto","shares");
481
482 The resulting Finance::Quote object will also arrange for your functions
483 to be callable without using fetch. This syntax is strongly discouraged,
484 as it results in pollution of the Finance::Quote namespace and provides
485 little advantages over the fetch() method:
486
487 my %loaninfo = $quoter->dodgyloans("car","boat","loan");
488
489 This mainly exists to maintain compatibility with previous versions of
490 Finance::Quote.
491
492
493 4. How to write a Finance::Quote::CurrencyRates module
494 ======================================================
495
496 Currency Rate modules carryout a single task: Given two currencies, return
497 multipliers that describe the relative value of the currencies. In the
498 simplest case, one multiplier is "1.0" and the other multiplier is the exchange
499 rate between the two currencies. In the other case, the multipliers reflect the
500 relative value of each currency to a third base currency.
501
502 For example, on 2020-10-30, the European Central Bank reported that 1.1698 USD
503 (United States Dollar) was equivalent to 1.00 EUR (Euro). It also reported that
504 0.90208 GBP (Pound Sterling) was equivalent to 1.00 EUR. To convert from USD to
505 GBP, the following from/to multipliers are equivalent:
506
507 from to Base Currency
508 -------- -------- -------------
509 1.16980 0.90208 EUR
510 1.00000 0.77114 USD
511 1.29678 1.00000 GBP
512
513 Depending on the source of the currency rates, it may be more convenient for a
514 currency rate module to return multipliers that are relative to a third
515 currency (first row), the conversion rate (the second row), or the inverse
516 conversion rate (the third row).
517
518 Currency rate modules should avoid carrying out arithmetic and leave it to
519 Finance::Quote to use the multipliers to compute the final conversion value.
520
521 4.1. The package name
522 ---------------------
523
524 Currency rate modules belong in the Finance::Quote::CurrencyRates namespace and
525 the name of the module should be descriptive. Use PascalCase (upper CamelCase)
526 for the module name.
527
528 4.2. The new() constructor
529 --------------------------
530
531 Currency rate modules are called from a Finance::Quote quoter object when
532 currency conversion is either explicitly requested from a caller or to convert
533 a fetched security price to a user specified currency. When instantiating a
534 quoter object, the caller may pass module specific parameters. Suppose the
535 currency rate module ExampleRates requires an API key and also optionally
536 supports caching rates during the lifetime of the quoter object. Then the
537 code
538
539 my $q = Finance::Quote->new('currency_rates' =>
540 {order => ['ExampleRates', 'ECB'],
541 examplerates => {API_KEY => 'x01234566',
542 cache => True}});
543
544 instantiates a quoter that uses ExampleRates first and falls back the ECB
545 for currency rates. When ExampleRates is used, the API_KEY can cache options
546 are configured through a module specific hash.
547
548 Finance::Quote passes just the hash containing the API_KEY and cache key/value
549 pairs to ExampleRates->new() as a hash reference. If the caller does not
550 specify an ExampleRates key in the currency_rates hash, then ExampleRates->new()
551 is called with no arguments. ExamplesRates may, but is not required, to
552 alternately read the API key from an environment variable.
553
554 Clearly document all currency rate module options in the POD documentation for
555 the module.
556
557 4.3. The multipliers() function
558 -------------------------------
559
560 Finance::Quote calls the currency rate object method multipliers() with the
561 arguments (object, user_agent, from, to), where
562
563 object was previously instantiated with a call to new()
564 user_agent is a LWP::UserAgent
565 from is an ISO currency code
566 to is an ISO currency code
567
568 multipliers() must a pair of multipliers (from_multiplier, to_multiplier),
569 where
570
571 from_multiplier is a floating point value
572 to_multiplier is a floating point value
573
574 and to_multiplier/from_multiplier will convert a value in currency "from"
575 into currency "to".
576
577 On error, multipliers() must return undef.
578
579 5. How to contribute your module to the world
580 =============================================
581
582 Check list:
583
584 o Created a well-named module file in the Finance directory tree
585 o Included POD documentation at the bottom of module file
586 o Included a VERSION comment just *after* the use module statements
587 o Added the module name in alphabetical order to the @MODULES or @CURRENCY_MODULES variable in Quote.pm
588 o Added the module name in alphabetical order in the SEE ALSO section of the Quote.pm POD
589 o Created a well-named test file in the t/ directory, including a test that succeeds and a test that fails. PLEASE indicate where to find valid symbols if your test file contains symbols for securities that expire such as Funds. Failing tests will cause your module to be removed from Finance::Quote if it is too difficult for maintainers to find valid symbols for tests.
590 o Ensure tests 00-store-date.t, 01-pod.t, 02-pod-coverage.t, 03-kwalitee.t, 04-critic.t, 05-data-dumper.t pass
591 o Added module information in alphabetical order to the Modules-README.yml file
592 o Add a quick description of the change/addition to the Changes file. Most recent changes at the top of the list. Add new lines just below the {{$NEXT}} label at the top of the file.
593
594 Contributions to Finance-Quote are best presented as pull requests on GitHub.com.
595
596 A. Create a GitHub account and sign-in
597 B. Go to https://github.com/finance-quote/finance-quote
598 C. Click "Fork" in the upper-right corner
599 D. Commit your new module and other code changes to the fork
600 E. Click "New Pull Request" on https://github.com/finance-quote/finance-quote
601 F. Click "Compare Across Forks"
602 G. Select the appropriate commits and create the merge request
603
604 Contact developers at [email protected] to discuss
605 new modules, ask questions, or get help opening a merge request.
606
607
608 6. How to find out more
609 =======================
610 The Finance::Quote GitHub page is located at
611
612 https://github.com/finance-quote/finance-quote
613
614 and contains information about the project and links to older SourceForge
615 documentation.
616
617
618 7. How to join the mailing lists
619 ================================
620
621 There are two mailing lists for Finance::Quote. These can both be accessed
622 from:
623
624 http://sourceforge.net/mail/?group_id=4232
625
626