Debian Codegraph @

README

raw
1 NAME
2 Test::Inter - framework for more readable interactive test scripts
3
4 DESCRIPTION
5 This is another framework for writing test scripts. Much of the syntax
6 is loosely inspired by Test::More, and Test::Inter has most of it's
7 functionality, but it is not a drop-in replacement.
8
9 Test::More (and other existing test frameworks) suffer from two
10 weaknesses, both of which have prevented me from ever using them:
11
12 None offer the ability to access specific tests in
13 a reasonably interactive fashion, primarily for
14 debugging purposes
15
16 None offer the ability to write the tests in
17 whatever format would make the tests the most
18 readable
19
20 The way I write and use test scripts, existing Test::* modules are not
21 nearly as useful as they could be.
22
23 Test scripts written using Test::More work fine when running as part of
24 the test suite, but debugging an individual test requires extra steps,
25 and the tests themselves are not as readable as they should be.
26
27 INTERACTIVE EXECUTION
28 One requirement that I have of a test framework is the ability to
29 interact with it.
30
31 I do most of my debugging using test scripts. When I find a bug, I write
32 a test case for it (typically by adding it to an existing test script)
33 and then debug it using the test script. Then I leave the test there to
34 ensure that the bug won't come back (hopefully).
35
36 Since I use test scripts in a very interactive way (often in the
37 debugger), I want to be able to do the following trivially:
38
39 Easy access to a specific test or tests
40 I'd like to be able to run only a single test, or a subset of tests.
41
42 Easily set breakpoints in the debugger
43 Setting a breakpoint in the debugger to run up to the start of the
44 Nth test is one of the most common tasks I want to do when I'm
45 debugging a failed test.
46
47 To illustrate the first point, in Test::More, a series of tests might be
48 specified in a test script as shown in the following example (line
49 numbers added for convenience):
50
51 ...
52
53 100: # test 1
54 101: $result = func("apples","bushels");
55 102: is($result, "enough");
56 103:
57 104: # test 2
58 105: $result = func("grapefruit","tons");
59 106: is($result, "enough");
60 107:
61 108: # test 3
62 109: $result = func("oranges","boatloads");
63 110: is($result, "insufficient");
64 111:
65 112: # tests 4-6
66 113: foreach my $arg (qw(pears plums pineapple)) {
67 114: $result = func($arg,"boxes");
68 115: is($result, "enough");
69 116: }
70
71 ...
72
73 Say you ran the test suite, and test 3 failed. To debug it you have to
74 open up the test script, find the 3rd test, and set the appropriate
75 breakpoint. In this case, you'll want to break at line 109.
76
77 None of these steps are impossible of course, but it will take some time
78 to get it right. It becomes harder when there are lots of tests (imagine
79 that you want to test the 117th test instead of the 3rd test) or when
80 tests are wrapped up in loops, embedded in subroutines, or other similar
81 situations.
82
83 As an example, what if it's the 5th test that fails in the example
84 above. Now the break point will be a conditional one, so you have to
85 figure out not only the line, but the condition the appropriate state
86 during that test. In this case, you need to stop at line 114 when $arg
87 is 'plums'.
88
89 Wouldn't it be far better to set a break point in func when the Nth test
90 is reached? With Test::Inter, you can.
91
92 So for the above script, the debugger commands that you would use to
93 debug the 3rd test are:
94
95 Test::More : b 109
96 Test::Inter: b func ($::TI_NUM==3)
97
98 and the 5th test are:
99
100 Test::More : b 114 ($arg eq 'plums')
101 Test::Inter: b func ($::TI_NUM==5)
102
103 It would also be nice to be able to skip the first two tests... perhaps
104 they take a long time to run, and I want to get right to work on test 3.
105 You can do this easily too by setting the $::TI_START variable.
106
107 There are some other variables that can be used to specify which test or
108 tests to run described in the "TEST::INTER VARIABLES" section below.
109
110 The other thing I want to do when I run the test scripts interactively
111 is to see more information which will assist in debugging a failed test.
112
113 This can be controlled with variables such as TI_QUIET, TI_MODE, and
114 TI_WIDTH described below in the "TEST::INTER VARIABLES" section.
115
116 READABLE TESTS
117 The other feature that I wanted in a test suite is the ability to define
118 the tests in a format that is natural and readable FOR THE TESTS. In
119 almost every case, it is best to think of a test script as consisting of
120 two separate parts: a script part, and a test part.
121
122 The script part of a test script is the least important part! It's
123 usually fairly trivial, rarely needs to be changed, and is not the focus
124 of the test script.
125
126 The tests part of the script IS the important part, and these should be
127 expressed in a form that is natural to them, easy to maintain, easy to
128 read, and easy to modify, and none of these should involve modifying the
129 script portion of the test script in general. Because the content of the
130 tests is the important part of the script, the emphasis should be in
131 making them more readable, even at the expense of the script portion. As
132 a general rule, if the script portion of the test script obscures the
133 tests in any way, it's not written correctly!
134
135 The solution to this is well understood, and is common to many other
136 systems where you are mixing two "languages". The task of correctly
137 specifying both the tests and the test script is virtually identical to
138 the task of creating a PHP script which consists of a mixture of PHP and
139 HTML, or the task of creating a template file using some templating
140 system where the file consists of a mixture of text to be displayed and
141 templating commands. It is well understood in each of these cases that
142 the more the two "languages" are interwoven, the less readable both are,
143 and the harder it is to maintain. The more you are able to separate the
144 two, the easier both are to read and maintain.
145
146 As often as possible, I want the tests to be written in some sort of
147 text format which can be easily viewed and modified (usually as a simple
148 table) with no perl commands interspersed. I want to the freedom to
149 define the tests in one section (a long string, the DATA section, or
150 even in a separate file) which is easily readable. This may introduce
151 the necessity of parsing it, but it makes it significantly easier to
152 maintain the tests.
153
154 This flexibility makes it much easier to read the tests (as opposed to
155 the script) which is the fundamental content of a test script.
156
157 Looking again at the example test script, you can see that there is far
158 too much perl interspersed with the tests.
159
160 It's difficult to read the tests individually in this script because
161 there is too much perl code among them, and virtually impossible to look
162 at them as a whole.
163
164 It is true that looking at this particular example, it is very simple...
165 but the script ISN'T the content you're interested in (and bear in mind
166 that many test scripts are nowhere near this simple). The REAL content
167 of this script are the tests, which consist of the function arguments
168 and the expected result. Although it's not impossible to see each of
169 these in the script above, it's not in a format that is conducive to
170 studying the tests, and especially not for examining the list of tests
171 as a whole.
172
173 Now, look at an alternate way of specifying the tests using this module:
174
175 $tests = "
176
177 apples bushels => enough
178
179 grapefruit tons => enough
180
181 oranges boatloads => insufficient
182
183 pears boxes => enough
184
185 plums boxes => enough
186
187 pineapple boxes => enough
188
189 ";
190
191 $o->tests(tests => $tests,
192 func => \&func);
193
194 Here, it's easy to see the list of tests, and adding additional tests is
195 a breeze.
196
197 CREATING A TEST
198 This module supports a number of methods for defining tests, so you can
199 use whichever one is most convenient (including methods that are
200 identical to Test::More if that really is the best method).
201
202 Every test may have several pieces of information:
203
204 A name
205 Every test is automatically assigned a number, but it may be useful
206 to specify a name of a test (which is actually a short description
207 of the test). Whenever a test result is reported, the name will be
208 given (if one was specified).
209
210 The name may not have a '#' in it.
211
212 The name is completely optional, but can serve to make the results
213 more readable.
214
215 An expected result
216 In order to test something, you need to know what result was
217 expected (or in some cases, what result was NOT expected).
218
219 A function and arguments OR a result
220 You also need to know the results that you're comparing to the
221 expected results.
222
223 This can be obtained by simply working with a set of results, or a
224 function name and a set of arguments to pass to it.
225
226 Conditions
227 It is useful to be able to specify state information at the start of
228 the test suite (for example, to see if certain features are
229 available), and some tests may only run if those conditions are met.
230
231 If no conditions are set for a test, it will always run.
232
233 Todo tests
234 Some tests may be marked as 'todo' tests. These are test which are
235 allowed to fail (meaning that they have been put in place for an
236 as-yet unimplemented feature). Since it is expected that the test
237 will fail, the test suite will still pass, even if these tests fail.
238
239 The tests will still run and if they pass, a message is issued
240 saying that the feature is now implemented, and the tests should be
241 graduated to non-todo state.
242
243 BASE METHODS
244 new
245 $o = new Test::Inter [$name] [%options];
246
247 This creates a new test framework. There are several options which
248 may be used to specify which tests are run, how they are run, and
249 what output is given.
250
251 The entire test script can be named by passing in $name.
252
253 All options can be set in four different ways.
254
255 First, you can pass in a hash of OPT = VAL> pairs in the new method.
256 So, to set the start option, the %options) hash would contain:
257
258 start => VALUE
259
260 Second, you can set an environment variable. This will override any
261 value passed in the first way. The environment variable is named
262 TI_XXX where XXX is the fully capitalized option. So:
263
264 $ENV{TI_START} = VALUE
265
266 The third method, which overrides the previous two, is to set a
267 global variable. It is also named TI_XXX in the main namespace, so
268 to set it this way, set:
269
270 $::TI_START = VALUE
271
272 The final method is to call one of the methods below and these
273 override all other methods.
274
275 Each of the allowed options are described below in the following
276 base methods:
277
278 start
279 end
280 testnum
281 plan
282 abort
283 quiet
284 mode
285 skip_all
286 width
287 use_lib
288
289 version
290 $o->version();
291
292 Returns the version of the module.
293
294 encoding
295 $o->encoding($encoding);
296
297 $encoding is any value that can be passed as an encoding to perl's
298 Encode::decode function.
299
300 Use this if your test strings contain characters in other encodings.
301
302 start
303 $o = new Test::Inter 'start' => $N;
304 $o->start($N)
305
306 To define which test you want to start with, set the start option as
307 described in the new method above.
308
309 When the start test is defined, most tests numbered less than N are
310 completely ignored. If the tests are being run quietly (see the
311 quiet method below), nothing is printed out for these tests.
312 Otherwise, a skip message is printed out.
313
314 One class of tests IS still executed. Tests run using the require_ok
315 or use_ok methods (to test the loading of modules) are still run.
316
317 If no value (or a value of 0) is used, tests run from the first
318 test.
319
320 end
321 $o = new Test::Inter 'end' => $M;
322 $o->end($M);
323
324 To define which test you want to end with, set the end option as
325 described in the new method above.
326
327 When the end test is defined, all tests numbered more than M are
328 completely ignored. If the tests are being run quietly (see the
329 quiet method below), nothing is printed out for these tests.
330 Otherwise, a skip message is printed out.
331
332 If no value is given, it defaults to 0 (which means that all
333 remaining tests are run).
334
335 testnum
336 $o = new Test::Inter 'testnum' => $N;
337 $o->testnum($N);
338
339 To run only a single test, set the testnum option as described in
340 the new method above.
341
342 It is equivalent to setting both the start and end tests to $N.
343
344 plan
345 done_testing
346 $o = new Test::Inter 'plan' => $N;
347 $o->plan($n);
348
349 $o->done_testing();
350 $o->done_testing($n);
351
352 The TAP API (the 'language' used to run a sequence of tests and see
353 which ones failed and which ones passed) requires a statement of the
354 number of tests that are expected to run.
355
356 This statement can appear at the start of the test suite, or at the
357 end.
358
359 If you know in advance how many tests should run in the test script,
360 you can set the plan option as described in the new method above to
361 the number of tests.
362
363 If you know how many tests should run at the end of the test script,
364 you can pass in a non-zero integer to the done_testing method.
365
366 Frequently, you don't really care how many tests are in the script
367 (especially if new tests are added on a regular basis). In this
368 case, you still need to include a statement that says that the
369 number of tests expected is however many were run. To do this, call
370 the done_testing method with no argument.
371
372 NOTE: if the plan method is used, it MUST be used before any tests
373 are run (including those that test the loading of modules). If the
374 done_testing method is used, it MUST be called after all tests are
375 run. You must specify a plan or use a done_testing statement, but
376 you cannot do both.
377
378 It is NOT strictly required to set a plan if the script is only run
379 interactively, so if for some reason this module is used for test
380 scripts which are not part of a standard perl test suite, the plan
381 and done_testing statements are optional. As a matter of fact, the
382 script will run just fine without them... but a perl installer will
383 report a failure in the test suite.
384
385 abort
386 $o = new Test::Inter 'abort' => 0/1/2;
387 $o->abort(0/1/2);
388
389 To define how you want a failure to be treated, set the abort option
390 as described in the new method above. The abort option can take a
391 value of 0, 1, or 2.
392
393 If this is set to 1, the test script will run unmodified until a
394 test fails. At that point, all remaining tests will be skipped. If
395 it is set to 2, the test script will run until a test fails at which
396 point it will exit with an error code of 1. With a value of 0,
397 failed tests will be reported, but the script will continue.
398
399 In both cases, todo tests will NOT trigger the abort behavior.
400
401 quiet
402 $o = new Test::Inter 'quiet' => 0/1/2;
403 $o->quiet(0/1/2);
404
405 To define how you want failures to be reported, set the quiet option
406 as described in the new method above. The quiet option can take a
407 value of 0, 1, or 2.
408
409 If this is set to 0 (the default), all information will be printed
410 out. If it is set to 1, some optional information will not be
411 printed. If it is set to 2, all optional information will not be
412 printed.
413
414 mode
415 $o = new Test::Inter 'mode' => MODE;
416 $o->mode(MODE);
417
418 Test::Inter scripts can be run in either an interactive mode, or as
419 part of a test suite with different behaviors. To select the mode,
420 set the mode option as described in the new method above. The mode
421 option can take a value of 'inter' or 'test'.
422
423 When run in test mode, it prints out the results using the TAP
424 grammar (i.e. 'ok 1', 'not ok 3', etc.).
425
426 When run in interactive mode, it prints out results in a more human
427 readable format.
428
429 width
430 $o = new Test::Inter 'width' => WIDTH;
431 $o->width(WIDTH);
432
433 The width option can be set as described in the new method above.
434
435 WIDTH is the width of the terminal (for printing out failed test
436 information). It defaults to 80, but it can be set to any width (and
437 lines longer then this are truncated). If WIDTH is set to 0, no
438 truncation is done.
439
440 use_lib
441 $o = new Test::Inter 'use_lib' => VALUE;
442 $o->use_lib(VALUE);
443 $o->use_lib();
444
445 By default, the library included in the module distribution will be
446 added to the search path for modules, so a 'use MODULE' line should
447 find the version stored in this module distribution.
448
449 If VALUE is set to 'off', the search path will not be modified
450 automatically.
451
452 You may add the library path at a later time by calling:
453
454 $o->use_lib('on');
455 $o->use_lib();
456
457 Note: both calls must be used. The first sets the option, the second
458 actually modifies the search path.
459
460 skip_all
461 $o = new Test::Inter 'skip_all' => REASON;
462 $o->skip_all(REASON);
463
464 The skip_all option can be set as described in the new method above.
465
466 If this is set, the entire test script will be skipped for the
467 reason given. This must be done before any test is run, and before
468 any plan number is set.
469
470 The skip_all can also be called at any point during the script (i.e.
471 after tests have been run). In this case, all remaining scripts will
472 be skipped.
473
474 $o->skip_all(REASON,FEATURE,FEATURE,...);
475 $o->skip_all('',FEATURE,FEATURE,...);
476
477 This will skip all tests (or all remaining tests) unless all
478 <FEATURE>s are available. REASON can be entered as an empty string
479 and the reason the tests are skipped will be a message about the
480 missing feature.
481
482 feature
483 $o->feature($feature,$val);
484
485 This defines a feature. If $val is non-zero, the feature is
486 available. Otherwise it is not.
487
488 diag
489 note
490 $o->diag($message);
491 $o->note($message);
492
493 Both of these print an optional message. Messages printed with the
494 "note" method are always optional and will be omitted if the quiet
495 option is set to 1 or 2. Messages printed with the "diag" method are
496 optional and will not be printed if the quiet option is set to 2,
497 but they will be printed if the quiet method is set to 1.
498
499 testdir
500 $o->testdir();
501 $o->testdir('mod');
502 $o->testdir('lib');
503
504 Occasionally, it may be necessary to know the directory where
505 Test::Inter gets some of it's information. By default, the directory
506 containing the tests will be returned, but if the optional argument
507 'mod' is included, it will return the path to the module
508 distribution (which should include both a lib and t subdirerctory).
509 If the argument 'lib' is included, it will return the directory
510 where the libraries are stored.
511
512 METHODS FOR LOADING MODULES
513 Test scripts can load other modules (using either the perl "use" or
514 "require" commands). There are three different modes for doing this
515 which determine how this is done.
516
517 required
518 By default, this is used to test for a module that is required for
519 all tests in the test script.
520
521 Loading the module is treated as an actual test in the test suite.
522 The test is to determine whether the module is available and can be
523 loaded. If it can be loaded, it is, and it is reported as a
524 successful test. If it cannot be loaded, it is reported as a failed
525 test.
526
527 In the result of a failed test, all remaining tests will be skipped
528 automatically (except for other tests which load modules).
529
530 feature
531 In feature mode, loading the module is not treated as a test (i.e.
532 it will not print out an 'ok' or 'not ok' line. Instead, it will set
533 a feature (named the same as the module) which can be used to
534 determine whether other tests should run or not.
535
536 forbid
537 In a few very rare cases, we may want to test for a module but
538 expect that it not be present. This is the exact opposite of the
539 required mode.
540
541 Successfully loading the module is treated as a test failure. In the
542 event of a failure, all remaining tests will be skipped.
543
544 The methods available are:
545
546 require_ok
547 $o->require_ok($module [,$mode]);
548
549 This is used to load a module using the perl "require" function. If
550 $mode is not passed in, the default mode (required) is used to test
551 the existence of the module.
552
553 If $mode is passed in, it must be either the string 'forbid' or
554 'feature'.
555
556 If $mode is 'feature', a feature named $module is set if the module
557 was able to be loaded.
558
559 use_ok
560 $o->use_ok(@args [,$mode]);
561
562 This is used to load a module with "use", or check a perl version.
563
564 BEGIN { $o->use_ok('5.010'); }
565 BEGIN { $o->use_ok('Some::Module'); }
566 BEGIN { $o->use_ok('Some::Module',2.05); }
567 BEGIN { $o->use_ok('Some::Module','foo','bar'); }
568 BEGIN { $o->use_ok('Some::Module',2.05,'foo','bar'); }
569
570 are the same as:
571
572 use 5.010;
573 use Some::Module;
574 use Some::Module 2.05;
575 use Some::Module qw(foo bar);
576 use Some::Module 2.05 qw(foo bar);
577
578 Putting the use_ok call in a BEGIN block allows the functions to be
579 imported at compile-time and prototypes are properly honored. You'll
580 also need to load the Test::Inter module, and create the object in a
581 BEGIN block.
582
583 $mode acts the same as in the require_ok method.
584
585 METHODS FOR RUNNING TEST
586 There are several methods for running tests. The ok, is, and isnt
587 methods are included for those already comfortable with Test::More and
588 wishing to stick with the same format of test script. The tests method
589 is the suggested method though since it makes use of the full power of
590 this module.
591
592 ok
593 $o->ok(TESTS);
594
595 A test run with ok looks at a result, and if it evaluates to 0 (or
596 false), it fails. If it evaluates to non-zero (or true), it passes.
597
598 These tests do not require you to specify the expected results. If
599 expected results are given, they will be compared against the result
600 received, and if they differ, a diagnostic message will be printed,
601 but the test will still succeed or fail based only on the actual
602 result produced.
603
604 These tests require a single result and either zero or one expected
605 results.
606
607 To run a single test, use any of the following:
608
609 $o->ok(); # always succeeds
610
611 $o->ok($result);
612 $o->ok($result,$name);
613 $o->ok($result,$expected,$name);
614
615 $o->ok(\&func);
616 $o->ok(\&func,$name);
617 $o->ok(\&func,$expected,$name);
618
619 $o->ok(\&func,\@args);
620 $o->ok(\&func,\@args,$name);
621 $o->ok(\&func,\@args,$expected,$name);
622
623 If $result is a scalar, the test passes if $result is true. If
624 $result is a list reference, the test succeeds if the list contains
625 any defined values. If $result is a hash reference, the test
626 succeeds if the hash contains any key with a value that is not
627 "undef".
628
629 If "\&func" and "\@args" are passed in, then $result is generated by
630 passing @args to &func and behaves identically to the calls where
631 $result is passed in. If "\&func" is passed in but no arguments, the
632 function takes no arguments, but still produces a result.
633
634 If an expected value is passed in and the result does not match it,
635 a diagnostic warning will be printed, even if the test passes.
636
637 is
638 isnt
639 $o->is(TESTS);
640 $o->isnt(TESTS);
641
642 A test run with is looks at a result and tests to see if it is
643 identical to an expected result. If it is, the test passes.
644 Otherwise it fails. In the case of a failure, a diagnostic message
645 will show what result was actually obtained and what was expected.
646
647 A test run with isnt looks at a result and tests to see if the
648 result obtained is different than an expected result. If it is
649 different, the test passes. Otherwise it fails.
650
651 The is method can be called in any of the following ways:
652
653 $o->is($result,$expected);
654 $o->is($result,$expected,$name);
655
656 $o->is(\&func,$expected);
657 $o->is(\&func,$expected,$name);
658
659 $o->is(\&func,\@args,$expected);
660 $o->is(\&func,\@args,$expected,$name);
661
662 The isnt method can be called in exactly the same way.
663
664 As with the ok method, the result can be a scalar, hashref, or
665 listref. If it is a hashref or listref, the entire structure must
666 match the expected value.
667
668 tests
669 $o->tests($opt=>$val, $opt=>$val, ...)
670
671 The options available are described in the following section.
672
673 file
674 $o->file($func,$input,$outputdir,$expected,$name [,@args]);
675
676 Sometimes it may be easiest to store the input, output, and expected
677 output from a test in a text file. In this case, each line of output
678 will be treated as a single test, so the output and expected output
679 must match up exactly.
680
681 $func is a reference to a function which will produce a temporary
682 output file.
683
684 If $input is specified, it is the name of the input file. If it is
685 empty, no input file will be used. The input file can be fully
686 specified, or it can be relative to the test directory.
687
688 If $outputdir is passed in, it is the directory where the output
689 file will be written. It can be fully specified, or relative to the
690 test directory. If $outputdir is left blank, the temporary file will
691 be written to the test directory.
692
693 $expected is the name of a file which contains the expected output.
694 It can be fully specified, or it will be checked for in the test
695 directory.
696
697 $name is the name of this series of tests.
698
699 @args are extra arguments to pass to the test function.
700
701 The function will be called with the arguments:
702
703 &$func( [$input,] $output,@args);
704
705 $input is only passed in if it was passed in to this method. If no
706 input file is specified, nothing will be passed to the function.
707
708 $output is the name of a temporary file where the output will be
709 written to.
710
711 USING THE TESTS METHOD
712 It is expected that most tests (except for those that load a module)
713 will be run using the tests method called as:
714
715 $o->tests(%options);
716
717 The following options are available:
718
719 name
720 name => NAME
721
722 This sets the name of this set of tests. All tests will be given the
723 same name.
724
725 tests
726 func
727 expected
728 In order to specify a series of tests, you have to specify either a
729 function and a list of arguments, or a list of results.
730
731 Specifying the function and list of arguments can be done using the
732 pair:
733
734 func => \&FUNCTION
735 tests => TESTS
736
737 If the func option is not set, tests contains a list of results.
738
739 A list of expected results may also be given. They can be included
740 in the
741
742 tests => TESTS
743
744 option or included separately as:
745
746 expected => RESULTS
747
748 The way to specify these are covered in the next section SPECIFYING
749 THE TESTS.
750
751 feature
752 disable
753 feature => [FEATURE1, FEATURE2, ...]
754
755 disable => [FEATURE1, FEATURE2, ...]
756
757 The default set of tests to run is determined using the start, end,
758 and skip_all methods discussed above. Using those methods, a list of
759 tests is obtained, and it is expected that these will run.
760
761 The feature and disable options modify the list.
762
763 If the feature option is included, the tests given in this call will
764 only run if ALL of the features listed are available.
765
766 If the disable option is included, the tests will be run unless ANY
767 of the features listed are available.
768
769 skip
770 skip => REASON
771
772 Skip these tests for the reason given.
773
774 todo
775 todo => 0/1
776
777 Setting this to 1 says that these tests are allowed to fail. They
778 represent a feature that is not yet implemented.
779
780 If the tests succeed, a message will be printed notifying the
781 developer that the tests are now ready to promote to actual use.
782
783 SPECIFYING THE TESTS
784 A series of tests can be specified in two different ways. The tests can
785 be written in a very simple string format, or stored as a list.
786
787 Demonstrating how this can be done is best done by example, so let's say
788 that there is a function (func) which takes two arguments, and returns a
789 single value. Let's say that the expected output (and the actual output)
790 from 3 different sets of arguments is:
791
792 Input Expected Output Actual Output
793 ----- --------------- -------------
794 1,2 a a
795 3,4 b x
796 5,6 c c
797
798 (so in this case, the first and third tests pass, but the 2nd one will
799 fail).
800
801 Specifying these tests as lists could be done as:
802
803 $o->tests(
804 func => &func,
805 tests => [ [1,2], [3,4], [5,6] ],
806 expected => [ [a], [b], [c] ],
807 );
808
809 Here, the tests are stored as a list, and each element in the list is a
810 listref containing the set of arguments.
811
812 If the func option is not passed in, the tests option is set to a list
813 of results to compare with the expected results, so the following is
814 equivalent to the above:
815
816 $o->tests(
817 tests => [ [a], [x], [c] ],
818 expected => [ [a], [b], [c] ],
819 );
820
821 If an argument (or actual result) or an expected result is only a single
822 value, it can be entered as a scalar instead of a list ref, so the
823 following is also equivalent:
824
825 $o->tests(
826 func => &func,
827 tests => [ [1,2], [3,4], [5,6] ],
828 expected => [ a, b, [c] ],
829 );
830
831 The only exception to this is if the single value is itself a list
832 reference. In this case it MUST be included as a reference. In other
833 words, if you have a single test, and the expected value for this test
834 is a list reference, it must be passed in as:
835
836 expected => [ [ \@r ] ]
837
838 NOT as:
839
840 expected => [ \@r ]
841
842 Passing in a set of expected results is optional. If none are passed in,
843 the tests are treated as if they had been passed to the ok method (i.e.
844 if they return something true, they pass, otherwise they fail).
845
846 The second way to specify tests is as a string. The string is a
847 multi-line string with each tests being separate from the next test by a
848 blank line. Comments (lines which begin with '#') are allowed, and are
849 ignored. Whitespace at the start and end of the line is ignored.
850
851 The string may contain the results directly, or results may be passed in
852 separately. For example, the following all give the same sets of tests
853 as the example above:
854
855 $o->tests(
856 func => &func,
857 tests => "
858 # Test 1
859 1 2 => a
860
861 # Test 2
862 3 4 => b
863
864 5 6 => c
865 ",
866 );
867
868 $o->tests(
869 func => &func,
870 tests => "
871 1 2
872
873 3 4
874
875 5 6
876 ",
877 expected => [ [a], [b], [c] ]
878 );
879
880 $o->tests(
881 func => &func,
882 tests => [ [1,2], [3,4], [5,6] ],
883 expected => "
884 a
885
886 b
887
888 c
889 ",
890 );
891
892 $o->tests(
893 func => &func,
894 tests => "
895 1 2
896
897 3 4
898
899 5 6
900 ",
901 expected => "
902 a
903
904 b
905
906 c
907 ",
908 );
909
910 The expected results may also consist of only a single set of results
911 (in this case, it must be passed in as a listref). In this case, all of
912 the tests are expected to have the same results.
913
914 So, the following are equivalent:
915
916 $o->tests(
917 func => &func,
918 tests => "
919 1 2 => a b
920
921 3 4 => a b
922
923 5 6 => a b
924 ",
925 );
926
927 $o->tests(
928 func => &func,
929 tests => "
930 1 2
931
932 3 4
933
934 5 6
935 ",
936 expected => [ [a, b] ],
937 );
938
939 $o->tests(
940 func => &func,
941 tests => "
942 1 2
943
944 3 4
945
946 5 6
947 ",
948 expected => "a b",
949 );
950
951 The number of expected values must either be 1 (i.e. all of the tests
952 are expected to produce the same value) or exactly the same number as
953 the number of tests.
954
955 The parser is actually quite powerful, and can handle multi-line tests,
956 quoted strings, and nested data structures.
957
958 The test may be split across any number of lines, provided there is not
959 a completely blank line (which signals the end of the test), so the
960 following are equivalent:
961
962 tests => "a b c",
963 tests => "a b
964 c",
965
966 Arguments (or expected results) may include data structures. For
967 example, the following are equivalent:
968
969 tests => "[ a b ] { a 1 b 2 }"
970 tests => [ [ [a,b], { a=>1, b=>2 } ] ]
971
972 Whitespace is mostly optional, but there is one exception. An item must
973 end with some kind of delimiter, so the following will fail:
974
975 tests => "[a b][c d]"
976
977 The first element (the list ref [a b]) must be separated from the second
978 element by the delimiter (which is whitespace in this case), so it must
979 be written as:
980
981 tests => "[a b] [c d]"
982
983 As already demonstrated, hashrefs and listrefs may be included and
984 nested. Elements may also be included inside parens, but this is
985 optional since all arguments and expected results are already treated as
986 lists, so the following are equivalent:
987
988 tests => "a b c"
989 tests => "(a b) c"
990
991 Although parens are optional, they may make things more readable, and
992 allow you to use something other than whitespace as the delimiter. Since
993 parens are actually ignored, a string '()' is also ignored, so do not
994 use empty parentheses.
995
996 If the character immediately following the opening paren, brace, or
997 bracket is a punctuation mark, then it is used as the delimiter instead
998 of whitespace. For example, the following are all equivalent:
999
1000 [ a b c ]
1001 [a b c]
1002 [, a,b,c ]
1003 [, a, b, c ]
1004
1005 A delimiter is a single character, and the following may not be used as
1006 a delimiter:
1007
1008 any opening/closing characters () [] {}
1009 single or double quotes
1010 alphanumeric characters
1011 underscore
1012
1013 Whitespace (including newlines) around the delimiter is ignored, so the
1014 following is valid:
1015
1016 [, a,
1017 b,
1018 c ]
1019
1020 Two delimiters next to each other or a trailing delimiter produce an
1021 empty string.
1022
1023 "(,a,b,)" => (a, b, '')
1024 "(,a,,b)" => (a, '', b)
1025
1026 Hashrefs may be specified by braces and the following are equivalent:
1027
1028 { a 1 b 2 }
1029 {, a,1,b,2 }
1030 {, a,1,b,2, }
1031
1032 Note that a trailing delimiter is ignored if there are already an even
1033 number of elements, or an empty string otherwise.
1034
1035 Nested structures are allowed:
1036
1037 "[ [1 2] [3 4] ]"
1038
1039 For example,
1040
1041 $o->tests(
1042 func => &func,
1043 tests => "a [ b c ] { d 1 e 2 } => x y"
1044 );
1045
1046 is equivalent to:
1047
1048 $o->tests(
1049 func => &func,
1050 tests => [ [a, [b,c], {d=>1,e=>2}] ],
1051 results => [ [x,y] ],
1052 );
1053
1054 Any single value can be surrounded by single or double quotes in order
1055 to include the delimiter. So:
1056
1057 "(, a,'b,c',e )"
1058
1059 is equivalent to:
1060
1061 "( a b,c e )"
1062
1063 Any single value can be the string '__undef__' which will be turned into
1064 an actual undef. If the value is '__blank__' it is turned into an empty
1065 string (''), though it can also be specified as '' directly. Any value
1066 can have an embedded newline by including a __nl__ in the value, but the
1067 value must be written on a single line.
1068
1069 Expected results are separated from arguments by ' => '.
1070
1071 TEST::INTER VARIABLES
1072 To summarize the information above, the following variables are used by
1073 Test::Inter. Each variable can be set in two different ways: as an
1074 environment variable and as a perl variable in the main namespace.
1075
1076 For example, the TI_END variable can be set as:
1077
1078 $::TI_END
1079 $ENV{TI_END}
1080
1081 The following variables can be used to define which tests are run:
1082
1083 TI_START
1084 Set this to define the test you want to start with.
1085
1086 Example: If you have a perl test script and you want to start
1087 running it at test 12, run the following shell commands:
1088
1089 TI_START=12
1090 ./my_test_script.t
1091
1092 TI_END
1093 Set this to define the test you want to end with.
1094
1095 TI_TESTNUM
1096 Set this to run only a single test
1097
1098 There is also a variable TI_NUM (available only as $::TI_NUM) which is
1099 set automatically by Test::Inter to be the test currently being run.
1100
1101 The following variables control what is output from the tests, and how
1102 it is formatted:
1103
1104 TI_QUIET
1105 How verbose the test script is. Values are 0 (most verbose) to 2
1106 (least verbose).
1107
1108 TI_MODE
1109 How the output is formatted. Values are 'inter' (interactive mode)
1110 or 'test' (test suite mode). Interactive mode is easier to read.
1111 Test mode is for running as part of a test suite.
1112
1113 TI_WIDTH
1114 The width of the terminal.
1115
1116 The following variables control how some tests are run:
1117
1118 TI_NOCLEAN
1119 When running a file test, the temporary output file will not be
1120 removed if this is set.
1121
1122 HISTORY
1123 The history of this module dates back to 1996 when I needed to write a
1124 test suite for my Date::Manip module. At that time, none of the Test::*
1125 modules currently available in CPAN existed (the earliest ones didn't
1126 come along until 1998), so I was left completely on my own writing my
1127 test scripts.
1128
1129 I wrote a very basic version of my test framework which allowed me to
1130 write all of the tests as a string, it would parse the string, count the
1131 tests, and then run them.
1132
1133 Over the years, the functionality I wanted grew, and periodically, I'd
1134 go back and reexamine other Test frameworks (primarily Test::More) to
1135 see if I could replace my framework with an existing module... and I've
1136 always found them wanting, and chosen to extend my existing framework
1137 instead.
1138
1139 As I've written other modules, I've wanted to use the framework in them
1140 too, so I've always just copied it in, but this is obviously tedious and
1141 error prone. I'm not sure why it took me so long... but in 2010, I
1142 finally decided it was time to rework the framework in a module form.
1143
1144 I loosely based my module on Test::More. I like the functionality of
1145 that module, and wanted most of it (and I plan on adding more in future
1146 versions). So this module uses some similar syntax to Test::More (though
1147 it allows a great deal more flexibility in how the tests are specified).
1148
1149 One thing to note is that I may have been able to write this module as
1150 an extension to Test::More, but after looking into that possibility, I
1151 decided that it would be faster to not do that. I did "borrow" a couple
1152 of routines from it (though they've been modified quite heavily) as a
1153 starting point for a few of the functions in this module, and I thank
1154 the authors of Test::More for their work.
1155
1156 KNOWN BUGS AND LIMITATIONS
1157 None known.
1158
1159 SEE ALSO
1160 Test::More - the 'industry standard' of perl test frameworks
1161
1162 BUGS AND QUESTIONS
1163 If you find a bug in Test::Inter, there are three ways to send it to me.
1164 Any of them are fine, so use the method that is easiest for you.
1165
1166 Direct email
1167 You are welcome to send it directly to me by email. The email
1168 address to use is: [email protected].
1169
1170 CPAN Bug Tracking
1171 You can submit it using the CPAN tracking too. This can be done at
1172 the following URL:
1173
1174 <http://rt.cpan.org/Public/Dist/Display.html?Name=Test-Inter>
1175
1176 GitHub
1177 You can submit it as an issue on GitHub. This can be done at the
1178 following URL:
1179
1180 <https://github.com/SBECK-github/Test-Inter>
1181
1182 Please do not use other means to report bugs (such as forums for a
1183 specific OS or Linux distribution) as it is impossible for me to keep up
1184 with all of them.
1185
1186 When filing a bug report, please include the following information:
1187
1188 Test::Inter version
1189 Please include the version of Test::Inter you are using. You can get
1190 this by using the script:
1191
1192 use Test::Inter;
1193 print $Test::Inter::VERSION,"\n";
1194
1195 If you want to report missing or incorrect codes, you must be running
1196 the most recent version of Test::Inter.
1197
1198 If you find any problems with the documentation (errors, typos, or items
1199 that are not clear), please send them to me. I welcome any suggestions
1200 that will allow me to improve the documentation.
1201
1202 LICENSE
1203 This script is free software; you can redistribute it and/or modify it
1204 under the same terms as Perl itself.
1205
1206 AUTHOR
1207 Sullivan Beck ([email protected])
1208