| 1 |
NAME |
| 2 |
|
| 3 |
Test::MemoryGrowth - assert that code does not cause growth in memory |
| 4 |
usage |
| 5 |
|
| 6 |
SYNOPSIS |
| 7 |
|
| 8 |
use Test::More; |
| 9 |
use Test::MemoryGrowth; |
| 10 |
|
| 11 |
use Some::Class; |
| 12 |
|
| 13 |
no_growth { |
| 14 |
my $obj = Some::Class->new; |
| 15 |
} 'Constructing Some::Class does not grow memory'; |
| 16 |
|
| 17 |
my $obj = Some::Class->new; |
| 18 |
no_growth { |
| 19 |
$obj->do_thing; |
| 20 |
} 'Some::Class->do_thing does not grow memory'; |
| 21 |
|
| 22 |
|
| 23 |
#### This test will fail #### |
| 24 |
my @list; |
| 25 |
no_growth { |
| 26 |
push @list, "Hello world"; |
| 27 |
} 'pushing to an array does not grow memory'; |
| 28 |
|
| 29 |
done_testing; |
| 30 |
|
| 31 |
DESCRIPTION |
| 32 |
|
| 33 |
This module provides a function to check that a given block of code |
| 34 |
does not result in the process consuming extra memory once it has |
| 35 |
finished. Despite the name of this module it does not, in the strictest |
| 36 |
sense of the word, test for a memory leak: that term is specifically |
| 37 |
applied to cases where memory has been allocated but all record of it |
| 38 |
has been lost, so it cannot possibly be reclaimed. While the method |
| 39 |
employed by this module can detect such bugs, it can also detect cases |
| 40 |
where memory is still referenced and reachable, but the usage has grown |
| 41 |
more than would be expected or necessary. |
| 42 |
|
| 43 |
The block of code will be run a large number of times (by default |
| 44 |
10,000), and the difference in memory usage by the process before and |
| 45 |
after is compared. If the memory usage has now increased by more than |
| 46 |
one byte per call, then the test fails. |
| 47 |
|
| 48 |
In order to give the code a chance to load initial resources it needs, |
| 49 |
it will be run a few times first (by default 10); giving it a chance to |
| 50 |
load files, AUTOLOADs, caches, or any other information that it |
| 51 |
requires. Any extra memory usage here will not count against it. |
| 52 |
|
| 53 |
This simple method is not a guaranteed indicator of the absence of |
| 54 |
memory resource bugs from a piece of code; it has the possibility to |
| 55 |
fail in both a false-negative and a false-positive way. |
| 56 |
|
| 57 |
False Negative |
| 58 |
|
| 59 |
It is possible that a piece of code causes memory usage growth that |
| 60 |
this module does not detect. Because it only detects memory growth of |
| 61 |
at least one byte per call, it cannot detect cases of linear memory |
| 62 |
growth at lower rates than this. Most memory usage growth comes |
| 63 |
either from Perl-level or C-level bugs where memory objects are |
| 64 |
created at every call and not reclaimed again. (These are either |
| 65 |
genuine memory leaks, or needless allocations of objects that are |
| 66 |
stored somewhere and never reclaimed). It is unlikely such a bug |
| 67 |
would result in a growth rate smaller than one byte per call. |
| 68 |
|
| 69 |
A second failure case comes from the fact that memory usage is taken |
| 70 |
from the Operating System's measure of the process's Virtual Memory |
| 71 |
size, so as to be able to detect memory usage growth in C libraries |
| 72 |
or XS-level wrapping code, as well as Perl functions. Because Perl |
| 73 |
does not aggressively return unused memory to the Operating System, |
| 74 |
it is possible that a piece of code could use un-allocated but |
| 75 |
un-reclaimed memory to grow into; resulting in an increase in its |
| 76 |
requirements despite not requesting extra memory from the Operating |
| 77 |
System. |
| 78 |
|
| 79 |
False Positive |
| 80 |
|
| 81 |
It is possible that the test will claim that a function grows in |
| 82 |
memory, when the behaviour is in fact perfectly normal for the code |
| 83 |
in question. For example, the code could simply be some function |
| 84 |
whose behaviour is required to store extra state; for example, adding |
| 85 |
a new item into a list. In this case it is in fact expected that the |
| 86 |
memory usage of the process will increase. |
| 87 |
|
| 88 |
By careful use of this test module, false indications can be minimised. |
| 89 |
By splitting tests across many test scripts, each one can be started in |
| 90 |
a new process state, where most of the memory assigned from the |
| 91 |
Operating System is in use by Perl, so anything extra that the code |
| 92 |
requires will have to request more. This should reduce the false |
| 93 |
negative indications. |
| 94 |
|
| 95 |
By keeping in mind that the module simply measures the change in |
| 96 |
allocated memory size, false positives can be minimised, by not |
| 97 |
attempting to assert that certain pieces of code do not grow in memory, |
| 98 |
when in fact it would be expected that they do. |
| 99 |
|
| 100 |
Devel::Gladiator Integration |
| 101 |
|
| 102 |
Since version 0.04. |
| 103 |
|
| 104 |
If Devel::Gladiator is installed, this test module will use it as a |
| 105 |
second potential source of detecting memory growth. A walk of the Perl |
| 106 |
memory heap is taken before running the code, in order to count the |
| 107 |
number of every kind of object present. This is then compared to a |
| 108 |
second count taken afterwards. Any object types that have increased by |
| 109 |
at least one per call are reported. |
| 110 |
|
| 111 |
For example, the output might contain the following extra lines of |
| 112 |
diagnostic output: |
| 113 |
|
| 114 |
# Growths in arena object counts: |
| 115 |
# ARRAY 1735 -> 11735 (1.00 per call) |
| 116 |
# HASH 459 -> 10459 (1.00 per call) |
| 117 |
# REF 1387 -> 21387 (2.00 per call) |
| 118 |
# REF-ARRAY 163 -> 10163 (1.00 per call) |
| 119 |
# REF-HASH 66 -> 10066 (1.00 per call) |
| 120 |
# WithContainerSlots 10 -> 10010 (1.00 per call) |
| 121 |
|
| 122 |
Devel::MAT Integration |
| 123 |
|
| 124 |
If Devel::MAT is installed, this test module will use it to dump the |
| 125 |
state of the memory after a failure. It will create a .pmat file named |
| 126 |
the same as the unit test, but with the trailing .t suffix replaced |
| 127 |
with -TEST.pmat where TEST is the number of the test that failed (in |
| 128 |
case there was more than one). It will then run the code under test one |
| 129 |
more time, before writing another file whose name is suffixed with |
| 130 |
-TEST-after.pmat. This pair of files may be useful for differential |
| 131 |
analysis. |
| 132 |
|
| 133 |
FUNCTIONS |
| 134 |
|
| 135 |
no_growth |
| 136 |
|
| 137 |
no_growth { CODE } %opts, $name; |
| 138 |
|
| 139 |
Assert that the code block does not consume extra memory. |
| 140 |
|
| 141 |
Takes the following named arguments: |
| 142 |
|
| 143 |
calls => INT |
| 144 |
|
| 145 |
The number of times to call the code during growth testing. |
| 146 |
|
| 147 |
burn_in => INT |
| 148 |
|
| 149 |
The number of times to call the code initially, before watching for |
| 150 |
memory usage. |
| 151 |
|
| 152 |
TODO |
| 153 |
|
| 154 |
* More OS portability |
| 155 |
|
| 156 |
Currently, this module uses OS-specific methods of determining |
| 157 |
process memory usage (namely, by inspecting /proc/self/status on |
| 158 |
Linux or calling procstat on FreeBSD). It would be nice to support |
| 159 |
more OSes, and potentially find a better abstraction for doing so. |
| 160 |
|
| 161 |
Currently I am unaware of a simple portable mechanism to query this. |
| 162 |
Patches very much welcome. :) |
| 163 |
|
| 164 |
AUTHOR |
| 165 |
|
| 166 |
Paul Evans <[email protected]> |
| 167 |
|