| 1 |
.\" -*- mode: troff; coding: utf-8 -*- |
| 2 |
.\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45) |
| 3 |
.\" |
| 4 |
.\" Standard preamble: |
| 5 |
.\" ======================================================================== |
| 6 |
.de Sp \" Vertical space (when we can't use .PP) |
| 7 |
.if t .sp .5v |
| 8 |
.if n .sp |
| 9 |
.. |
| 10 |
.de Vb \" Begin verbatim text |
| 11 |
.ft CW |
| 12 |
.nf |
| 13 |
.ne \\$1 |
| 14 |
.. |
| 15 |
.de Ve \" End verbatim text |
| 16 |
.ft R |
| 17 |
.fi |
| 18 |
.. |
| 19 |
.\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. |
| 20 |
.ie n \{\ |
| 21 |
. ds C` "" |
| 22 |
. ds C' "" |
| 23 |
'br\} |
| 24 |
.el\{\ |
| 25 |
. ds C` |
| 26 |
. ds C' |
| 27 |
'br\} |
| 28 |
.\" |
| 29 |
.\" Escape single quotes in literal strings from groff's Unicode transform. |
| 30 |
.ie \n(.g .ds Aq \(aq |
| 31 |
.el .ds Aq ' |
| 32 |
.\" |
| 33 |
.\" If the F register is >0, we'll generate index entries on stderr for |
| 34 |
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index |
| 35 |
.\" entries marked with X<> in POD. Of course, you'll have to process the |
| 36 |
.\" output yourself in some meaningful fashion. |
| 37 |
.\" |
| 38 |
.\" Avoid warning from groff about undefined register 'F'. |
| 39 |
.de IX |
| 40 |
.. |
| 41 |
.nr rF 0 |
| 42 |
.if \n(.g .if rF .nr rF 1 |
| 43 |
.if (\n(rF:(\n(.g==0)) \{\ |
| 44 |
. if \nF \{\ |
| 45 |
. de IX |
| 46 |
. tm Index:\\$1\t\\n%\t"\\$2" |
| 47 |
.. |
| 48 |
. if !\nF==2 \{\ |
| 49 |
. nr % 0 |
| 50 |
. nr F 2 |
| 51 |
. \} |
| 52 |
. \} |
| 53 |
.\} |
| 54 |
.rr rF |
| 55 |
.\" ======================================================================== |
| 56 |
.\" |
| 57 |
.IX Title "PERLPERF 1" |
| 58 |
.TH PERLPERF 1 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide" |
| 59 |
.\" For nroff, turn off justification. Always turn off hyphenation; it makes |
| 60 |
.\" way too many mistakes in technical documents. |
| 61 |
.if n .ad l |
| 62 |
.nh |
| 63 |
.SH NAME |
| 64 |
perlperf \- Perl Performance and Optimization Techniques |
| 65 |
.SH DESCRIPTION |
| 66 |
.IX Header "DESCRIPTION" |
| 67 |
This is an introduction to the use of performance and optimization techniques |
| 68 |
which can be used with particular reference to perl programs. While many perl |
| 69 |
developers have come from other languages, and can use their prior knowledge |
| 70 |
where appropriate, there are many other people who might benefit from a few |
| 71 |
perl specific pointers. If you want the condensed version, perhaps the best |
| 72 |
advice comes from the renowned Japanese Samurai, Miyamoto Musashi, who said: |
| 73 |
.PP |
| 74 |
.Vb 1 |
| 75 |
\& "Do Not Engage in Useless Activity" |
| 76 |
.Ve |
| 77 |
.PP |
| 78 |
in 1645. |
| 79 |
.SH OVERVIEW |
| 80 |
.IX Header "OVERVIEW" |
| 81 |
Perhaps the most common mistake programmers make is to attempt to optimize |
| 82 |
their code before a program actually does anything useful \- this is a bad idea. |
| 83 |
There's no point in having an extremely fast program that doesn't work. The |
| 84 |
first job is to get a program to \fIcorrectly\fR do something \fBuseful\fR, (not to |
| 85 |
mention ensuring the test suite is fully functional), and only then to consider |
| 86 |
optimizing it. Having decided to optimize existing working code, there are |
| 87 |
several simple but essential steps to consider which are intrinsic to any |
| 88 |
optimization process. |
| 89 |
.SS "ONE STEP SIDEWAYS" |
| 90 |
.IX Subsection "ONE STEP SIDEWAYS" |
| 91 |
Firstly, you need to establish a baseline time for the existing code, which |
| 92 |
timing needs to be reliable and repeatable. You'll probably want to use the |
| 93 |
\&\f(CW\*(C`Benchmark\*(C'\fR or \f(CW\*(C`Devel::NYTProf\*(C'\fR modules, or something similar, for this step, |
| 94 |
or perhaps the Unix system \f(CW\*(C`time\*(C'\fR utility, whichever is appropriate. See the |
| 95 |
base of this document for a longer list of benchmarking and profiling modules, |
| 96 |
and recommended further reading. |
| 97 |
.SS "ONE STEP FORWARD" |
| 98 |
.IX Subsection "ONE STEP FORWARD" |
| 99 |
Next, having examined the program for \fIhot spots\fR, (places where the code |
| 100 |
seems to run slowly), change the code with the intention of making it run |
| 101 |
faster. Using version control software, like \f(CW\*(C`subversion\*(C'\fR, will ensure no |
| 102 |
changes are irreversible. It's too easy to fiddle here and fiddle there \- |
| 103 |
don't change too much at any one time or you might not discover which piece of |
| 104 |
code \fBreally\fR was the slow bit. |
| 105 |
.SS "ANOTHER STEP SIDEWAYS" |
| 106 |
.IX Subsection "ANOTHER STEP SIDEWAYS" |
| 107 |
It's not enough to say: "that will make it run faster", you have to check it. |
| 108 |
Rerun the code under control of the benchmarking or profiling modules, from the |
| 109 |
first step above, and check that the new code executed the \fBsame task\fR in |
| 110 |
\&\fIless time\fR. Save your work and repeat... |
| 111 |
.SH "GENERAL GUIDELINES" |
| 112 |
.IX Header "GENERAL GUIDELINES" |
| 113 |
The critical thing when considering performance is to remember there is no such |
| 114 |
thing as a \f(CW\*(C`Golden Bullet\*(C'\fR, which is why there are no rules, only guidelines. |
| 115 |
.PP |
| 116 |
It is clear that inline code is going to be faster than subroutine or method |
| 117 |
calls, because there is less overhead, but this approach has the disadvantage |
| 118 |
of being less maintainable and comes at the cost of greater memory usage \- |
| 119 |
there is no such thing as a free lunch. If you are searching for an element in |
| 120 |
a list, it can be more efficient to store the data in a hash structure, and |
| 121 |
then simply look to see whether the key is defined, rather than to loop through |
| 122 |
the entire array using \fBgrep()\fR for instance. \fBsubstr()\fR may be (a lot) faster |
| 123 |
than \fBgrep()\fR but not as flexible, so you have another trade-off to access. Your |
| 124 |
code may contain a line which takes 0.01 of a second to execute which if you |
| 125 |
call it 1,000 times, quite likely in a program parsing even medium sized files |
| 126 |
for instance, you already have a 10 second delay, in just one single code |
| 127 |
location, and if you call that line 100,000 times, your entire program will |
| 128 |
slow down to an unbearable crawl. |
| 129 |
.PP |
| 130 |
Using a subroutine as part of your sort is a powerful way to get exactly what |
| 131 |
you want, but will usually be slower than the built-in \fIalphabetic\fR \f(CW\*(C`cmp\*(C'\fR and |
| 132 |
\&\fInumeric\fR \f(CW\*(C`<=>\*(C'\fR sort operators. It is possible to make multiple |
| 133 |
passes over your data, building indices to make the upcoming sort more |
| 134 |
efficient, and to use what is known as the \f(CW\*(C`OM\*(C'\fR (Orcish Maneuver) to cache the |
| 135 |
sort keys in advance. The cache lookup, while a good idea, can itself be a |
| 136 |
source of slowdown by enforcing a double pass over the data \- once to setup the |
| 137 |
cache, and once to sort the data. Using \f(CWpack()\fR to extract the required sort |
| 138 |
key into a consistent string can be an efficient way to build a single string |
| 139 |
to compare, instead of using multiple sort keys, which makes it possible to use |
| 140 |
the standard, written in \f(CW\*(C`c\*(C'\fR and fast, perl \f(CWsort()\fR function on the output, |
| 141 |
and is the basis of the \f(CW\*(C`GRT\*(C'\fR (Guttman Rossler Transform). Some string |
| 142 |
combinations can slow the \f(CW\*(C`GRT\*(C'\fR down, by just being too plain complex for its |
| 143 |
own good. |
| 144 |
.PP |
| 145 |
For applications using database backends, the standard \f(CW\*(C`DBIx\*(C'\fR namespace has |
| 146 |
tried to help with keeping things nippy, not least because it tries to \fInot\fR |
| 147 |
query the database until the latest possible moment, but always read the docs |
| 148 |
which come with your choice of libraries. Among the many issues facing |
| 149 |
developers dealing with databases should remain aware of is to always use |
| 150 |
\&\f(CW\*(C`SQL\*(C'\fR placeholders and to consider pre-fetching data sets when this might |
| 151 |
prove advantageous. Splitting up a large file by assigning multiple processes |
| 152 |
to parsing a single file, using say \f(CW\*(C`POE\*(C'\fR, \f(CW\*(C`threads\*(C'\fR or \f(CW\*(C`fork\*(C'\fR can also be a |
| 153 |
useful way of optimizing your usage of the available \f(CW\*(C`CPU\*(C'\fR resources, though |
| 154 |
this technique is fraught with concurrency issues and demands high attention to |
| 155 |
detail. |
| 156 |
.PP |
| 157 |
Every case has a specific application and one or more exceptions, and there is |
| 158 |
no replacement for running a few tests and finding out which method works best |
| 159 |
for your particular environment, this is why writing optimal code is not an |
| 160 |
exact science, and why we love using Perl so much \- TMTOWTDI. |
| 161 |
.SH BENCHMARKS |
| 162 |
.IX Header "BENCHMARKS" |
| 163 |
Here are a few examples to demonstrate usage of Perl's benchmarking tools. |
| 164 |
.SS "Assigning and Dereferencing Variables" |
| 165 |
.IX Subsection "Assigning and Dereferencing Variables" |
| 166 |
I'm sure most of us have seen code which looks like, (or worse than), this: |
| 167 |
.PP |
| 168 |
.Vb 2 |
| 169 |
\& if ( $obj\->{_ref}\->{_myscore} >= $obj\->{_ref}\->{_yourscore} ) { |
| 170 |
\& ... |
| 171 |
.Ve |
| 172 |
.PP |
| 173 |
This sort of code can be a real eyesore to read, as well as being very |
| 174 |
sensitive to typos, and it's much clearer to dereference the variable |
| 175 |
explicitly. We're side-stepping the issue of working with object-oriented |
| 176 |
programming techniques to encapsulate variable access via methods, only |
| 177 |
accessible through an object. Here we're just discussing the technical |
| 178 |
implementation of choice, and whether this has an effect on performance. We |
| 179 |
can see whether this dereferencing operation, has any overhead by putting |
| 180 |
comparative code in a file and running a \f(CW\*(C`Benchmark\*(C'\fR test. |
| 181 |
.PP |
| 182 |
# dereference |
| 183 |
.PP |
| 184 |
.Vb 1 |
| 185 |
\& #!/usr/bin/perl |
| 186 |
\& |
| 187 |
\& use v5.36; |
| 188 |
\& |
| 189 |
\& use Benchmark; |
| 190 |
\& |
| 191 |
\& my $ref = { |
| 192 |
\& \*(Aqref\*(Aq => { |
| 193 |
\& _myscore => \*(Aq100 + 1\*(Aq, |
| 194 |
\& _yourscore => \*(Aq102 \- 1\*(Aq, |
| 195 |
\& }, |
| 196 |
\& }; |
| 197 |
\& |
| 198 |
\& timethese(1000000, { |
| 199 |
\& \*(Aqdirect\*(Aq => sub { |
| 200 |
\& my $x = $ref\->{ref}\->{_myscore} . $ref\->{ref}\->{_yourscore} ; |
| 201 |
\& }, |
| 202 |
\& \*(Aqdereference\*(Aq => sub { |
| 203 |
\& my $ref = $ref\->{ref}; |
| 204 |
\& my $myscore = $ref\->{_myscore}; |
| 205 |
\& my $yourscore = $ref\->{_yourscore}; |
| 206 |
\& my $x = $myscore . $yourscore; |
| 207 |
\& }, |
| 208 |
\& }); |
| 209 |
.Ve |
| 210 |
.PP |
| 211 |
It's essential to run any timing measurements a sufficient number of times so |
| 212 |
the numbers settle on a numerical average, otherwise each run will naturally |
| 213 |
fluctuate due to variations in the environment, to reduce the effect of |
| 214 |
contention for \f(CW\*(C`CPU\*(C'\fR resources and network bandwidth for instance. Running |
| 215 |
the above code for one million iterations, we can take a look at the report |
| 216 |
output by the \f(CW\*(C`Benchmark\*(C'\fR module, to see which approach is the most effective. |
| 217 |
.PP |
| 218 |
.Vb 1 |
| 219 |
\& $> perl dereference |
| 220 |
\& |
| 221 |
\& Benchmark: timing 1000000 iterations of dereference, direct... |
| 222 |
\& dereference: 2 wallclock secs ( 1.59 usr + 0.00 sys = 1.59 CPU) @ 628930.82/s (n=1000000) |
| 223 |
\& direct: 1 wallclock secs ( 1.20 usr + 0.00 sys = 1.20 CPU) @ 833333.33/s (n=1000000) |
| 224 |
.Ve |
| 225 |
.PP |
| 226 |
The difference is clear to see and the dereferencing approach is slower. While |
| 227 |
it managed to execute an average of 628,930 times a second during our test, the |
| 228 |
direct approach managed to run an additional 204,403 times, unfortunately. |
| 229 |
Unfortunately, because there are many examples of code written using the |
| 230 |
multiple layer direct variable access, and it's usually horrible. It is, |
| 231 |
however, minusculy faster. The question remains whether the minute gain is |
| 232 |
actually worth the eyestrain, or the loss of maintainability. |
| 233 |
.SS "Search and replace or tr" |
| 234 |
.IX Subsection "Search and replace or tr" |
| 235 |
If we have a string which needs to be modified, while a regex will almost |
| 236 |
always be much more flexible, \f(CW\*(C`tr\*(C'\fR, an oft underused tool, can still be a |
| 237 |
useful. One scenario might be replace all vowels with another character. The |
| 238 |
regex solution might look like this: |
| 239 |
.PP |
| 240 |
.Vb 1 |
| 241 |
\& $str =~ s/[aeiou]/x/g |
| 242 |
.Ve |
| 243 |
.PP |
| 244 |
The \f(CW\*(C`tr\*(C'\fR alternative might look like this: |
| 245 |
.PP |
| 246 |
.Vb 1 |
| 247 |
\& $str =~ tr/aeiou/xxxxx/ |
| 248 |
.Ve |
| 249 |
.PP |
| 250 |
We can put that into a test file which we can run to check which approach is |
| 251 |
the fastest, using a global \f(CW$STR\fR variable to assign to the \f(CW\*(C`my $str\*(C'\fR |
| 252 |
variable so as to avoid perl trying to optimize any of the work away by |
| 253 |
noticing it's assigned only the once. |
| 254 |
.PP |
| 255 |
# regex-transliterate |
| 256 |
.PP |
| 257 |
.Vb 1 |
| 258 |
\& #!/usr/bin/perl |
| 259 |
\& |
| 260 |
\& use v5.36; |
| 261 |
\& |
| 262 |
\& use Benchmark; |
| 263 |
\& |
| 264 |
\& my $STR = "$$\-this and that"; |
| 265 |
\& |
| 266 |
\& timethese( 1000000, { |
| 267 |
\& \*(Aqsr\*(Aq => sub { my $str = $STR; $str =~ s/[aeiou]/x/g; return $str; }, |
| 268 |
\& \*(Aqtr\*(Aq => sub { my $str = $STR; $str =~ tr/aeiou/xxxxx/; return $str; }, |
| 269 |
\& }); |
| 270 |
.Ve |
| 271 |
.PP |
| 272 |
Running the code gives us our results: |
| 273 |
.PP |
| 274 |
.Vb 1 |
| 275 |
\& $> perl regex\-transliterate |
| 276 |
\& |
| 277 |
\& Benchmark: timing 1000000 iterations of sr, tr... |
| 278 |
\& sr: 2 wallclock secs ( 1.19 usr + 0.00 sys = 1.19 CPU) @ 840336.13/s (n=1000000) |
| 279 |
\& tr: 0 wallclock secs ( 0.49 usr + 0.00 sys = 0.49 CPU) @ 2040816.33/s (n=1000000) |
| 280 |
.Ve |
| 281 |
.PP |
| 282 |
The \f(CW\*(C`tr\*(C'\fR version is a clear winner. One solution is flexible, the other is |
| 283 |
fast \- and it's appropriately the programmer's choice which to use. |
| 284 |
.PP |
| 285 |
Check the \f(CW\*(C`Benchmark\*(C'\fR docs for further useful techniques. |
| 286 |
.SH "PROFILING TOOLS" |
| 287 |
.IX Header "PROFILING TOOLS" |
| 288 |
A slightly larger piece of code will provide something on which a profiler can |
| 289 |
produce more extensive reporting statistics. This example uses the simplistic |
| 290 |
\&\f(CW\*(C`wordmatch\*(C'\fR program which parses a given input file and spews out a short |
| 291 |
report on the contents. |
| 292 |
.PP |
| 293 |
# wordmatch |
| 294 |
.PP |
| 295 |
.Vb 1 |
| 296 |
\& #!/usr/bin/perl |
| 297 |
\& |
| 298 |
\& use v5.36; |
| 299 |
\& |
| 300 |
\& =head1 NAME |
| 301 |
\& |
| 302 |
\& filewords \- word analysis of input file |
| 303 |
\& |
| 304 |
\& =head1 SYNOPSIS |
| 305 |
\& |
| 306 |
\& filewords \-f inputfilename [\-d] |
| 307 |
\& |
| 308 |
\& =head1 DESCRIPTION |
| 309 |
\& |
| 310 |
\& This program parses the given filename, specified with C<\-f>, and |
| 311 |
\& displays a simple analysis of the words found therein. Use the C<\-d> |
| 312 |
\& switch to enable debugging messages. |
| 313 |
\& |
| 314 |
\& =cut |
| 315 |
\& |
| 316 |
\& use FileHandle; |
| 317 |
\& use Getopt::Long; |
| 318 |
\& |
| 319 |
\& my $debug = 0; |
| 320 |
\& my $file = \*(Aq\*(Aq; |
| 321 |
\& |
| 322 |
\& my $result = GetOptions ( |
| 323 |
\& \*(Aqdebug\*(Aq => \e$debug, |
| 324 |
\& \*(Aqfile=s\*(Aq => \e$file, |
| 325 |
\& ); |
| 326 |
\& die("invalid args") unless $result; |
| 327 |
\& |
| 328 |
\& unless ( \-f $file ) { |
| 329 |
\& die("Usage: $0 \-f filename [\-d]"); |
| 330 |
\& } |
| 331 |
\& my $FH = FileHandle\->new("< $file") |
| 332 |
\& or die("unable to open file($file): $!"); |
| 333 |
\& |
| 334 |
\& my $i_LINES = 0; |
| 335 |
\& my $i_WORDS = 0; |
| 336 |
\& my %count = (); |
| 337 |
\& |
| 338 |
\& my @lines = <$FH>; |
| 339 |
\& foreach my $line ( @lines ) { |
| 340 |
\& $i_LINES++; |
| 341 |
\& $line =~ s/\en//; |
| 342 |
\& my @words = split(/ +/, $line); |
| 343 |
\& my $i_words = scalar(@words); |
| 344 |
\& $i_WORDS = $i_WORDS + $i_words; |
| 345 |
\& debug("line: $i_LINES supplying $i_words words: @words"); |
| 346 |
\& my $i_word = 0; |
| 347 |
\& foreach my $word ( @words ) { |
| 348 |
\& $i_word++; |
| 349 |
\& $count{$i_LINES}{spec} += matches($i_word, $word, |
| 350 |
\& \*(Aq[^a\-zA\-Z0\-9]\*(Aq); |
| 351 |
\& $count{$i_LINES}{only} += matches($i_word, $word, |
| 352 |
\& \*(Aq^[^a\-zA\-Z0\-9]+$\*(Aq); |
| 353 |
\& $count{$i_LINES}{cons} += matches($i_word, $word, |
| 354 |
\& \*(Aq^[(?i:bcdfghjklmnpqrstvwxyz)]+$\*(Aq); |
| 355 |
\& $count{$i_LINES}{vows} += matches($i_word, $word, |
| 356 |
\& \*(Aq^[(?i:aeiou)]+$\*(Aq); |
| 357 |
\& $count{$i_LINES}{caps} += matches($i_word, $word, |
| 358 |
\& \*(Aq^[(A\-Z)]+$\*(Aq); |
| 359 |
\& } |
| 360 |
\& } |
| 361 |
\& |
| 362 |
\& print report( %count ); |
| 363 |
\& |
| 364 |
\& sub matches { |
| 365 |
\& my $i_wd = shift; |
| 366 |
\& my $word = shift; |
| 367 |
\& my $regex = shift; |
| 368 |
\& my $has = 0; |
| 369 |
\& |
| 370 |
\& if ( $word =~ /($regex)/ ) { |
| 371 |
\& $has++ if $1; |
| 372 |
\& } |
| 373 |
\& |
| 374 |
\& debug( "word: $i_wd " |
| 375 |
\& . ($has ? \*(Aqmatches\*(Aq : \*(Aqdoes not match\*(Aq) |
| 376 |
\& . " chars: /$regex/"); |
| 377 |
\& |
| 378 |
\& return $has; |
| 379 |
\& } |
| 380 |
\& |
| 381 |
\& sub report { |
| 382 |
\& my %report = @_; |
| 383 |
\& my %rep; |
| 384 |
\& |
| 385 |
\& foreach my $line ( keys %report ) { |
| 386 |
\& foreach my $key ( keys $report{$line}\->%* ) { |
| 387 |
\& $rep{$key} += $report{$line}{$key}; |
| 388 |
\& } |
| 389 |
\& } |
| 390 |
\& |
| 391 |
\& my $report = qq| |
| 392 |
\& $0 report for $file: |
| 393 |
\& lines in file: $i_LINES |
| 394 |
\& words in file: $i_WORDS |
| 395 |
\& words with special (non\-word) characters: $i_spec |
| 396 |
\& words with only special (non\-word) characters: $i_only |
| 397 |
\& words with only consonants: $i_cons |
| 398 |
\& words with only capital letters: $i_caps |
| 399 |
\& words with only vowels: $i_vows |
| 400 |
\& |; |
| 401 |
\& |
| 402 |
\& return $report; |
| 403 |
\& } |
| 404 |
\& |
| 405 |
\& sub debug { |
| 406 |
\& my $message = shift; |
| 407 |
\& |
| 408 |
\& if ( $debug ) { |
| 409 |
\& print STDERR "DBG: $message\en"; |
| 410 |
\& } |
| 411 |
\& } |
| 412 |
\& |
| 413 |
\& exit 0; |
| 414 |
.Ve |
| 415 |
.SS Devel::DProf |
| 416 |
.IX Subsection "Devel::DProf" |
| 417 |
This venerable module has been the de-facto standard for Perl code profiling |
| 418 |
for more than a decade, but has been replaced by a number of other modules |
| 419 |
which have brought us back to the 21st century. Although you're recommended to |
| 420 |
evaluate your tool from the several mentioned here and from the CPAN list at |
| 421 |
the base of this document, (and currently Devel::NYTProf seems to be the |
| 422 |
weapon of choice \- see below), we'll take a quick look at the output from |
| 423 |
Devel::DProf first, to set a baseline for Perl profiling tools. Run the |
| 424 |
above program under the control of \f(CW\*(C`Devel::DProf\*(C'\fR by using the \f(CW\*(C`\-d\*(C'\fR switch on |
| 425 |
the command-line. |
| 426 |
.PP |
| 427 |
.Vb 1 |
| 428 |
\& $> perl \-d:DProf wordmatch \-f perl5db.pl |
| 429 |
\& |
| 430 |
\& <...multiple lines snipped...> |
| 431 |
\& |
| 432 |
\& wordmatch report for perl5db.pl: |
| 433 |
\& lines in file: 9428 |
| 434 |
\& words in file: 50243 |
| 435 |
\& words with special (non\-word) characters: 20480 |
| 436 |
\& words with only special (non\-word) characters: 7790 |
| 437 |
\& words with only consonants: 4801 |
| 438 |
\& words with only capital letters: 1316 |
| 439 |
\& words with only vowels: 1701 |
| 440 |
.Ve |
| 441 |
.PP |
| 442 |
\&\f(CW\*(C`Devel::DProf\*(C'\fR produces a special file, called \fItmon.out\fR by default, and |
| 443 |
this file is read by the \f(CW\*(C`dprofpp\*(C'\fR program, which is already installed as part |
| 444 |
of the \f(CW\*(C`Devel::DProf\*(C'\fR distribution. If you call \f(CW\*(C`dprofpp\*(C'\fR with no options, |
| 445 |
it will read the \fItmon.out\fR file in the current directory and produce a human |
| 446 |
readable statistics report of the run of your program. Note that this may take |
| 447 |
a little time. |
| 448 |
.PP |
| 449 |
.Vb 1 |
| 450 |
\& $> dprofpp |
| 451 |
\& |
| 452 |
\& Total Elapsed Time = 2.951677 Seconds |
| 453 |
\& User+System Time = 2.871677 Seconds |
| 454 |
\& Exclusive Times |
| 455 |
\& %Time ExclSec CumulS #Calls sec/call Csec/c Name |
| 456 |
\& 102. 2.945 3.003 251215 0.0000 0.0000 main::matches |
| 457 |
\& 2.40 0.069 0.069 260643 0.0000 0.0000 main::debug |
| 458 |
\& 1.74 0.050 0.050 1 0.0500 0.0500 main::report |
| 459 |
\& 1.04 0.030 0.049 4 0.0075 0.0123 main::BEGIN |
| 460 |
\& 0.35 0.010 0.010 3 0.0033 0.0033 Exporter::as_heavy |
| 461 |
\& 0.35 0.010 0.010 7 0.0014 0.0014 IO::File::BEGIN |
| 462 |
\& 0.00 \- \-0.000 1 \- \- Getopt::Long::FindOption |
| 463 |
\& 0.00 \- \-0.000 1 \- \- Symbol::BEGIN |
| 464 |
\& 0.00 \- \-0.000 1 \- \- Fcntl::BEGIN |
| 465 |
\& 0.00 \- \-0.000 1 \- \- Fcntl::bootstrap |
| 466 |
\& 0.00 \- \-0.000 1 \- \- warnings::BEGIN |
| 467 |
\& 0.00 \- \-0.000 1 \- \- IO::bootstrap |
| 468 |
\& 0.00 \- \-0.000 1 \- \- Getopt::Long::ConfigDefaults |
| 469 |
\& 0.00 \- \-0.000 1 \- \- Getopt::Long::Configure |
| 470 |
\& 0.00 \- \-0.000 1 \- \- Symbol::gensym |
| 471 |
.Ve |
| 472 |
.PP |
| 473 |
\&\f(CW\*(C`dprofpp\*(C'\fR will produce some quite detailed reporting on the activity of the |
| 474 |
\&\f(CW\*(C`wordmatch\*(C'\fR program. The wallclock, user and system, times are at the top of |
| 475 |
the analysis, and after this are the main columns defining which define the |
| 476 |
report. Check the \f(CW\*(C`dprofpp\*(C'\fR docs for details of the many options it supports. |
| 477 |
.PP |
| 478 |
See also \f(CW\*(C`Apache::DProf\*(C'\fR which hooks \f(CW\*(C`Devel::DProf\*(C'\fR into \f(CW\*(C`mod_perl\*(C'\fR. |
| 479 |
.SS Devel::Profiler |
| 480 |
.IX Subsection "Devel::Profiler" |
| 481 |
Let's take a look at the same program using a different profiler: |
| 482 |
\&\f(CW\*(C`Devel::Profiler\*(C'\fR, a drop-in Perl-only replacement for \f(CW\*(C`Devel::DProf\*(C'\fR. The |
| 483 |
usage is very slightly different in that instead of using the special \f(CW\*(C`\-d:\*(C'\fR |
| 484 |
flag, you pull \f(CW\*(C`Devel::Profiler\*(C'\fR in directly as a module using \f(CW\*(C`\-M\*(C'\fR. |
| 485 |
.PP |
| 486 |
.Vb 1 |
| 487 |
\& $> perl \-MDevel::Profiler wordmatch \-f perl5db.pl |
| 488 |
\& |
| 489 |
\& <...multiple lines snipped...> |
| 490 |
\& |
| 491 |
\& wordmatch report for perl5db.pl: |
| 492 |
\& lines in file: 9428 |
| 493 |
\& words in file: 50243 |
| 494 |
\& words with special (non\-word) characters: 20480 |
| 495 |
\& words with only special (non\-word) characters: 7790 |
| 496 |
\& words with only consonants: 4801 |
| 497 |
\& words with only capital letters: 1316 |
| 498 |
\& words with only vowels: 1701 |
| 499 |
.Ve |
| 500 |
.PP |
| 501 |
\&\f(CW\*(C`Devel::Profiler\*(C'\fR generates a tmon.out file which is compatible with the |
| 502 |
\&\f(CW\*(C`dprofpp\*(C'\fR program, thus saving the construction of a dedicated statistics |
| 503 |
reader program. \f(CW\*(C`dprofpp\*(C'\fR usage is therefore identical to the above example. |
| 504 |
.PP |
| 505 |
.Vb 1 |
| 506 |
\& $> dprofpp |
| 507 |
\& |
| 508 |
\& Total Elapsed Time = 20.984 Seconds |
| 509 |
\& User+System Time = 19.981 Seconds |
| 510 |
\& Exclusive Times |
| 511 |
\& %Time ExclSec CumulS #Calls sec/call Csec/c Name |
| 512 |
\& 49.0 9.792 14.509 251215 0.0000 0.0001 main::matches |
| 513 |
\& 24.4 4.887 4.887 260643 0.0000 0.0000 main::debug |
| 514 |
\& 0.25 0.049 0.049 1 0.0490 0.0490 main::report |
| 515 |
\& 0.00 0.000 0.000 1 0.0000 0.0000 Getopt::Long::GetOptions |
| 516 |
\& 0.00 0.000 0.000 2 0.0000 0.0000 Getopt::Long::ParseOptionSpec |
| 517 |
\& 0.00 0.000 0.000 1 0.0000 0.0000 Getopt::Long::FindOption |
| 518 |
\& 0.00 0.000 0.000 1 0.0000 0.0000 IO::File::new |
| 519 |
\& 0.00 0.000 0.000 1 0.0000 0.0000 IO::Handle::new |
| 520 |
\& 0.00 0.000 0.000 1 0.0000 0.0000 Symbol::gensym |
| 521 |
\& 0.00 0.000 0.000 1 0.0000 0.0000 IO::File::open |
| 522 |
.Ve |
| 523 |
.PP |
| 524 |
Interestingly we get slightly different results, which is mostly because the |
| 525 |
algorithm which generates the report is different, even though the output file |
| 526 |
format was allegedly identical. The elapsed, user and system times are clearly |
| 527 |
showing the time it took for \f(CW\*(C`Devel::Profiler\*(C'\fR to execute its own run, but |
| 528 |
the column listings feel more accurate somehow than the ones we had earlier |
| 529 |
from \f(CW\*(C`Devel::DProf\*(C'\fR. The 102% figure has disappeared, for example. This is |
| 530 |
where we have to use the tools at our disposal, and recognise their pros and |
| 531 |
cons, before using them. Interestingly, the numbers of calls for each |
| 532 |
subroutine are identical in the two reports, it's the percentages which differ. |
| 533 |
As the author of \f(CW\*(C`Devel::Profiler\*(C'\fR writes: |
| 534 |
.PP |
| 535 |
.Vb 6 |
| 536 |
\& ...running HTML::Template\*(Aqs test suite under Devel::DProf shows |
| 537 |
\& output() taking NO time but Devel::Profiler shows around 10% of the |
| 538 |
\& time is in output(). I don\*(Aqt know which to trust but my gut tells me |
| 539 |
\& something is wrong with Devel::DProf. HTML::Template::output() is a |
| 540 |
\& big routine that\*(Aqs called for every test. Either way, something needs |
| 541 |
\& fixing. |
| 542 |
.Ve |
| 543 |
.PP |
| 544 |
YMMV. |
| 545 |
.PP |
| 546 |
See also \f(CW\*(C`Devel::Apache::Profiler\*(C'\fR which hooks \f(CW\*(C`Devel::Profiler\*(C'\fR |
| 547 |
into \f(CW\*(C`mod_perl\*(C'\fR. |
| 548 |
.SS Devel::SmallProf |
| 549 |
.IX Subsection "Devel::SmallProf" |
| 550 |
The \f(CW\*(C`Devel::SmallProf\*(C'\fR profiler examines the runtime of your Perl program and |
| 551 |
produces a line-by-line listing to show how many times each line was called, |
| 552 |
and how long each line took to execute. It is called by supplying the familiar |
| 553 |
\&\f(CW\*(C`\-d\*(C'\fR flag to Perl at runtime. |
| 554 |
.PP |
| 555 |
.Vb 1 |
| 556 |
\& $> perl \-d:SmallProf wordmatch \-f perl5db.pl |
| 557 |
\& |
| 558 |
\& <...multiple lines snipped...> |
| 559 |
\& |
| 560 |
\& wordmatch report for perl5db.pl: |
| 561 |
\& lines in file: 9428 |
| 562 |
\& words in file: 50243 |
| 563 |
\& words with special (non\-word) characters: 20480 |
| 564 |
\& words with only special (non\-word) characters: 7790 |
| 565 |
\& words with only consonants: 4801 |
| 566 |
\& words with only capital letters: 1316 |
| 567 |
\& words with only vowels: 1701 |
| 568 |
.Ve |
| 569 |
.PP |
| 570 |
\&\f(CW\*(C`Devel::SmallProf\*(C'\fR writes its output into a file called \fIsmallprof.out\fR, by |
| 571 |
default. The format of the file looks like this: |
| 572 |
.PP |
| 573 |
.Vb 1 |
| 574 |
\& <num> <time> <ctime> <line>:<text> |
| 575 |
.Ve |
| 576 |
.PP |
| 577 |
When the program has terminated, the output may be examined and sorted using |
| 578 |
any standard text filtering utilities. Something like the following may be |
| 579 |
sufficient: |
| 580 |
.PP |
| 581 |
.Vb 1 |
| 582 |
\& $> cat smallprof.out | grep \ed*: | sort \-k3 | tac | head \-n20 |
| 583 |
\& |
| 584 |
\& 251215 1.65674 7.68000 75: if ( $word =~ /($regex)/ ) { |
| 585 |
\& 251215 0.03264 4.40000 79: debug("word: $i_wd ".($has ? \*(Aqmatches\*(Aq : |
| 586 |
\& 251215 0.02693 4.10000 81: return $has; |
| 587 |
\& 260643 0.02841 4.07000 128: if ( $debug ) { |
| 588 |
\& 260643 0.02601 4.04000 126: my $message = shift; |
| 589 |
\& 251215 0.02641 3.91000 73: my $has = 0; |
| 590 |
\& 251215 0.03311 3.71000 70: my $i_wd = shift; |
| 591 |
\& 251215 0.02699 3.69000 72: my $regex = shift; |
| 592 |
\& 251215 0.02766 3.68000 71: my $word = shift; |
| 593 |
\& 50243 0.59726 1.00000 59: $count{$i_LINES}{cons} = |
| 594 |
\& 50243 0.48175 0.92000 61: $count{$i_LINES}{spec} = |
| 595 |
\& 50243 0.00644 0.89000 56: my $i_cons = matches($i_word, $word, |
| 596 |
\& 50243 0.48837 0.88000 63: $count{$i_LINES}{caps} = |
| 597 |
\& 50243 0.00516 0.88000 58: my $i_caps = matches($i_word, $word, \*(Aq^[(A\- |
| 598 |
\& 50243 0.00631 0.81000 54: my $i_spec = matches($i_word, $word, \*(Aq[^a\- |
| 599 |
\& 50243 0.00496 0.80000 57: my $i_vows = matches($i_word, $word, |
| 600 |
\& 50243 0.00688 0.80000 53: $i_word++; |
| 601 |
\& 50243 0.48469 0.79000 62: $count{$i_LINES}{only} = |
| 602 |
\& 50243 0.48928 0.77000 60: $count{$i_LINES}{vows} = |
| 603 |
\& 50243 0.00683 0.75000 55: my $i_only = matches($i_word, $word, \*(Aq^[^a\- |
| 604 |
.Ve |
| 605 |
.PP |
| 606 |
You can immediately see a slightly different focus to the subroutine profiling |
| 607 |
modules, and we start to see exactly which line of code is taking the most |
| 608 |
time. That regex line is looking a bit suspicious, for example. Remember that |
| 609 |
these tools are supposed to be used together, there is no single best way to |
| 610 |
profile your code, you need to use the best tools for the job. |
| 611 |
.PP |
| 612 |
See also \f(CW\*(C`Apache::SmallProf\*(C'\fR which hooks \f(CW\*(C`Devel::SmallProf\*(C'\fR into |
| 613 |
\&\f(CW\*(C`mod_perl\*(C'\fR. |
| 614 |
.SS Devel::FastProf |
| 615 |
.IX Subsection "Devel::FastProf" |
| 616 |
\&\f(CW\*(C`Devel::FastProf\*(C'\fR is another Perl line profiler. This was written with a view |
| 617 |
to getting a faster line profiler, than is possible with for example |
| 618 |
\&\f(CW\*(C`Devel::SmallProf\*(C'\fR, because it's written in \f(CW\*(C`C\*(C'\fR. To use \f(CW\*(C`Devel::FastProf\*(C'\fR, |
| 619 |
supply the \f(CW\*(C`\-d\*(C'\fR argument to Perl: |
| 620 |
.PP |
| 621 |
.Vb 1 |
| 622 |
\& $> perl \-d:FastProf wordmatch \-f perl5db.pl |
| 623 |
\& |
| 624 |
\& <...multiple lines snipped...> |
| 625 |
\& |
| 626 |
\& wordmatch report for perl5db.pl: |
| 627 |
\& lines in file: 9428 |
| 628 |
\& words in file: 50243 |
| 629 |
\& words with special (non\-word) characters: 20480 |
| 630 |
\& words with only special (non\-word) characters: 7790 |
| 631 |
\& words with only consonants: 4801 |
| 632 |
\& words with only capital letters: 1316 |
| 633 |
\& words with only vowels: 1701 |
| 634 |
.Ve |
| 635 |
.PP |
| 636 |
\&\f(CW\*(C`Devel::FastProf\*(C'\fR writes statistics to the file \fIfastprof.out\fR in the current |
| 637 |
directory. The output file, which can be specified, can be interpreted by using |
| 638 |
the \f(CW\*(C`fprofpp\*(C'\fR command-line program. |
| 639 |
.PP |
| 640 |
.Vb 1 |
| 641 |
\& $> fprofpp | head \-n20 |
| 642 |
\& |
| 643 |
\& # fprofpp output format is: |
| 644 |
\& # filename:line time count: source |
| 645 |
\& wordmatch:75 3.93338 251215: if ( $word =~ /($regex)/ ) { |
| 646 |
\& wordmatch:79 1.77774 251215: debug("word: $i_wd ".($has ? \*(Aqmatches\*(Aq : \*(Aqdoes not match\*(Aq)." chars: /$regex/"); |
| 647 |
\& wordmatch:81 1.47604 251215: return $has; |
| 648 |
\& wordmatch:126 1.43441 260643: my $message = shift; |
| 649 |
\& wordmatch:128 1.42156 260643: if ( $debug ) { |
| 650 |
\& wordmatch:70 1.36824 251215: my $i_wd = shift; |
| 651 |
\& wordmatch:71 1.36739 251215: my $word = shift; |
| 652 |
\& wordmatch:72 1.35939 251215: my $regex = shift; |
| 653 |
.Ve |
| 654 |
.PP |
| 655 |
Straightaway we can see that the number of times each line has been called is |
| 656 |
identical to the \f(CW\*(C`Devel::SmallProf\*(C'\fR output, and the sequence is only very |
| 657 |
slightly different based on the ordering of the amount of time each line took |
| 658 |
to execute, \f(CW\*(C`if ( $debug ) { \*(C'\fR and \f(CW\*(C`my $message = shift;\*(C'\fR, for example. The |
| 659 |
differences in the actual times recorded might be in the algorithm used |
| 660 |
internally, or it could be due to system resource limitations or contention. |
| 661 |
.PP |
| 662 |
See also the DBIx::Profile which will profile database queries running |
| 663 |
under the \f(CW\*(C`DBIx::*\*(C'\fR namespace. |
| 664 |
.SS Devel::NYTProf |
| 665 |
.IX Subsection "Devel::NYTProf" |
| 666 |
\&\f(CW\*(C`Devel::NYTProf\*(C'\fR is the \fBnext generation\fR of Perl code profiler, fixing many |
| 667 |
shortcomings in other tools and implementing many cool features. First of all it |
| 668 |
can be used as either a \fIline\fR profiler, a \fIblock\fR or a \fIsubroutine\fR |
| 669 |
profiler, all at once. It can also use sub-microsecond (100ns) resolution on |
| 670 |
systems which provide \f(CWclock_gettime()\fR. It can be started and stopped even |
| 671 |
by the program being profiled. It's a one-line entry to profile \f(CW\*(C`mod_perl\*(C'\fR |
| 672 |
applications. It's written in \f(CW\*(C`c\*(C'\fR and is probably the fastest profiler |
| 673 |
available for Perl. The list of coolness just goes on. Enough of that, let's |
| 674 |
see how to it works \- just use the familiar \f(CW\*(C`\-d\*(C'\fR switch to plug it in and run |
| 675 |
the code. |
| 676 |
.PP |
| 677 |
.Vb 1 |
| 678 |
\& $> perl \-d:NYTProf wordmatch \-f perl5db.pl |
| 679 |
\& |
| 680 |
\& wordmatch report for perl5db.pl: |
| 681 |
\& lines in file: 9427 |
| 682 |
\& words in file: 50243 |
| 683 |
\& words with special (non\-word) characters: 20480 |
| 684 |
\& words with only special (non\-word) characters: 7790 |
| 685 |
\& words with only consonants: 4801 |
| 686 |
\& words with only capital letters: 1316 |
| 687 |
\& words with only vowels: 1701 |
| 688 |
.Ve |
| 689 |
.PP |
| 690 |
\&\f(CW\*(C`NYTProf\*(C'\fR will generate a report database into the file \fInytprof.out\fR by |
| 691 |
default. Human readable reports can be generated from here by using the |
| 692 |
supplied \f(CW\*(C`nytprofhtml\*(C'\fR (HTML output) and \f(CW\*(C`nytprofcsv\*(C'\fR (CSV output) programs. |
| 693 |
We've used the Unix system \f(CW\*(C`html2text\*(C'\fR utility to convert the |
| 694 |
\&\fInytprof/index.html\fR file for convenience here. |
| 695 |
.PP |
| 696 |
.Vb 1 |
| 697 |
\& $> html2text nytprof/index.html |
| 698 |
\& |
| 699 |
\& Performance Profile Index |
| 700 |
\& For wordmatch |
| 701 |
\& Run on Fri Sep 26 13:46:39 2008 |
| 702 |
\& Reported on Fri Sep 26 13:47:23 2008 |
| 703 |
\& |
| 704 |
\& Top 15 Subroutines \-\- ordered by exclusive time |
| 705 |
\& |Calls |P |F |Inclusive|Exclusive|Subroutine | |
| 706 |
\& | | | |Time |Time | | |
| 707 |
\& |251215|5 |1 |13.09263 |10.47692 |main:: |matches | |
| 708 |
\& |260642|2 |1 |2.71199 |2.71199 |main:: |debug | |
| 709 |
\& |1 |1 |1 |0.21404 |0.21404 |main:: |report | |
| 710 |
\& |2 |2 |2 |0.00511 |0.00511 |XSLoader:: |load (xsub) | |
| 711 |
\& |14 |14|7 |0.00304 |0.00298 |Exporter:: |import | |
| 712 |
\& |3 |1 |1 |0.00265 |0.00254 |Exporter:: |as_heavy | |
| 713 |
\& |10 |10|4 |0.00140 |0.00140 |vars:: |import | |
| 714 |
\& |13 |13|1 |0.00129 |0.00109 |constant:: |import | |
| 715 |
\& |1 |1 |1 |0.00360 |0.00096 |FileHandle:: |import | |
| 716 |
\& |3 |3 |3 |0.00086 |0.00074 |warnings::register::|import | |
| 717 |
\& |9 |3 |1 |0.00036 |0.00036 |strict:: |bits | |
| 718 |
\& |13 |13|13|0.00032 |0.00029 |strict:: |import | |
| 719 |
\& |2 |2 |2 |0.00020 |0.00020 |warnings:: |import | |
| 720 |
\& |2 |1 |1 |0.00020 |0.00020 |Getopt::Long:: |ParseOptionSpec| |
| 721 |
\& |7 |7 |6 |0.00043 |0.00020 |strict:: |unimport | |
| 722 |
\& |
| 723 |
\& For more information see the full list of 189 subroutines. |
| 724 |
.Ve |
| 725 |
.PP |
| 726 |
The first part of the report already shows the critical information regarding |
| 727 |
which subroutines are using the most time. The next gives some statistics |
| 728 |
about the source files profiled. |
| 729 |
.PP |
| 730 |
.Vb 10 |
| 731 |
\& Source Code Files \-\- ordered by exclusive time then name |
| 732 |
\& |Stmts |Exclusive|Avg. |Reports |Source File | |
| 733 |
\& | |Time | | | | |
| 734 |
\& |2699761|15.66654 |6e\-06 |line . block . sub|wordmatch | |
| 735 |
\& |35 |0.02187 |0.00062|line . block . sub|IO/Handle.pm | |
| 736 |
\& |274 |0.01525 |0.00006|line . block . sub|Getopt/Long.pm | |
| 737 |
\& |20 |0.00585 |0.00029|line . block . sub|Fcntl.pm | |
| 738 |
\& |128 |0.00340 |0.00003|line . block . sub|Exporter/Heavy.pm | |
| 739 |
\& |42 |0.00332 |0.00008|line . block . sub|IO/File.pm | |
| 740 |
\& |261 |0.00308 |0.00001|line . block . sub|Exporter.pm | |
| 741 |
\& |323 |0.00248 |8e\-06 |line . block . sub|constant.pm | |
| 742 |
\& |12 |0.00246 |0.00021|line . block . sub|File/Spec/Unix.pm | |
| 743 |
\& |191 |0.00240 |0.00001|line . block . sub|vars.pm | |
| 744 |
\& |77 |0.00201 |0.00003|line . block . sub|FileHandle.pm | |
| 745 |
\& |12 |0.00198 |0.00016|line . block . sub|Carp.pm | |
| 746 |
\& |14 |0.00175 |0.00013|line . block . sub|Symbol.pm | |
| 747 |
\& |15 |0.00130 |0.00009|line . block . sub|IO.pm | |
| 748 |
\& |22 |0.00120 |0.00005|line . block . sub|IO/Seekable.pm | |
| 749 |
\& |198 |0.00085 |4e\-06 |line . block . sub|warnings/register.pm| |
| 750 |
\& |114 |0.00080 |7e\-06 |line . block . sub|strict.pm | |
| 751 |
\& |47 |0.00068 |0.00001|line . block . sub|warnings.pm | |
| 752 |
\& |27 |0.00054 |0.00002|line . block . sub|overload.pm | |
| 753 |
\& |9 |0.00047 |0.00005|line . block . sub|SelectSaver.pm | |
| 754 |
\& |13 |0.00045 |0.00003|line . block . sub|File/Spec.pm | |
| 755 |
\& |2701595|15.73869 | |Total | |
| 756 |
\& |128647 |0.74946 | |Average | |
| 757 |
\& | |0.00201 |0.00003|Median | |
| 758 |
\& | |0.00121 |0.00003|Deviation | |
| 759 |
\& |
| 760 |
\& Report produced by the NYTProf 2.03 Perl profiler, developed by Tim Bunce and |
| 761 |
\& Adam Kaplan. |
| 762 |
.Ve |
| 763 |
.PP |
| 764 |
At this point, if you're using the \fIhtml\fR report, you can click through the |
| 765 |
various links to bore down into each subroutine and each line of code. Because |
| 766 |
we're using the text reporting here, and there's a whole directory full of |
| 767 |
reports built for each source file, we'll just display a part of the |
| 768 |
corresponding \fIwordmatch\-line.html\fR file, sufficient to give an idea of the |
| 769 |
sort of output you can expect from this cool tool. |
| 770 |
.PP |
| 771 |
.Vb 1 |
| 772 |
\& $> html2text nytprof/wordmatch\-line.html |
| 773 |
\& |
| 774 |
\& Performance Profile \-\- \-block view\-.\-line view\-.\-sub view\- |
| 775 |
\& For wordmatch |
| 776 |
\& Run on Fri Sep 26 13:46:39 2008 |
| 777 |
\& Reported on Fri Sep 26 13:47:22 2008 |
| 778 |
\& |
| 779 |
\& File wordmatch |
| 780 |
\& |
| 781 |
\& Subroutines \-\- ordered by exclusive time |
| 782 |
\& |Calls |P|F|Inclusive|Exclusive|Subroutine | |
| 783 |
\& | | | |Time |Time | | |
| 784 |
\& |251215|5|1|13.09263 |10.47692 |main::|matches| |
| 785 |
\& |260642|2|1|2.71199 |2.71199 |main::|debug | |
| 786 |
\& |1 |1|1|0.21404 |0.21404 |main::|report | |
| 787 |
\& |0 |0|0|0 |0 |main::|BEGIN | |
| 788 |
\& |
| 789 |
\& |
| 790 |
\& |Line|Stmts.|Exclusive|Avg. |Code | |
| 791 |
\& | | |Time | | | |
| 792 |
\& |1 | | | |#!/usr/bin/perl | |
| 793 |
\& |2 | | | | | |
| 794 |
\& | | | | |use strict; | |
| 795 |
\& |3 |3 |0.00086 |0.00029|# spent 0.00003s making 1 calls to strict:: | |
| 796 |
\& | | | | |import | |
| 797 |
\& | | | | |use warnings; | |
| 798 |
\& |4 |3 |0.01563 |0.00521|# spent 0.00012s making 1 calls to warnings:: | |
| 799 |
\& | | | | |import | |
| 800 |
\& |5 | | | | | |
| 801 |
\& |6 | | | |=head1 NAME | |
| 802 |
\& |7 | | | | | |
| 803 |
\& |8 | | | |filewords \- word analysis of input file | |
| 804 |
\& <...snip...> |
| 805 |
\& |62 |1 |0.00445 |0.00445|print report( %count ); | |
| 806 |
\& | | | | |# spent 0.21404s making 1 calls to main::report| |
| 807 |
\& |63 | | | | | |
| 808 |
\& | | | | |# spent 23.56955s (10.47692+2.61571) within | |
| 809 |
\& | | | | |main::matches which was called 251215 times, | |
| 810 |
\& | | | | |avg 0.00005s/call: # 50243 times | |
| 811 |
\& | | | | |(2.12134+0.51939s) at line 57 of wordmatch, avg| |
| 812 |
\& | | | | |0.00005s/call # 50243 times (2.17735+0.54550s) | |
| 813 |
\& |64 | | | |at line 56 of wordmatch, avg 0.00005s/call # | |
| 814 |
\& | | | | |50243 times (2.10992+0.51797s) at line 58 of | |
| 815 |
\& | | | | |wordmatch, avg 0.00005s/call # 50243 times | |
| 816 |
\& | | | | |(2.12696+0.51598s) at line 55 of wordmatch, avg| |
| 817 |
\& | | | | |0.00005s/call # 50243 times (1.94134+0.51687s) | |
| 818 |
\& | | | | |at line 54 of wordmatch, avg 0.00005s/call | |
| 819 |
\& | | | | |sub matches { | |
| 820 |
\& <...snip...> |
| 821 |
\& |102 | | | | | |
| 822 |
\& | | | | |# spent 2.71199s within main::debug which was | |
| 823 |
\& | | | | |called 260642 times, avg 0.00001s/call: # | |
| 824 |
\& | | | | |251215 times (2.61571+0s) by main::matches at | |
| 825 |
\& |103 | | | |line 74 of wordmatch, avg 0.00001s/call # 9427 | |
| 826 |
\& | | | | |times (0.09628+0s) at line 50 of wordmatch, avg| |
| 827 |
\& | | | | |0.00001s/call | |
| 828 |
\& | | | | |sub debug { | |
| 829 |
\& |104 |260642|0.58496 |2e\-06 |my $message = shift; | |
| 830 |
\& |105 | | | | | |
| 831 |
\& |106 |260642|1.09917 |4e\-06 |if ( $debug ) { | |
| 832 |
\& |107 | | | |print STDERR "DBG: $message\en"; | |
| 833 |
\& |108 | | | |} | |
| 834 |
\& |109 | | | |} | |
| 835 |
\& |110 | | | | | |
| 836 |
\& |111 |1 |0.01501 |0.01501|exit 0; | |
| 837 |
\& |112 | | | | | |
| 838 |
.Ve |
| 839 |
.PP |
| 840 |
Oodles of very useful information in there \- this seems to be the way forward. |
| 841 |
.PP |
| 842 |
See also \f(CW\*(C`Devel::NYTProf::Apache\*(C'\fR which hooks \f(CW\*(C`Devel::NYTProf\*(C'\fR into |
| 843 |
\&\f(CW\*(C`mod_perl\*(C'\fR. |
| 844 |
.SH SORTING |
| 845 |
.IX Header "SORTING" |
| 846 |
Perl modules are not the only tools a performance analyst has at their |
| 847 |
disposal, system tools like \f(CW\*(C`time\*(C'\fR should not be overlooked as the next |
| 848 |
example shows, where we take a quick look at sorting. Many books, theses and |
| 849 |
articles, have been written about efficient sorting algorithms, and this is not |
| 850 |
the place to repeat such work, there's several good sorting modules which |
| 851 |
deserve taking a look at too: \f(CW\*(C`Sort::Maker\*(C'\fR, \f(CW\*(C`Sort::Key\*(C'\fR spring to mind. |
| 852 |
However, it's still possible to make some observations on certain Perl specific |
| 853 |
interpretations on issues relating to sorting data sets and give an example or |
| 854 |
two with regard to how sorting large data volumes can effect performance. |
| 855 |
Firstly, an often overlooked point when sorting large amounts of data, one can |
| 856 |
attempt to reduce the data set to be dealt with and in many cases \f(CWgrep()\fR can |
| 857 |
be quite useful as a simple filter: |
| 858 |
.PP |
| 859 |
.Vb 1 |
| 860 |
\& @data = sort grep { /$filter/ } @incoming |
| 861 |
.Ve |
| 862 |
.PP |
| 863 |
A command such as this can vastly reduce the volume of material to actually |
| 864 |
sort through in the first place, and should not be too lightly disregarded |
| 865 |
purely on the basis of its simplicity. The \f(CW\*(C`KISS\*(C'\fR principle is too often |
| 866 |
overlooked \- the next example uses the simple system \f(CW\*(C`time\*(C'\fR utility to |
| 867 |
demonstrate. Let's take a look at an actual example of sorting the contents of |
| 868 |
a large file, an apache logfile would do. This one has over a quarter of a |
| 869 |
million lines, is 50M in size, and a snippet of it looks like this: |
| 870 |
.PP |
| 871 |
# logfile |
| 872 |
.PP |
| 873 |
.Vb 10 |
| 874 |
\& 188.209\-65\-87.adsl\-dyn.isp.belgacom.be \- \- [08/Feb/2007:12:57:16 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "\-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" |
| 875 |
\& 188.209\-65\-87.adsl\-dyn.isp.belgacom.be \- \- [08/Feb/2007:12:57:16 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "\-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" |
| 876 |
\& 151.56.71.198 \- \- [08/Feb/2007:12:57:41 +0000] "GET /suse\-on\-vaio.html HTTP/1.1" 200 2858 "http://www.linux\-on\-laptops.com/sony.html" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en\-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1" |
| 877 |
\& 151.56.71.198 \- \- [08/Feb/2007:12:57:42 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse\-on\-vaio.html" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en\-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1" |
| 878 |
\& 151.56.71.198 \- \- [08/Feb/2007:12:57:43 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "\-" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en\-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1" |
| 879 |
\& 217.113.68.60 \- \- [08/Feb/2007:13:02:15 +0000] "GET / HTTP/1.1" 304 \- "\-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" |
| 880 |
\& 217.113.68.60 \- \- [08/Feb/2007:13:02:16 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" |
| 881 |
\& debora.to.isac.cnr.it \- \- [08/Feb/2007:13:03:58 +0000] "GET /suse\-on\-vaio.html HTTP/1.1" 200 2858 "http://www.linux\-on\-laptops.com/sony.html" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)" |
| 882 |
\& debora.to.isac.cnr.it \- \- [08/Feb/2007:13:03:58 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse\-on\-vaio.html" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)" |
| 883 |
\& debora.to.isac.cnr.it \- \- [08/Feb/2007:13:03:58 +0000] "GET /favicon.ico HTTP/1.1" 404 209 "\-" "Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.0 (like Gecko)" |
| 884 |
\& 195.24.196.99 \- \- [08/Feb/2007:13:26:48 +0000] "GET / HTTP/1.0" 200 3309 "\-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9" |
| 885 |
\& 195.24.196.99 \- \- [08/Feb/2007:13:26:58 +0000] "GET /data/css HTTP/1.0" 404 206 "http://www.rfi.net/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9" |
| 886 |
\& 195.24.196.99 \- \- [08/Feb/2007:13:26:59 +0000] "GET /favicon.ico HTTP/1.0" 404 209 "\-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9" |
| 887 |
\& crawl1.cosmixcorp.com \- \- [08/Feb/2007:13:27:57 +0000] "GET /robots.txt HTTP/1.0" 200 179 "\-" "voyager/1.0" |
| 888 |
\& crawl1.cosmixcorp.com \- \- [08/Feb/2007:13:28:25 +0000] "GET /links.html HTTP/1.0" 200 3413 "\-" "voyager/1.0" |
| 889 |
\& fhm226.internetdsl.tpnet.pl \- \- [08/Feb/2007:13:37:32 +0000] "GET /suse\-on\-vaio.html HTTP/1.1" 200 2858 "http://www.linux\-on\-laptops.com/sony.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" |
| 890 |
\& fhm226.internetdsl.tpnet.pl \- \- [08/Feb/2007:13:37:34 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net/suse\-on\-vaio.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" |
| 891 |
\& 80.247.140.134 \- \- [08/Feb/2007:13:57:35 +0000] "GET / HTTP/1.1" 200 3309 "\-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)" |
| 892 |
\& 80.247.140.134 \- \- [08/Feb/2007:13:57:37 +0000] "GET /data/css HTTP/1.1" 404 206 "http://www.rfi.net" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)" |
| 893 |
\& pop.compuscan.co.za \- \- [08/Feb/2007:14:10:43 +0000] "GET / HTTP/1.1" 200 3309 "\-" "www.clamav.net" |
| 894 |
\& livebot\-207\-46\-98\-57.search.live.com \- \- [08/Feb/2007:14:12:04 +0000] "GET /robots.txt HTTP/1.0" 200 179 "\-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" |
| 895 |
\& livebot\-207\-46\-98\-57.search.live.com \- \- [08/Feb/2007:14:12:04 +0000] "GET /html/oracle.html HTTP/1.0" 404 214 "\-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" |
| 896 |
\& dslb\-088\-064\-005\-154.pools.arcor\-ip.net \- \- [08/Feb/2007:14:12:15 +0000] "GET / HTTP/1.1" 200 3309 "\-" "www.clamav.net" |
| 897 |
\& 196.201.92.41 \- \- [08/Feb/2007:14:15:01 +0000] "GET / HTTP/1.1" 200 3309 "\-" "MOT\-L7/08.B7.DCR MIB/2.2.1 Profile/MIDP\-2.0 Configuration/CLDC\-1.1" |
| 898 |
.Ve |
| 899 |
.PP |
| 900 |
The specific task here is to sort the 286,525 lines of this file by Response |
| 901 |
Code, Query, Browser, Referring Url, and lastly Date. One solution might be to |
| 902 |
use the following code, which iterates over the files given on the |
| 903 |
command-line. |
| 904 |
.PP |
| 905 |
# sort-apache-log |
| 906 |
.PP |
| 907 |
.Vb 1 |
| 908 |
\& #!/usr/bin/perl \-n |
| 909 |
\& |
| 910 |
\& use v5.36; |
| 911 |
\& |
| 912 |
\& my @data; |
| 913 |
\& |
| 914 |
\& LINE: |
| 915 |
\& while ( <> ) { |
| 916 |
\& my $line = $_; |
| 917 |
\& if ( |
| 918 |
\& $line =~ m/^( |
| 919 |
\& ([\ew\e.\e\-]+) # client |
| 920 |
\& \es*\-\es*\-\es*\e[ |
| 921 |
\& ([^]]+) # date |
| 922 |
\& \e]\es*"\ew+\es* |
| 923 |
\& (\eS+) # query |
| 924 |
\& [^"]+"\es* |
| 925 |
\& (\ed+) # status |
| 926 |
\& \es+\eS+\es+"[^"]*"\es+" |
| 927 |
\& ([^"]*) # browser |
| 928 |
\& " |
| 929 |
\& .* |
| 930 |
\& )$/x |
| 931 |
\& ) { |
| 932 |
\& my @chunks = split(/ +/, $line); |
| 933 |
\& my $ip = $1; |
| 934 |
\& my $date = $2; |
| 935 |
\& my $query = $3; |
| 936 |
\& my $status = $4; |
| 937 |
\& my $browser = $5; |
| 938 |
\& |
| 939 |
\& push(@data, [$ip, $date, $query, $status, $browser, $line]); |
| 940 |
\& } |
| 941 |
\& } |
| 942 |
\& |
| 943 |
\& my @sorted = sort { |
| 944 |
\& $a\->[3] cmp $b\->[3] |
| 945 |
\& || |
| 946 |
\& $a\->[2] cmp $b\->[2] |
| 947 |
\& || |
| 948 |
\& $a\->[0] cmp $b\->[0] |
| 949 |
\& || |
| 950 |
\& $a\->[1] cmp $b\->[1] |
| 951 |
\& || |
| 952 |
\& $a\->[4] cmp $b\->[4] |
| 953 |
\& } @data; |
| 954 |
\& |
| 955 |
\& foreach my $data ( @sorted ) { |
| 956 |
\& print $data\->[5]; |
| 957 |
\& } |
| 958 |
\& |
| 959 |
\& exit 0; |
| 960 |
.Ve |
| 961 |
.PP |
| 962 |
When running this program, redirect \f(CW\*(C`STDOUT\*(C'\fR so it is possible to check the |
| 963 |
output is correct from following test runs and use the system \f(CW\*(C`time\*(C'\fR utility |
| 964 |
to check the overall runtime. |
| 965 |
.PP |
| 966 |
.Vb 1 |
| 967 |
\& $> time ./sort\-apache\-log logfile > out\-sort |
| 968 |
\& |
| 969 |
\& real 0m17.371s |
| 970 |
\& user 0m15.757s |
| 971 |
\& sys 0m0.592s |
| 972 |
.Ve |
| 973 |
.PP |
| 974 |
The program took just over 17 wallclock seconds to run. Note the different |
| 975 |
values \f(CW\*(C`time\*(C'\fR outputs, it's important to always use the same one, and to not |
| 976 |
confuse what each one means. |
| 977 |
.IP "Elapsed Real Time" 4 |
| 978 |
.IX Item "Elapsed Real Time" |
| 979 |
The overall, or wallclock, time between when \f(CW\*(C`time\*(C'\fR was called, and when it |
| 980 |
terminates. The elapsed time includes both user and system times, and time |
| 981 |
spent waiting for other users and processes on the system. Inevitably, this is |
| 982 |
the most approximate of the measurements given. |
| 983 |
.IP "User CPU Time" 4 |
| 984 |
.IX Item "User CPU Time" |
| 985 |
The user time is the amount of time the entire process spent on behalf of the |
| 986 |
user on this system executing this program. |
| 987 |
.IP "System CPU Time" 4 |
| 988 |
.IX Item "System CPU Time" |
| 989 |
The system time is the amount of time the kernel itself spent executing |
| 990 |
routines, or system calls, on behalf of this process user. |
| 991 |
.PP |
| 992 |
Running this same process as a \f(CW\*(C`Schwarzian Transform\*(C'\fR it is possible to |
| 993 |
eliminate the input and output arrays for storing all the data, and work on the |
| 994 |
input directly as it arrives too. Otherwise, the code looks fairly similar: |
| 995 |
.PP |
| 996 |
# sort-apache-log-schwarzian |
| 997 |
.PP |
| 998 |
.Vb 1 |
| 999 |
\& #!/usr/bin/perl \-n |
| 1000 |
\& |
| 1001 |
\& use v5.36; |
| 1002 |
\& |
| 1003 |
\& print |
| 1004 |
\& |
| 1005 |
\& map $_\->[0] => |
| 1006 |
\& |
| 1007 |
\& sort { |
| 1008 |
\& $a\->[4] cmp $b\->[4] |
| 1009 |
\& || |
| 1010 |
\& $a\->[3] cmp $b\->[3] |
| 1011 |
\& || |
| 1012 |
\& $a\->[1] cmp $b\->[1] |
| 1013 |
\& || |
| 1014 |
\& $a\->[2] cmp $b\->[2] |
| 1015 |
\& || |
| 1016 |
\& $a\->[5] cmp $b\->[5] |
| 1017 |
\& } |
| 1018 |
\& map [ $_, m/^( |
| 1019 |
\& ([\ew\e.\e\-]+) # client |
| 1020 |
\& \es*\-\es*\-\es*\e[ |
| 1021 |
\& ([^]]+) # date |
| 1022 |
\& \e]\es*"\ew+\es* |
| 1023 |
\& (\eS+) # query |
| 1024 |
\& [^"]+"\es* |
| 1025 |
\& (\ed+) # status |
| 1026 |
\& \es+\eS+\es+"[^"]*"\es+" |
| 1027 |
\& ([^"]*) # browser |
| 1028 |
\& " |
| 1029 |
\& .* |
| 1030 |
\& )$/xo ] |
| 1031 |
\& |
| 1032 |
\& => <>; |
| 1033 |
\& |
| 1034 |
\& exit 0; |
| 1035 |
.Ve |
| 1036 |
.PP |
| 1037 |
Run the new code against the same logfile, as above, to check the new time. |
| 1038 |
.PP |
| 1039 |
.Vb 1 |
| 1040 |
\& $> time ./sort\-apache\-log\-schwarzian logfile > out\-schwarz |
| 1041 |
\& |
| 1042 |
\& real 0m9.664s |
| 1043 |
\& user 0m8.873s |
| 1044 |
\& sys 0m0.704s |
| 1045 |
.Ve |
| 1046 |
.PP |
| 1047 |
The time has been cut in half, which is a respectable speed improvement by any |
| 1048 |
standard. Naturally, it is important to check the output is consistent with |
| 1049 |
the first program run, this is where the Unix system \f(CW\*(C`cksum\*(C'\fR utility comes in. |
| 1050 |
.PP |
| 1051 |
.Vb 3 |
| 1052 |
\& $> cksum out\-sort out\-schwarz |
| 1053 |
\& 3044173777 52029194 out\-sort |
| 1054 |
\& 3044173777 52029194 out\-schwarz |
| 1055 |
.Ve |
| 1056 |
.PP |
| 1057 |
BTW. Beware too of pressure from managers who see you speed a program up by 50% |
| 1058 |
of the runtime once, only to get a request one month later to do the same again |
| 1059 |
(true story) \- you'll just have to point out you're only human, even if you are a |
| 1060 |
Perl programmer, and you'll see what you can do... |
| 1061 |
.SH LOGGING |
| 1062 |
.IX Header "LOGGING" |
| 1063 |
An essential part of any good development process is appropriate error handling |
| 1064 |
with appropriately informative messages, however there exists a school of |
| 1065 |
thought which suggests that log files should be \fIchatty\fR, as if the chain of |
| 1066 |
unbroken output somehow ensures the survival of the program. If speed is in |
| 1067 |
any way an issue, this approach is wrong. |
| 1068 |
.PP |
| 1069 |
A common sight is code which looks something like this: |
| 1070 |
.PP |
| 1071 |
.Vb 2 |
| 1072 |
\& logger\->debug( "A logging message via process\-id: $$ INC: " |
| 1073 |
\& . Dumper(\e%INC) ) |
| 1074 |
.Ve |
| 1075 |
.PP |
| 1076 |
The problem is that this code will always be parsed and executed, even when the |
| 1077 |
debug level set in the logging configuration file is zero. Once the \fBdebug()\fR |
| 1078 |
subroutine has been entered, and the internal \f(CW$debug\fR variable confirmed to |
| 1079 |
be zero, for example, the message which has been sent in will be discarded and |
| 1080 |
the program will continue. In the example given though, the \f(CW\*(C`\e%INC\*(C'\fR hash will |
| 1081 |
already have been dumped, and the message string constructed, all of which work |
| 1082 |
could be bypassed by a debug variable at the statement level, like this: |
| 1083 |
.PP |
| 1084 |
.Vb 2 |
| 1085 |
\& logger\->debug( "A logging message via process\-id: $$ INC: " |
| 1086 |
\& . Dumper(\e%INC) ) if $DEBUG; |
| 1087 |
.Ve |
| 1088 |
.PP |
| 1089 |
This effect can be demonstrated by setting up a test script with both forms, |
| 1090 |
including a \f(CWdebug()\fR subroutine to emulate typical \f(CWlogger()\fR functionality. |
| 1091 |
.PP |
| 1092 |
# ifdebug |
| 1093 |
.PP |
| 1094 |
.Vb 1 |
| 1095 |
\& #!/usr/bin/perl |
| 1096 |
\& |
| 1097 |
\& use v5.36; |
| 1098 |
\& |
| 1099 |
\& use Benchmark; |
| 1100 |
\& use Data::Dumper; |
| 1101 |
\& my $DEBUG = 0; |
| 1102 |
\& |
| 1103 |
\& sub debug { |
| 1104 |
\& my $msg = shift; |
| 1105 |
\& |
| 1106 |
\& if ( $DEBUG ) { |
| 1107 |
\& print "DEBUG: $msg\en"; |
| 1108 |
\& } |
| 1109 |
\& }; |
| 1110 |
\& |
| 1111 |
\& timethese(100000, { |
| 1112 |
\& \*(Aqdebug\*(Aq => sub { |
| 1113 |
\& debug( "A $0 logging message via process\-id: $$" . Dumper(\e%INC) ) |
| 1114 |
\& }, |
| 1115 |
\& \*(Aqifdebug\*(Aq => sub { |
| 1116 |
\& debug( "A $0 logging message via process\-id: $$" . Dumper(\e%INC) ) if $DEBUG |
| 1117 |
\& }, |
| 1118 |
\& }); |
| 1119 |
.Ve |
| 1120 |
.PP |
| 1121 |
Let's see what \f(CW\*(C`Benchmark\*(C'\fR makes of this: |
| 1122 |
.PP |
| 1123 |
.Vb 5 |
| 1124 |
\& $> perl ifdebug |
| 1125 |
\& Benchmark: timing 100000 iterations of constant, sub... |
| 1126 |
\& ifdebug: 0 wallclock secs ( 0.01 usr + 0.00 sys = 0.01 CPU) @ 10000000.00/s (n=100000) |
| 1127 |
\& (warning: too few iterations for a reliable count) |
| 1128 |
\& debug: 14 wallclock secs (13.18 usr + 0.04 sys = 13.22 CPU) @ 7564.30/s (n=100000) |
| 1129 |
.Ve |
| 1130 |
.PP |
| 1131 |
In the one case the code, which does exactly the same thing as far as |
| 1132 |
outputting any debugging information is concerned, in other words nothing, |
| 1133 |
takes 14 seconds, and in the other case the code takes one hundredth of a |
| 1134 |
second. Looks fairly definitive. Use a \f(CW$DEBUG\fR variable BEFORE you call the |
| 1135 |
subroutine, rather than relying on the smart functionality inside it. |
| 1136 |
.SS "Logging if DEBUG (constant)" |
| 1137 |
.IX Subsection "Logging if DEBUG (constant)" |
| 1138 |
It's possible to take the previous idea a little further, by using a compile |
| 1139 |
time \f(CW\*(C`DEBUG\*(C'\fR constant. |
| 1140 |
.PP |
| 1141 |
# ifdebug-constant |
| 1142 |
.PP |
| 1143 |
.Vb 1 |
| 1144 |
\& #!/usr/bin/perl |
| 1145 |
\& |
| 1146 |
\& use v5.36; |
| 1147 |
\& |
| 1148 |
\& use Benchmark; |
| 1149 |
\& use Data::Dumper; |
| 1150 |
\& use constant |
| 1151 |
\& DEBUG => 0 |
| 1152 |
\& ; |
| 1153 |
\& |
| 1154 |
\& sub debug { |
| 1155 |
\& if ( DEBUG ) { |
| 1156 |
\& my $msg = shift; |
| 1157 |
\& print "DEBUG: $msg\en"; |
| 1158 |
\& } |
| 1159 |
\& }; |
| 1160 |
\& |
| 1161 |
\& timethese(100000, { |
| 1162 |
\& \*(Aqdebug\*(Aq => sub { |
| 1163 |
\& debug( "A $0 logging message via process\-id: $$" . Dumper(\e%INC) ) |
| 1164 |
\& }, |
| 1165 |
\& \*(Aqconstant\*(Aq => sub { |
| 1166 |
\& debug( "A $0 logging message via process\-id: $$" . Dumper(\e%INC) ) if DEBUG |
| 1167 |
\& }, |
| 1168 |
\& }); |
| 1169 |
.Ve |
| 1170 |
.PP |
| 1171 |
Running this program produces the following output: |
| 1172 |
.PP |
| 1173 |
.Vb 5 |
| 1174 |
\& $> perl ifdebug\-constant |
| 1175 |
\& Benchmark: timing 100000 iterations of constant, sub... |
| 1176 |
\& constant: 0 wallclock secs (\-0.00 usr + 0.00 sys = \-0.00 CPU) @ \-7205759403792793600000.00/s (n=100000) |
| 1177 |
\& (warning: too few iterations for a reliable count) |
| 1178 |
\& sub: 14 wallclock secs (13.09 usr + 0.00 sys = 13.09 CPU) @ 7639.42/s (n=100000) |
| 1179 |
.Ve |
| 1180 |
.PP |
| 1181 |
The \f(CW\*(C`DEBUG\*(C'\fR constant wipes the floor with even the \f(CW$debug\fR variable, |
| 1182 |
clocking in at minus zero seconds, and generates a "warning: too few iterations |
| 1183 |
for a reliable count" message into the bargain. To see what is really going |
| 1184 |
on, and why we had too few iterations when we thought we asked for 100000, we |
| 1185 |
can use the very useful \f(CW\*(C`B::Deparse\*(C'\fR to inspect the new code: |
| 1186 |
.PP |
| 1187 |
.Vb 1 |
| 1188 |
\& $> perl \-MO=Deparse ifdebug\-constant |
| 1189 |
\& |
| 1190 |
\& use Benchmark; |
| 1191 |
\& use Data::Dumper; |
| 1192 |
\& use constant (\*(AqDEBUG\*(Aq, 0); |
| 1193 |
\& sub debug { |
| 1194 |
\& use warnings; |
| 1195 |
\& use strict \*(Aqrefs\*(Aq; |
| 1196 |
\& 0; |
| 1197 |
\& } |
| 1198 |
\& use warnings; |
| 1199 |
\& use strict \*(Aqrefs\*(Aq; |
| 1200 |
\& timethese(100000, {\*(Aqsub\*(Aq, sub { |
| 1201 |
\& debug "A $0 logging message via process\-id: $$" . Dumper(\e%INC); |
| 1202 |
\& } |
| 1203 |
\& , \*(Aqconstant\*(Aq, sub { |
| 1204 |
\& 0; |
| 1205 |
\& } |
| 1206 |
\& }); |
| 1207 |
\& ifdebug\-constant syntax OK |
| 1208 |
.Ve |
| 1209 |
.PP |
| 1210 |
The output shows the \fBconstant()\fR subroutine we're testing being replaced with |
| 1211 |
the value of the \f(CW\*(C`DEBUG\*(C'\fR constant: zero. The line to be tested has been |
| 1212 |
completely optimized away, and you can't get much more efficient than that. |
| 1213 |
.SH POSTSCRIPT |
| 1214 |
.IX Header "POSTSCRIPT" |
| 1215 |
This document has provided several way to go about identifying hot-spots, and |
| 1216 |
checking whether any modifications have improved the runtime of the code. |
| 1217 |
.PP |
| 1218 |
As a final thought, remember that it's not (at the time of writing) possible to |
| 1219 |
produce a useful program which will run in zero or negative time and this basic |
| 1220 |
principle can be written as: \fIuseful programs are slow\fR by their very |
| 1221 |
definition. It is of course possible to write a nearly instantaneous program, |
| 1222 |
but it's not going to do very much, here's a very efficient one: |
| 1223 |
.PP |
| 1224 |
.Vb 1 |
| 1225 |
\& $> perl \-e 0 |
| 1226 |
.Ve |
| 1227 |
.PP |
| 1228 |
Optimizing that any further is a job for \f(CW\*(C`p5p\*(C'\fR. |
| 1229 |
.SH "SEE ALSO" |
| 1230 |
.IX Header "SEE ALSO" |
| 1231 |
Further reading can be found using the modules and links below. |
| 1232 |
.SS PERLDOCS |
| 1233 |
.IX Subsection "PERLDOCS" |
| 1234 |
For example: \f(CW\*(C`perldoc \-f sort\*(C'\fR. |
| 1235 |
.PP |
| 1236 |
perlfaq4. |
| 1237 |
.PP |
| 1238 |
perlfork, perlfunc, perlretut, perlthrtut. |
| 1239 |
.PP |
| 1240 |
threads. |
| 1241 |
.SS "MAN PAGES" |
| 1242 |
.IX Subsection "MAN PAGES" |
| 1243 |
\&\f(CW\*(C`time\*(C'\fR. |
| 1244 |
.SS MODULES |
| 1245 |
.IX Subsection "MODULES" |
| 1246 |
It's not possible to individually showcase all the performance related code for |
| 1247 |
Perl here, naturally, but here's a short list of modules from the CPAN which |
| 1248 |
deserve further attention. |
| 1249 |
.PP |
| 1250 |
.Vb 10 |
| 1251 |
\& Apache::DProf |
| 1252 |
\& Apache::SmallProf |
| 1253 |
\& Benchmark |
| 1254 |
\& DBIx::Profile |
| 1255 |
\& Devel::AutoProfiler |
| 1256 |
\& Devel::DProf |
| 1257 |
\& Devel::DProfLB |
| 1258 |
\& Devel::FastProf |
| 1259 |
\& Devel::GraphVizProf |
| 1260 |
\& Devel::NYTProf |
| 1261 |
\& Devel::NYTProf::Apache |
| 1262 |
\& Devel::Profiler |
| 1263 |
\& Devel::Profile |
| 1264 |
\& Devel::Profit |
| 1265 |
\& Devel::SmallProf |
| 1266 |
\& Devel::WxProf |
| 1267 |
\& POE::Devel::Profiler |
| 1268 |
\& Sort::Key |
| 1269 |
\& Sort::Maker |
| 1270 |
.Ve |
| 1271 |
.SS URLS |
| 1272 |
.IX Subsection "URLS" |
| 1273 |
Very useful online reference material: |
| 1274 |
.PP |
| 1275 |
.Vb 1 |
| 1276 |
\& https://web.archive.org/web/20120515021937/http://www.ccl4.org/~nick/P/Fast_Enough/ |
| 1277 |
\& |
| 1278 |
\& https://web.archive.org/web/20050706081718/http://www\-106.ibm.com/developerworks/library/l\-optperl.html |
| 1279 |
\& |
| 1280 |
\& https://perlbuzz.com/2007/11/14/bind_output_variables_in_dbi_for_speed_and_safety/ |
| 1281 |
\& |
| 1282 |
\& http://en.wikipedia.org/wiki/Performance_analysis |
| 1283 |
\& |
| 1284 |
\& http://apache.perl.org/docs/1.0/guide/performance.html |
| 1285 |
\& |
| 1286 |
\& http://perlgolf.sourceforge.net/ |
| 1287 |
\& |
| 1288 |
\& http://www.sysarch.com/Perl/sort_paper.html |
| 1289 |
.Ve |
| 1290 |
.SH AUTHOR |
| 1291 |
.IX Header "AUTHOR" |
| 1292 |
Richard Foley <[email protected]> Copyright (c) 2008 |