| 1 |
.\" -*- mode: troff; coding: utf-8 -*- |
| 2 |
.\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45) |
| 3 |
.\" |
| 4 |
.\" Standard preamble: |
| 5 |
.\" ======================================================================== |
| 6 |
.de Sp \" Vertical space (when we can't use .PP) |
| 7 |
.if t .sp .5v |
| 8 |
.if n .sp |
| 9 |
.. |
| 10 |
.de Vb \" Begin verbatim text |
| 11 |
.ft CW |
| 12 |
.nf |
| 13 |
.ne \\$1 |
| 14 |
.. |
| 15 |
.de Ve \" End verbatim text |
| 16 |
.ft R |
| 17 |
.fi |
| 18 |
.. |
| 19 |
.\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. |
| 20 |
.ie n \{\ |
| 21 |
. ds C` "" |
| 22 |
. ds C' "" |
| 23 |
'br\} |
| 24 |
.el\{\ |
| 25 |
. ds C` |
| 26 |
. ds C' |
| 27 |
'br\} |
| 28 |
.\" |
| 29 |
.\" Escape single quotes in literal strings from groff's Unicode transform. |
| 30 |
.ie \n(.g .ds Aq \(aq |
| 31 |
.el .ds Aq ' |
| 32 |
.\" |
| 33 |
.\" If the F register is >0, we'll generate index entries on stderr for |
| 34 |
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index |
| 35 |
.\" entries marked with X<> in POD. Of course, you'll have to process the |
| 36 |
.\" output yourself in some meaningful fashion. |
| 37 |
.\" |
| 38 |
.\" Avoid warning from groff about undefined register 'F'. |
| 39 |
.de IX |
| 40 |
.. |
| 41 |
.nr rF 0 |
| 42 |
.if \n(.g .if rF .nr rF 1 |
| 43 |
.if (\n(rF:(\n(.g==0)) \{\ |
| 44 |
. if \nF \{\ |
| 45 |
. de IX |
| 46 |
. tm Index:\\$1\t\\n%\t"\\$2" |
| 47 |
.. |
| 48 |
. if !\nF==2 \{\ |
| 49 |
. nr % 0 |
| 50 |
. nr F 2 |
| 51 |
. \} |
| 52 |
. \} |
| 53 |
.\} |
| 54 |
.rr rF |
| 55 |
.\" ======================================================================== |
| 56 |
.\" |
| 57 |
.IX Title "PERLFILTER 1" |
| 58 |
.TH PERLFILTER 1 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide" |
| 59 |
.\" For nroff, turn off justification. Always turn off hyphenation; it makes |
| 60 |
.\" way too many mistakes in technical documents. |
| 61 |
.if n .ad l |
| 62 |
.nh |
| 63 |
.SH NAME |
| 64 |
perlfilter \- Source Filters |
| 65 |
.SH DESCRIPTION |
| 66 |
.IX Header "DESCRIPTION" |
| 67 |
This article is about a little-known feature of Perl called |
| 68 |
\&\fIsource filters\fR. Source filters alter the program text of a module |
| 69 |
before Perl sees it, much as a C preprocessor alters the source text of |
| 70 |
a C program before the compiler sees it. This article tells you more |
| 71 |
about what source filters are, how they work, and how to write your |
| 72 |
own. |
| 73 |
.PP |
| 74 |
The original purpose of source filters was to let you encrypt your |
| 75 |
program source to prevent casual reading. This isn't all they can do, as |
| 76 |
you'll soon learn. But first, the basics. |
| 77 |
.SH CONCEPTS |
| 78 |
.IX Header "CONCEPTS" |
| 79 |
Before the Perl interpreter can execute a Perl script, it must first |
| 80 |
read it from a file into memory for parsing and compilation. If that |
| 81 |
script itself includes other scripts with a \f(CW\*(C`use\*(C'\fR or \f(CW\*(C`require\*(C'\fR |
| 82 |
statement, then each of those scripts will have to be read from their |
| 83 |
respective files as well. |
| 84 |
.PP |
| 85 |
Now think of each logical connection between the Perl parser and an |
| 86 |
individual file as a \fIsource stream\fR. A source stream is created when |
| 87 |
the Perl parser opens a file, it continues to exist as the source code |
| 88 |
is read into memory, and it is destroyed when Perl is finished parsing |
| 89 |
the file. If the parser encounters a \f(CW\*(C`require\*(C'\fR or \f(CW\*(C`use\*(C'\fR statement in |
| 90 |
a source stream, a new and distinct stream is created just for that |
| 91 |
file. |
| 92 |
.PP |
| 93 |
The diagram below represents a single source stream, with the flow of |
| 94 |
source from a Perl script file on the left into the Perl parser on the |
| 95 |
right. This is how Perl normally operates. |
| 96 |
.PP |
| 97 |
.Vb 1 |
| 98 |
\& file \-\-\-\-\-\-\-> parser |
| 99 |
.Ve |
| 100 |
.PP |
| 101 |
There are two important points to remember: |
| 102 |
.IP 1. 5 |
| 103 |
Although there can be any number of source streams in existence at any |
| 104 |
given time, only one will be active. |
| 105 |
.IP 2. 5 |
| 106 |
Every source stream is associated with only one file. |
| 107 |
.PP |
| 108 |
A source filter is a special kind of Perl module that intercepts and |
| 109 |
modifies a source stream before it reaches the parser. A source filter |
| 110 |
changes our diagram like this: |
| 111 |
.PP |
| 112 |
.Vb 1 |
| 113 |
\& file \-\-\-\-> filter \-\-\-\-> parser |
| 114 |
.Ve |
| 115 |
.PP |
| 116 |
If that doesn't make much sense, consider the analogy of a command |
| 117 |
pipeline. Say you have a shell script stored in the compressed file |
| 118 |
\&\fItrial.gz\fR. The simple pipeline command below runs the script without |
| 119 |
needing to create a temporary file to hold the uncompressed file. |
| 120 |
.PP |
| 121 |
.Vb 1 |
| 122 |
\& gunzip \-c trial.gz | sh |
| 123 |
.Ve |
| 124 |
.PP |
| 125 |
In this case, the data flow from the pipeline can be represented as follows: |
| 126 |
.PP |
| 127 |
.Vb 1 |
| 128 |
\& trial.gz \-\-\-\-> gunzip \-\-\-\-> sh |
| 129 |
.Ve |
| 130 |
.PP |
| 131 |
With source filters, you can store the text of your script compressed and use a source filter to uncompress it for Perl's parser: |
| 132 |
.PP |
| 133 |
.Vb 2 |
| 134 |
\& compressed gunzip |
| 135 |
\& Perl program \-\-\-> source filter \-\-\-> parser |
| 136 |
.Ve |
| 137 |
.SH "USING FILTERS" |
| 138 |
.IX Header "USING FILTERS" |
| 139 |
So how do you use a source filter in a Perl script? Above, I said that |
| 140 |
a source filter is just a special kind of module. Like all Perl |
| 141 |
modules, a source filter is invoked with a use statement. |
| 142 |
.PP |
| 143 |
Say you want to pass your Perl source through the C preprocessor before |
| 144 |
execution. As it happens, the source filters distribution comes with a C |
| 145 |
preprocessor filter module called Filter::cpp. |
| 146 |
.PP |
| 147 |
Below is an example program, \f(CW\*(C`cpp_test\*(C'\fR, which makes use of this filter. |
| 148 |
Line numbers have been added to allow specific lines to be referenced |
| 149 |
easily. |
| 150 |
.PP |
| 151 |
.Vb 4 |
| 152 |
\& 1: use Filter::cpp; |
| 153 |
\& 2: #define TRUE 1 |
| 154 |
\& 3: $a = TRUE; |
| 155 |
\& 4: print "a = $a\en"; |
| 156 |
.Ve |
| 157 |
.PP |
| 158 |
When you execute this script, Perl creates a source stream for the |
| 159 |
file. Before the parser processes any of the lines from the file, the |
| 160 |
source stream looks like this: |
| 161 |
.PP |
| 162 |
.Vb 1 |
| 163 |
\& cpp_test \-\-\-\-\-\-\-\-\-> parser |
| 164 |
.Ve |
| 165 |
.PP |
| 166 |
Line 1, \f(CW\*(C`use Filter::cpp\*(C'\fR, includes and installs the \f(CW\*(C`cpp\*(C'\fR filter |
| 167 |
module. All source filters work this way. The use statement is compiled |
| 168 |
and executed at compile time, before any more of the file is read, and |
| 169 |
it attaches the cpp filter to the source stream behind the scenes. Now |
| 170 |
the data flow looks like this: |
| 171 |
.PP |
| 172 |
.Vb 1 |
| 173 |
\& cpp_test \-\-\-\-> cpp filter \-\-\-\-> parser |
| 174 |
.Ve |
| 175 |
.PP |
| 176 |
As the parser reads the second and subsequent lines from the source |
| 177 |
stream, it feeds those lines through the \f(CW\*(C`cpp\*(C'\fR source filter before |
| 178 |
processing them. The \f(CW\*(C`cpp\*(C'\fR filter simply passes each line through the |
| 179 |
real C preprocessor. The output from the C preprocessor is then |
| 180 |
inserted back into the source stream by the filter. |
| 181 |
.PP |
| 182 |
.Vb 5 |
| 183 |
\& .\-> cpp \-\-. |
| 184 |
\& | | |
| 185 |
\& | | |
| 186 |
\& | <\-\*(Aq |
| 187 |
\& cpp_test \-\-\-\-> cpp filter \-\-\-\-> parser |
| 188 |
.Ve |
| 189 |
.PP |
| 190 |
The parser then sees the following code: |
| 191 |
.PP |
| 192 |
.Vb 3 |
| 193 |
\& use Filter::cpp; |
| 194 |
\& $a = 1; |
| 195 |
\& print "a = $a\en"; |
| 196 |
.Ve |
| 197 |
.PP |
| 198 |
Let's consider what happens when the filtered code includes another |
| 199 |
module with use: |
| 200 |
.PP |
| 201 |
.Vb 5 |
| 202 |
\& 1: use Filter::cpp; |
| 203 |
\& 2: #define TRUE 1 |
| 204 |
\& 3: use Fred; |
| 205 |
\& 4: $a = TRUE; |
| 206 |
\& 5: print "a = $a\en"; |
| 207 |
.Ve |
| 208 |
.PP |
| 209 |
The \f(CW\*(C`cpp\*(C'\fR filter does not apply to the text of the Fred module, only |
| 210 |
to the text of the file that used it (\f(CW\*(C`cpp_test\*(C'\fR). Although the use |
| 211 |
statement on line 3 will pass through the cpp filter, the module that |
| 212 |
gets included (\f(CW\*(C`Fred\*(C'\fR) will not. The source streams look like this |
| 213 |
after line 3 has been parsed and before line 4 is parsed: |
| 214 |
.PP |
| 215 |
.Vb 1 |
| 216 |
\& cpp_test \-\-\-> cpp filter \-\-\-> parser (INACTIVE) |
| 217 |
\& |
| 218 |
\& Fred.pm \-\-\-\-> parser |
| 219 |
.Ve |
| 220 |
.PP |
| 221 |
As you can see, a new stream has been created for reading the source |
| 222 |
from \f(CW\*(C`Fred.pm\*(C'\fR. This stream will remain active until all of \f(CW\*(C`Fred.pm\*(C'\fR |
| 223 |
has been parsed. The source stream for \f(CW\*(C`cpp_test\*(C'\fR will still exist, |
| 224 |
but is inactive. Once the parser has finished reading Fred.pm, the |
| 225 |
source stream associated with it will be destroyed. The source stream |
| 226 |
for \f(CW\*(C`cpp_test\*(C'\fR then becomes active again and the parser reads line 4 |
| 227 |
and subsequent lines from \f(CW\*(C`cpp_test\*(C'\fR. |
| 228 |
.PP |
| 229 |
You can use more than one source filter on a single file. Similarly, |
| 230 |
you can reuse the same filter in as many files as you like. |
| 231 |
.PP |
| 232 |
For example, if you have a uuencoded and compressed source file, it is |
| 233 |
possible to stack a uudecode filter and an uncompression filter like |
| 234 |
this: |
| 235 |
.PP |
| 236 |
.Vb 4 |
| 237 |
\& use Filter::uudecode; use Filter::uncompress; |
| 238 |
\& M\*(AqXL(".H<US4\*(Aq\*(AqV9I;F%L\*(Aq)Q;>7/;1I;_>_I3=&E=%:F*I"T?22Q/ |
| 239 |
\& M6]9*<IQCO*XFT"0[PL%%\*(AqY+IG?WN^ZYN\-$\*(AqJ.[.JE$,20/?K=_[> |
| 240 |
\& ... |
| 241 |
.Ve |
| 242 |
.PP |
| 243 |
Once the first line has been processed, the flow will look like this: |
| 244 |
.PP |
| 245 |
.Vb 2 |
| 246 |
\& file \-\-\-> uudecode \-\-\-> uncompress \-\-\-> parser |
| 247 |
\& filter filter |
| 248 |
.Ve |
| 249 |
.PP |
| 250 |
Data flows through filters in the same order they appear in the source |
| 251 |
file. The uudecode filter appeared before the uncompress filter, so the |
| 252 |
source file will be uudecoded before it's uncompressed. |
| 253 |
.SH "WRITING A SOURCE FILTER" |
| 254 |
.IX Header "WRITING A SOURCE FILTER" |
| 255 |
There are three ways to write your own source filter. You can write it |
| 256 |
in C, use an external program as a filter, or write the filter in Perl. |
| 257 |
I won't cover the first two in any great detail, so I'll get them out |
| 258 |
of the way first. Writing the filter in Perl is most convenient, so |
| 259 |
I'll devote the most space to it. |
| 260 |
.SH "WRITING A SOURCE FILTER IN C" |
| 261 |
.IX Header "WRITING A SOURCE FILTER IN C" |
| 262 |
The first of the three available techniques is to write the filter |
| 263 |
completely in C. The external module you create interfaces directly |
| 264 |
with the source filter hooks provided by Perl. |
| 265 |
.PP |
| 266 |
The advantage of this technique is that you have complete control over |
| 267 |
the implementation of your filter. The big disadvantage is the |
| 268 |
increased complexity required to write the filter \- not only do you |
| 269 |
need to understand the source filter hooks, but you also need a |
| 270 |
reasonable knowledge of Perl guts. One of the few times it is worth |
| 271 |
going to this trouble is when writing a source scrambler. The |
| 272 |
\&\f(CW\*(C`decrypt\*(C'\fR filter (which unscrambles the source before Perl parses it) |
| 273 |
included with the source filter distribution is an example of a C |
| 274 |
source filter (see Decryption Filters, below). |
| 275 |
.IP "\fBDecryption Filters\fR" 5 |
| 276 |
.IX Item "Decryption Filters" |
| 277 |
All decryption filters work on the principle of "security through |
| 278 |
obscurity." Regardless of how well you write a decryption filter and |
| 279 |
how strong your encryption algorithm is, anyone determined enough can |
| 280 |
retrieve the original source code. The reason is quite simple: |
| 281 |
in order to execute your program Perl must parse its source code. |
| 282 |
This means that Perl must have all the information needed to decrypt |
| 283 |
your program, and that means that that information is also available to |
| 284 |
anyone able to run the program. |
| 285 |
.Sp |
| 286 |
That said, there are a number of steps that can be taken to make life |
| 287 |
difficult for the potential reader. The most important: Write your |
| 288 |
decryption filter in C and statically link the decryption module into |
| 289 |
the Perl binary. For further tips to make life difficult for the |
| 290 |
potential reader, see the file \fIdecrypt.pm\fR in the source filters |
| 291 |
distribution. |
| 292 |
.SH "CREATING A SOURCE FILTER AS A SEPARATE EXECUTABLE" |
| 293 |
.IX Header "CREATING A SOURCE FILTER AS A SEPARATE EXECUTABLE" |
| 294 |
An alternative to writing the filter in C is to create a separate |
| 295 |
executable in the language of your choice. The separate executable |
| 296 |
reads from standard input, does whatever processing is necessary, and |
| 297 |
writes the filtered data to standard output. \f(CW\*(C`Filter::cpp\*(C'\fR is an |
| 298 |
example of a source filter implemented as a separate executable \- the |
| 299 |
executable is the C preprocessor bundled with your C compiler. |
| 300 |
.PP |
| 301 |
The source filter distribution includes two modules that simplify this |
| 302 |
task: \f(CW\*(C`Filter::exec\*(C'\fR and \f(CW\*(C`Filter::sh\*(C'\fR. Both allow you to run any |
| 303 |
external executable. Both use a coprocess to control the flow of data |
| 304 |
into and out of the external executable. (For details on coprocesses, |
| 305 |
see Stephens, W.R., "Advanced Programming in the UNIX Environment." |
| 306 |
Addison-Wesley, ISBN 0\-210\-56317\-7, pages 441\-445.) The difference |
| 307 |
between them is that \f(CW\*(C`Filter::exec\*(C'\fR spawns the external command |
| 308 |
directly, while \f(CW\*(C`Filter::sh\*(C'\fR spawns a shell to execute the external |
| 309 |
command. (Unix uses the Bourne shell; NT uses the cmd shell.) Spawning |
| 310 |
a shell allows you to make use of the shell metacharacters and |
| 311 |
redirection facilities. |
| 312 |
.PP |
| 313 |
Here is an example script that uses \f(CW\*(C`Filter::sh\*(C'\fR: |
| 314 |
.PP |
| 315 |
.Vb 3 |
| 316 |
\& use Filter::sh \*(Aqtr XYZ PQR\*(Aq; |
| 317 |
\& $a = 1; |
| 318 |
\& print "XYZ a = $a\en"; |
| 319 |
.Ve |
| 320 |
.PP |
| 321 |
The output you'll get when the script is executed: |
| 322 |
.PP |
| 323 |
.Vb 1 |
| 324 |
\& PQR a = 1 |
| 325 |
.Ve |
| 326 |
.PP |
| 327 |
Writing a source filter as a separate executable works fine, but a |
| 328 |
small performance penalty is incurred. For example, if you execute the |
| 329 |
small example above, a separate subprocess will be created to run the |
| 330 |
Unix \f(CW\*(C`tr\*(C'\fR command. Each use of the filter requires its own subprocess. |
| 331 |
If creating subprocesses is expensive on your system, you might want to |
| 332 |
consider one of the other options for creating source filters. |
| 333 |
.SH "WRITING A SOURCE FILTER IN PERL" |
| 334 |
.IX Header "WRITING A SOURCE FILTER IN PERL" |
| 335 |
The easiest and most portable option available for creating your own |
| 336 |
source filter is to write it completely in Perl. To distinguish this |
| 337 |
from the previous two techniques, I'll call it a Perl source filter. |
| 338 |
.PP |
| 339 |
To help understand how to write a Perl source filter we need an example |
| 340 |
to study. Here is a complete source filter that performs rot13 |
| 341 |
decoding. (Rot13 is a very simple encryption scheme used in Usenet |
| 342 |
postings to hide the contents of offensive posts. It moves every letter |
| 343 |
forward thirteen places, so that A becomes N, B becomes O, and Z |
| 344 |
becomes M.) |
| 345 |
.PP |
| 346 |
.Vb 1 |
| 347 |
\& package Rot13; |
| 348 |
\& |
| 349 |
\& use Filter::Util::Call; |
| 350 |
\& |
| 351 |
\& sub import { |
| 352 |
\& my ($type) = @_; |
| 353 |
\& my ($ref) = []; |
| 354 |
\& filter_add(bless $ref); |
| 355 |
\& } |
| 356 |
\& |
| 357 |
\& sub filter { |
| 358 |
\& my ($self) = @_; |
| 359 |
\& my ($status); |
| 360 |
\& |
| 361 |
\& tr/n\-za\-mN\-ZA\-M/a\-zA\-Z/ |
| 362 |
\& if ($status = filter_read()) > 0; |
| 363 |
\& $status; |
| 364 |
\& } |
| 365 |
\& |
| 366 |
\& 1; |
| 367 |
.Ve |
| 368 |
.PP |
| 369 |
All Perl source filters are implemented as Perl classes and have the |
| 370 |
same basic structure as the example above. |
| 371 |
.PP |
| 372 |
First, we include the \f(CW\*(C`Filter::Util::Call\*(C'\fR module, which exports a |
| 373 |
number of functions into your filter's namespace. The filter shown |
| 374 |
above uses two of these functions, \f(CWfilter_add()\fR and |
| 375 |
\&\f(CWfilter_read()\fR. |
| 376 |
.PP |
| 377 |
Next, we create the filter object and associate it with the source |
| 378 |
stream by defining the \f(CW\*(C`import\*(C'\fR function. If you know Perl well |
| 379 |
enough, you know that \f(CW\*(C`import\*(C'\fR is called automatically every time a |
| 380 |
module is included with a use statement. This makes \f(CW\*(C`import\*(C'\fR the ideal |
| 381 |
place to both create and install a filter object. |
| 382 |
.PP |
| 383 |
In the example filter, the object (\f(CW$ref\fR) is blessed just like any |
| 384 |
other Perl object. Our example uses an anonymous array, but this isn't |
| 385 |
a requirement. Because this example doesn't need to store any context |
| 386 |
information, we could have used a scalar or hash reference just as |
| 387 |
well. The next section demonstrates context data. |
| 388 |
.PP |
| 389 |
The association between the filter object and the source stream is made |
| 390 |
with the \f(CWfilter_add()\fR function. This takes a filter object as a |
| 391 |
parameter (\f(CW$ref\fR in this case) and installs it in the source stream. |
| 392 |
.PP |
| 393 |
Finally, there is the code that actually does the filtering. For this |
| 394 |
type of Perl source filter, all the filtering is done in a method |
| 395 |
called \f(CWfilter()\fR. (It is also possible to write a Perl source filter |
| 396 |
using a closure. See the \f(CW\*(C`Filter::Util::Call\*(C'\fR manual page for more |
| 397 |
details.) It's called every time the Perl parser needs another line of |
| 398 |
source to process. The \f(CWfilter()\fR method, in turn, reads lines from |
| 399 |
the source stream using the \f(CWfilter_read()\fR function. |
| 400 |
.PP |
| 401 |
If a line was available from the source stream, \f(CWfilter_read()\fR |
| 402 |
returns a status value greater than zero and appends the line to \f(CW$_\fR. |
| 403 |
A status value of zero indicates end-of-file, less than zero means an |
| 404 |
error. The filter function itself is expected to return its status in |
| 405 |
the same way, and put the filtered line it wants written to the source |
| 406 |
stream in \f(CW$_\fR. The use of \f(CW$_\fR accounts for the brevity of most Perl |
| 407 |
source filters. |
| 408 |
.PP |
| 409 |
In order to make use of the rot13 filter we need some way of encoding |
| 410 |
the source file in rot13 format. The script below, \f(CW\*(C`mkrot13\*(C'\fR, does |
| 411 |
just that. |
| 412 |
.PP |
| 413 |
.Vb 5 |
| 414 |
\& die "usage mkrot13 filename\en" unless @ARGV; |
| 415 |
\& my $in = $ARGV[0]; |
| 416 |
\& my $out = "$in.tmp"; |
| 417 |
\& open(IN, "<$in") or die "Cannot open file $in: $!\en"; |
| 418 |
\& open(OUT, ">$out") or die "Cannot open file $out: $!\en"; |
| 419 |
\& |
| 420 |
\& print OUT "use Rot13;\en"; |
| 421 |
\& while (<IN>) { |
| 422 |
\& tr/a\-zA\-Z/n\-za\-mN\-ZA\-M/; |
| 423 |
\& print OUT; |
| 424 |
\& } |
| 425 |
\& |
| 426 |
\& close IN; |
| 427 |
\& close OUT; |
| 428 |
\& unlink $in; |
| 429 |
\& rename $out, $in; |
| 430 |
.Ve |
| 431 |
.PP |
| 432 |
If we encrypt this with \f(CW\*(C`mkrot13\*(C'\fR: |
| 433 |
.PP |
| 434 |
.Vb 1 |
| 435 |
\& print " hello fred \en"; |
| 436 |
.Ve |
| 437 |
.PP |
| 438 |
the result will be this: |
| 439 |
.PP |
| 440 |
.Vb 2 |
| 441 |
\& use Rot13; |
| 442 |
\& cevag "uryyb serq\ea"; |
| 443 |
.Ve |
| 444 |
.PP |
| 445 |
Running it produces this output: |
| 446 |
.PP |
| 447 |
.Vb 1 |
| 448 |
\& hello fred |
| 449 |
.Ve |
| 450 |
.SH "USING CONTEXT: THE DEBUG FILTER" |
| 451 |
.IX Header "USING CONTEXT: THE DEBUG FILTER" |
| 452 |
The rot13 example was a trivial example. Here's another demonstration |
| 453 |
that shows off a few more features. |
| 454 |
.PP |
| 455 |
Say you wanted to include a lot of debugging code in your Perl script |
| 456 |
during development, but you didn't want it available in the released |
| 457 |
product. Source filters offer a solution. In order to keep the example |
| 458 |
simple, let's say you wanted the debugging output to be controlled by |
| 459 |
an environment variable, \f(CW\*(C`DEBUG\*(C'\fR. Debugging code is enabled if the |
| 460 |
variable exists, otherwise it is disabled. |
| 461 |
.PP |
| 462 |
Two special marker lines will bracket debugging code, like this: |
| 463 |
.PP |
| 464 |
.Vb 5 |
| 465 |
\& ## DEBUG_BEGIN |
| 466 |
\& if ($year > 1999) { |
| 467 |
\& warn "Debug: millennium bug in year $year\en"; |
| 468 |
\& } |
| 469 |
\& ## DEBUG_END |
| 470 |
.Ve |
| 471 |
.PP |
| 472 |
The filter ensures that Perl parses the code between the <DEBUG_BEGIN> |
| 473 |
and \f(CW\*(C`DEBUG_END\*(C'\fR markers only when the \f(CW\*(C`DEBUG\*(C'\fR environment variable |
| 474 |
exists. That means that when \f(CW\*(C`DEBUG\*(C'\fR does exist, the code above |
| 475 |
should be passed through the filter unchanged. The marker lines can |
| 476 |
also be passed through as-is, because the Perl parser will see them as |
| 477 |
comment lines. When \f(CW\*(C`DEBUG\*(C'\fR isn't set, we need a way to disable the |
| 478 |
debug code. A simple way to achieve that is to convert the lines |
| 479 |
between the two markers into comments: |
| 480 |
.PP |
| 481 |
.Vb 5 |
| 482 |
\& ## DEBUG_BEGIN |
| 483 |
\& #if ($year > 1999) { |
| 484 |
\& # warn "Debug: millennium bug in year $year\en"; |
| 485 |
\& #} |
| 486 |
\& ## DEBUG_END |
| 487 |
.Ve |
| 488 |
.PP |
| 489 |
Here is the complete Debug filter: |
| 490 |
.PP |
| 491 |
.Vb 1 |
| 492 |
\& package Debug; |
| 493 |
\& |
| 494 |
\& use v5.36; |
| 495 |
\& use Filter::Util::Call; |
| 496 |
\& |
| 497 |
\& use constant TRUE => 1; |
| 498 |
\& use constant FALSE => 0; |
| 499 |
\& |
| 500 |
\& sub import { |
| 501 |
\& my ($type) = @_; |
| 502 |
\& my (%context) = ( |
| 503 |
\& Enabled => defined $ENV{DEBUG}, |
| 504 |
\& InTraceBlock => FALSE, |
| 505 |
\& Filename => (caller)[1], |
| 506 |
\& LineNo => 0, |
| 507 |
\& LastBegin => 0, |
| 508 |
\& ); |
| 509 |
\& filter_add(bless \e%context); |
| 510 |
\& } |
| 511 |
\& |
| 512 |
\& sub Die { |
| 513 |
\& my ($self) = shift; |
| 514 |
\& my ($message) = shift; |
| 515 |
\& my ($line_no) = shift || $self\->{LastBegin}; |
| 516 |
\& die "$message at $self\->{Filename} line $line_no.\en" |
| 517 |
\& } |
| 518 |
\& |
| 519 |
\& sub filter { |
| 520 |
\& my ($self) = @_; |
| 521 |
\& my ($status); |
| 522 |
\& $status = filter_read(); |
| 523 |
\& ++ $self\->{LineNo}; |
| 524 |
\& |
| 525 |
\& # deal with EOF/error first |
| 526 |
\& if ($status <= 0) { |
| 527 |
\& $self\->Die("DEBUG_BEGIN has no DEBUG_END") |
| 528 |
\& if $self\->{InTraceBlock}; |
| 529 |
\& return $status; |
| 530 |
\& } |
| 531 |
\& |
| 532 |
\& if ($self\->{InTraceBlock}) { |
| 533 |
\& if (/^\es*##\es*DEBUG_BEGIN/ ) { |
| 534 |
\& $self\->Die("Nested DEBUG_BEGIN", $self\->{LineNo}) |
| 535 |
\& } elsif (/^\es*##\es*DEBUG_END/) { |
| 536 |
\& $self\->{InTraceBlock} = FALSE; |
| 537 |
\& } |
| 538 |
\& |
| 539 |
\& # comment out the debug lines when the filter is disabled |
| 540 |
\& s/^/#/ if ! $self\->{Enabled}; |
| 541 |
\& } elsif ( /^\es*##\es*DEBUG_BEGIN/ ) { |
| 542 |
\& $self\->{InTraceBlock} = TRUE; |
| 543 |
\& $self\->{LastBegin} = $self\->{LineNo}; |
| 544 |
\& } elsif ( /^\es*##\es*DEBUG_END/ ) { |
| 545 |
\& $self\->Die("DEBUG_END has no DEBUG_BEGIN", $self\->{LineNo}); |
| 546 |
\& } |
| 547 |
\& return $status; |
| 548 |
\& } |
| 549 |
\& |
| 550 |
\& 1; |
| 551 |
.Ve |
| 552 |
.PP |
| 553 |
The big difference between this filter and the previous example is the |
| 554 |
use of context data in the filter object. The filter object is based on |
| 555 |
a hash reference, and is used to keep various pieces of context |
| 556 |
information between calls to the filter function. All but two of the |
| 557 |
hash fields are used for error reporting. The first of those two, |
| 558 |
Enabled, is used by the filter to determine whether the debugging code |
| 559 |
should be given to the Perl parser. The second, InTraceBlock, is true |
| 560 |
when the filter has encountered a \f(CW\*(C`DEBUG_BEGIN\*(C'\fR line, but has not yet |
| 561 |
encountered the following \f(CW\*(C`DEBUG_END\*(C'\fR line. |
| 562 |
.PP |
| 563 |
If you ignore all the error checking that most of the code does, the |
| 564 |
essence of the filter is as follows: |
| 565 |
.PP |
| 566 |
.Vb 4 |
| 567 |
\& sub filter { |
| 568 |
\& my ($self) = @_; |
| 569 |
\& my ($status); |
| 570 |
\& $status = filter_read(); |
| 571 |
\& |
| 572 |
\& # deal with EOF/error first |
| 573 |
\& return $status if $status <= 0; |
| 574 |
\& if ($self\->{InTraceBlock}) { |
| 575 |
\& if (/^\es*##\es*DEBUG_END/) { |
| 576 |
\& $self\->{InTraceBlock} = FALSE |
| 577 |
\& } |
| 578 |
\& |
| 579 |
\& # comment out debug lines when the filter is disabled |
| 580 |
\& s/^/#/ if ! $self\->{Enabled}; |
| 581 |
\& } elsif ( /^\es*##\es*DEBUG_BEGIN/ ) { |
| 582 |
\& $self\->{InTraceBlock} = TRUE; |
| 583 |
\& } |
| 584 |
\& return $status; |
| 585 |
\& } |
| 586 |
.Ve |
| 587 |
.PP |
| 588 |
Be warned: just as the C\-preprocessor doesn't know C, the Debug filter |
| 589 |
doesn't know Perl. It can be fooled quite easily: |
| 590 |
.PP |
| 591 |
.Vb 3 |
| 592 |
\& print <<EOM; |
| 593 |
\& ##DEBUG_BEGIN |
| 594 |
\& EOM |
| 595 |
.Ve |
| 596 |
.PP |
| 597 |
Such things aside, you can see that a lot can be achieved with a modest |
| 598 |
amount of code. |
| 599 |
.SH CONCLUSION |
| 600 |
.IX Header "CONCLUSION" |
| 601 |
You now have better understanding of what a source filter is, and you |
| 602 |
might even have a possible use for them. If you feel like playing with |
| 603 |
source filters but need a bit of inspiration, here are some extra |
| 604 |
features you could add to the Debug filter. |
| 605 |
.PP |
| 606 |
First, an easy one. Rather than having debugging code that is |
| 607 |
all-or-nothing, it would be much more useful to be able to control |
| 608 |
which specific blocks of debugging code get included. Try extending the |
| 609 |
syntax for debug blocks to allow each to be identified. The contents of |
| 610 |
the \f(CW\*(C`DEBUG\*(C'\fR environment variable can then be used to control which |
| 611 |
blocks get included. |
| 612 |
.PP |
| 613 |
Once you can identify individual blocks, try allowing them to be |
| 614 |
nested. That isn't difficult either. |
| 615 |
.PP |
| 616 |
Here is an interesting idea that doesn't involve the Debug filter. |
| 617 |
Currently Perl subroutines have fairly limited support for formal |
| 618 |
parameter lists. You can specify the number of parameters and their |
| 619 |
type, but you still have to manually take them out of the \f(CW@_\fR array |
| 620 |
yourself. Write a source filter that allows you to have a named |
| 621 |
parameter list. Such a filter would turn this: |
| 622 |
.PP |
| 623 |
.Vb 1 |
| 624 |
\& sub MySub ($first, $second, @rest) { ... } |
| 625 |
.Ve |
| 626 |
.PP |
| 627 |
into this: |
| 628 |
.PP |
| 629 |
.Vb 6 |
| 630 |
\& sub MySub($$@) { |
| 631 |
\& my ($first) = shift; |
| 632 |
\& my ($second) = shift; |
| 633 |
\& my (@rest) = @_; |
| 634 |
\& ... |
| 635 |
\& } |
| 636 |
.Ve |
| 637 |
.PP |
| 638 |
Finally, if you feel like a real challenge, have a go at writing a |
| 639 |
full-blown Perl macro preprocessor as a source filter. Borrow the |
| 640 |
useful features from the C preprocessor and any other macro processors |
| 641 |
you know. The tricky bit will be choosing how much knowledge of Perl's |
| 642 |
syntax you want your filter to have. |
| 643 |
.SH LIMITATIONS |
| 644 |
.IX Header "LIMITATIONS" |
| 645 |
Source filters only work on the string level, thus are highly limited |
| 646 |
in its ability to change source code on the fly. It cannot detect |
| 647 |
comments, quoted strings, heredocs, it is no replacement for a real |
| 648 |
parser. |
| 649 |
The only stable usage for source filters are encryption, compression, |
| 650 |
or the byteloader, to translate binary code back to source code. |
| 651 |
.PP |
| 652 |
See for example the limitations in Switch, which uses source filters, |
| 653 |
and thus is does not work inside a string eval, the presence of |
| 654 |
regexes with embedded newlines that are specified with raw \f(CW\*(C`/.../\*(C'\fR |
| 655 |
delimiters and don't have a modifier \f(CW\*(C`//x\*(C'\fR are indistinguishable from |
| 656 |
code chunks beginning with the division operator \f(CW\*(C`/\*(C'\fR. As a workaround |
| 657 |
you must use \f(CW\*(C`m/.../\*(C'\fR or \f(CW\*(C`m?...?\*(C'\fR for such patterns. Also, the presence of |
| 658 |
regexes specified with raw \f(CW\*(C`?...?\*(C'\fR delimiters may cause mysterious |
| 659 |
errors. The workaround is to use \f(CW\*(C`m?...?\*(C'\fR instead. See |
| 660 |
<https://metacpan.org/pod/Switch#LIMITATIONS>. |
| 661 |
.PP |
| 662 |
Currently the content of the \f(CW\*(C`_\|_DATA_\|_\*(C'\fR block is not filtered. |
| 663 |
.PP |
| 664 |
Currently internal buffer lengths are limited to 32\-bit only. |
| 665 |
.SH "THINGS TO LOOK OUT FOR" |
| 666 |
.IX Header "THINGS TO LOOK OUT FOR" |
| 667 |
.ie n .IP "Some Filters Clobber the ""DATA"" Handle" 5 |
| 668 |
.el .IP "Some Filters Clobber the \f(CWDATA\fR Handle" 5 |
| 669 |
.IX Item "Some Filters Clobber the DATA Handle" |
| 670 |
Some source filters use the \f(CW\*(C`DATA\*(C'\fR handle to read the calling program. |
| 671 |
When using these source filters you cannot rely on this handle, nor expect |
| 672 |
any particular kind of behavior when operating on it. Filters based on |
| 673 |
Filter::Util::Call (and therefore Filter::Simple) do not alter the \f(CW\*(C`DATA\*(C'\fR |
| 674 |
filehandle, but on the other hand totally ignore the text after \f(CW\*(C`_\|_DATA_\|_\*(C'\fR. |
| 675 |
.SH REQUIREMENTS |
| 676 |
.IX Header "REQUIREMENTS" |
| 677 |
The Source Filters distribution is available on CPAN, in |
| 678 |
.PP |
| 679 |
.Vb 1 |
| 680 |
\& CPAN/modules/by\-module/Filter |
| 681 |
.Ve |
| 682 |
.PP |
| 683 |
Starting from Perl 5.8 Filter::Util::Call (the core part of the |
| 684 |
Source Filters distribution) is part of the standard Perl distribution. |
| 685 |
Also included is a friendlier interface called Filter::Simple, by |
| 686 |
Damian Conway. |
| 687 |
.SH AUTHOR |
| 688 |
.IX Header "AUTHOR" |
| 689 |
Paul Marquess <[email protected]> |
| 690 |
.PP |
| 691 |
Reini Urban <[email protected]> |
| 692 |
.SH Copyrights |
| 693 |
.IX Header "Copyrights" |
| 694 |
The first version of this article originally appeared in The Perl |
| 695 |
Journal #11, and is copyright 1998 The Perl Journal. It appears |
| 696 |
courtesy of Jon Orwant and The Perl Journal. This document may be |
| 697 |
distributed under the same terms as Perl itself. |