| 1 |
.\" -*- mode: troff; coding: utf-8 -*- |
| 2 |
.\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45) |
| 3 |
.\" |
| 4 |
.\" Standard preamble: |
| 5 |
.\" ======================================================================== |
| 6 |
.de Sp \" Vertical space (when we can't use .PP) |
| 7 |
.if t .sp .5v |
| 8 |
.if n .sp |
| 9 |
.. |
| 10 |
.de Vb \" Begin verbatim text |
| 11 |
.ft CW |
| 12 |
.nf |
| 13 |
.ne \\$1 |
| 14 |
.. |
| 15 |
.de Ve \" End verbatim text |
| 16 |
.ft R |
| 17 |
.fi |
| 18 |
.. |
| 19 |
.\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. |
| 20 |
.ie n \{\ |
| 21 |
. ds C` "" |
| 22 |
. ds C' "" |
| 23 |
'br\} |
| 24 |
.el\{\ |
| 25 |
. ds C` |
| 26 |
. ds C' |
| 27 |
'br\} |
| 28 |
.\" |
| 29 |
.\" Escape single quotes in literal strings from groff's Unicode transform. |
| 30 |
.ie \n(.g .ds Aq \(aq |
| 31 |
.el .ds Aq ' |
| 32 |
.\" |
| 33 |
.\" If the F register is >0, we'll generate index entries on stderr for |
| 34 |
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index |
| 35 |
.\" entries marked with X<> in POD. Of course, you'll have to process the |
| 36 |
.\" output yourself in some meaningful fashion. |
| 37 |
.\" |
| 38 |
.\" Avoid warning from groff about undefined register 'F'. |
| 39 |
.de IX |
| 40 |
.. |
| 41 |
.nr rF 0 |
| 42 |
.if \n(.g .if rF .nr rF 1 |
| 43 |
.if (\n(rF:(\n(.g==0)) \{\ |
| 44 |
. if \nF \{\ |
| 45 |
. de IX |
| 46 |
. tm Index:\\$1\t\\n%\t"\\$2" |
| 47 |
.. |
| 48 |
. if !\nF==2 \{\ |
| 49 |
. nr % 0 |
| 50 |
. nr F 2 |
| 51 |
. \} |
| 52 |
. \} |
| 53 |
.\} |
| 54 |
.rr rF |
| 55 |
.\" ======================================================================== |
| 56 |
.\" |
| 57 |
.IX Title "PERLRETUT 1" |
| 58 |
.TH PERLRETUT 1 2025-11-16 "perl v5.40.1" "Perl Programmers Reference Guide" |
| 59 |
.\" For nroff, turn off justification. Always turn off hyphenation; it makes |
| 60 |
.\" way too many mistakes in technical documents. |
| 61 |
.if n .ad l |
| 62 |
.nh |
| 63 |
.SH NAME |
| 64 |
perlretut \- Perl regular expressions tutorial |
| 65 |
.SH DESCRIPTION |
| 66 |
.IX Header "DESCRIPTION" |
| 67 |
This page provides a basic tutorial on understanding, creating and |
| 68 |
using regular expressions in Perl. It serves as a complement to the |
| 69 |
reference page on regular expressions perlre. Regular expressions |
| 70 |
are an integral part of the \f(CW\*(C`m//\*(C'\fR, \f(CW\*(C`s///\*(C'\fR, \f(CW\*(C`qr//\*(C'\fR and \f(CW\*(C`split\*(C'\fR |
| 71 |
operators and so this tutorial also overlaps with |
| 72 |
"Regexp Quote-Like Operators" in perlop and "split" in perlfunc. |
| 73 |
.PP |
| 74 |
Perl is widely renowned for excellence in text processing, and regular |
| 75 |
expressions are one of the big factors behind this fame. Perl regular |
| 76 |
expressions display an efficiency and flexibility unknown in most |
| 77 |
other computer languages. Mastering even the basics of regular |
| 78 |
expressions will allow you to manipulate text with surprising ease. |
| 79 |
.PP |
| 80 |
What is a regular expression? At its most basic, a regular expression |
| 81 |
is a template that is used to determine if a string has certain |
| 82 |
characteristics. The string is most often some text, such as a line, |
| 83 |
sentence, web page, or even a whole book, but it doesn't have to be. It |
| 84 |
could be binary data, for example. Biologists often use Perl to look |
| 85 |
for patterns in long DNA sequences. |
| 86 |
.PP |
| 87 |
Suppose we want to determine if the text in variable, \f(CW$var\fR contains |
| 88 |
the sequence of characters \f(CW\*(C`m\ u\ s\ h\ r\ o\ o\ m\*(C'\fR |
| 89 |
(blanks added for legibility). We can write in Perl |
| 90 |
.PP |
| 91 |
.Vb 1 |
| 92 |
\& $var =~ m/mushroom/ |
| 93 |
.Ve |
| 94 |
.PP |
| 95 |
The value of this expression will be TRUE if \f(CW$var\fR contains that |
| 96 |
sequence of characters anywhere within it, and FALSE otherwise. The |
| 97 |
portion enclosed in \f(CW\*(Aq/\*(Aq\fR characters denotes the characteristic we |
| 98 |
are looking for. |
| 99 |
We use the term \fIpattern\fR for it. The process of looking to see if the |
| 100 |
pattern occurs in the string is called \fImatching\fR, and the \f(CW"=~"\fR |
| 101 |
operator along with the \f(CW\*(C`m//\*(C'\fR tell Perl to try to match the pattern |
| 102 |
against the string. Note that the pattern is also a string, but a very |
| 103 |
special kind of one, as we will see. Patterns are in common use these |
| 104 |
days; |
| 105 |
examples are the patterns typed into a search engine to find web pages |
| 106 |
and the patterns used to list files in a directory, \fIe.g.\fR, "\f(CW\*(C`ls *.txt\*(C'\fR" |
| 107 |
or "\f(CW\*(C`dir *.*\*(C'\fR". In Perl, the patterns described by regular expressions |
| 108 |
are used not only to search strings, but to also extract desired parts |
| 109 |
of strings, and to do search and replace operations. |
| 110 |
.PP |
| 111 |
Regular expressions have the undeserved reputation of being abstract |
| 112 |
and difficult to understand. This really stems simply because the |
| 113 |
notation used to express them tends to be terse and dense, and not |
| 114 |
because of inherent complexity. We recommend using the \f(CW\*(C`/x\*(C'\fR regular |
| 115 |
expression modifier (described below) along with plenty of white space |
| 116 |
to make them less dense, and easier to read. Regular expressions are |
| 117 |
constructed using |
| 118 |
simple concepts like conditionals and loops and are no more difficult |
| 119 |
to understand than the corresponding \f(CW\*(C`if\*(C'\fR conditionals and \f(CW\*(C`while\*(C'\fR |
| 120 |
loops in the Perl language itself. |
| 121 |
.PP |
| 122 |
This tutorial flattens the learning curve by discussing regular |
| 123 |
expression concepts, along with their notation, one at a time and with |
| 124 |
many examples. The first part of the tutorial will progress from the |
| 125 |
simplest word searches to the basic regular expression concepts. If |
| 126 |
you master the first part, you will have all the tools needed to solve |
| 127 |
about 98% of your needs. The second part of the tutorial is for those |
| 128 |
comfortable with the basics, and hungry for more power tools. It |
| 129 |
discusses the more advanced regular expression operators and |
| 130 |
introduces the latest cutting-edge innovations. |
| 131 |
.PP |
| 132 |
A note: to save time, "regular expression" is often abbreviated as |
| 133 |
regexp or regex. Regexp is a more natural abbreviation than regex, but |
| 134 |
is harder to pronounce. The Perl pod documentation is evenly split on |
| 135 |
regexp vs regex; in Perl, there is more than one way to abbreviate it. |
| 136 |
We'll use regexp in this tutorial. |
| 137 |
.PP |
| 138 |
New in v5.22, \f(CW\*(C`use re \*(Aqstrict\*(Aq\*(C'\fR applies stricter |
| 139 |
rules than otherwise when compiling regular expression patterns. It can |
| 140 |
find things that, while legal, may not be what you intended. |
| 141 |
.SH "Part 1: The basics" |
| 142 |
.IX Header "Part 1: The basics" |
| 143 |
.SS "Simple word matching" |
| 144 |
.IX Subsection "Simple word matching" |
| 145 |
The simplest regexp is simply a word, or more generally, a string of |
| 146 |
characters. A regexp consisting of just a word matches any string that |
| 147 |
contains that word: |
| 148 |
.PP |
| 149 |
.Vb 1 |
| 150 |
\& "Hello World" =~ /World/; # matches |
| 151 |
.Ve |
| 152 |
.PP |
| 153 |
What is this Perl statement all about? \f(CW"Hello World"\fR is a simple |
| 154 |
double-quoted string. \f(CW\*(C`World\*(C'\fR is the regular expression and the |
| 155 |
\&\f(CW\*(C`//\*(C'\fR enclosing \f(CW\*(C`/World/\*(C'\fR tells Perl to search a string for a match. |
| 156 |
The operator \f(CW\*(C`=~\*(C'\fR associates the string with the regexp match and |
| 157 |
produces a true value if the regexp matched, or false if the regexp |
| 158 |
did not match. In our case, \f(CW\*(C`World\*(C'\fR matches the second word in |
| 159 |
\&\f(CW"Hello World"\fR, so the expression is true. Expressions like this |
| 160 |
are useful in conditionals: |
| 161 |
.PP |
| 162 |
.Vb 6 |
| 163 |
\& if ("Hello World" =~ /World/) { |
| 164 |
\& print "It matches\en"; |
| 165 |
\& } |
| 166 |
\& else { |
| 167 |
\& print "It doesn\*(Aqt match\en"; |
| 168 |
\& } |
| 169 |
.Ve |
| 170 |
.PP |
| 171 |
There are useful variations on this theme. The sense of the match can |
| 172 |
be reversed by using the \f(CW\*(C`!~\*(C'\fR operator: |
| 173 |
.PP |
| 174 |
.Vb 6 |
| 175 |
\& if ("Hello World" !~ /World/) { |
| 176 |
\& print "It doesn\*(Aqt match\en"; |
| 177 |
\& } |
| 178 |
\& else { |
| 179 |
\& print "It matches\en"; |
| 180 |
\& } |
| 181 |
.Ve |
| 182 |
.PP |
| 183 |
The literal string in the regexp can be replaced by a variable: |
| 184 |
.PP |
| 185 |
.Vb 7 |
| 186 |
\& my $greeting = "World"; |
| 187 |
\& if ("Hello World" =~ /$greeting/) { |
| 188 |
\& print "It matches\en"; |
| 189 |
\& } |
| 190 |
\& else { |
| 191 |
\& print "It doesn\*(Aqt match\en"; |
| 192 |
\& } |
| 193 |
.Ve |
| 194 |
.PP |
| 195 |
If you're matching against the special default variable \f(CW$_\fR, the |
| 196 |
\&\f(CW\*(C`$_ =~\*(C'\fR part can be omitted: |
| 197 |
.PP |
| 198 |
.Vb 7 |
| 199 |
\& $_ = "Hello World"; |
| 200 |
\& if (/World/) { |
| 201 |
\& print "It matches\en"; |
| 202 |
\& } |
| 203 |
\& else { |
| 204 |
\& print "It doesn\*(Aqt match\en"; |
| 205 |
\& } |
| 206 |
.Ve |
| 207 |
.PP |
| 208 |
And finally, the \f(CW\*(C`//\*(C'\fR default delimiters for a match can be changed |
| 209 |
to arbitrary delimiters by putting an \f(CW\*(Aqm\*(Aq\fR out front: |
| 210 |
.PP |
| 211 |
.Vb 4 |
| 212 |
\& "Hello World" =~ m!World!; # matches, delimited by \*(Aq!\*(Aq |
| 213 |
\& "Hello World" =~ m{World}; # matches, note the paired \*(Aq{}\*(Aq |
| 214 |
\& "/usr/bin/perl" =~ m"/perl"; # matches after \*(Aq/usr/bin\*(Aq, |
| 215 |
\& # \*(Aq/\*(Aq becomes an ordinary char |
| 216 |
.Ve |
| 217 |
.PP |
| 218 |
\&\f(CW\*(C`/World/\*(C'\fR, \f(CW\*(C`m!World!\*(C'\fR, and \f(CW\*(C`m{World}\*(C'\fR all represent the |
| 219 |
same thing. When, \fIe.g.\fR, the quote (\f(CW\*(Aq"\*(Aq\fR) is used as a delimiter, the forward |
| 220 |
slash \f(CW\*(Aq/\*(Aq\fR becomes an ordinary character and can be used in this regexp |
| 221 |
without trouble. |
| 222 |
.PP |
| 223 |
Let's consider how different regexps would match \f(CW"Hello World"\fR: |
| 224 |
.PP |
| 225 |
.Vb 4 |
| 226 |
\& "Hello World" =~ /world/; # doesn\*(Aqt match |
| 227 |
\& "Hello World" =~ /o W/; # matches |
| 228 |
\& "Hello World" =~ /oW/; # doesn\*(Aqt match |
| 229 |
\& "Hello World" =~ /World /; # doesn\*(Aqt match |
| 230 |
.Ve |
| 231 |
.PP |
| 232 |
The first regexp \f(CW\*(C`world\*(C'\fR doesn't match because regexps are by default |
| 233 |
case-sensitive. The second regexp matches because the substring |
| 234 |
\&\f(CW\*(Aqo\ W\*(Aq\fR occurs in the string \f(CW"Hello\ World"\fR. The space |
| 235 |
character \f(CW\*(Aq \*(Aq\fR is treated like any other character in a regexp and is |
| 236 |
needed to match in this case. The lack of a space character is the |
| 237 |
reason the third regexp \f(CW\*(AqoW\*(Aq\fR doesn't match. The fourth regexp |
| 238 |
"\f(CW\*(C`World \*(C'\fR" doesn't match because there is a space at the end of the |
| 239 |
regexp, but not at the end of the string. The lesson here is that |
| 240 |
regexps must match a part of the string \fIexactly\fR in order for the |
| 241 |
statement to be true. |
| 242 |
.PP |
| 243 |
If a regexp matches in more than one place in the string, Perl will |
| 244 |
always match at the earliest possible point in the string: |
| 245 |
.PP |
| 246 |
.Vb 2 |
| 247 |
\& "Hello World" =~ /o/; # matches \*(Aqo\*(Aq in \*(AqHello\*(Aq |
| 248 |
\& "That hat is red" =~ /hat/; # matches \*(Aqhat\*(Aq in \*(AqThat\*(Aq |
| 249 |
.Ve |
| 250 |
.PP |
| 251 |
With respect to character matching, there are a few more points you |
| 252 |
need to know about. First of all, not all characters can be used |
| 253 |
"as-is" in a match. Some characters, called \fImetacharacters\fR, are |
| 254 |
generally reserved for use in regexp notation. The metacharacters are |
| 255 |
.PP |
| 256 |
.Vb 1 |
| 257 |
\& {}[]()^$.|*+?\-#\e |
| 258 |
.Ve |
| 259 |
.PP |
| 260 |
This list is not as definitive as it may appear (or be claimed to be in |
| 261 |
other documentation). For example, \f(CW"#"\fR is a metacharacter only when |
| 262 |
the \f(CW\*(C`/x\*(C'\fR pattern modifier (described below) is used, and both \f(CW"}"\fR |
| 263 |
and \f(CW"]"\fR are metacharacters only when paired with opening \f(CW"{"\fR or |
| 264 |
\&\f(CW"["\fR respectively; other gotchas apply. |
| 265 |
.PP |
| 266 |
The significance of each of these will be explained |
| 267 |
in the rest of the tutorial, but for now, it is important only to know |
| 268 |
that a metacharacter can be matched as-is by putting a backslash before |
| 269 |
it: |
| 270 |
.PP |
| 271 |
.Vb 5 |
| 272 |
\& "2+2=4" =~ /2+2/; # doesn\*(Aqt match, + is a metacharacter |
| 273 |
\& "2+2=4" =~ /2\e+2/; # matches, \e+ is treated like an ordinary + |
| 274 |
\& "The interval is [0,1)." =~ /[0,1)./ # is a syntax error! |
| 275 |
\& "The interval is [0,1)." =~ /\e[0,1\e)\e./ # matches |
| 276 |
\& "#!/usr/bin/perl" =~ /#!\e/usr\e/bin\e/perl/; # matches |
| 277 |
.Ve |
| 278 |
.PP |
| 279 |
In the last regexp, the forward slash \f(CW\*(Aq/\*(Aq\fR is also backslashed, |
| 280 |
because it is used to delimit the regexp. This can lead to LTS |
| 281 |
(leaning toothpick syndrome), however, and it is often more readable |
| 282 |
to change delimiters. |
| 283 |
.PP |
| 284 |
.Vb 1 |
| 285 |
\& "#!/usr/bin/perl" =~ m!#\e!/usr/bin/perl!; # easier to read |
| 286 |
.Ve |
| 287 |
.PP |
| 288 |
The backslash character \f(CW\*(Aq\e\*(Aq\fR is a metacharacter itself and needs to |
| 289 |
be backslashed: |
| 290 |
.PP |
| 291 |
.Vb 1 |
| 292 |
\& \*(AqC:\eWIN32\*(Aq =~ /C:\e\eWIN/; # matches |
| 293 |
.Ve |
| 294 |
.PP |
| 295 |
In situations where it doesn't make sense for a particular metacharacter |
| 296 |
to mean what it normally does, it automatically loses its |
| 297 |
metacharacter-ness and becomes an ordinary character that is to be |
| 298 |
matched literally. For example, the \f(CW\*(Aq}\*(Aq\fR is a metacharacter only when |
| 299 |
it is the mate of a \f(CW\*(Aq{\*(Aq\fR metacharacter. Otherwise it is treated as a |
| 300 |
literal RIGHT CURLY BRACKET. This may lead to unexpected results. |
| 301 |
\&\f(CW\*(C`use re \*(Aqstrict\*(Aq\*(C'\fR can catch some of these. |
| 302 |
.PP |
| 303 |
In addition to the metacharacters, there are some ASCII characters |
| 304 |
which don't have printable character equivalents and are instead |
| 305 |
represented by \fIescape sequences\fR. Common examples are \f(CW\*(C`\et\*(C'\fR for a |
| 306 |
tab, \f(CW\*(C`\en\*(C'\fR for a newline, \f(CW\*(C`\er\*(C'\fR for a carriage return and \f(CW\*(C`\ea\*(C'\fR for a |
| 307 |
bell (or alert). If your string is better thought of as a sequence of arbitrary |
| 308 |
bytes, the octal escape sequence, \fIe.g.\fR, \f(CW\*(C`\e033\*(C'\fR, or hexadecimal escape |
| 309 |
sequence, \fIe.g.\fR, \f(CW\*(C`\ex1B\*(C'\fR may be a more natural representation for your |
| 310 |
bytes. Here are some examples of escapes: |
| 311 |
.PP |
| 312 |
.Vb 5 |
| 313 |
\& "1000\et2000" =~ m(0\et2) # matches |
| 314 |
\& "1000\en2000" =~ /0\en20/ # matches |
| 315 |
\& "1000\et2000" =~ /\e000\et2/ # doesn\*(Aqt match, "0" ne "\e000" |
| 316 |
\& "cat" =~ /\eo{143}\ex61\ex74/ # matches in ASCII, but a weird way |
| 317 |
\& # to spell cat |
| 318 |
.Ve |
| 319 |
.PP |
| 320 |
If you've been around Perl a while, all this talk of escape sequences |
| 321 |
may seem familiar. Similar escape sequences are used in double-quoted |
| 322 |
strings and in fact the regexps in Perl are mostly treated as |
| 323 |
double-quoted strings. This means that variables can be used in |
| 324 |
regexps as well. Just like double-quoted strings, the values of the |
| 325 |
variables in the regexp will be substituted in before the regexp is |
| 326 |
evaluated for matching purposes. So we have: |
| 327 |
.PP |
| 328 |
.Vb 4 |
| 329 |
\& $foo = \*(Aqhouse\*(Aq; |
| 330 |
\& \*(Aqhousecat\*(Aq =~ /$foo/; # matches |
| 331 |
\& \*(Aqcathouse\*(Aq =~ /cat$foo/; # matches |
| 332 |
\& \*(Aqhousecat\*(Aq =~ /${foo}cat/; # matches |
| 333 |
.Ve |
| 334 |
.PP |
| 335 |
So far, so good. With the knowledge above you can already perform |
| 336 |
searches with just about any literal string regexp you can dream up. |
| 337 |
Here is a \fIvery simple\fR emulation of the Unix grep program: |
| 338 |
.PP |
| 339 |
.Vb 7 |
| 340 |
\& % cat > simple_grep |
| 341 |
\& #!/usr/bin/perl |
| 342 |
\& $regexp = shift; |
| 343 |
\& while (<>) { |
| 344 |
\& print if /$regexp/; |
| 345 |
\& } |
| 346 |
\& ^D |
| 347 |
\& |
| 348 |
\& % chmod +x simple_grep |
| 349 |
\& |
| 350 |
\& % simple_grep abba /usr/dict/words |
| 351 |
\& Babbage |
| 352 |
\& cabbage |
| 353 |
\& cabbages |
| 354 |
\& sabbath |
| 355 |
\& Sabbathize |
| 356 |
\& Sabbathizes |
| 357 |
\& sabbatical |
| 358 |
\& scabbard |
| 359 |
\& scabbards |
| 360 |
.Ve |
| 361 |
.PP |
| 362 |
This program is easy to understand. \f(CW\*(C`#!/usr/bin/perl\*(C'\fR is the standard |
| 363 |
way to invoke a perl program from the shell. |
| 364 |
\&\f(CW\*(C`$regexp\ =\ shift;\*(C'\fR saves the first command line argument as the |
| 365 |
regexp to be used, leaving the rest of the command line arguments to |
| 366 |
be treated as files. \f(CW\*(C`while\ (<>)\*(C'\fR loops over all the lines in |
| 367 |
all the files. For each line, \f(CW\*(C`print\ if\ /$regexp/;\*(C'\fR prints the |
| 368 |
line if the regexp matches the line. In this line, both \f(CW\*(C`print\*(C'\fR and |
| 369 |
\&\f(CW\*(C`/$regexp/\*(C'\fR use the default variable \f(CW$_\fR implicitly. |
| 370 |
.PP |
| 371 |
With all of the regexps above, if the regexp matched anywhere in the |
| 372 |
string, it was considered a match. Sometimes, however, we'd like to |
| 373 |
specify \fIwhere\fR in the string the regexp should try to match. To do |
| 374 |
this, we would use the \fIanchor\fR metacharacters \f(CW\*(Aq^\*(Aq\fR and \f(CW\*(Aq$\*(Aq\fR. The |
| 375 |
anchor \f(CW\*(Aq^\*(Aq\fR means match at the beginning of the string and the anchor |
| 376 |
\&\f(CW\*(Aq$\*(Aq\fR means match at the end of the string, or before a newline at the |
| 377 |
end of the string. Here is how they are used: |
| 378 |
.PP |
| 379 |
.Vb 4 |
| 380 |
\& "housekeeper" =~ /keeper/; # matches |
| 381 |
\& "housekeeper" =~ /^keeper/; # doesn\*(Aqt match |
| 382 |
\& "housekeeper" =~ /keeper$/; # matches |
| 383 |
\& "housekeeper\en" =~ /keeper$/; # matches |
| 384 |
.Ve |
| 385 |
.PP |
| 386 |
The second regexp doesn't match because \f(CW\*(Aq^\*(Aq\fR constrains \f(CW\*(C`keeper\*(C'\fR to |
| 387 |
match only at the beginning of the string, but \f(CW"housekeeper"\fR has |
| 388 |
keeper starting in the middle. The third regexp does match, since the |
| 389 |
\&\f(CW\*(Aq$\*(Aq\fR constrains \f(CW\*(C`keeper\*(C'\fR to match only at the end of the string. |
| 390 |
.PP |
| 391 |
When both \f(CW\*(Aq^\*(Aq\fR and \f(CW\*(Aq$\*(Aq\fR are used at the same time, the regexp has to |
| 392 |
match both the beginning and the end of the string, \fIi.e.\fR, the regexp |
| 393 |
matches the whole string. Consider |
| 394 |
.PP |
| 395 |
.Vb 3 |
| 396 |
\& "keeper" =~ /^keep$/; # doesn\*(Aqt match |
| 397 |
\& "keeper" =~ /^keeper$/; # matches |
| 398 |
\& "" =~ /^$/; # ^$ matches an empty string |
| 399 |
.Ve |
| 400 |
.PP |
| 401 |
The first regexp doesn't match because the string has more to it than |
| 402 |
\&\f(CW\*(C`keep\*(C'\fR. Since the second regexp is exactly the string, it |
| 403 |
matches. Using both \f(CW\*(Aq^\*(Aq\fR and \f(CW\*(Aq$\*(Aq\fR in a regexp forces the complete |
| 404 |
string to match, so it gives you complete control over which strings |
| 405 |
match and which don't. Suppose you are looking for a fellow named |
| 406 |
bert, off in a string by himself: |
| 407 |
.PP |
| 408 |
.Vb 1 |
| 409 |
\& "dogbert" =~ /bert/; # matches, but not what you want |
| 410 |
\& |
| 411 |
\& "dilbert" =~ /^bert/; # doesn\*(Aqt match, but .. |
| 412 |
\& "bertram" =~ /^bert/; # matches, so still not good enough |
| 413 |
\& |
| 414 |
\& "bertram" =~ /^bert$/; # doesn\*(Aqt match, good |
| 415 |
\& "dilbert" =~ /^bert$/; # doesn\*(Aqt match, good |
| 416 |
\& "bert" =~ /^bert$/; # matches, perfect |
| 417 |
.Ve |
| 418 |
.PP |
| 419 |
Of course, in the case of a literal string, one could just as easily |
| 420 |
use the string comparison \f(CW\*(C`$string\ eq\ \*(Aqbert\*(Aq\*(C'\fR and it would be |
| 421 |
more efficient. The \f(CW\*(C`^...$\*(C'\fR regexp really becomes useful when we |
| 422 |
add in the more powerful regexp tools below. |
| 423 |
.SS "Using character classes" |
| 424 |
.IX Subsection "Using character classes" |
| 425 |
Although one can already do quite a lot with the literal string |
| 426 |
regexps above, we've only scratched the surface of regular expression |
| 427 |
technology. In this and subsequent sections we will introduce regexp |
| 428 |
concepts (and associated metacharacter notations) that will allow a |
| 429 |
regexp to represent not just a single character sequence, but a \fIwhole |
| 430 |
class\fR of them. |
| 431 |
.PP |
| 432 |
One such concept is that of a \fIcharacter class\fR. A character class |
| 433 |
allows a set of possible characters, rather than just a single |
| 434 |
character, to match at a particular point in a regexp. You can define |
| 435 |
your own custom character classes. These |
| 436 |
are denoted by brackets \f(CW\*(C`[...]\*(C'\fR, with the set of characters |
| 437 |
to be possibly matched inside. Here are some examples: |
| 438 |
.PP |
| 439 |
.Vb 4 |
| 440 |
\& /cat/; # matches \*(Aqcat\*(Aq |
| 441 |
\& /[bcr]at/; # matches \*(Aqbat, \*(Aqcat\*(Aq, or \*(Aqrat\*(Aq |
| 442 |
\& /item[0123456789]/; # matches \*(Aqitem0\*(Aq or ... or \*(Aqitem9\*(Aq |
| 443 |
\& "abc" =~ /[cab]/; # matches \*(Aqa\*(Aq |
| 444 |
.Ve |
| 445 |
.PP |
| 446 |
In the last statement, even though \f(CW\*(Aqc\*(Aq\fR is the first character in |
| 447 |
the class, \f(CW\*(Aqa\*(Aq\fR matches because the first character position in the |
| 448 |
string is the earliest point at which the regexp can match. |
| 449 |
.PP |
| 450 |
.Vb 2 |
| 451 |
\& /[yY][eE][sS]/; # match \*(Aqyes\*(Aq in a case\-insensitive way |
| 452 |
\& # \*(Aqyes\*(Aq, \*(AqYes\*(Aq, \*(AqYES\*(Aq, etc. |
| 453 |
.Ve |
| 454 |
.PP |
| 455 |
This regexp displays a common task: perform a case-insensitive |
| 456 |
match. Perl provides a way of avoiding all those brackets by simply |
| 457 |
appending an \f(CW\*(Aqi\*(Aq\fR to the end of the match. Then \f(CW\*(C`/[yY][eE][sS]/;\*(C'\fR |
| 458 |
can be rewritten as \f(CW\*(C`/yes/i;\*(C'\fR. The \f(CW\*(Aqi\*(Aq\fR stands for |
| 459 |
case-insensitive and is an example of a \fImodifier\fR of the matching |
| 460 |
operation. We will meet other modifiers later in the tutorial. |
| 461 |
.PP |
| 462 |
We saw in the section above that there were ordinary characters, which |
| 463 |
represented themselves, and special characters, which needed a |
| 464 |
backslash \f(CW\*(Aq\e\*(Aq\fR to represent themselves. The same is true in a |
| 465 |
character class, but the sets of ordinary and special characters |
| 466 |
inside a character class are different than those outside a character |
| 467 |
class. The special characters for a character class are \f(CW\*(C`\-]\e^$\*(C'\fR (and |
| 468 |
the pattern delimiter, whatever it is). |
| 469 |
\&\f(CW\*(Aq]\*(Aq\fR is special because it denotes the end of a character class. \f(CW\*(Aq$\*(Aq\fR is |
| 470 |
special because it denotes a scalar variable. \f(CW\*(Aq\e\*(Aq\fR is special because |
| 471 |
it is used in escape sequences, just like above. Here is how the |
| 472 |
special characters \f(CW\*(C`]$\e\*(C'\fR are handled: |
| 473 |
.PP |
| 474 |
.Vb 5 |
| 475 |
\& /[\e]c]def/; # matches \*(Aq]def\*(Aq or \*(Aqcdef\*(Aq |
| 476 |
\& $x = \*(Aqbcr\*(Aq; |
| 477 |
\& /[$x]at/; # matches \*(Aqbat\*(Aq, \*(Aqcat\*(Aq, or \*(Aqrat\*(Aq |
| 478 |
\& /[\e$x]at/; # matches \*(Aq$at\*(Aq or \*(Aqxat\*(Aq |
| 479 |
\& /[\e\e$x]at/; # matches \*(Aq\eat\*(Aq, \*(Aqbat, \*(Aqcat\*(Aq, or \*(Aqrat\*(Aq |
| 480 |
.Ve |
| 481 |
.PP |
| 482 |
The last two are a little tricky. In \f(CW\*(C`[\e$x]\*(C'\fR, the backslash protects |
| 483 |
the dollar sign, so the character class has two members \f(CW\*(Aq$\*(Aq\fR and \f(CW\*(Aqx\*(Aq\fR. |
| 484 |
In \f(CW\*(C`[\e\e$x]\*(C'\fR, the backslash is protected, so \f(CW$x\fR is treated as a |
| 485 |
variable and substituted in double quote fashion. |
| 486 |
.PP |
| 487 |
The special character \f(CW\*(Aq\-\*(Aq\fR acts as a range operator within character |
| 488 |
classes, so that a contiguous set of characters can be written as a |
| 489 |
range. With ranges, the unwieldy \f(CW\*(C`[0123456789]\*(C'\fR and \f(CW\*(C`[abc...xyz]\*(C'\fR |
| 490 |
become the svelte \f(CW\*(C`[0\-9]\*(C'\fR and \f(CW\*(C`[a\-z]\*(C'\fR. Some examples are |
| 491 |
.PP |
| 492 |
.Vb 6 |
| 493 |
\& /item[0\-9]/; # matches \*(Aqitem0\*(Aq or ... or \*(Aqitem9\*(Aq |
| 494 |
\& /[0\-9bx\-z]aa/; # matches \*(Aq0aa\*(Aq, ..., \*(Aq9aa\*(Aq, |
| 495 |
\& # \*(Aqbaa\*(Aq, \*(Aqxaa\*(Aq, \*(Aqyaa\*(Aq, or \*(Aqzaa\*(Aq |
| 496 |
\& /[0\-9a\-fA\-F]/; # matches a hexadecimal digit |
| 497 |
\& /[0\-9a\-zA\-Z_]/; # matches a "word" character, |
| 498 |
\& # like those in a Perl variable name |
| 499 |
.Ve |
| 500 |
.PP |
| 501 |
If \f(CW\*(Aq\-\*(Aq\fR is the first or last character in a character class, it is |
| 502 |
treated as an ordinary character; \f(CW\*(C`[\-ab]\*(C'\fR, \f(CW\*(C`[ab\-]\*(C'\fR and \f(CW\*(C`[a\e\-b]\*(C'\fR are |
| 503 |
all equivalent. |
| 504 |
.PP |
| 505 |
The special character \f(CW\*(Aq^\*(Aq\fR in the first position of a character class |
| 506 |
denotes a \fInegated character class\fR, which matches any character but |
| 507 |
those in the brackets. Both \f(CW\*(C`[...]\*(C'\fR and \f(CW\*(C`[^...]\*(C'\fR must match a |
| 508 |
character, or the match fails. Then |
| 509 |
.PP |
| 510 |
.Vb 4 |
| 511 |
\& /[^a]at/; # doesn\*(Aqt match \*(Aqaat\*(Aq or \*(Aqat\*(Aq, but matches |
| 512 |
\& # all other \*(Aqbat\*(Aq, \*(Aqcat, \*(Aq0at\*(Aq, \*(Aq%at\*(Aq, etc. |
| 513 |
\& /[^0\-9]/; # matches a non\-numeric character |
| 514 |
\& /[a^]at/; # matches \*(Aqaat\*(Aq or \*(Aq^at\*(Aq; here \*(Aq^\*(Aq is ordinary |
| 515 |
.Ve |
| 516 |
.PP |
| 517 |
Now, even \f(CW\*(C`[0\-9]\*(C'\fR can be a bother to write multiple times, so in the |
| 518 |
interest of saving keystrokes and making regexps more readable, Perl |
| 519 |
has several abbreviations for common character classes, as shown below. |
| 520 |
Since the introduction of Unicode, unless the \f(CW\*(C`/a\*(C'\fR modifier is in |
| 521 |
effect, these character classes match more than just a few characters in |
| 522 |
the ASCII range. |
| 523 |
.IP \(bu 4 |
| 524 |
\&\f(CW\*(C`\ed\*(C'\fR matches a digit, not just \f(CW\*(C`[0\-9]\*(C'\fR but also digits from non-roman scripts |
| 525 |
.IP \(bu 4 |
| 526 |
\&\f(CW\*(C`\es\*(C'\fR matches a whitespace character, the set \f(CW\*(C`[\e \et\er\en\ef]\*(C'\fR and others |
| 527 |
.IP \(bu 4 |
| 528 |
\&\f(CW\*(C`\ew\*(C'\fR matches a word character (alphanumeric or \f(CW\*(Aq_\*(Aq\fR), not just \f(CW\*(C`[0\-9a\-zA\-Z_]\*(C'\fR |
| 529 |
but also digits and characters from non-roman scripts |
| 530 |
.IP \(bu 4 |
| 531 |
\&\f(CW\*(C`\eD\*(C'\fR is a negated \f(CW\*(C`\ed\*(C'\fR; it represents any other character than a digit, or \f(CW\*(C`[^\ed]\*(C'\fR |
| 532 |
.IP \(bu 4 |
| 533 |
\&\f(CW\*(C`\eS\*(C'\fR is a negated \f(CW\*(C`\es\*(C'\fR; it represents any non-whitespace character \f(CW\*(C`[^\es]\*(C'\fR |
| 534 |
.IP \(bu 4 |
| 535 |
\&\f(CW\*(C`\eW\*(C'\fR is a negated \f(CW\*(C`\ew\*(C'\fR; it represents any non-word character \f(CW\*(C`[^\ew]\*(C'\fR |
| 536 |
.IP \(bu 4 |
| 537 |
The period \f(CW\*(Aq.\*(Aq\fR matches any character but \f(CW"\en"\fR (unless the modifier \f(CW\*(C`/s\*(C'\fR is |
| 538 |
in effect, as explained below). |
| 539 |
.IP \(bu 4 |
| 540 |
\&\f(CW\*(C`\eN\*(C'\fR, like the period, matches any character but \f(CW"\en"\fR, but it does so |
| 541 |
regardless of whether the modifier \f(CW\*(C`/s\*(C'\fR is in effect. |
| 542 |
.PP |
| 543 |
The \f(CW\*(C`/a\*(C'\fR modifier, available starting in Perl 5.14, is used to |
| 544 |
restrict the matches of \f(CW\*(C`\ed\*(C'\fR, \f(CW\*(C`\es\*(C'\fR, and \f(CW\*(C`\ew\*(C'\fR to just those in the ASCII range. |
| 545 |
It is useful to keep your program from being needlessly exposed to full |
| 546 |
Unicode (and its accompanying security considerations) when all you want |
| 547 |
is to process English-like text. (The "a" may be doubled, \f(CW\*(C`/aa\*(C'\fR, to |
| 548 |
provide even more restrictions, preventing case-insensitive matching of |
| 549 |
ASCII with non-ASCII characters; otherwise a Unicode "Kelvin Sign" |
| 550 |
would caselessly match a "k" or "K".) |
| 551 |
.PP |
| 552 |
The \f(CW\*(C`\ed\es\ew\eD\eS\eW\*(C'\fR abbreviations can be used both inside and outside |
| 553 |
of bracketed character classes. Here are some in use: |
| 554 |
.PP |
| 555 |
.Vb 7 |
| 556 |
\& /\ed\ed:\ed\ed:\ed\ed/; # matches a hh:mm:ss time format |
| 557 |
\& /[\ed\es]/; # matches any digit or whitespace character |
| 558 |
\& /\ew\eW\ew/; # matches a word char, followed by a |
| 559 |
\& # non\-word char, followed by a word char |
| 560 |
\& /..rt/; # matches any two chars, followed by \*(Aqrt\*(Aq |
| 561 |
\& /end\e./; # matches \*(Aqend.\*(Aq |
| 562 |
\& /end[.]/; # same thing, matches \*(Aqend.\*(Aq |
| 563 |
.Ve |
| 564 |
.PP |
| 565 |
Because a period is a metacharacter, it needs to be escaped to match |
| 566 |
as an ordinary period. Because, for example, \f(CW\*(C`\ed\*(C'\fR and \f(CW\*(C`\ew\*(C'\fR are sets |
| 567 |
of characters, it is incorrect to think of \f(CW\*(C`[^\ed\ew]\*(C'\fR as \f(CW\*(C`[\eD\eW]\*(C'\fR; in |
| 568 |
fact \f(CW\*(C`[^\ed\ew]\*(C'\fR is the same as \f(CW\*(C`[^\ew]\*(C'\fR, which is the same as |
| 569 |
\&\f(CW\*(C`[\eW]\*(C'\fR. Think De Morgan's laws. |
| 570 |
.PP |
| 571 |
In actuality, the period and \f(CW\*(C`\ed\es\ew\eD\eS\eW\*(C'\fR abbreviations are |
| 572 |
themselves types of character classes, so the ones surrounded by |
| 573 |
brackets are just one type of character class. When we need to make a |
| 574 |
distinction, we refer to them as "bracketed character classes." |
| 575 |
.PP |
| 576 |
An anchor useful in basic regexps is the \fIword anchor\fR |
| 577 |
\&\f(CW\*(C`\eb\*(C'\fR. This matches a boundary between a word character and a non-word |
| 578 |
character \f(CW\*(C`\ew\eW\*(C'\fR or \f(CW\*(C`\eW\ew\*(C'\fR: |
| 579 |
.PP |
| 580 |
.Vb 5 |
| 581 |
\& $x = "Housecat catenates house and cat"; |
| 582 |
\& $x =~ /cat/; # matches cat in \*(Aqhousecat\*(Aq |
| 583 |
\& $x =~ /\ebcat/; # matches cat in \*(Aqcatenates\*(Aq |
| 584 |
\& $x =~ /cat\eb/; # matches cat in \*(Aqhousecat\*(Aq |
| 585 |
\& $x =~ /\ebcat\eb/; # matches \*(Aqcat\*(Aq at end of string |
| 586 |
.Ve |
| 587 |
.PP |
| 588 |
Note in the last example, the end of the string is considered a word |
| 589 |
boundary. |
| 590 |
.PP |
| 591 |
For natural language processing (so that, for example, apostrophes are |
| 592 |
included in words), use instead \f(CW\*(C`\eb{wb}\*(C'\fR |
| 593 |
.PP |
| 594 |
.Vb 1 |
| 595 |
\& "don\*(Aqt" =~ / .+? \eb{wb} /x; # matches the whole string |
| 596 |
.Ve |
| 597 |
.PP |
| 598 |
You might wonder why \f(CW\*(Aq.\*(Aq\fR matches everything but \f(CW"\en"\fR \- why not |
| 599 |
every character? The reason is that often one is matching against |
| 600 |
lines and would like to ignore the newline characters. For instance, |
| 601 |
while the string \f(CW"\en"\fR represents one line, we would like to think |
| 602 |
of it as empty. Then |
| 603 |
.PP |
| 604 |
.Vb 2 |
| 605 |
\& "" =~ /^$/; # matches |
| 606 |
\& "\en" =~ /^$/; # matches, $ anchors before "\en" |
| 607 |
\& |
| 608 |
\& "" =~ /./; # doesn\*(Aqt match; it needs a char |
| 609 |
\& "" =~ /^.$/; # doesn\*(Aqt match; it needs a char |
| 610 |
\& "\en" =~ /^.$/; # doesn\*(Aqt match; it needs a char other than "\en" |
| 611 |
\& "a" =~ /^.$/; # matches |
| 612 |
\& "a\en" =~ /^.$/; # matches, $ anchors before "\en" |
| 613 |
.Ve |
| 614 |
.PP |
| 615 |
This behavior is convenient, because we usually want to ignore |
| 616 |
newlines when we count and match characters in a line. Sometimes, |
| 617 |
however, we want to keep track of newlines. We might even want \f(CW\*(Aq^\*(Aq\fR |
| 618 |
and \f(CW\*(Aq$\*(Aq\fR to anchor at the beginning and end of lines within the |
| 619 |
string, rather than just the beginning and end of the string. Perl |
| 620 |
allows us to choose between ignoring and paying attention to newlines |
| 621 |
by using the \f(CW\*(C`/s\*(C'\fR and \f(CW\*(C`/m\*(C'\fR modifiers. \f(CW\*(C`/s\*(C'\fR and \f(CW\*(C`/m\*(C'\fR stand for |
| 622 |
single line and multi-line and they determine whether a string is to |
| 623 |
be treated as one continuous string, or as a set of lines. The two |
| 624 |
modifiers affect two aspects of how the regexp is interpreted: 1) how |
| 625 |
the \f(CW\*(Aq.\*(Aq\fR character class is defined, and 2) where the anchors \f(CW\*(Aq^\*(Aq\fR |
| 626 |
and \f(CW\*(Aq$\*(Aq\fR are able to match. Here are the four possible combinations: |
| 627 |
.IP \(bu 4 |
| 628 |
no modifiers: Default behavior. \f(CW\*(Aq.\*(Aq\fR matches any character |
| 629 |
except \f(CW"\en"\fR. \f(CW\*(Aq^\*(Aq\fR matches only at the beginning of the string and |
| 630 |
\&\f(CW\*(Aq$\*(Aq\fR matches only at the end or before a newline at the end. |
| 631 |
.IP \(bu 4 |
| 632 |
s modifier (\f(CW\*(C`/s\*(C'\fR): Treat string as a single long line. \f(CW\*(Aq.\*(Aq\fR matches |
| 633 |
any character, even \f(CW"\en"\fR. \f(CW\*(Aq^\*(Aq\fR matches only at the beginning of |
| 634 |
the string and \f(CW\*(Aq$\*(Aq\fR matches only at the end or before a newline at the |
| 635 |
end. |
| 636 |
.IP \(bu 4 |
| 637 |
m modifier (\f(CW\*(C`/m\*(C'\fR): Treat string as a set of multiple lines. \f(CW\*(Aq.\*(Aq\fR |
| 638 |
matches any character except \f(CW"\en"\fR. \f(CW\*(Aq^\*(Aq\fR and \f(CW\*(Aq$\*(Aq\fR are able to match |
| 639 |
at the start or end of \fIany\fR line within the string. |
| 640 |
.IP \(bu 4 |
| 641 |
both s and m modifiers (\f(CW\*(C`/sm\*(C'\fR): Treat string as a single long line, but |
| 642 |
detect multiple lines. \f(CW\*(Aq.\*(Aq\fR matches any character, even |
| 643 |
\&\f(CW"\en"\fR. \f(CW\*(Aq^\*(Aq\fR and \f(CW\*(Aq$\*(Aq\fR, however, are able to match at the start or end |
| 644 |
of \fIany\fR line within the string. |
| 645 |
.PP |
| 646 |
Here are examples of \f(CW\*(C`/s\*(C'\fR and \f(CW\*(C`/m\*(C'\fR in action: |
| 647 |
.PP |
| 648 |
.Vb 1 |
| 649 |
\& $x = "There once was a girl\enWho programmed in Perl\en"; |
| 650 |
\& |
| 651 |
\& $x =~ /^Who/; # doesn\*(Aqt match, "Who" not at start of string |
| 652 |
\& $x =~ /^Who/s; # doesn\*(Aqt match, "Who" not at start of string |
| 653 |
\& $x =~ /^Who/m; # matches, "Who" at start of second line |
| 654 |
\& $x =~ /^Who/sm; # matches, "Who" at start of second line |
| 655 |
\& |
| 656 |
\& $x =~ /girl.Who/; # doesn\*(Aqt match, "." doesn\*(Aqt match "\en" |
| 657 |
\& $x =~ /girl.Who/s; # matches, "." matches "\en" |
| 658 |
\& $x =~ /girl.Who/m; # doesn\*(Aqt match, "." doesn\*(Aqt match "\en" |
| 659 |
\& $x =~ /girl.Who/sm; # matches, "." matches "\en" |
| 660 |
.Ve |
| 661 |
.PP |
| 662 |
Most of the time, the default behavior is what is wanted, but \f(CW\*(C`/s\*(C'\fR and |
| 663 |
\&\f(CW\*(C`/m\*(C'\fR are occasionally very useful. If \f(CW\*(C`/m\*(C'\fR is being used, the start |
| 664 |
of the string can still be matched with \f(CW\*(C`\eA\*(C'\fR and the end of the string |
| 665 |
can still be matched with the anchors \f(CW\*(C`\eZ\*(C'\fR (matches both the end and |
| 666 |
the newline before, like \f(CW\*(Aq$\*(Aq\fR), and \f(CW\*(C`\ez\*(C'\fR (matches only the end): |
| 667 |
.PP |
| 668 |
.Vb 2 |
| 669 |
\& $x =~ /^Who/m; # matches, "Who" at start of second line |
| 670 |
\& $x =~ /\eAWho/m; # doesn\*(Aqt match, "Who" is not at start of string |
| 671 |
\& |
| 672 |
\& $x =~ /girl$/m; # matches, "girl" at end of first line |
| 673 |
\& $x =~ /girl\eZ/m; # doesn\*(Aqt match, "girl" is not at end of string |
| 674 |
\& |
| 675 |
\& $x =~ /Perl\eZ/m; # matches, "Perl" is at newline before end |
| 676 |
\& $x =~ /Perl\ez/m; # doesn\*(Aqt match, "Perl" is not at end of string |
| 677 |
.Ve |
| 678 |
.PP |
| 679 |
We now know how to create choices among classes of characters in a |
| 680 |
regexp. What about choices among words or character strings? Such |
| 681 |
choices are described in the next section. |
| 682 |
.SS "Matching this or that" |
| 683 |
.IX Subsection "Matching this or that" |
| 684 |
Sometimes we would like our regexp to be able to match different |
| 685 |
possible words or character strings. This is accomplished by using |
| 686 |
the \fIalternation\fR metacharacter \f(CW\*(Aq|\*(Aq\fR. To match \f(CW\*(C`dog\*(C'\fR or \f(CW\*(C`cat\*(C'\fR, we |
| 687 |
form the regexp \f(CW\*(C`dog|cat\*(C'\fR. As before, Perl will try to match the |
| 688 |
regexp at the earliest possible point in the string. At each |
| 689 |
character position, Perl will first try to match the first |
| 690 |
alternative, \f(CW\*(C`dog\*(C'\fR. If \f(CW\*(C`dog\*(C'\fR doesn't match, Perl will then try the |
| 691 |
next alternative, \f(CW\*(C`cat\*(C'\fR. If \f(CW\*(C`cat\*(C'\fR doesn't match either, then the |
| 692 |
match fails and Perl moves to the next position in the string. Some |
| 693 |
examples: |
| 694 |
.PP |
| 695 |
.Vb 2 |
| 696 |
\& "cats and dogs" =~ /cat|dog|bird/; # matches "cat" |
| 697 |
\& "cats and dogs" =~ /dog|cat|bird/; # matches "cat" |
| 698 |
.Ve |
| 699 |
.PP |
| 700 |
Even though \f(CW\*(C`dog\*(C'\fR is the first alternative in the second regexp, |
| 701 |
\&\f(CW\*(C`cat\*(C'\fR is able to match earlier in the string. |
| 702 |
.PP |
| 703 |
.Vb 2 |
| 704 |
\& "cats" =~ /c|ca|cat|cats/; # matches "c" |
| 705 |
\& "cats" =~ /cats|cat|ca|c/; # matches "cats" |
| 706 |
.Ve |
| 707 |
.PP |
| 708 |
Here, all the alternatives match at the first string position, so the |
| 709 |
first alternative is the one that matches. If some of the |
| 710 |
alternatives are truncations of the others, put the longest ones first |
| 711 |
to give them a chance to match. |
| 712 |
.PP |
| 713 |
.Vb 2 |
| 714 |
\& "cab" =~ /a|b|c/ # matches "c" |
| 715 |
\& # /a|b|c/ == /[abc]/ |
| 716 |
.Ve |
| 717 |
.PP |
| 718 |
The last example points out that character classes are like |
| 719 |
alternations of characters. At a given character position, the first |
| 720 |
alternative that allows the regexp match to succeed will be the one |
| 721 |
that matches. |
| 722 |
.SS "Grouping things and hierarchical matching" |
| 723 |
.IX Subsection "Grouping things and hierarchical matching" |
| 724 |
Alternation allows a regexp to choose among alternatives, but by |
| 725 |
itself it is unsatisfying. The reason is that each alternative is a whole |
| 726 |
regexp, but sometime we want alternatives for just part of a |
| 727 |
regexp. For instance, suppose we want to search for housecats or |
| 728 |
housekeepers. The regexp \f(CW\*(C`housecat|housekeeper\*(C'\fR fits the bill, but is |
| 729 |
inefficient because we had to type \f(CW\*(C`house\*(C'\fR twice. It would be nice to |
| 730 |
have parts of the regexp be constant, like \f(CW\*(C`house\*(C'\fR, and some |
| 731 |
parts have alternatives, like \f(CW\*(C`cat|keeper\*(C'\fR. |
| 732 |
.PP |
| 733 |
The \fIgrouping\fR metacharacters \f(CW\*(C`()\*(C'\fR solve this problem. Grouping |
| 734 |
allows parts of a regexp to be treated as a single unit. Parts of a |
| 735 |
regexp are grouped by enclosing them in parentheses. Thus we could solve |
| 736 |
the \f(CW\*(C`housecat|housekeeper\*(C'\fR by forming the regexp as |
| 737 |
\&\f(CWhouse(cat|keeper)\fR. The regexp \f(CWhouse(cat|keeper)\fR means match |
| 738 |
\&\f(CW\*(C`house\*(C'\fR followed by either \f(CW\*(C`cat\*(C'\fR or \f(CW\*(C`keeper\*(C'\fR. Some more examples |
| 739 |
are |
| 740 |
.PP |
| 741 |
.Vb 4 |
| 742 |
\& /(a|b)b/; # matches \*(Aqab\*(Aq or \*(Aqbb\*(Aq |
| 743 |
\& /(ac|b)b/; # matches \*(Aqacb\*(Aq or \*(Aqbb\*(Aq |
| 744 |
\& /(^a|b)c/; # matches \*(Aqac\*(Aq at start of string or \*(Aqbc\*(Aq anywhere |
| 745 |
\& /(a|[bc])d/; # matches \*(Aqad\*(Aq, \*(Aqbd\*(Aq, or \*(Aqcd\*(Aq |
| 746 |
\& |
| 747 |
\& /house(cat|)/; # matches either \*(Aqhousecat\*(Aq or \*(Aqhouse\*(Aq |
| 748 |
\& /house(cat(s|)|)/; # matches either \*(Aqhousecats\*(Aq or \*(Aqhousecat\*(Aq or |
| 749 |
\& # \*(Aqhouse\*(Aq. Note groups can be nested. |
| 750 |
\& |
| 751 |
\& /(19|20|)\ed\ed/; # match years 19xx, 20xx, or the Y2K problem, xx |
| 752 |
\& "20" =~ /(19|20|)\ed\ed/; # matches the null alternative \*(Aq()\ed\ed\*(Aq, |
| 753 |
\& # because \*(Aq20\ed\ed\*(Aq can\*(Aqt match |
| 754 |
.Ve |
| 755 |
.PP |
| 756 |
Alternations behave the same way in groups as out of them: at a given |
| 757 |
string position, the leftmost alternative that allows the regexp to |
| 758 |
match is taken. So in the last example at the first string position, |
| 759 |
\&\f(CW"20"\fR matches the second alternative, but there is nothing left over |
| 760 |
to match the next two digits \f(CW\*(C`\ed\ed\*(C'\fR. So Perl moves on to the next |
| 761 |
alternative, which is the null alternative and that works, since |
| 762 |
\&\f(CW"20"\fR is two digits. |
| 763 |
.PP |
| 764 |
The process of trying one alternative, seeing if it matches, and |
| 765 |
moving on to the next alternative, while going back in the string |
| 766 |
from where the previous alternative was tried, if it doesn't, is called |
| 767 |
\&\fIbacktracking\fR. The term "backtracking" comes from the idea that |
| 768 |
matching a regexp is like a walk in the woods. Successfully matching |
| 769 |
a regexp is like arriving at a destination. There are many possible |
| 770 |
trailheads, one for each string position, and each one is tried in |
| 771 |
order, left to right. From each trailhead there may be many paths, |
| 772 |
some of which get you there, and some which are dead ends. When you |
| 773 |
walk along a trail and hit a dead end, you have to backtrack along the |
| 774 |
trail to an earlier point to try another trail. If you hit your |
| 775 |
destination, you stop immediately and forget about trying all the |
| 776 |
other trails. You are persistent, and only if you have tried all the |
| 777 |
trails from all the trailheads and not arrived at your destination, do |
| 778 |
you declare failure. To be concrete, here is a step-by-step analysis |
| 779 |
of what Perl does when it tries to match the regexp |
| 780 |
.PP |
| 781 |
.Vb 1 |
| 782 |
\& "abcde" =~ /(abd|abc)(df|d|de)/; |
| 783 |
.Ve |
| 784 |
.IP 1. 4 |
| 785 |
Start with the first letter in the string \f(CW\*(Aqa\*(Aq\fR. |
| 786 |
.IP 2. 4 |
| 787 |
Try the first alternative in the first group \f(CW\*(Aqabd\*(Aq\fR. |
| 788 |
.IP 3. 4 |
| 789 |
Match \f(CW\*(Aqa\*(Aq\fR followed by \f(CW\*(Aqb\*(Aq\fR. So far so good. |
| 790 |
.IP 4. 4 |
| 791 |
\&\f(CW\*(Aqd\*(Aq\fR in the regexp doesn't match \f(CW\*(Aqc\*(Aq\fR in the string \- a |
| 792 |
dead end. So backtrack two characters and pick the second alternative |
| 793 |
in the first group \f(CW\*(Aqabc\*(Aq\fR. |
| 794 |
.IP 5. 4 |
| 795 |
Match \f(CW\*(Aqa\*(Aq\fR followed by \f(CW\*(Aqb\*(Aq\fR followed by \f(CW\*(Aqc\*(Aq\fR. We are on a roll |
| 796 |
and have satisfied the first group. Set \f(CW$1\fR to \f(CW\*(Aqabc\*(Aq\fR. |
| 797 |
.IP 6. 4 |
| 798 |
Move on to the second group and pick the first alternative \f(CW\*(Aqdf\*(Aq\fR. |
| 799 |
.IP 7. 4 |
| 800 |
Match the \f(CW\*(Aqd\*(Aq\fR. |
| 801 |
.IP 8. 4 |
| 802 |
\&\f(CW\*(Aqf\*(Aq\fR in the regexp doesn't match \f(CW\*(Aqe\*(Aq\fR in the string, so a dead |
| 803 |
end. Backtrack one character and pick the second alternative in the |
| 804 |
second group \f(CW\*(Aqd\*(Aq\fR. |
| 805 |
.IP 9. 4 |
| 806 |
\&\f(CW\*(Aqd\*(Aq\fR matches. The second grouping is satisfied, so set |
| 807 |
\&\f(CW$2\fR to \f(CW\*(Aqd\*(Aq\fR. |
| 808 |
.IP 10. 4 |
| 809 |
We are at the end of the regexp, so we are done! We have |
| 810 |
matched \f(CW\*(Aqabcd\*(Aq\fR out of the string \f(CW"abcde"\fR. |
| 811 |
.PP |
| 812 |
There are a couple of things to note about this analysis. First, the |
| 813 |
third alternative in the second group \f(CW\*(Aqde\*(Aq\fR also allows a match, but we |
| 814 |
stopped before we got to it \- at a given character position, leftmost |
| 815 |
wins. Second, we were able to get a match at the first character |
| 816 |
position of the string \f(CW\*(Aqa\*(Aq\fR. If there were no matches at the first |
| 817 |
position, Perl would move to the second character position \f(CW\*(Aqb\*(Aq\fR and |
| 818 |
attempt the match all over again. Only when all possible paths at all |
| 819 |
possible character positions have been exhausted does Perl give |
| 820 |
up and declare \f(CW\*(C`$string\ =~\ /(abd|abc)(df|d|de)/;\*(C'\fR to be false. |
| 821 |
.PP |
| 822 |
Even with all this work, regexp matching happens remarkably fast. To |
| 823 |
speed things up, Perl compiles the regexp into a compact sequence of |
| 824 |
opcodes that can often fit inside a processor cache. When the code is |
| 825 |
executed, these opcodes can then run at full throttle and search very |
| 826 |
quickly. |
| 827 |
.SS "Extracting matches" |
| 828 |
.IX Subsection "Extracting matches" |
| 829 |
The grouping metacharacters \f(CW\*(C`()\*(C'\fR also serve another completely |
| 830 |
different function: they allow the extraction of the parts of a string |
| 831 |
that matched. This is very useful to find out what matched and for |
| 832 |
text processing in general. For each grouping, the part that matched |
| 833 |
inside goes into the special variables \f(CW$1\fR, \f(CW$2\fR, \fIetc\fR. They can be |
| 834 |
used just as ordinary variables: |
| 835 |
.PP |
| 836 |
.Vb 6 |
| 837 |
\& # extract hours, minutes, seconds |
| 838 |
\& if ($time =~ /(\ed\ed):(\ed\ed):(\ed\ed)/) { # match hh:mm:ss format |
| 839 |
\& $hours = $1; |
| 840 |
\& $minutes = $2; |
| 841 |
\& $seconds = $3; |
| 842 |
\& } |
| 843 |
.Ve |
| 844 |
.PP |
| 845 |
Now, we know that in scalar context, |
| 846 |
\&\f(CW\*(C`$time\ =~\ /(\ed\ed):(\ed\ed):(\ed\ed)/\*(C'\fR returns a true or false |
| 847 |
value. In list context, however, it returns the list of matched values |
| 848 |
\&\f(CW\*(C`($1,$2,$3)\*(C'\fR. So we could write the code more compactly as |
| 849 |
.PP |
| 850 |
.Vb 2 |
| 851 |
\& # extract hours, minutes, seconds |
| 852 |
\& ($hours, $minutes, $second) = ($time =~ /(\ed\ed):(\ed\ed):(\ed\ed)/); |
| 853 |
.Ve |
| 854 |
.PP |
| 855 |
If the groupings in a regexp are nested, \f(CW$1\fR gets the group with the |
| 856 |
leftmost opening parenthesis, \f(CW$2\fR the next opening parenthesis, |
| 857 |
\&\fIetc\fR. Here is a regexp with nested groups: |
| 858 |
.PP |
| 859 |
.Vb 2 |
| 860 |
\& /(ab(cd|ef)((gi)|j))/; |
| 861 |
\& 1 2 34 |
| 862 |
.Ve |
| 863 |
.PP |
| 864 |
If this regexp matches, \f(CW$1\fR contains a string starting with |
| 865 |
\&\f(CW\*(Aqab\*(Aq\fR, \f(CW$2\fR is either set to \f(CW\*(Aqcd\*(Aq\fR or \f(CW\*(Aqef\*(Aq\fR, \f(CW$3\fR equals either |
| 866 |
\&\f(CW\*(Aqgi\*(Aq\fR or \f(CW\*(Aqj\*(Aq\fR, and \f(CW$4\fR is either set to \f(CW\*(Aqgi\*(Aq\fR, just like \f(CW$3\fR, |
| 867 |
or it remains undefined. |
| 868 |
.PP |
| 869 |
For convenience, Perl sets \f(CW$+\fR to the string held by the highest numbered |
| 870 |
\&\f(CW$1\fR, \f(CW$2\fR,... that got assigned (and, somewhat related, \f(CW$^N\fR to the |
| 871 |
value of the \f(CW$1\fR, \f(CW$2\fR,... most-recently assigned; \fIi.e.\fR the \f(CW$1\fR, |
| 872 |
\&\f(CW$2\fR,... associated with the rightmost closing parenthesis used in the |
| 873 |
match). |
| 874 |
.SS Backreferences |
| 875 |
.IX Subsection "Backreferences" |
| 876 |
Closely associated with the matching variables \f(CW$1\fR, \f(CW$2\fR, ... are |
| 877 |
the \fIbackreferences\fR \f(CW\*(C`\eg1\*(C'\fR, \f(CW\*(C`\eg2\*(C'\fR,... Backreferences are simply |
| 878 |
matching variables that can be used \fIinside\fR a regexp. This is a |
| 879 |
really nice feature; what matches later in a regexp is made to depend on |
| 880 |
what matched earlier in the regexp. Suppose we wanted to look |
| 881 |
for doubled words in a text, like "the the". The following regexp finds |
| 882 |
all 3\-letter doubles with a space in between: |
| 883 |
.PP |
| 884 |
.Vb 1 |
| 885 |
\& /\eb(\ew\ew\ew)\es\eg1\eb/; |
| 886 |
.Ve |
| 887 |
.PP |
| 888 |
The grouping assigns a value to \f(CW\*(C`\eg1\*(C'\fR, so that the same 3\-letter sequence |
| 889 |
is used for both parts. |
| 890 |
.PP |
| 891 |
A similar task is to find words consisting of two identical parts: |
| 892 |
.PP |
| 893 |
.Vb 7 |
| 894 |
\& % simple_grep \*(Aq^(\ew\ew\ew\ew|\ew\ew\ew|\ew\ew|\ew)\eg1$\*(Aq /usr/dict/words |
| 895 |
\& beriberi |
| 896 |
\& booboo |
| 897 |
\& coco |
| 898 |
\& mama |
| 899 |
\& murmur |
| 900 |
\& papa |
| 901 |
.Ve |
| 902 |
.PP |
| 903 |
The regexp has a single grouping which considers 4\-letter |
| 904 |
combinations, then 3\-letter combinations, \fIetc\fR., and uses \f(CW\*(C`\eg1\*(C'\fR to look for |
| 905 |
a repeat. Although \f(CW$1\fR and \f(CW\*(C`\eg1\*(C'\fR represent the same thing, care should be |
| 906 |
taken to use matched variables \f(CW$1\fR, \f(CW$2\fR,... only \fIoutside\fR a regexp |
| 907 |
and backreferences \f(CW\*(C`\eg1\*(C'\fR, \f(CW\*(C`\eg2\*(C'\fR,... only \fIinside\fR a regexp; not doing |
| 908 |
so may lead to surprising and unsatisfactory results. |
| 909 |
.SS "Relative backreferences" |
| 910 |
.IX Subsection "Relative backreferences" |
| 911 |
Counting the opening parentheses to get the correct number for a |
| 912 |
backreference is error-prone as soon as there is more than one |
| 913 |
capturing group. A more convenient technique became available |
| 914 |
with Perl 5.10: relative backreferences. To refer to the immediately |
| 915 |
preceding capture group one now may write \f(CW\*(C`\eg\-1\*(C'\fR or \f(CW\*(C`\eg{\-1}\*(C'\fR, the next but |
| 916 |
last is available via \f(CW\*(C`\eg\-2\*(C'\fR or \f(CW\*(C`\eg{\-2}\*(C'\fR, and so on. |
| 917 |
.PP |
| 918 |
Another good reason in addition to readability and maintainability |
| 919 |
for using relative backreferences is illustrated by the following example, |
| 920 |
where a simple pattern for matching peculiar strings is used: |
| 921 |
.PP |
| 922 |
.Vb 1 |
| 923 |
\& $a99a = \*(Aq([a\-z])(\ed)\eg2\eg1\*(Aq; # matches a11a, g22g, x33x, etc. |
| 924 |
.Ve |
| 925 |
.PP |
| 926 |
Now that we have this pattern stored as a handy string, we might feel |
| 927 |
tempted to use it as a part of some other pattern: |
| 928 |
.PP |
| 929 |
.Vb 6 |
| 930 |
\& $line = "code=e99e"; |
| 931 |
\& if ($line =~ /^(\ew+)=$a99a$/){ # unexpected behavior! |
| 932 |
\& print "$1 is valid\en"; |
| 933 |
\& } else { |
| 934 |
\& print "bad line: \*(Aq$line\*(Aq\en"; |
| 935 |
\& } |
| 936 |
.Ve |
| 937 |
.PP |
| 938 |
But this doesn't match, at least not the way one might expect. Only |
| 939 |
after inserting the interpolated \f(CW$a99a\fR and looking at the resulting |
| 940 |
full text of the regexp is it obvious that the backreferences have |
| 941 |
backfired. The subexpression \f(CW\*(C`(\ew+)\*(C'\fR has snatched number 1 and |
| 942 |
demoted the groups in \f(CW$a99a\fR by one rank. This can be avoided by |
| 943 |
using relative backreferences: |
| 944 |
.PP |
| 945 |
.Vb 1 |
| 946 |
\& $a99a = \*(Aq([a\-z])(\ed)\eg{\-1}\eg{\-2}\*(Aq; # safe for being interpolated |
| 947 |
.Ve |
| 948 |
.SS "Named backreferences" |
| 949 |
.IX Subsection "Named backreferences" |
| 950 |
Perl 5.10 also introduced named capture groups and named backreferences. |
| 951 |
To attach a name to a capturing group, you write either |
| 952 |
\&\f(CW\*(C`(?<name>...)\*(C'\fR or \f(CW\*(C`(?\*(Aqname\*(Aq...)\*(C'\fR. The backreference may |
| 953 |
then be written as \f(CW\*(C`\eg{name}\*(C'\fR. It is permissible to attach the |
| 954 |
same name to more than one group, but then only the leftmost one of the |
| 955 |
eponymous set can be referenced. Outside of the pattern a named |
| 956 |
capture group is accessible through the \f(CW\*(C`%+\*(C'\fR hash. |
| 957 |
.PP |
| 958 |
Assuming that we have to match calendar dates which may be given in one |
| 959 |
of the three formats yyyy-mm-dd, mm/dd/yyyy or dd.mm.yyyy, we can write |
| 960 |
three suitable patterns where we use \f(CW\*(Aqd\*(Aq\fR, \f(CW\*(Aqm\*(Aq\fR and \f(CW\*(Aqy\*(Aq\fR respectively as the |
| 961 |
names of the groups capturing the pertaining components of a date. The |
| 962 |
matching operation combines the three patterns as alternatives: |
| 963 |
.PP |
| 964 |
.Vb 8 |
| 965 |
\& $fmt1 = \*(Aq(?<y>\ed\ed\ed\ed)\-(?<m>\ed\ed)\-(?<d>\ed\ed)\*(Aq; |
| 966 |
\& $fmt2 = \*(Aq(?<m>\ed\ed)/(?<d>\ed\ed)/(?<y>\ed\ed\ed\ed)\*(Aq; |
| 967 |
\& $fmt3 = \*(Aq(?<d>\ed\ed)\e.(?<m>\ed\ed)\e.(?<y>\ed\ed\ed\ed)\*(Aq; |
| 968 |
\& for my $d (qw(2006\-10\-21 15.01.2007 10/31/2005)) { |
| 969 |
\& if ( $d =~ m{$fmt1|$fmt2|$fmt3} ){ |
| 970 |
\& print "day=$+{d} month=$+{m} year=$+{y}\en"; |
| 971 |
\& } |
| 972 |
\& } |
| 973 |
.Ve |
| 974 |
.PP |
| 975 |
If any of the alternatives matches, the hash \f(CW\*(C`%+\*(C'\fR is bound to contain the |
| 976 |
three key-value pairs. |
| 977 |
.SS "Alternative capture group numbering" |
| 978 |
.IX Subsection "Alternative capture group numbering" |
| 979 |
Yet another capturing group numbering technique (also as from Perl 5.10) |
| 980 |
deals with the problem of referring to groups within a set of alternatives. |
| 981 |
Consider a pattern for matching a time of the day, civil or military style: |
| 982 |
.PP |
| 983 |
.Vb 3 |
| 984 |
\& if ( $time =~ /(\ed\ed|\ed):(\ed\ed)|(\ed\ed)(\ed\ed)/ ){ |
| 985 |
\& # process hour and minute |
| 986 |
\& } |
| 987 |
.Ve |
| 988 |
.PP |
| 989 |
Processing the results requires an additional if statement to determine |
| 990 |
whether \f(CW$1\fR and \f(CW$2\fR or \f(CW$3\fR and \f(CW$4\fR contain the goodies. It would |
| 991 |
be easier if we could use group numbers 1 and 2 in second alternative as |
| 992 |
well, and this is exactly what the parenthesized construct \f(CW\*(C`(?|...)\*(C'\fR, |
| 993 |
set around an alternative achieves. Here is an extended version of the |
| 994 |
previous pattern: |
| 995 |
.PP |
| 996 |
.Vb 3 |
| 997 |
\& if($time =~ /(?|(\ed\ed|\ed):(\ed\ed)|(\ed\ed)(\ed\ed))\es+([A\-Z][A\-Z][A\-Z])/){ |
| 998 |
\& print "hour=$1 minute=$2 zone=$3\en"; |
| 999 |
\& } |
| 1000 |
.Ve |
| 1001 |
.PP |
| 1002 |
Within the alternative numbering group, group numbers start at the same |
| 1003 |
position for each alternative. After the group, numbering continues |
| 1004 |
with one higher than the maximum reached across all the alternatives. |
| 1005 |
.SS "Position information" |
| 1006 |
.IX Subsection "Position information" |
| 1007 |
In addition to what was matched, Perl also provides the |
| 1008 |
positions of what was matched as contents of the \f(CW\*(C`@\-\*(C'\fR and \f(CW\*(C`@+\*(C'\fR |
| 1009 |
arrays. \f(CW\*(C`$\-[0]\*(C'\fR is the position of the start of the entire match and |
| 1010 |
\&\f(CW$+[0]\fR is the position of the end. Similarly, \f(CW\*(C`$\-[n]\*(C'\fR is the |
| 1011 |
position of the start of the \f(CW$n\fR match and \f(CW$+[n]\fR is the position |
| 1012 |
of the end. If \f(CW$n\fR is undefined, so are \f(CW\*(C`$\-[n]\*(C'\fR and \f(CW$+[n]\fR. Then |
| 1013 |
this code |
| 1014 |
.PP |
| 1015 |
.Vb 6 |
| 1016 |
\& $x = "Mmm...donut, thought Homer"; |
| 1017 |
\& $x =~ /^(Mmm|Yech)\e.\e.\e.(donut|peas)/; # matches |
| 1018 |
\& foreach $exp (1..$#\-) { |
| 1019 |
\& no strict \*(Aqrefs\*(Aq; |
| 1020 |
\& print "Match $exp: \*(Aq$$exp\*(Aq at position ($\-[$exp],$+[$exp])\en"; |
| 1021 |
\& } |
| 1022 |
.Ve |
| 1023 |
.PP |
| 1024 |
prints |
| 1025 |
.PP |
| 1026 |
.Vb 2 |
| 1027 |
\& Match 1: \*(AqMmm\*(Aq at position (0,3) |
| 1028 |
\& Match 2: \*(Aqdonut\*(Aq at position (6,11) |
| 1029 |
.Ve |
| 1030 |
.PP |
| 1031 |
Even if there are no groupings in a regexp, it is still possible to |
| 1032 |
find out what exactly matched in a string. If you use them, Perl |
| 1033 |
will set \f(CW\*(C`$\`\*(C'\fR to the part of the string before the match, will set \f(CW$&\fR |
| 1034 |
to the part of the string that matched, and will set \f(CW\*(C`$\*(Aq\*(C'\fR to the part |
| 1035 |
of the string after the match. An example: |
| 1036 |
.PP |
| 1037 |
.Vb 3 |
| 1038 |
\& $x = "the cat caught the mouse"; |
| 1039 |
\& $x =~ /cat/; # $\` = \*(Aqthe \*(Aq, $& = \*(Aqcat\*(Aq, $\*(Aq = \*(Aq caught the mouse\*(Aq |
| 1040 |
\& $x =~ /the/; # $\` = \*(Aq\*(Aq, $& = \*(Aqthe\*(Aq, $\*(Aq = \*(Aq cat caught the mouse\*(Aq |
| 1041 |
.Ve |
| 1042 |
.PP |
| 1043 |
In the second match, \f(CW\*(C`$\`\*(C'\fR equals \f(CW\*(Aq\*(Aq\fR because the regexp matched at the |
| 1044 |
first character position in the string and stopped; it never saw the |
| 1045 |
second "the". |
| 1046 |
.PP |
| 1047 |
If your code is to run on Perl versions earlier than |
| 1048 |
5.20, it is worthwhile to note that using \f(CW\*(C`$\`\*(C'\fR and \f(CW\*(C`$\*(Aq\*(C'\fR |
| 1049 |
slows down regexp matching quite a bit, while \f(CW$&\fR slows it down to a |
| 1050 |
lesser extent, because if they are used in one regexp in a program, |
| 1051 |
they are generated for \fIall\fR regexps in the program. So if raw |
| 1052 |
performance is a goal of your application, they should be avoided. |
| 1053 |
If you need to extract the corresponding substrings, use \f(CW\*(C`@\-\*(C'\fR and |
| 1054 |
\&\f(CW\*(C`@+\*(C'\fR instead: |
| 1055 |
.PP |
| 1056 |
.Vb 3 |
| 1057 |
\& $\` is the same as substr( $x, 0, $\-[0] ) |
| 1058 |
\& $& is the same as substr( $x, $\-[0], $+[0]\-$\-[0] ) |
| 1059 |
\& $\*(Aq is the same as substr( $x, $+[0] ) |
| 1060 |
.Ve |
| 1061 |
.PP |
| 1062 |
As of Perl 5.10, the \f(CW\*(C`${^PREMATCH}\*(C'\fR, \f(CW\*(C`${^MATCH}\*(C'\fR and \f(CW\*(C`${^POSTMATCH}\*(C'\fR |
| 1063 |
variables may be used. These are only set if the \f(CW\*(C`/p\*(C'\fR modifier is |
| 1064 |
present. Consequently they do not penalize the rest of the program. In |
| 1065 |
Perl 5.20, \f(CW\*(C`${^PREMATCH}\*(C'\fR, \f(CW\*(C`${^MATCH}\*(C'\fR and \f(CW\*(C`${^POSTMATCH}\*(C'\fR are available |
| 1066 |
whether the \f(CW\*(C`/p\*(C'\fR has been used or not (the modifier is ignored), and |
| 1067 |
\&\f(CW\*(C`$\`\*(C'\fR, \f(CW\*(C`$\*(Aq\*(C'\fR and \f(CW$&\fR do not cause any speed difference. |
| 1068 |
.SS "Non-capturing groupings" |
| 1069 |
.IX Subsection "Non-capturing groupings" |
| 1070 |
A group that is required to bundle a set of alternatives may or may not be |
| 1071 |
useful as a capturing group. If it isn't, it just creates a superfluous |
| 1072 |
addition to the set of available capture group values, inside as well as |
| 1073 |
outside the regexp. Non-capturing groupings, denoted by \f(CW\*(C`(?:regexp)\*(C'\fR, |
| 1074 |
still allow the regexp to be treated as a single unit, but don't establish |
| 1075 |
a capturing group at the same time. Both capturing and non-capturing |
| 1076 |
groupings are allowed to co-exist in the same regexp. Because there is |
| 1077 |
no extraction, non-capturing groupings are faster than capturing |
| 1078 |
groupings. Non-capturing groupings are also handy for choosing exactly |
| 1079 |
which parts of a regexp are to be extracted to matching variables: |
| 1080 |
.PP |
| 1081 |
.Vb 2 |
| 1082 |
\& # match a number, $1\-$4 are set, but we only want $1 |
| 1083 |
\& /([+\-]?\e *(\ed+(\e.\ed*)?|\e.\ed+)([eE][+\-]?\ed+)?)/; |
| 1084 |
\& |
| 1085 |
\& # match a number faster , only $1 is set |
| 1086 |
\& /([+\-]?\e *(?:\ed+(?:\e.\ed*)?|\e.\ed+)(?:[eE][+\-]?\ed+)?)/; |
| 1087 |
\& |
| 1088 |
\& # match a number, get $1 = whole number, $2 = exponent |
| 1089 |
\& /([+\-]?\e *(?:\ed+(?:\e.\ed*)?|\e.\ed+)(?:[eE]([+\-]?\ed+))?)/; |
| 1090 |
.Ve |
| 1091 |
.PP |
| 1092 |
Non-capturing groupings are also useful for removing nuisance |
| 1093 |
elements gathered from a split operation where parentheses are |
| 1094 |
required for some reason: |
| 1095 |
.PP |
| 1096 |
.Vb 3 |
| 1097 |
\& $x = \*(Aq12aba34ba5\*(Aq; |
| 1098 |
\& @num = split /(a|b)+/, $x; # @num = (\*(Aq12\*(Aq,\*(Aqa\*(Aq,\*(Aq34\*(Aq,\*(Aqa\*(Aq,\*(Aq5\*(Aq) |
| 1099 |
\& @num = split /(?:a|b)+/, $x; # @num = (\*(Aq12\*(Aq,\*(Aq34\*(Aq,\*(Aq5\*(Aq) |
| 1100 |
.Ve |
| 1101 |
.PP |
| 1102 |
In Perl 5.22 and later, all groups within a regexp can be set to |
| 1103 |
non-capturing by using the new \f(CW\*(C`/n\*(C'\fR flag: |
| 1104 |
.PP |
| 1105 |
.Vb 1 |
| 1106 |
\& "hello" =~ /(hi|hello)/n; # $1 is not set! |
| 1107 |
.Ve |
| 1108 |
.PP |
| 1109 |
See "n" in perlre for more information. |
| 1110 |
.SS "Matching repetitions" |
| 1111 |
.IX Subsection "Matching repetitions" |
| 1112 |
The examples in the previous section display an annoying weakness. We |
| 1113 |
were only matching 3\-letter words, or chunks of words of 4 letters or |
| 1114 |
less. We'd like to be able to match words or, more generally, strings |
| 1115 |
of any length, without writing out tedious alternatives like |
| 1116 |
\&\f(CW\*(C`\ew\ew\ew\ew|\ew\ew\ew|\ew\ew|\ew\*(C'\fR. |
| 1117 |
.PP |
| 1118 |
This is exactly the problem the \fIquantifier\fR metacharacters \f(CW\*(Aq?\*(Aq\fR, |
| 1119 |
\&\f(CW\*(Aq*\*(Aq\fR, \f(CW\*(Aq+\*(Aq\fR, and \f(CW\*(C`{}\*(C'\fR were created for. They allow us to delimit the |
| 1120 |
number of repeats for a portion of a regexp we consider to be a |
| 1121 |
match. Quantifiers are put immediately after the character, character |
| 1122 |
class, or grouping that we want to specify. They have the following |
| 1123 |
meanings: |
| 1124 |
.IP \(bu 4 |
| 1125 |
\&\f(CW\*(C`a?\*(C'\fR means: match \f(CW\*(Aqa\*(Aq\fR 1 or 0 times |
| 1126 |
.IP \(bu 4 |
| 1127 |
\&\f(CW\*(C`a*\*(C'\fR means: match \f(CW\*(Aqa\*(Aq\fR 0 or more times, \fIi.e.\fR, any number of times |
| 1128 |
.IP \(bu 4 |
| 1129 |
\&\f(CW\*(C`a+\*(C'\fR means: match \f(CW\*(Aqa\*(Aq\fR 1 or more times, \fIi.e.\fR, at least once |
| 1130 |
.IP \(bu 4 |
| 1131 |
\&\f(CW\*(C`a{n,m}\*(C'\fR means: match at least \f(CW\*(C`n\*(C'\fR times, but not more than \f(CW\*(C`m\*(C'\fR |
| 1132 |
times. |
| 1133 |
.IP \(bu 4 |
| 1134 |
\&\f(CW\*(C`a{n,}\*(C'\fR means: match at least \f(CW\*(C`n\*(C'\fR or more times |
| 1135 |
.IP \(bu 4 |
| 1136 |
\&\f(CW\*(C`a{,n}\*(C'\fR means: match at most \f(CW\*(C`n\*(C'\fR times, or fewer |
| 1137 |
.IP \(bu 4 |
| 1138 |
\&\f(CW\*(C`a{n}\*(C'\fR means: match exactly \f(CW\*(C`n\*(C'\fR times |
| 1139 |
.PP |
| 1140 |
If you like, you can add blanks (tab or space characters) within the |
| 1141 |
braces, but adjacent to them, and/or next to the comma (if any). |
| 1142 |
.PP |
| 1143 |
Here are some examples: |
| 1144 |
.PP |
| 1145 |
.Vb 10 |
| 1146 |
\& /[a\-z]+\es+\ed*/; # match a lowercase word, at least one space, and |
| 1147 |
\& # any number of digits |
| 1148 |
\& /(\ew+)\es+\eg1/; # match doubled words of arbitrary length |
| 1149 |
\& /y(es)?/i; # matches \*(Aqy\*(Aq, \*(AqY\*(Aq, or a case\-insensitive \*(Aqyes\*(Aq |
| 1150 |
\& $year =~ /^\ed{2,4}$/; # make sure year is at least 2 but not more |
| 1151 |
\& # than 4 digits |
| 1152 |
\& $year =~ /^\ed{ 2, 4 }$/; # Same; for those who like wide open |
| 1153 |
\& # spaces. |
| 1154 |
\& $year =~ /^\ed{2, 4}$/; # Same. |
| 1155 |
\& $year =~ /^\ed{4}$|^\ed{2}$/; # better match; throw out 3\-digit dates |
| 1156 |
\& $year =~ /^\ed{2}(\ed{2})?$/; # same thing written differently. |
| 1157 |
\& # However, this captures the last two |
| 1158 |
\& # digits in $1 and the other does not. |
| 1159 |
\& |
| 1160 |
\& % simple_grep \*(Aq^(\ew+)\eg1$\*(Aq /usr/dict/words # isn\*(Aqt this easier? |
| 1161 |
\& beriberi |
| 1162 |
\& booboo |
| 1163 |
\& coco |
| 1164 |
\& mama |
| 1165 |
\& murmur |
| 1166 |
\& papa |
| 1167 |
.Ve |
| 1168 |
.PP |
| 1169 |
For all of these quantifiers, Perl will try to match as much of the |
| 1170 |
string as possible, while still allowing the regexp to succeed. Thus |
| 1171 |
with \f(CW\*(C`/a?.../\*(C'\fR, Perl will first try to match the regexp with the \f(CW\*(Aqa\*(Aq\fR |
| 1172 |
present; if that fails, Perl will try to match the regexp without the |
| 1173 |
\&\f(CW\*(Aqa\*(Aq\fR present. For the quantifier \f(CW\*(Aq*\*(Aq\fR, we get the following: |
| 1174 |
.PP |
| 1175 |
.Vb 5 |
| 1176 |
\& $x = "the cat in the hat"; |
| 1177 |
\& $x =~ /^(.*)(cat)(.*)$/; # matches, |
| 1178 |
\& # $1 = \*(Aqthe \*(Aq |
| 1179 |
\& # $2 = \*(Aqcat\*(Aq |
| 1180 |
\& # $3 = \*(Aq in the hat\*(Aq |
| 1181 |
.Ve |
| 1182 |
.PP |
| 1183 |
Which is what we might expect, the match finds the only \f(CW\*(C`cat\*(C'\fR in the |
| 1184 |
string and locks onto it. Consider, however, this regexp: |
| 1185 |
.PP |
| 1186 |
.Vb 4 |
| 1187 |
\& $x =~ /^(.*)(at)(.*)$/; # matches, |
| 1188 |
\& # $1 = \*(Aqthe cat in the h\*(Aq |
| 1189 |
\& # $2 = \*(Aqat\*(Aq |
| 1190 |
\& # $3 = \*(Aq\*(Aq (0 characters match) |
| 1191 |
.Ve |
| 1192 |
.PP |
| 1193 |
One might initially guess that Perl would find the \f(CW\*(C`at\*(C'\fR in \f(CW\*(C`cat\*(C'\fR and |
| 1194 |
stop there, but that wouldn't give the longest possible string to the |
| 1195 |
first quantifier \f(CW\*(C`.*\*(C'\fR. Instead, the first quantifier \f(CW\*(C`.*\*(C'\fR grabs as |
| 1196 |
much of the string as possible while still having the regexp match. In |
| 1197 |
this example, that means having the \f(CW\*(C`at\*(C'\fR sequence with the final \f(CW\*(C`at\*(C'\fR |
| 1198 |
in the string. The other important principle illustrated here is that, |
| 1199 |
when there are two or more elements in a regexp, the \fIleftmost\fR |
| 1200 |
quantifier, if there is one, gets to grab as much of the string as |
| 1201 |
possible, leaving the rest of the regexp to fight over scraps. Thus in |
| 1202 |
our example, the first quantifier \f(CW\*(C`.*\*(C'\fR grabs most of the string, while |
| 1203 |
the second quantifier \f(CW\*(C`.*\*(C'\fR gets the empty string. Quantifiers that |
| 1204 |
grab as much of the string as possible are called \fImaximal match\fR or |
| 1205 |
\&\fIgreedy\fR quantifiers. |
| 1206 |
.PP |
| 1207 |
When a regexp can match a string in several different ways, we can use |
| 1208 |
the principles above to predict which way the regexp will match: |
| 1209 |
.IP \(bu 4 |
| 1210 |
Principle 0: Taken as a whole, any regexp will be matched at the |
| 1211 |
earliest possible position in the string. |
| 1212 |
.IP \(bu 4 |
| 1213 |
Principle 1: In an alternation \f(CW\*(C`a|b|c...\*(C'\fR, the leftmost alternative |
| 1214 |
that allows a match for the whole regexp will be the one used. |
| 1215 |
.IP \(bu 4 |
| 1216 |
Principle 2: The maximal matching quantifiers \f(CW\*(Aq?\*(Aq\fR, \f(CW\*(Aq*\*(Aq\fR, \f(CW\*(Aq+\*(Aq\fR and |
| 1217 |
\&\f(CW\*(C`{n,m}\*(C'\fR will in general match as much of the string as possible while |
| 1218 |
still allowing the whole regexp to match. |
| 1219 |
.IP \(bu 4 |
| 1220 |
Principle 3: If there are two or more elements in a regexp, the |
| 1221 |
leftmost greedy quantifier, if any, will match as much of the string |
| 1222 |
as possible while still allowing the whole regexp to match. The next |
| 1223 |
leftmost greedy quantifier, if any, will try to match as much of the |
| 1224 |
string remaining available to it as possible, while still allowing the |
| 1225 |
whole regexp to match. And so on, until all the regexp elements are |
| 1226 |
satisfied. |
| 1227 |
.PP |
| 1228 |
As we have seen above, Principle 0 overrides the others. The regexp |
| 1229 |
will be matched as early as possible, with the other principles |
| 1230 |
determining how the regexp matches at that earliest character |
| 1231 |
position. |
| 1232 |
.PP |
| 1233 |
Here is an example of these principles in action: |
| 1234 |
.PP |
| 1235 |
.Vb 5 |
| 1236 |
\& $x = "The programming republic of Perl"; |
| 1237 |
\& $x =~ /^(.+)(e|r)(.*)$/; # matches, |
| 1238 |
\& # $1 = \*(AqThe programming republic of Pe\*(Aq |
| 1239 |
\& # $2 = \*(Aqr\*(Aq |
| 1240 |
\& # $3 = \*(Aql\*(Aq |
| 1241 |
.Ve |
| 1242 |
.PP |
| 1243 |
This regexp matches at the earliest string position, \f(CW\*(AqT\*(Aq\fR. One |
| 1244 |
might think that \f(CW\*(Aqe\*(Aq\fR, being leftmost in the alternation, would be |
| 1245 |
matched, but \f(CW\*(Aqr\*(Aq\fR produces the longest string in the first quantifier. |
| 1246 |
.PP |
| 1247 |
.Vb 3 |
| 1248 |
\& $x =~ /(m{1,2})(.*)$/; # matches, |
| 1249 |
\& # $1 = \*(Aqmm\*(Aq |
| 1250 |
\& # $2 = \*(Aqing republic of Perl\*(Aq |
| 1251 |
.Ve |
| 1252 |
.PP |
| 1253 |
Here, The earliest possible match is at the first \f(CW\*(Aqm\*(Aq\fR in |
| 1254 |
\&\f(CW\*(C`programming\*(C'\fR. \f(CW\*(C`m{1,2}\*(C'\fR is the first quantifier, so it gets to match |
| 1255 |
a maximal \f(CW\*(C`mm\*(C'\fR. |
| 1256 |
.PP |
| 1257 |
.Vb 3 |
| 1258 |
\& $x =~ /.*(m{1,2})(.*)$/; # matches, |
| 1259 |
\& # $1 = \*(Aqm\*(Aq |
| 1260 |
\& # $2 = \*(Aqing republic of Perl\*(Aq |
| 1261 |
.Ve |
| 1262 |
.PP |
| 1263 |
Here, the regexp matches at the start of the string. The first |
| 1264 |
quantifier \f(CW\*(C`.*\*(C'\fR grabs as much as possible, leaving just a single |
| 1265 |
\&\f(CW\*(Aqm\*(Aq\fR for the second quantifier \f(CW\*(C`m{1,2}\*(C'\fR. |
| 1266 |
.PP |
| 1267 |
.Vb 4 |
| 1268 |
\& $x =~ /(.?)(m{1,2})(.*)$/; # matches, |
| 1269 |
\& # $1 = \*(Aqa\*(Aq |
| 1270 |
\& # $2 = \*(Aqmm\*(Aq |
| 1271 |
\& # $3 = \*(Aqing republic of Perl\*(Aq |
| 1272 |
.Ve |
| 1273 |
.PP |
| 1274 |
Here, \f(CW\*(C`.?\*(C'\fR eats its maximal one character at the earliest possible |
| 1275 |
position in the string, \f(CW\*(Aqa\*(Aq\fR in \f(CW\*(C`programming\*(C'\fR, leaving \f(CW\*(C`m{1,2}\*(C'\fR |
| 1276 |
the opportunity to match both \f(CW\*(Aqm\*(Aq\fR's. Finally, |
| 1277 |
.PP |
| 1278 |
.Vb 1 |
| 1279 |
\& "aXXXb" =~ /(X*)/; # matches with $1 = \*(Aq\*(Aq |
| 1280 |
.Ve |
| 1281 |
.PP |
| 1282 |
because it can match zero copies of \f(CW\*(AqX\*(Aq\fR at the beginning of the |
| 1283 |
string. If you definitely want to match at least one \f(CW\*(AqX\*(Aq\fR, use |
| 1284 |
\&\f(CW\*(C`X+\*(C'\fR, not \f(CW\*(C`X*\*(C'\fR. |
| 1285 |
.PP |
| 1286 |
Sometimes greed is not good. At times, we would like quantifiers to |
| 1287 |
match a \fIminimal\fR piece of string, rather than a maximal piece. For |
| 1288 |
this purpose, Larry Wall created the \fIminimal match\fR or |
| 1289 |
\&\fInon-greedy\fR quantifiers \f(CW\*(C`??\*(C'\fR, \f(CW\*(C`*?\*(C'\fR, \f(CW\*(C`+?\*(C'\fR, and \f(CW\*(C`{}?\*(C'\fR. These are |
| 1290 |
the usual quantifiers with a \f(CW\*(Aq?\*(Aq\fR appended to them. They have the |
| 1291 |
following meanings: |
| 1292 |
.IP \(bu 4 |
| 1293 |
\&\f(CW\*(C`a??\*(C'\fR means: match \f(CW\*(Aqa\*(Aq\fR 0 or 1 times. Try 0 first, then 1. |
| 1294 |
.IP \(bu 4 |
| 1295 |
\&\f(CW\*(C`a*?\*(C'\fR means: match \f(CW\*(Aqa\*(Aq\fR 0 or more times, \fIi.e.\fR, any number of times, |
| 1296 |
but as few times as possible |
| 1297 |
.IP \(bu 4 |
| 1298 |
\&\f(CW\*(C`a+?\*(C'\fR means: match \f(CW\*(Aqa\*(Aq\fR 1 or more times, \fIi.e.\fR, at least once, but |
| 1299 |
as few times as possible |
| 1300 |
.IP \(bu 4 |
| 1301 |
\&\f(CW\*(C`a{n,m}?\*(C'\fR means: match at least \f(CW\*(C`n\*(C'\fR times, not more than \f(CW\*(C`m\*(C'\fR |
| 1302 |
times, as few times as possible |
| 1303 |
.IP \(bu 4 |
| 1304 |
\&\f(CW\*(C`a{n,}?\*(C'\fR means: match at least \f(CW\*(C`n\*(C'\fR times, but as few times as |
| 1305 |
possible |
| 1306 |
.IP \(bu 4 |
| 1307 |
\&\f(CW\*(C`a{,n}?\*(C'\fR means: match at most \f(CW\*(C`n\*(C'\fR times, but as few times as |
| 1308 |
possible |
| 1309 |
.IP \(bu 4 |
| 1310 |
\&\f(CW\*(C`a{n}?\*(C'\fR means: match exactly \f(CW\*(C`n\*(C'\fR times. Because we match exactly |
| 1311 |
\&\f(CW\*(C`n\*(C'\fR times, \f(CW\*(C`a{n}?\*(C'\fR is equivalent to \f(CW\*(C`a{n}\*(C'\fR and is just there for |
| 1312 |
notational consistency. |
| 1313 |
.PP |
| 1314 |
Let's look at the example above, but with minimal quantifiers: |
| 1315 |
.PP |
| 1316 |
.Vb 5 |
| 1317 |
\& $x = "The programming republic of Perl"; |
| 1318 |
\& $x =~ /^(.+?)(e|r)(.*)$/; # matches, |
| 1319 |
\& # $1 = \*(AqTh\*(Aq |
| 1320 |
\& # $2 = \*(Aqe\*(Aq |
| 1321 |
\& # $3 = \*(Aq programming republic of Perl\*(Aq |
| 1322 |
.Ve |
| 1323 |
.PP |
| 1324 |
The minimal string that will allow both the start of the string \f(CW\*(Aq^\*(Aq\fR |
| 1325 |
and the alternation to match is \f(CW\*(C`Th\*(C'\fR, with the alternation \f(CW\*(C`e|r\*(C'\fR |
| 1326 |
matching \f(CW\*(Aqe\*(Aq\fR. The second quantifier \f(CW\*(C`.*\*(C'\fR is free to gobble up the |
| 1327 |
rest of the string. |
| 1328 |
.PP |
| 1329 |
.Vb 3 |
| 1330 |
\& $x =~ /(m{1,2}?)(.*?)$/; # matches, |
| 1331 |
\& # $1 = \*(Aqm\*(Aq |
| 1332 |
\& # $2 = \*(Aqming republic of Perl\*(Aq |
| 1333 |
.Ve |
| 1334 |
.PP |
| 1335 |
The first string position that this regexp can match is at the first |
| 1336 |
\&\f(CW\*(Aqm\*(Aq\fR in \f(CW\*(C`programming\*(C'\fR. At this position, the minimal \f(CW\*(C`m{1,2}?\*(C'\fR |
| 1337 |
matches just one \f(CW\*(Aqm\*(Aq\fR. Although the second quantifier \f(CW\*(C`.*?\*(C'\fR would |
| 1338 |
prefer to match no characters, it is constrained by the end-of-string |
| 1339 |
anchor \f(CW\*(Aq$\*(Aq\fR to match the rest of the string. |
| 1340 |
.PP |
| 1341 |
.Vb 4 |
| 1342 |
\& $x =~ /(.*?)(m{1,2}?)(.*)$/; # matches, |
| 1343 |
\& # $1 = \*(AqThe progra\*(Aq |
| 1344 |
\& # $2 = \*(Aqm\*(Aq |
| 1345 |
\& # $3 = \*(Aqming republic of Perl\*(Aq |
| 1346 |
.Ve |
| 1347 |
.PP |
| 1348 |
In this regexp, you might expect the first minimal quantifier \f(CW\*(C`.*?\*(C'\fR |
| 1349 |
to match the empty string, because it is not constrained by a \f(CW\*(Aq^\*(Aq\fR |
| 1350 |
anchor to match the beginning of the word. Principle 0 applies here, |
| 1351 |
however. Because it is possible for the whole regexp to match at the |
| 1352 |
start of the string, it \fIwill\fR match at the start of the string. Thus |
| 1353 |
the first quantifier has to match everything up to the first \f(CW\*(Aqm\*(Aq\fR. The |
| 1354 |
second minimal quantifier matches just one \f(CW\*(Aqm\*(Aq\fR and the third |
| 1355 |
quantifier matches the rest of the string. |
| 1356 |
.PP |
| 1357 |
.Vb 4 |
| 1358 |
\& $x =~ /(.??)(m{1,2})(.*)$/; # matches, |
| 1359 |
\& # $1 = \*(Aqa\*(Aq |
| 1360 |
\& # $2 = \*(Aqmm\*(Aq |
| 1361 |
\& # $3 = \*(Aqing republic of Perl\*(Aq |
| 1362 |
.Ve |
| 1363 |
.PP |
| 1364 |
Just as in the previous regexp, the first quantifier \f(CW\*(C`.??\*(C'\fR can match |
| 1365 |
earliest at position \f(CW\*(Aqa\*(Aq\fR, so it does. The second quantifier is |
| 1366 |
greedy, so it matches \f(CW\*(C`mm\*(C'\fR, and the third matches the rest of the |
| 1367 |
string. |
| 1368 |
.PP |
| 1369 |
We can modify principle 3 above to take into account non-greedy |
| 1370 |
quantifiers: |
| 1371 |
.IP \(bu 4 |
| 1372 |
Principle 3: If there are two or more elements in a regexp, the |
| 1373 |
leftmost greedy (non-greedy) quantifier, if any, will match as much |
| 1374 |
(little) of the string as possible while still allowing the whole |
| 1375 |
regexp to match. The next leftmost greedy (non-greedy) quantifier, if |
| 1376 |
any, will try to match as much (little) of the string remaining |
| 1377 |
available to it as possible, while still allowing the whole regexp to |
| 1378 |
match. And so on, until all the regexp elements are satisfied. |
| 1379 |
.PP |
| 1380 |
Just like alternation, quantifiers are also susceptible to |
| 1381 |
backtracking. Here is a step-by-step analysis of the example |
| 1382 |
.PP |
| 1383 |
.Vb 5 |
| 1384 |
\& $x = "the cat in the hat"; |
| 1385 |
\& $x =~ /^(.*)(at)(.*)$/; # matches, |
| 1386 |
\& # $1 = \*(Aqthe cat in the h\*(Aq |
| 1387 |
\& # $2 = \*(Aqat\*(Aq |
| 1388 |
\& # $3 = \*(Aq\*(Aq (0 matches) |
| 1389 |
.Ve |
| 1390 |
.IP 1. 4 |
| 1391 |
Start with the first letter in the string \f(CW\*(Aqt\*(Aq\fR. |
| 1392 |
.IP 2. 4 |
| 1393 |
The first quantifier \f(CW\*(Aq.*\*(Aq\fR starts out by matching the whole |
| 1394 |
string \f(CW"the cat in the hat"\fR. |
| 1395 |
.IP 3. 4 |
| 1396 |
\&\f(CW\*(Aqa\*(Aq\fR in the regexp element \f(CW\*(Aqat\*(Aq\fR doesn't match the end |
| 1397 |
of the string. Backtrack one character. |
| 1398 |
.IP 4. 4 |
| 1399 |
\&\f(CW\*(Aqa\*(Aq\fR in the regexp element \f(CW\*(Aqat\*(Aq\fR still doesn't match |
| 1400 |
the last letter of the string \f(CW\*(Aqt\*(Aq\fR, so backtrack one more character. |
| 1401 |
.IP 5. 4 |
| 1402 |
Now we can match the \f(CW\*(Aqa\*(Aq\fR and the \f(CW\*(Aqt\*(Aq\fR. |
| 1403 |
.IP 6. 4 |
| 1404 |
Move on to the third element \f(CW\*(Aq.*\*(Aq\fR. Since we are at the |
| 1405 |
end of the string and \f(CW\*(Aq.*\*(Aq\fR can match 0 times, assign it the empty |
| 1406 |
string. |
| 1407 |
.IP 7. 4 |
| 1408 |
We are done! |
| 1409 |
.PP |
| 1410 |
Most of the time, all this moving forward and backtracking happens |
| 1411 |
quickly and searching is fast. There are some pathological regexps, |
| 1412 |
however, whose execution time exponentially grows with the size of the |
| 1413 |
string. A typical structure that blows up in your face is of the form |
| 1414 |
.PP |
| 1415 |
.Vb 1 |
| 1416 |
\& /(a|b+)*/; |
| 1417 |
.Ve |
| 1418 |
.PP |
| 1419 |
The problem is the nested indeterminate quantifiers. There are many |
| 1420 |
different ways of partitioning a string of length n between the \f(CW\*(Aq+\*(Aq\fR |
| 1421 |
and \f(CW\*(Aq*\*(Aq\fR: one repetition with \f(CW\*(C`b+\*(C'\fR of length n, two repetitions with |
| 1422 |
the first \f(CW\*(C`b+\*(C'\fR length k and the second with length n\-k, m repetitions |
| 1423 |
whose bits add up to length n, \fIetc\fR. In fact there are an exponential |
| 1424 |
number of ways to partition a string as a function of its length. A |
| 1425 |
regexp may get lucky and match early in the process, but if there is |
| 1426 |
no match, Perl will try \fIevery\fR possibility before giving up. So be |
| 1427 |
careful with nested \f(CW\*(Aq*\*(Aq\fR's, \f(CW\*(C`{n,m}\*(C'\fR's, and \f(CW\*(Aq+\*(Aq\fR's. The book |
| 1428 |
\&\fIMastering Regular Expressions\fR by Jeffrey Friedl gives a wonderful |
| 1429 |
discussion of this and other efficiency issues. |
| 1430 |
.SS "Possessive quantifiers" |
| 1431 |
.IX Subsection "Possessive quantifiers" |
| 1432 |
Backtracking during the relentless search for a match may be a waste |
| 1433 |
of time, particularly when the match is bound to fail. Consider |
| 1434 |
the simple pattern |
| 1435 |
.PP |
| 1436 |
.Vb 1 |
| 1437 |
\& /^\ew+\es+\ew+$/; # a word, spaces, a word |
| 1438 |
.Ve |
| 1439 |
.PP |
| 1440 |
Whenever this is applied to a string which doesn't quite meet the |
| 1441 |
pattern's expectations such as \f(CW"abc\ \ "\fR or \f(CW"abc\ \ def\ "\fR, |
| 1442 |
the regexp engine will backtrack, approximately once for each character |
| 1443 |
in the string. But we know that there is no way around taking \fIall\fR |
| 1444 |
of the initial word characters to match the first repetition, that \fIall\fR |
| 1445 |
spaces must be eaten by the middle part, and the same goes for the second |
| 1446 |
word. |
| 1447 |
.PP |
| 1448 |
With the introduction of the \fIpossessive quantifiers\fR in Perl 5.10, we |
| 1449 |
have a way of instructing the regexp engine not to backtrack, with the |
| 1450 |
usual quantifiers with a \f(CW\*(Aq+\*(Aq\fR appended to them. This makes them greedy as |
| 1451 |
well as stingy; once they succeed they won't give anything back to permit |
| 1452 |
another solution. They have the following meanings: |
| 1453 |
.IP \(bu 4 |
| 1454 |
\&\f(CW\*(C`a{n,m}+\*(C'\fR means: match at least \f(CW\*(C`n\*(C'\fR times, not more than \f(CW\*(C`m\*(C'\fR times, |
| 1455 |
as many times as possible, and don't give anything up. \f(CW\*(C`a?+\*(C'\fR is short |
| 1456 |
for \f(CW\*(C`a{0,1}+\*(C'\fR |
| 1457 |
.IP \(bu 4 |
| 1458 |
\&\f(CW\*(C`a{n,}+\*(C'\fR means: match at least \f(CW\*(C`n\*(C'\fR times, but as many times as possible, |
| 1459 |
and don't give anything up. \f(CW\*(C`a++\*(C'\fR is short for \f(CW\*(C`a{1,}+\*(C'\fR. |
| 1460 |
.IP \(bu 4 |
| 1461 |
\&\f(CW\*(C`a{,n}+\*(C'\fR means: match as many times as possible up to at most \f(CW\*(C`n\*(C'\fR |
| 1462 |
times, and don't give anything up. \f(CW\*(C`a*+\*(C'\fR is short for \f(CW\*(C`a{0,}+\*(C'\fR. |
| 1463 |
.IP \(bu 4 |
| 1464 |
\&\f(CW\*(C`a{n}+\*(C'\fR means: match exactly \f(CW\*(C`n\*(C'\fR times. It is just there for |
| 1465 |
notational consistency. |
| 1466 |
.PP |
| 1467 |
These possessive quantifiers represent a special case of a more general |
| 1468 |
concept, the \fIindependent subexpression\fR, see below. |
| 1469 |
.PP |
| 1470 |
As an example where a possessive quantifier is suitable we consider |
| 1471 |
matching a quoted string, as it appears in several programming languages. |
| 1472 |
The backslash is used as an escape character that indicates that the |
| 1473 |
next character is to be taken literally, as another character for the |
| 1474 |
string. Therefore, after the opening quote, we expect a (possibly |
| 1475 |
empty) sequence of alternatives: either some character except an |
| 1476 |
unescaped quote or backslash or an escaped character. |
| 1477 |
.PP |
| 1478 |
.Vb 1 |
| 1479 |
\& /"(?:[^"\e\e]++|\e\e.)*+"/; |
| 1480 |
.Ve |
| 1481 |
.SS "Building a regexp" |
| 1482 |
.IX Subsection "Building a regexp" |
| 1483 |
At this point, we have all the basic regexp concepts covered, so let's |
| 1484 |
give a more involved example of a regular expression. We will build a |
| 1485 |
regexp that matches numbers. |
| 1486 |
.PP |
| 1487 |
The first task in building a regexp is to decide what we want to match |
| 1488 |
and what we want to exclude. In our case, we want to match both |
| 1489 |
integers and floating point numbers and we want to reject any string |
| 1490 |
that isn't a number. |
| 1491 |
.PP |
| 1492 |
The next task is to break the problem down into smaller problems that |
| 1493 |
are easily converted into a regexp. |
| 1494 |
.PP |
| 1495 |
The simplest case is integers. These consist of a sequence of digits, |
| 1496 |
with an optional sign in front. The digits we can represent with |
| 1497 |
\&\f(CW\*(C`\ed+\*(C'\fR and the sign can be matched with \f(CW\*(C`[+\-]\*(C'\fR. Thus the integer |
| 1498 |
regexp is |
| 1499 |
.PP |
| 1500 |
.Vb 1 |
| 1501 |
\& /[+\-]?\ed+/; # matches integers |
| 1502 |
.Ve |
| 1503 |
.PP |
| 1504 |
A floating point number potentially has a sign, an integral part, a |
| 1505 |
decimal point, a fractional part, and an exponent. One or more of these |
| 1506 |
parts is optional, so we need to check out the different |
| 1507 |
possibilities. Floating point numbers which are in proper form include |
| 1508 |
123., 0.345, .34, \-1e6, and 25.4E\-72. As with integers, the sign out |
| 1509 |
front is completely optional and can be matched by \f(CW\*(C`[+\-]?\*(C'\fR. We can |
| 1510 |
see that if there is no exponent, floating point numbers must have a |
| 1511 |
decimal point, otherwise they are integers. We might be tempted to |
| 1512 |
model these with \f(CW\*(C`\ed*\e.\ed*\*(C'\fR, but this would also match just a single |
| 1513 |
decimal point, which is not a number. So the three cases of floating |
| 1514 |
point number without exponent are |
| 1515 |
.PP |
| 1516 |
.Vb 3 |
| 1517 |
\& /[+\-]?\ed+\e./; # 1., 321., etc. |
| 1518 |
\& /[+\-]?\e.\ed+/; # .1, .234, etc. |
| 1519 |
\& /[+\-]?\ed+\e.\ed+/; # 1.0, 30.56, etc. |
| 1520 |
.Ve |
| 1521 |
.PP |
| 1522 |
These can be combined into a single regexp with a three-way alternation: |
| 1523 |
.PP |
| 1524 |
.Vb 1 |
| 1525 |
\& /[+\-]?(\ed+\e.\ed+|\ed+\e.|\e.\ed+)/; # floating point, no exponent |
| 1526 |
.Ve |
| 1527 |
.PP |
| 1528 |
In this alternation, it is important to put \f(CW\*(Aq\ed+\e.\ed+\*(Aq\fR before |
| 1529 |
\&\f(CW\*(Aq\ed+\e.\*(Aq\fR. If \f(CW\*(Aq\ed+\e.\*(Aq\fR were first, the regexp would happily match that |
| 1530 |
and ignore the fractional part of the number. |
| 1531 |
.PP |
| 1532 |
Now consider floating point numbers with exponents. The key |
| 1533 |
observation here is that \fIboth\fR integers and numbers with decimal |
| 1534 |
points are allowed in front of an exponent. Then exponents, like the |
| 1535 |
overall sign, are independent of whether we are matching numbers with |
| 1536 |
or without decimal points, and can be "decoupled" from the |
| 1537 |
mantissa. The overall form of the regexp now becomes clear: |
| 1538 |
.PP |
| 1539 |
.Vb 1 |
| 1540 |
\& /^(optional sign)(integer | f.p. mantissa)(optional exponent)$/; |
| 1541 |
.Ve |
| 1542 |
.PP |
| 1543 |
The exponent is an \f(CW\*(Aqe\*(Aq\fR or \f(CW\*(AqE\*(Aq\fR, followed by an integer. So the |
| 1544 |
exponent regexp is |
| 1545 |
.PP |
| 1546 |
.Vb 1 |
| 1547 |
\& /[eE][+\-]?\ed+/; # exponent |
| 1548 |
.Ve |
| 1549 |
.PP |
| 1550 |
Putting all the parts together, we get a regexp that matches numbers: |
| 1551 |
.PP |
| 1552 |
.Vb 1 |
| 1553 |
\& /^[+\-]?(\ed+\e.\ed+|\ed+\e.|\e.\ed+|\ed+)([eE][+\-]?\ed+)?$/; # Ta da! |
| 1554 |
.Ve |
| 1555 |
.PP |
| 1556 |
Long regexps like this may impress your friends, but can be hard to |
| 1557 |
decipher. In complex situations like this, the \f(CW\*(C`/x\*(C'\fR modifier for a |
| 1558 |
match is invaluable. It allows one to put nearly arbitrary whitespace |
| 1559 |
and comments into a regexp without affecting their meaning. Using it, |
| 1560 |
we can rewrite our "extended" regexp in the more pleasing form |
| 1561 |
.PP |
| 1562 |
.Vb 10 |
| 1563 |
\& /^ |
| 1564 |
\& [+\-]? # first, match an optional sign |
| 1565 |
\& ( # then match integers or f.p. mantissas: |
| 1566 |
\& \ed+\e.\ed+ # mantissa of the form a.b |
| 1567 |
\& |\ed+\e. # mantissa of the form a. |
| 1568 |
\& |\e.\ed+ # mantissa of the form .b |
| 1569 |
\& |\ed+ # integer of the form a |
| 1570 |
\& ) |
| 1571 |
\& ( [eE] [+\-]? \ed+ )? # finally, optionally match an exponent |
| 1572 |
\& $/x; |
| 1573 |
.Ve |
| 1574 |
.PP |
| 1575 |
If whitespace is mostly irrelevant, how does one include space |
| 1576 |
characters in an extended regexp? The answer is to backslash it |
| 1577 |
\&\f(CW\*(Aq\e\ \*(Aq\fR or put it in a character class \f(CW\*(C`[\ ]\*(C'\fR. The same thing |
| 1578 |
goes for pound signs: use \f(CW\*(C`\e#\*(C'\fR or \f(CW\*(C`[#]\*(C'\fR. For instance, Perl allows |
| 1579 |
a space between the sign and the mantissa or integer, and we could add |
| 1580 |
this to our regexp as follows: |
| 1581 |
.PP |
| 1582 |
.Vb 10 |
| 1583 |
\& /^ |
| 1584 |
\& [+\-]?\e * # first, match an optional sign *and space* |
| 1585 |
\& ( # then match integers or f.p. mantissas: |
| 1586 |
\& \ed+\e.\ed+ # mantissa of the form a.b |
| 1587 |
\& |\ed+\e. # mantissa of the form a. |
| 1588 |
\& |\e.\ed+ # mantissa of the form .b |
| 1589 |
\& |\ed+ # integer of the form a |
| 1590 |
\& ) |
| 1591 |
\& ( [eE] [+\-]? \ed+ )? # finally, optionally match an exponent |
| 1592 |
\& $/x; |
| 1593 |
.Ve |
| 1594 |
.PP |
| 1595 |
In this form, it is easier to see a way to simplify the |
| 1596 |
alternation. Alternatives 1, 2, and 4 all start with \f(CW\*(C`\ed+\*(C'\fR, so it |
| 1597 |
could be factored out: |
| 1598 |
.PP |
| 1599 |
.Vb 11 |
| 1600 |
\& /^ |
| 1601 |
\& [+\-]?\e * # first, match an optional sign |
| 1602 |
\& ( # then match integers or f.p. mantissas: |
| 1603 |
\& \ed+ # start out with a ... |
| 1604 |
\& ( |
| 1605 |
\& \e.\ed* # mantissa of the form a.b or a. |
| 1606 |
\& )? # ? takes care of integers of the form a |
| 1607 |
\& |\e.\ed+ # mantissa of the form .b |
| 1608 |
\& ) |
| 1609 |
\& ( [eE] [+\-]? \ed+ )? # finally, optionally match an exponent |
| 1610 |
\& $/x; |
| 1611 |
.Ve |
| 1612 |
.PP |
| 1613 |
Starting in Perl v5.26, specifying \f(CW\*(C`/xx\*(C'\fR changes the square-bracketed |
| 1614 |
portions of a pattern to ignore tabs and space characters unless they |
| 1615 |
are escaped by preceding them with a backslash. So, we could write |
| 1616 |
.PP |
| 1617 |
.Vb 11 |
| 1618 |
\& /^ |
| 1619 |
\& [ + \- ]?\e * # first, match an optional sign |
| 1620 |
\& ( # then match integers or f.p. mantissas: |
| 1621 |
\& \ed+ # start out with a ... |
| 1622 |
\& ( |
| 1623 |
\& \e.\ed* # mantissa of the form a.b or a. |
| 1624 |
\& )? # ? takes care of integers of the form a |
| 1625 |
\& |\e.\ed+ # mantissa of the form .b |
| 1626 |
\& ) |
| 1627 |
\& ( [ e E ] [ + \- ]? \ed+ )? # finally, optionally match an exponent |
| 1628 |
\& $/xx; |
| 1629 |
.Ve |
| 1630 |
.PP |
| 1631 |
This doesn't really improve the legibility of this example, but it's |
| 1632 |
available in case you want it. Squashing the pattern down to the |
| 1633 |
compact form, we have |
| 1634 |
.PP |
| 1635 |
.Vb 1 |
| 1636 |
\& /^[+\-]?\e *(\ed+(\e.\ed*)?|\e.\ed+)([eE][+\-]?\ed+)?$/; |
| 1637 |
.Ve |
| 1638 |
.PP |
| 1639 |
This is our final regexp. To recap, we built a regexp by |
| 1640 |
.IP \(bu 4 |
| 1641 |
specifying the task in detail, |
| 1642 |
.IP \(bu 4 |
| 1643 |
breaking down the problem into smaller parts, |
| 1644 |
.IP \(bu 4 |
| 1645 |
translating the small parts into regexps, |
| 1646 |
.IP \(bu 4 |
| 1647 |
combining the regexps, |
| 1648 |
.IP \(bu 4 |
| 1649 |
and optimizing the final combined regexp. |
| 1650 |
.PP |
| 1651 |
These are also the typical steps involved in writing a computer |
| 1652 |
program. This makes perfect sense, because regular expressions are |
| 1653 |
essentially programs written in a little computer language that specifies |
| 1654 |
patterns. |
| 1655 |
.SS "Using regular expressions in Perl" |
| 1656 |
.IX Subsection "Using regular expressions in Perl" |
| 1657 |
The last topic of Part 1 briefly covers how regexps are used in Perl |
| 1658 |
programs. Where do they fit into Perl syntax? |
| 1659 |
.PP |
| 1660 |
We have already introduced the matching operator in its default |
| 1661 |
\&\f(CW\*(C`/regexp/\*(C'\fR and arbitrary delimiter \f(CW\*(C`m!regexp!\*(C'\fR forms. We have used |
| 1662 |
the binding operator \f(CW\*(C`=~\*(C'\fR and its negation \f(CW\*(C`!~\*(C'\fR to test for string |
| 1663 |
matches. Associated with the matching operator, we have discussed the |
| 1664 |
single line \f(CW\*(C`/s\*(C'\fR, multi-line \f(CW\*(C`/m\*(C'\fR, case-insensitive \f(CW\*(C`/i\*(C'\fR and |
| 1665 |
extended \f(CW\*(C`/x\*(C'\fR modifiers. There are a few more things you might |
| 1666 |
want to know about matching operators. |
| 1667 |
.PP |
| 1668 |
\fIProhibiting substitution\fR |
| 1669 |
.IX Subsection "Prohibiting substitution" |
| 1670 |
.PP |
| 1671 |
If you change \f(CW$pattern\fR after the first substitution happens, Perl |
| 1672 |
will ignore it. If you don't want any substitutions at all, use the |
| 1673 |
special delimiter \f(CW\*(C`m\*(Aq\*(Aq\*(C'\fR: |
| 1674 |
.PP |
| 1675 |
.Vb 4 |
| 1676 |
\& @pattern = (\*(AqSeuss\*(Aq); |
| 1677 |
\& while (<>) { |
| 1678 |
\& print if m\*(Aq@pattern\*(Aq; # matches literal \*(Aq@pattern\*(Aq, not \*(AqSeuss\*(Aq |
| 1679 |
\& } |
| 1680 |
.Ve |
| 1681 |
.PP |
| 1682 |
Similar to strings, \f(CW\*(C`m\*(Aq\*(Aq\*(C'\fR acts like apostrophes on a regexp; all other |
| 1683 |
\&\f(CW\*(Aqm\*(Aq\fR delimiters act like quotes. If the regexp evaluates to the empty string, |
| 1684 |
the regexp in the \fIlast successful match\fR is used instead. So we have |
| 1685 |
.PP |
| 1686 |
.Vb 2 |
| 1687 |
\& "dog" =~ /d/; # \*(Aqd\*(Aq matches |
| 1688 |
\& "dogbert" =~ //; # this matches the \*(Aqd\*(Aq regexp used before |
| 1689 |
.Ve |
| 1690 |
.PP |
| 1691 |
\fIGlobal matching\fR |
| 1692 |
.IX Subsection "Global matching" |
| 1693 |
.PP |
| 1694 |
The final two modifiers we will discuss here, |
| 1695 |
\&\f(CW\*(C`/g\*(C'\fR and \f(CW\*(C`/c\*(C'\fR, concern multiple matches. |
| 1696 |
The modifier \f(CW\*(C`/g\*(C'\fR stands for global matching and allows the |
| 1697 |
matching operator to match within a string as many times as possible. |
| 1698 |
In scalar context, successive invocations against a string will have |
| 1699 |
\&\f(CW\*(C`/g\*(C'\fR jump from match to match, keeping track of position in the |
| 1700 |
string as it goes along. You can get or set the position with the |
| 1701 |
\&\f(CWpos()\fR function. |
| 1702 |
.PP |
| 1703 |
The use of \f(CW\*(C`/g\*(C'\fR is shown in the following example. Suppose we have |
| 1704 |
a string that consists of words separated by spaces. If we know how |
| 1705 |
many words there are in advance, we could extract the words using |
| 1706 |
groupings: |
| 1707 |
.PP |
| 1708 |
.Vb 5 |
| 1709 |
\& $x = "cat dog house"; # 3 words |
| 1710 |
\& $x =~ /^\es*(\ew+)\es+(\ew+)\es+(\ew+)\es*$/; # matches, |
| 1711 |
\& # $1 = \*(Aqcat\*(Aq |
| 1712 |
\& # $2 = \*(Aqdog\*(Aq |
| 1713 |
\& # $3 = \*(Aqhouse\*(Aq |
| 1714 |
.Ve |
| 1715 |
.PP |
| 1716 |
But what if we had an indeterminate number of words? This is the sort |
| 1717 |
of task \f(CW\*(C`/g\*(C'\fR was made for. To extract all words, form the simple |
| 1718 |
regexp \f(CW\*(C`(\ew+)\*(C'\fR and loop over all matches with \f(CW\*(C`/(\ew+)/g\*(C'\fR: |
| 1719 |
.PP |
| 1720 |
.Vb 3 |
| 1721 |
\& while ($x =~ /(\ew+)/g) { |
| 1722 |
\& print "Word is $1, ends at position ", pos $x, "\en"; |
| 1723 |
\& } |
| 1724 |
.Ve |
| 1725 |
.PP |
| 1726 |
prints |
| 1727 |
.PP |
| 1728 |
.Vb 3 |
| 1729 |
\& Word is cat, ends at position 3 |
| 1730 |
\& Word is dog, ends at position 7 |
| 1731 |
\& Word is house, ends at position 13 |
| 1732 |
.Ve |
| 1733 |
.PP |
| 1734 |
A failed match or changing the target string resets the position. If |
| 1735 |
you don't want the position reset after failure to match, add the |
| 1736 |
\&\f(CW\*(C`/c\*(C'\fR, as in \f(CW\*(C`/regexp/gc\*(C'\fR. The current position in the string is |
| 1737 |
associated with the string, not the regexp. This means that different |
| 1738 |
strings have different positions and their respective positions can be |
| 1739 |
set or read independently. |
| 1740 |
.PP |
| 1741 |
In list context, \f(CW\*(C`/g\*(C'\fR returns a list of matched groupings, or if |
| 1742 |
there are no groupings, a list of matches to the whole regexp. So if |
| 1743 |
we wanted just the words, we could use |
| 1744 |
.PP |
| 1745 |
.Vb 4 |
| 1746 |
\& @words = ($x =~ /(\ew+)/g); # matches, |
| 1747 |
\& # $words[0] = \*(Aqcat\*(Aq |
| 1748 |
\& # $words[1] = \*(Aqdog\*(Aq |
| 1749 |
\& # $words[2] = \*(Aqhouse\*(Aq |
| 1750 |
.Ve |
| 1751 |
.PP |
| 1752 |
Closely associated with the \f(CW\*(C`/g\*(C'\fR modifier is the \f(CW\*(C`\eG\*(C'\fR anchor. The |
| 1753 |
\&\f(CW\*(C`\eG\*(C'\fR anchor matches at the point where the previous \f(CW\*(C`/g\*(C'\fR match left |
| 1754 |
off. \f(CW\*(C`\eG\*(C'\fR allows us to easily do context-sensitive matching: |
| 1755 |
.PP |
| 1756 |
.Vb 12 |
| 1757 |
\& $metric = 1; # use metric units |
| 1758 |
\& ... |
| 1759 |
\& $x = <FILE>; # read in measurement |
| 1760 |
\& $x =~ /^([+\-]?\ed+)\es*/g; # get magnitude |
| 1761 |
\& $weight = $1; |
| 1762 |
\& if ($metric) { # error checking |
| 1763 |
\& print "Units error!" unless $x =~ /\eGkg\e./g; |
| 1764 |
\& } |
| 1765 |
\& else { |
| 1766 |
\& print "Units error!" unless $x =~ /\eGlbs\e./g; |
| 1767 |
\& } |
| 1768 |
\& $x =~ /\eG\es+(widget|sprocket)/g; # continue processing |
| 1769 |
.Ve |
| 1770 |
.PP |
| 1771 |
The combination of \f(CW\*(C`/g\*(C'\fR and \f(CW\*(C`\eG\*(C'\fR allows us to process the string a |
| 1772 |
bit at a time and use arbitrary Perl logic to decide what to do next. |
| 1773 |
Currently, the \f(CW\*(C`\eG\*(C'\fR anchor is only fully supported when used to anchor |
| 1774 |
to the start of the pattern. |
| 1775 |
.PP |
| 1776 |
\&\f(CW\*(C`\eG\*(C'\fR is also invaluable in processing fixed-length records with |
| 1777 |
regexps. Suppose we have a snippet of coding region DNA, encoded as |
| 1778 |
base pair letters \f(CW\*(C`ATCGTTGAAT...\*(C'\fR and we want to find all the stop |
| 1779 |
codons \f(CW\*(C`TGA\*(C'\fR. In a coding region, codons are 3\-letter sequences, so |
| 1780 |
we can think of the DNA snippet as a sequence of 3\-letter records. The |
| 1781 |
naive regexp |
| 1782 |
.PP |
| 1783 |
.Vb 3 |
| 1784 |
\& # expanded, this is "ATC GTT GAA TGC AAA TGA CAT GAC" |
| 1785 |
\& $dna = "ATCGTTGAATGCAAATGACATGAC"; |
| 1786 |
\& $dna =~ /TGA/; |
| 1787 |
.Ve |
| 1788 |
.PP |
| 1789 |
doesn't work; it may match a \f(CW\*(C`TGA\*(C'\fR, but there is no guarantee that |
| 1790 |
the match is aligned with codon boundaries, \fIe.g.\fR, the substring |
| 1791 |
\&\f(CW\*(C`GTT\ GAA\*(C'\fR gives a match. A better solution is |
| 1792 |
.PP |
| 1793 |
.Vb 3 |
| 1794 |
\& while ($dna =~ /(\ew\ew\ew)*?TGA/g) { # note the minimal *? |
| 1795 |
\& print "Got a TGA stop codon at position ", pos $dna, "\en"; |
| 1796 |
\& } |
| 1797 |
.Ve |
| 1798 |
.PP |
| 1799 |
which prints |
| 1800 |
.PP |
| 1801 |
.Vb 2 |
| 1802 |
\& Got a TGA stop codon at position 18 |
| 1803 |
\& Got a TGA stop codon at position 23 |
| 1804 |
.Ve |
| 1805 |
.PP |
| 1806 |
Position 18 is good, but position 23 is bogus. What happened? |
| 1807 |
.PP |
| 1808 |
The answer is that our regexp works well until we get past the last |
| 1809 |
real match. Then the regexp will fail to match a synchronized \f(CW\*(C`TGA\*(C'\fR |
| 1810 |
and start stepping ahead one character position at a time, not what we |
| 1811 |
want. The solution is to use \f(CW\*(C`\eG\*(C'\fR to anchor the match to the codon |
| 1812 |
alignment: |
| 1813 |
.PP |
| 1814 |
.Vb 3 |
| 1815 |
\& while ($dna =~ /\eG(\ew\ew\ew)*?TGA/g) { |
| 1816 |
\& print "Got a TGA stop codon at position ", pos $dna, "\en"; |
| 1817 |
\& } |
| 1818 |
.Ve |
| 1819 |
.PP |
| 1820 |
This prints |
| 1821 |
.PP |
| 1822 |
.Vb 1 |
| 1823 |
\& Got a TGA stop codon at position 18 |
| 1824 |
.Ve |
| 1825 |
.PP |
| 1826 |
which is the correct answer. This example illustrates that it is |
| 1827 |
important not only to match what is desired, but to reject what is not |
| 1828 |
desired. |
| 1829 |
.PP |
| 1830 |
(There are other regexp modifiers that are available, such as |
| 1831 |
\&\f(CW\*(C`/o\*(C'\fR, but their specialized uses are beyond the |
| 1832 |
scope of this introduction. ) |
| 1833 |
.PP |
| 1834 |
\fISearch and replace\fR |
| 1835 |
.IX Subsection "Search and replace" |
| 1836 |
.PP |
| 1837 |
Regular expressions also play a big role in \fIsearch and replace\fR |
| 1838 |
operations in Perl. Search and replace is accomplished with the |
| 1839 |
\&\f(CW\*(C`s///\*(C'\fR operator. The general form is |
| 1840 |
\&\f(CW\*(C`s/regexp/replacement/modifiers\*(C'\fR, with everything we know about |
| 1841 |
regexps and modifiers applying in this case as well. The |
| 1842 |
\&\fIreplacement\fR is a Perl double-quoted string that replaces in the |
| 1843 |
string whatever is matched with the \f(CW\*(C`regexp\*(C'\fR. The operator \f(CW\*(C`=~\*(C'\fR is |
| 1844 |
also used here to associate a string with \f(CW\*(C`s///\*(C'\fR. If matching |
| 1845 |
against \f(CW$_\fR, the \f(CW\*(C`$_\ =~\*(C'\fR can be dropped. If there is a match, |
| 1846 |
\&\f(CW\*(C`s///\*(C'\fR returns the number of substitutions made; otherwise it returns |
| 1847 |
false. Here are a few examples: |
| 1848 |
.PP |
| 1849 |
.Vb 8 |
| 1850 |
\& $x = "Time to feed the cat!"; |
| 1851 |
\& $x =~ s/cat/hacker/; # $x contains "Time to feed the hacker!" |
| 1852 |
\& if ($x =~ s/^(Time.*hacker)!$/$1 now!/) { |
| 1853 |
\& $more_insistent = 1; |
| 1854 |
\& } |
| 1855 |
\& $y = "\*(Aqquoted words\*(Aq"; |
| 1856 |
\& $y =~ s/^\*(Aq(.*)\*(Aq$/$1/; # strip single quotes, |
| 1857 |
\& # $y contains "quoted words" |
| 1858 |
.Ve |
| 1859 |
.PP |
| 1860 |
In the last example, the whole string was matched, but only the part |
| 1861 |
inside the single quotes was grouped. With the \f(CW\*(C`s///\*(C'\fR operator, the |
| 1862 |
matched variables \f(CW$1\fR, \f(CW$2\fR, \fIetc\fR. are immediately available for use |
| 1863 |
in the replacement expression, so we use \f(CW$1\fR to replace the quoted |
| 1864 |
string with just what was quoted. With the global modifier, \f(CW\*(C`s///g\*(C'\fR |
| 1865 |
will search and replace all occurrences of the regexp in the string: |
| 1866 |
.PP |
| 1867 |
.Vb 6 |
| 1868 |
\& $x = "I batted 4 for 4"; |
| 1869 |
\& $x =~ s/4/four/; # doesn\*(Aqt do it all: |
| 1870 |
\& # $x contains "I batted four for 4" |
| 1871 |
\& $x = "I batted 4 for 4"; |
| 1872 |
\& $x =~ s/4/four/g; # does it all: |
| 1873 |
\& # $x contains "I batted four for four" |
| 1874 |
.Ve |
| 1875 |
.PP |
| 1876 |
If you prefer "regex" over "regexp" in this tutorial, you could use |
| 1877 |
the following program to replace it: |
| 1878 |
.PP |
| 1879 |
.Vb 9 |
| 1880 |
\& % cat > simple_replace |
| 1881 |
\& #!/usr/bin/perl |
| 1882 |
\& $regexp = shift; |
| 1883 |
\& $replacement = shift; |
| 1884 |
\& while (<>) { |
| 1885 |
\& s/$regexp/$replacement/g; |
| 1886 |
\& print; |
| 1887 |
\& } |
| 1888 |
\& ^D |
| 1889 |
\& |
| 1890 |
\& % simple_replace regexp regex perlretut.pod |
| 1891 |
.Ve |
| 1892 |
.PP |
| 1893 |
In \f(CW\*(C`simple_replace\*(C'\fR we used the \f(CW\*(C`s///g\*(C'\fR modifier to replace all |
| 1894 |
occurrences of the regexp on each line. (Even though the regular |
| 1895 |
expression appears in a loop, Perl is smart enough to compile it |
| 1896 |
only once.) As with \f(CW\*(C`simple_grep\*(C'\fR, both the |
| 1897 |
\&\f(CW\*(C`print\*(C'\fR and the \f(CW\*(C`s/$regexp/$replacement/g\*(C'\fR use \f(CW$_\fR implicitly. |
| 1898 |
.PP |
| 1899 |
If you don't want \f(CW\*(C`s///\*(C'\fR to change your original variable you can use |
| 1900 |
the non-destructive substitute modifier, \f(CW\*(C`s///r\*(C'\fR. This changes the |
| 1901 |
behavior so that \f(CW\*(C`s///r\*(C'\fR returns the final substituted string |
| 1902 |
(instead of the number of substitutions): |
| 1903 |
.PP |
| 1904 |
.Vb 3 |
| 1905 |
\& $x = "I like dogs."; |
| 1906 |
\& $y = $x =~ s/dogs/cats/r; |
| 1907 |
\& print "$x $y\en"; |
| 1908 |
.Ve |
| 1909 |
.PP |
| 1910 |
That example will print "I like dogs. I like cats". Notice the original |
| 1911 |
\&\f(CW$x\fR variable has not been affected. The overall |
| 1912 |
result of the substitution is instead stored in \f(CW$y\fR. If the |
| 1913 |
substitution doesn't affect anything then the original string is |
| 1914 |
returned: |
| 1915 |
.PP |
| 1916 |
.Vb 3 |
| 1917 |
\& $x = "I like dogs."; |
| 1918 |
\& $y = $x =~ s/elephants/cougars/r; |
| 1919 |
\& print "$x $y\en"; # prints "I like dogs. I like dogs." |
| 1920 |
.Ve |
| 1921 |
.PP |
| 1922 |
One other interesting thing that the \f(CW\*(C`s///r\*(C'\fR flag allows is chaining |
| 1923 |
substitutions: |
| 1924 |
.PP |
| 1925 |
.Vb 4 |
| 1926 |
\& $x = "Cats are great."; |
| 1927 |
\& print $x =~ s/Cats/Dogs/r =~ s/Dogs/Frogs/r =~ |
| 1928 |
\& s/Frogs/Hedgehogs/r, "\en"; |
| 1929 |
\& # prints "Hedgehogs are great." |
| 1930 |
.Ve |
| 1931 |
.PP |
| 1932 |
A modifier available specifically to search and replace is the |
| 1933 |
\&\f(CW\*(C`s///e\*(C'\fR evaluation modifier. \f(CW\*(C`s///e\*(C'\fR treats the |
| 1934 |
replacement text as Perl code, rather than a double-quoted |
| 1935 |
string. The value that the code returns is substituted for the |
| 1936 |
matched substring. \f(CW\*(C`s///e\*(C'\fR is useful if you need to do a bit of |
| 1937 |
computation in the process of replacing text. This example counts |
| 1938 |
character frequencies in a line: |
| 1939 |
.PP |
| 1940 |
.Vb 4 |
| 1941 |
\& $x = "Bill the cat"; |
| 1942 |
\& $x =~ s/(.)/$chars{$1}++;$1/eg; # final $1 replaces char with itself |
| 1943 |
\& print "frequency of \*(Aq$_\*(Aq is $chars{$_}\en" |
| 1944 |
\& foreach (sort {$chars{$b} <=> $chars{$a}} keys %chars); |
| 1945 |
.Ve |
| 1946 |
.PP |
| 1947 |
This prints |
| 1948 |
.PP |
| 1949 |
.Vb 9 |
| 1950 |
\& frequency of \*(Aq \*(Aq is 2 |
| 1951 |
\& frequency of \*(Aqt\*(Aq is 2 |
| 1952 |
\& frequency of \*(Aql\*(Aq is 2 |
| 1953 |
\& frequency of \*(AqB\*(Aq is 1 |
| 1954 |
\& frequency of \*(Aqc\*(Aq is 1 |
| 1955 |
\& frequency of \*(Aqe\*(Aq is 1 |
| 1956 |
\& frequency of \*(Aqh\*(Aq is 1 |
| 1957 |
\& frequency of \*(Aqi\*(Aq is 1 |
| 1958 |
\& frequency of \*(Aqa\*(Aq is 1 |
| 1959 |
.Ve |
| 1960 |
.PP |
| 1961 |
As with the match \f(CW\*(C`m//\*(C'\fR operator, \f(CW\*(C`s///\*(C'\fR can use other delimiters, |
| 1962 |
such as \f(CW\*(C`s!!!\*(C'\fR and \f(CW\*(C`s{}{}\*(C'\fR, and even \f(CW\*(C`s{}//\*(C'\fR. If single quotes are |
| 1963 |
used \f(CW\*(C`s\*(Aq\*(Aq\*(Aq\*(C'\fR, then the regexp and replacement are |
| 1964 |
treated as single-quoted strings and there are no |
| 1965 |
variable substitutions. \f(CW\*(C`s///\*(C'\fR in list context |
| 1966 |
returns the same thing as in scalar context, \fIi.e.\fR, the number of |
| 1967 |
matches. |
| 1968 |
.PP |
| 1969 |
\fIThe split function\fR |
| 1970 |
.IX Subsection "The split function" |
| 1971 |
.PP |
| 1972 |
The \f(CWsplit()\fR function is another place where a regexp is used. |
| 1973 |
\&\f(CW\*(C`split /regexp/, string, limit\*(C'\fR separates the \f(CW\*(C`string\*(C'\fR operand into |
| 1974 |
a list of substrings and returns that list. The regexp must be designed |
| 1975 |
to match whatever constitutes the separators for the desired substrings. |
| 1976 |
The \f(CW\*(C`limit\*(C'\fR, if present, constrains splitting into no more than \f(CW\*(C`limit\*(C'\fR |
| 1977 |
number of strings. For example, to split a string into words, use |
| 1978 |
.PP |
| 1979 |
.Vb 4 |
| 1980 |
\& $x = "Calvin and Hobbes"; |
| 1981 |
\& @words = split /\es+/, $x; # $word[0] = \*(AqCalvin\*(Aq |
| 1982 |
\& # $word[1] = \*(Aqand\*(Aq |
| 1983 |
\& # $word[2] = \*(AqHobbes\*(Aq |
| 1984 |
.Ve |
| 1985 |
.PP |
| 1986 |
If the empty regexp \f(CW\*(C`//\*(C'\fR is used, the regexp always matches and |
| 1987 |
the string is split into individual characters. If the regexp has |
| 1988 |
groupings, then the resulting list contains the matched substrings from the |
| 1989 |
groupings as well. For instance, |
| 1990 |
.PP |
| 1991 |
.Vb 12 |
| 1992 |
\& $x = "/usr/bin/perl"; |
| 1993 |
\& @dirs = split m!/!, $x; # $dirs[0] = \*(Aq\*(Aq |
| 1994 |
\& # $dirs[1] = \*(Aqusr\*(Aq |
| 1995 |
\& # $dirs[2] = \*(Aqbin\*(Aq |
| 1996 |
\& # $dirs[3] = \*(Aqperl\*(Aq |
| 1997 |
\& @parts = split m!(/)!, $x; # $parts[0] = \*(Aq\*(Aq |
| 1998 |
\& # $parts[1] = \*(Aq/\*(Aq |
| 1999 |
\& # $parts[2] = \*(Aqusr\*(Aq |
| 2000 |
\& # $parts[3] = \*(Aq/\*(Aq |
| 2001 |
\& # $parts[4] = \*(Aqbin\*(Aq |
| 2002 |
\& # $parts[5] = \*(Aq/\*(Aq |
| 2003 |
\& # $parts[6] = \*(Aqperl\*(Aq |
| 2004 |
.Ve |
| 2005 |
.PP |
| 2006 |
Since the first character of \f(CW$x\fR matched the regexp, \f(CW\*(C`split\*(C'\fR prepended |
| 2007 |
an empty initial element to the list. |
| 2008 |
.PP |
| 2009 |
If you have read this far, congratulations! You now have all the basic |
| 2010 |
tools needed to use regular expressions to solve a wide range of text |
| 2011 |
processing problems. If this is your first time through the tutorial, |
| 2012 |
why not stop here and play around with regexps a while.... Part\ 2 |
| 2013 |
concerns the more esoteric aspects of regular expressions and those |
| 2014 |
concepts certainly aren't needed right at the start. |
| 2015 |
.SH "Part 2: Power tools" |
| 2016 |
.IX Header "Part 2: Power tools" |
| 2017 |
OK, you know the basics of regexps and you want to know more. If |
| 2018 |
matching regular expressions is analogous to a walk in the woods, then |
| 2019 |
the tools discussed in Part 1 are analogous to topo maps and a |
| 2020 |
compass, basic tools we use all the time. Most of the tools in part 2 |
| 2021 |
are analogous to flare guns and satellite phones. They aren't used |
| 2022 |
too often on a hike, but when we are stuck, they can be invaluable. |
| 2023 |
.PP |
| 2024 |
What follows are the more advanced, less used, or sometimes esoteric |
| 2025 |
capabilities of Perl regexps. In Part 2, we will assume you are |
| 2026 |
comfortable with the basics and concentrate on the advanced features. |
| 2027 |
.SS "More on characters, strings, and character classes" |
| 2028 |
.IX Subsection "More on characters, strings, and character classes" |
| 2029 |
There are a number of escape sequences and character classes that we |
| 2030 |
haven't covered yet. |
| 2031 |
.PP |
| 2032 |
There are several escape sequences that convert characters or strings |
| 2033 |
between upper and lower case, and they are also available within |
| 2034 |
patterns. \f(CW\*(C`\el\*(C'\fR and \f(CW\*(C`\eu\*(C'\fR convert the next character to lower or |
| 2035 |
upper case, respectively: |
| 2036 |
.PP |
| 2037 |
.Vb 4 |
| 2038 |
\& $x = "perl"; |
| 2039 |
\& $string =~ /\eu$x/; # matches \*(AqPerl\*(Aq in $string |
| 2040 |
\& $x = "M(rs?|s)\e\e."; # note the double backslash |
| 2041 |
\& $string =~ /\el$x/; # matches \*(Aqmr.\*(Aq, \*(Aqmrs.\*(Aq, and \*(Aqms.\*(Aq, |
| 2042 |
.Ve |
| 2043 |
.PP |
| 2044 |
A \f(CW\*(C`\eL\*(C'\fR or \f(CW\*(C`\eU\*(C'\fR indicates a lasting conversion of case, until |
| 2045 |
terminated by \f(CW\*(C`\eE\*(C'\fR or thrown over by another \f(CW\*(C`\eU\*(C'\fR or \f(CW\*(C`\eL\*(C'\fR: |
| 2046 |
.PP |
| 2047 |
.Vb 4 |
| 2048 |
\& $x = "This word is in lower case:\eL SHOUT\eE"; |
| 2049 |
\& $x =~ /shout/; # matches |
| 2050 |
\& $x = "I STILL KEYPUNCH CARDS FOR MY 360"; |
| 2051 |
\& $x =~ /\eUkeypunch/; # matches punch card string |
| 2052 |
.Ve |
| 2053 |
.PP |
| 2054 |
If there is no \f(CW\*(C`\eE\*(C'\fR, case is converted until the end of the |
| 2055 |
string. The regexps \f(CW\*(C`\eL\eu$word\*(C'\fR or \f(CW\*(C`\eu\eL$word\*(C'\fR convert the first |
| 2056 |
character of \f(CW$word\fR to uppercase and the rest of the characters to |
| 2057 |
lowercase. (Beyond ASCII characters, it gets somewhat more complicated; |
| 2058 |
\&\f(CW\*(C`\eu\*(C'\fR actually performs \fItitlecase\fR mapping, which for most characters |
| 2059 |
is the same as uppercase, but not for all; see |
| 2060 |
<https://unicode.org/faq/casemap_charprop.html#4>.) |
| 2061 |
.PP |
| 2062 |
Control characters can be escaped with \f(CW\*(C`\ec\*(C'\fR, so that a control-Z |
| 2063 |
character would be matched with \f(CW\*(C`\ecZ\*(C'\fR. The escape sequence |
| 2064 |
\&\f(CW\*(C`\eQ\*(C'\fR...\f(CW\*(C`\eE\*(C'\fR quotes, or protects most non-alphabetic characters. For |
| 2065 |
instance, |
| 2066 |
.PP |
| 2067 |
.Vb 2 |
| 2068 |
\& $x = "\eQThat !^*&%~& cat!"; |
| 2069 |
\& $x =~ /\eQ!^*&%~&\eE/; # check for rough language |
| 2070 |
.Ve |
| 2071 |
.PP |
| 2072 |
It does not protect \f(CW\*(Aq$\*(Aq\fR or \f(CW\*(Aq@\*(Aq\fR, so that variables can still be |
| 2073 |
substituted. |
| 2074 |
.PP |
| 2075 |
\&\f(CW\*(C`\eQ\*(C'\fR, \f(CW\*(C`\eL\*(C'\fR, \f(CW\*(C`\el\*(C'\fR, \f(CW\*(C`\eU\*(C'\fR, \f(CW\*(C`\eu\*(C'\fR and \f(CW\*(C`\eE\*(C'\fR are actually part of |
| 2076 |
double-quotish syntax, and not part of regexp syntax proper. They will |
| 2077 |
work if they appear in a regular expression embedded directly in a |
| 2078 |
program, but not when contained in a string that is interpolated in a |
| 2079 |
pattern. |
| 2080 |
.PP |
| 2081 |
Perl regexps can handle more than just the |
| 2082 |
standard ASCII character set. Perl supports \fIUnicode\fR, a standard |
| 2083 |
for representing the alphabets from virtually all of the world's written |
| 2084 |
languages, and a host of symbols. Perl's text strings are Unicode strings, so |
| 2085 |
they can contain characters with a value (codepoint or character number) higher |
| 2086 |
than 255. |
| 2087 |
.PP |
| 2088 |
What does this mean for regexps? Well, regexp users don't need to know |
| 2089 |
much about Perl's internal representation of strings. But they do need |
| 2090 |
to know 1) how to represent Unicode characters in a regexp and 2) that |
| 2091 |
a matching operation will treat the string to be searched as a sequence |
| 2092 |
of characters, not bytes. The answer to 1) is that Unicode characters |
| 2093 |
greater than \f(CWchr(255)\fR are represented using the \f(CW\*(C`\ex{hex}\*(C'\fR notation, because |
| 2094 |
\&\f(CW\*(C`\ex\*(C'\fR\fIXY\fR (without curly braces and \fIXY\fR are two hex digits) doesn't |
| 2095 |
go further than 255. (Starting in Perl 5.14, if you're an octal fan, |
| 2096 |
you can also use \f(CW\*(C`\eo{oct}\*(C'\fR.) |
| 2097 |
.PP |
| 2098 |
.Vb 2 |
| 2099 |
\& /\ex{263a}/; # match a Unicode smiley face :) |
| 2100 |
\& /\ex{ 263a }/; # Same |
| 2101 |
.Ve |
| 2102 |
.PP |
| 2103 |
\&\fBNOTE\fR: In Perl 5.6.0 it used to be that one needed to say \f(CW\*(C`use |
| 2104 |
utf8\*(C'\fR to use any Unicode features. This is no longer the case: for |
| 2105 |
almost all Unicode processing, the explicit \f(CW\*(C`utf8\*(C'\fR pragma is not |
| 2106 |
needed. (The only case where it matters is if your Perl script is in |
| 2107 |
Unicode and encoded in UTF\-8, then an explicit \f(CW\*(C`use utf8\*(C'\fR is needed.) |
| 2108 |
.PP |
| 2109 |
Figuring out the hexadecimal sequence of a Unicode character you want |
| 2110 |
or deciphering someone else's hexadecimal Unicode regexp is about as |
| 2111 |
much fun as programming in machine code. So another way to specify |
| 2112 |
Unicode characters is to use the \fInamed character\fR escape |
| 2113 |
sequence \f(CW\*(C`\eN{\fR\f(CIname\fR\f(CW}\*(C'\fR. \fIname\fR is a name for the Unicode character, as |
| 2114 |
specified in the Unicode standard. For instance, if we wanted to |
| 2115 |
represent or match the astrological sign for the planet Mercury, we |
| 2116 |
could use |
| 2117 |
.PP |
| 2118 |
.Vb 3 |
| 2119 |
\& $x = "abc\eN{MERCURY}def"; |
| 2120 |
\& $x =~ /\eN{MERCURY}/; # matches |
| 2121 |
\& $x =~ /\eN{ MERCURY }/; # Also matches |
| 2122 |
.Ve |
| 2123 |
.PP |
| 2124 |
One can also use "short" names: |
| 2125 |
.PP |
| 2126 |
.Vb 2 |
| 2127 |
\& print "\eN{GREEK SMALL LETTER SIGMA} is called sigma.\en"; |
| 2128 |
\& print "\eN{greek:Sigma} is an upper\-case sigma.\en"; |
| 2129 |
.Ve |
| 2130 |
.PP |
| 2131 |
You can also restrict names to a certain alphabet by specifying the |
| 2132 |
charnames pragma: |
| 2133 |
.PP |
| 2134 |
.Vb 2 |
| 2135 |
\& use charnames qw(greek); |
| 2136 |
\& print "\eN{sigma} is Greek sigma\en"; |
| 2137 |
.Ve |
| 2138 |
.PP |
| 2139 |
An index of character names is available on-line from the Unicode |
| 2140 |
Consortium, <https://www.unicode.org/charts/charindex.html>; explanatory |
| 2141 |
material with links to other resources at |
| 2142 |
<https://www.unicode.org/standard/where>. |
| 2143 |
.PP |
| 2144 |
Starting in Perl v5.32, an alternative to \f(CW\*(C`\eN{...}\*(C'\fR for full names is |
| 2145 |
available, and that is to say |
| 2146 |
.PP |
| 2147 |
.Vb 1 |
| 2148 |
\& /\ep{Name=greek small letter sigma}/ |
| 2149 |
.Ve |
| 2150 |
.PP |
| 2151 |
The casing of the character name is irrelevant when used in \f(CW\*(C`\ep{}\*(C'\fR, as |
| 2152 |
are most spaces, underscores and hyphens. (A few outlier characters |
| 2153 |
cause problems with ignoring all of them always. The details (which you |
| 2154 |
can look up when you get more proficient, and if ever needed) are in |
| 2155 |
<https://www.unicode.org/reports/tr44/tr44\-24.html#UAX44\-LM2>). |
| 2156 |
.PP |
| 2157 |
The answer to requirement 2) is that a regexp (mostly) |
| 2158 |
uses Unicode characters. The "mostly" is for messy backward |
| 2159 |
compatibility reasons, but starting in Perl 5.14, any regexp compiled in |
| 2160 |
the scope of a \f(CW\*(C`use feature \*(Aqunicode_strings\*(Aq\*(C'\fR (which is automatically |
| 2161 |
turned on within the scope of a \f(CW\*(C`use v5.12\*(C'\fR or higher) will turn that |
| 2162 |
"mostly" into "always". If you want to handle Unicode properly, you |
| 2163 |
should ensure that \f(CW\*(Aqunicode_strings\*(Aq\fR is turned on. |
| 2164 |
Internally, this is encoded to bytes using either UTF\-8 or a native 8 |
| 2165 |
bit encoding, depending on the history of the string, but conceptually |
| 2166 |
it is a sequence of characters, not bytes. See perlunitut for a |
| 2167 |
tutorial about that. |
| 2168 |
.PP |
| 2169 |
Let us now discuss Unicode character classes, most usually called |
| 2170 |
"character properties". These are represented by the \f(CW\*(C`\ep{\fR\f(CIname\fR\f(CW}\*(C'\fR |
| 2171 |
escape sequence. The negation of this is \f(CW\*(C`\eP{\fR\f(CIname\fR\f(CW}\*(C'\fR. For example, |
| 2172 |
to match lower and uppercase characters, |
| 2173 |
.PP |
| 2174 |
.Vb 5 |
| 2175 |
\& $x = "BOB"; |
| 2176 |
\& $x =~ /^\ep{IsUpper}/; # matches, uppercase char class |
| 2177 |
\& $x =~ /^\eP{IsUpper}/; # doesn\*(Aqt match, char class sans uppercase |
| 2178 |
\& $x =~ /^\ep{IsLower}/; # doesn\*(Aqt match, lowercase char class |
| 2179 |
\& $x =~ /^\eP{IsLower}/; # matches, char class sans lowercase |
| 2180 |
.Ve |
| 2181 |
.PP |
| 2182 |
(The "\f(CW\*(C`Is\*(C'\fR" is optional.) |
| 2183 |
.PP |
| 2184 |
There are many, many Unicode character properties. For the full list |
| 2185 |
see perluniprops. Most of them have synonyms with shorter names, |
| 2186 |
also listed there. Some synonyms are a single character. For these, |
| 2187 |
you can drop the braces. For instance, \f(CW\*(C`\epM\*(C'\fR is the same thing as |
| 2188 |
\&\f(CW\*(C`\ep{Mark}\*(C'\fR, meaning things like accent marks. |
| 2189 |
.PP |
| 2190 |
The Unicode \f(CW\*(C`\ep{Script}\*(C'\fR and \f(CW\*(C`\ep{Script_Extensions}\*(C'\fR properties are |
| 2191 |
used to categorize every Unicode character into the language script it |
| 2192 |
is written in. For example, |
| 2193 |
English, French, and a bunch of other European languages are written in |
| 2194 |
the Latin script. But there is also the Greek script, the Thai script, |
| 2195 |
the Katakana script, \fIetc\fR. (\f(CW\*(C`Script\*(C'\fR is an older, less advanced, |
| 2196 |
form of \f(CW\*(C`Script_Extensions\*(C'\fR, retained only for backwards |
| 2197 |
compatibility.) You can test whether a character is in a particular |
| 2198 |
script with, for example \f(CW\*(C`\ep{Latin}\*(C'\fR, \f(CW\*(C`\ep{Greek}\*(C'\fR, or |
| 2199 |
\&\f(CW\*(C`\ep{Katakana}\*(C'\fR. To test if it isn't in the Balinese script, you would |
| 2200 |
use \f(CW\*(C`\eP{Balinese}\*(C'\fR. (These all use \f(CW\*(C`Script_Extensions\*(C'\fR under the |
| 2201 |
hood, as that gives better results.) |
| 2202 |
.PP |
| 2203 |
What we have described so far is the single form of the \f(CW\*(C`\ep{...}\*(C'\fR character |
| 2204 |
classes. There is also a compound form which you may run into. These |
| 2205 |
look like \f(CW\*(C`\ep{\fR\f(CIname\fR\f(CW=\fR\f(CIvalue\fR\f(CW}\*(C'\fR or \f(CW\*(C`\ep{\fR\f(CIname\fR\f(CW:\fR\f(CIvalue\fR\f(CW}\*(C'\fR (the equals sign and colon |
| 2206 |
can be used interchangeably). These are more general than the single form, |
| 2207 |
and in fact most of the single forms are just Perl-defined shortcuts for common |
| 2208 |
compound forms. For example, the script examples in the previous paragraph |
| 2209 |
could be written equivalently as \f(CW\*(C`\ep{Script_Extensions=Latin}\*(C'\fR, \f(CW\*(C`\ep{Script_Extensions:Greek}\*(C'\fR, |
| 2210 |
\&\f(CW\*(C`\ep{script_extensions=katakana}\*(C'\fR, and \f(CW\*(C`\eP{script_extensions=balinese}\*(C'\fR (case is irrelevant |
| 2211 |
between the \f(CW\*(C`{}\*(C'\fR braces). You may |
| 2212 |
never have to use the compound forms, but sometimes it is necessary, and their |
| 2213 |
use can make your code easier to understand. |
| 2214 |
.PP |
| 2215 |
\&\f(CW\*(C`\eX\*(C'\fR is an abbreviation for a character class that comprises |
| 2216 |
a Unicode \fIextended grapheme cluster\fR. This represents a "logical character": |
| 2217 |
what appears to be a single character, but may be represented internally by more |
| 2218 |
than one. As an example, using the Unicode full names, \fIe.g.\fR, "A\ +\ COMBINING\ RING" is a grapheme cluster with base character "A" and combining character |
| 2219 |
"COMBINING\ RING, which translates in Danish to "A" with the circle atop it, |
| 2220 |
as in the word Ã…ngstrom. |
| 2221 |
.PP |
| 2222 |
For the full and latest information about Unicode see the latest |
| 2223 |
Unicode standard, or the Unicode Consortium's website <https://www.unicode.org> |
| 2224 |
.PP |
| 2225 |
As if all those classes weren't enough, Perl also defines POSIX-style |
| 2226 |
character classes. These have the form \f(CW\*(C`[:\fR\f(CIname\fR\f(CW:]\*(C'\fR, with \fIname\fR the |
| 2227 |
name of the POSIX class. The POSIX classes are \f(CW\*(C`alpha\*(C'\fR, \f(CW\*(C`alnum\*(C'\fR, |
| 2228 |
\&\f(CW\*(C`ascii\*(C'\fR, \f(CW\*(C`blank\*(C'\fR, \f(CW\*(C`cntrl\*(C'\fR, \f(CW\*(C`digit\*(C'\fR, \f(CW\*(C`graph\*(C'\fR, \f(CW\*(C`lower\*(C'\fR, \f(CW\*(C`print\*(C'\fR, |
| 2229 |
\&\f(CW\*(C`punct\*(C'\fR, \f(CW\*(C`space\*(C'\fR, \f(CW\*(C`upper\*(C'\fR, \f(CW\*(C`xdigit\*(C'\fR, and \f(CW\*(C`word\*(C'\fR (a Perl extension to |
| 2230 |
match \f(CW\*(C`\ew\*(C'\fR). The \f(CW\*(C`/a\*(C'\fR |
| 2231 |
modifier restricts these to matching just in the ASCII range; otherwise |
| 2232 |
they can match the same as their corresponding Perl Unicode classes: |
| 2233 |
\&\f(CW\*(C`[:upper:]\*(C'\fR is the same as \f(CW\*(C`\ep{IsUpper}\*(C'\fR, \fIetc\fR. (There are some |
| 2234 |
exceptions and gotchas with this; see perlrecharclass for a full |
| 2235 |
discussion.) The \f(CW\*(C`[:digit:]\*(C'\fR, \f(CW\*(C`[:word:]\*(C'\fR, and |
| 2236 |
\&\f(CW\*(C`[:space:]\*(C'\fR correspond to the familiar \f(CW\*(C`\ed\*(C'\fR, \f(CW\*(C`\ew\*(C'\fR, and \f(CW\*(C`\es\*(C'\fR |
| 2237 |
character classes. To negate a POSIX class, put a \f(CW\*(Aq^\*(Aq\fR in front of |
| 2238 |
the name, so that, \fIe.g.\fR, \f(CW\*(C`[:^digit:]\*(C'\fR corresponds to \f(CW\*(C`\eD\*(C'\fR and, under |
| 2239 |
Unicode, \f(CW\*(C`\eP{IsDigit}\*(C'\fR. The Unicode and POSIX character classes can |
| 2240 |
be used just like \f(CW\*(C`\ed\*(C'\fR, with the exception that POSIX character |
| 2241 |
classes can only be used inside of a character class: |
| 2242 |
.PP |
| 2243 |
.Vb 6 |
| 2244 |
\& /\es+[abc[:digit:]xyz]\es*/; # match a,b,c,x,y,z, or a digit |
| 2245 |
\& /^=item\es[[:digit:]]/; # match \*(Aq=item\*(Aq, |
| 2246 |
\& # followed by a space and a digit |
| 2247 |
\& /\es+[abc\ep{IsDigit}xyz]\es+/; # match a,b,c,x,y,z, or a digit |
| 2248 |
\& /^=item\es\ep{IsDigit}/; # match \*(Aq=item\*(Aq, |
| 2249 |
\& # followed by a space and a digit |
| 2250 |
.Ve |
| 2251 |
.PP |
| 2252 |
Whew! That is all the rest of the characters and character classes. |
| 2253 |
.SS "Compiling and saving regular expressions" |
| 2254 |
.IX Subsection "Compiling and saving regular expressions" |
| 2255 |
In Part 1 we mentioned that Perl compiles a regexp into a compact |
| 2256 |
sequence of opcodes. Thus, a compiled regexp is a data structure |
| 2257 |
that can be stored once and used again and again. The regexp quote |
| 2258 |
\&\f(CW\*(C`qr//\*(C'\fR does exactly that: \f(CW\*(C`qr/string/\*(C'\fR compiles the \f(CW\*(C`string\*(C'\fR as a |
| 2259 |
regexp and transforms the result into a form that can be assigned to a |
| 2260 |
variable: |
| 2261 |
.PP |
| 2262 |
.Vb 1 |
| 2263 |
\& $reg = qr/foo+bar?/; # reg contains a compiled regexp |
| 2264 |
.Ve |
| 2265 |
.PP |
| 2266 |
Then \f(CW$reg\fR can be used as a regexp: |
| 2267 |
.PP |
| 2268 |
.Vb 3 |
| 2269 |
\& $x = "fooooba"; |
| 2270 |
\& $x =~ $reg; # matches, just like /foo+bar?/ |
| 2271 |
\& $x =~ /$reg/; # same thing, alternate form |
| 2272 |
.Ve |
| 2273 |
.PP |
| 2274 |
\&\f(CW$reg\fR can also be interpolated into a larger regexp: |
| 2275 |
.PP |
| 2276 |
.Vb 1 |
| 2277 |
\& $x =~ /(abc)?$reg/; # still matches |
| 2278 |
.Ve |
| 2279 |
.PP |
| 2280 |
As with the matching operator, the regexp quote can use different |
| 2281 |
delimiters, \fIe.g.\fR, \f(CW\*(C`qr!!\*(C'\fR, \f(CW\*(C`qr{}\*(C'\fR or \f(CW\*(C`qr~~\*(C'\fR. Apostrophes |
| 2282 |
as delimiters (\f(CW\*(C`qr\*(Aq\*(Aq\*(C'\fR) inhibit any interpolation. |
| 2283 |
.PP |
| 2284 |
Pre-compiled regexps are useful for creating dynamic matches that |
| 2285 |
don't need to be recompiled each time they are encountered. Using |
| 2286 |
pre-compiled regexps, we write a \f(CW\*(C`grep_step\*(C'\fR program which greps |
| 2287 |
for a sequence of patterns, advancing to the next pattern as soon |
| 2288 |
as one has been satisfied. |
| 2289 |
.PP |
| 2290 |
.Vb 4 |
| 2291 |
\& % cat > grep_step |
| 2292 |
\& #!/usr/bin/perl |
| 2293 |
\& # grep_step \- match <number> regexps, one after the other |
| 2294 |
\& # usage: multi_grep <number> regexp1 regexp2 ... file1 file2 ... |
| 2295 |
\& |
| 2296 |
\& $number = shift; |
| 2297 |
\& $regexp[$_] = shift foreach (0..$number\-1); |
| 2298 |
\& @compiled = map qr/$_/, @regexp; |
| 2299 |
\& while ($line = <>) { |
| 2300 |
\& if ($line =~ /$compiled[0]/) { |
| 2301 |
\& print $line; |
| 2302 |
\& shift @compiled; |
| 2303 |
\& last unless @compiled; |
| 2304 |
\& } |
| 2305 |
\& } |
| 2306 |
\& ^D |
| 2307 |
\& |
| 2308 |
\& % grep_step 3 shift print last grep_step |
| 2309 |
\& $number = shift; |
| 2310 |
\& print $line; |
| 2311 |
\& last unless @compiled; |
| 2312 |
.Ve |
| 2313 |
.PP |
| 2314 |
Storing pre-compiled regexps in an array \f(CW@compiled\fR allows us to |
| 2315 |
simply loop through the regexps without any recompilation, thus gaining |
| 2316 |
flexibility without sacrificing speed. |
| 2317 |
.SS "Composing regular expressions at runtime" |
| 2318 |
.IX Subsection "Composing regular expressions at runtime" |
| 2319 |
Backtracking is more efficient than repeated tries with different regular |
| 2320 |
expressions. If there are several regular expressions and a match with |
| 2321 |
any of them is acceptable, then it is possible to combine them into a set |
| 2322 |
of alternatives. If the individual expressions are input data, this |
| 2323 |
can be done by programming a join operation. We'll exploit this idea in |
| 2324 |
an improved version of the \f(CW\*(C`simple_grep\*(C'\fR program: a program that matches |
| 2325 |
multiple patterns: |
| 2326 |
.PP |
| 2327 |
.Vb 4 |
| 2328 |
\& % cat > multi_grep |
| 2329 |
\& #!/usr/bin/perl |
| 2330 |
\& # multi_grep \- match any of <number> regexps |
| 2331 |
\& # usage: multi_grep <number> regexp1 regexp2 ... file1 file2 ... |
| 2332 |
\& |
| 2333 |
\& $number = shift; |
| 2334 |
\& $regexp[$_] = shift foreach (0..$number\-1); |
| 2335 |
\& $pattern = join \*(Aq|\*(Aq, @regexp; |
| 2336 |
\& |
| 2337 |
\& while ($line = <>) { |
| 2338 |
\& print $line if $line =~ /$pattern/; |
| 2339 |
\& } |
| 2340 |
\& ^D |
| 2341 |
\& |
| 2342 |
\& % multi_grep 2 shift for multi_grep |
| 2343 |
\& $number = shift; |
| 2344 |
\& $regexp[$_] = shift foreach (0..$number\-1); |
| 2345 |
.Ve |
| 2346 |
.PP |
| 2347 |
Sometimes it is advantageous to construct a pattern from the \fIinput\fR |
| 2348 |
that is to be analyzed and use the permissible values on the left |
| 2349 |
hand side of the matching operations. As an example for this somewhat |
| 2350 |
paradoxical situation, let's assume that our input contains a command |
| 2351 |
verb which should match one out of a set of available command verbs, |
| 2352 |
with the additional twist that commands may be abbreviated as long as |
| 2353 |
the given string is unique. The program below demonstrates the basic |
| 2354 |
algorithm. |
| 2355 |
.PP |
| 2356 |
.Vb 10 |
| 2357 |
\& % cat > keymatch |
| 2358 |
\& #!/usr/bin/perl |
| 2359 |
\& $kwds = \*(Aqcopy compare list print\*(Aq; |
| 2360 |
\& while( $cmd = <> ){ |
| 2361 |
\& $cmd =~ s/^\es+|\es+$//g; # trim leading and trailing spaces |
| 2362 |
\& if( ( @matches = $kwds =~ /\eb$cmd\ew*/g ) == 1 ){ |
| 2363 |
\& print "command: \*(Aq@matches\*(Aq\en"; |
| 2364 |
\& } elsif( @matches == 0 ){ |
| 2365 |
\& print "no such command: \*(Aq$cmd\*(Aq\en"; |
| 2366 |
\& } else { |
| 2367 |
\& print "not unique: \*(Aq$cmd\*(Aq (could be one of: @matches)\en"; |
| 2368 |
\& } |
| 2369 |
\& } |
| 2370 |
\& ^D |
| 2371 |
\& |
| 2372 |
\& % keymatch |
| 2373 |
\& li |
| 2374 |
\& command: \*(Aqlist\*(Aq |
| 2375 |
\& co |
| 2376 |
\& not unique: \*(Aqco\*(Aq (could be one of: copy compare) |
| 2377 |
\& printer |
| 2378 |
\& no such command: \*(Aqprinter\*(Aq |
| 2379 |
.Ve |
| 2380 |
.PP |
| 2381 |
Rather than trying to match the input against the keywords, we match the |
| 2382 |
combined set of keywords against the input. The pattern matching |
| 2383 |
operation \f(CW\*(C`$kwds\ =~\ /\eb($cmd\ew*)/g\*(C'\fR does several things at the |
| 2384 |
same time. It makes sure that the given command begins where a keyword |
| 2385 |
begins (\f(CW\*(C`\eb\*(C'\fR). It tolerates abbreviations due to the added \f(CW\*(C`\ew*\*(C'\fR. It |
| 2386 |
tells us the number of matches (\f(CW\*(C`scalar @matches\*(C'\fR) and all the keywords |
| 2387 |
that were actually matched. You could hardly ask for more. |
| 2388 |
.SS "Embedding comments and modifiers in a regular expression" |
| 2389 |
.IX Subsection "Embedding comments and modifiers in a regular expression" |
| 2390 |
Starting with this section, we will be discussing Perl's set of |
| 2391 |
\&\fIextended patterns\fR. These are extensions to the traditional regular |
| 2392 |
expression syntax that provide powerful new tools for pattern |
| 2393 |
matching. We have already seen extensions in the form of the minimal |
| 2394 |
matching constructs \f(CW\*(C`??\*(C'\fR, \f(CW\*(C`*?\*(C'\fR, \f(CW\*(C`+?\*(C'\fR, \f(CW\*(C`{n,m}?\*(C'\fR, \f(CW\*(C`{n,}?\*(C'\fR, and |
| 2395 |
\&\f(CW\*(C`{,n}?\*(C'\fR. Most of the extensions below have the form \f(CW\*(C`(?char...)\*(C'\fR, |
| 2396 |
where the \f(CW\*(C`char\*(C'\fR is a character that determines the type of extension. |
| 2397 |
.PP |
| 2398 |
The first extension is an embedded comment \f(CW\*(C`(?#text)\*(C'\fR. This embeds a |
| 2399 |
comment into the regular expression without affecting its meaning. The |
| 2400 |
comment should not have any closing parentheses in the text. An |
| 2401 |
example is |
| 2402 |
.PP |
| 2403 |
.Vb 1 |
| 2404 |
\& /(?# Match an integer:)[+\-]?\ed+/; |
| 2405 |
.Ve |
| 2406 |
.PP |
| 2407 |
This style of commenting has been largely superseded by the raw, |
| 2408 |
freeform commenting that is allowed with the \f(CW\*(C`/x\*(C'\fR modifier. |
| 2409 |
.PP |
| 2410 |
Most modifiers, such as \f(CW\*(C`/i\*(C'\fR, \f(CW\*(C`/m\*(C'\fR, \f(CW\*(C`/s\*(C'\fR and \f(CW\*(C`/x\*(C'\fR (or any |
| 2411 |
combination thereof) can also be embedded in |
| 2412 |
a regexp using \f(CW\*(C`(?i)\*(C'\fR, \f(CW\*(C`(?m)\*(C'\fR, \f(CW\*(C`(?s)\*(C'\fR, and \f(CW\*(C`(?x)\*(C'\fR. For instance, |
| 2413 |
.PP |
| 2414 |
.Vb 7 |
| 2415 |
\& /(?i)yes/; # match \*(Aqyes\*(Aq case insensitively |
| 2416 |
\& /yes/i; # same thing |
| 2417 |
\& /(?x)( # freeform version of an integer regexp |
| 2418 |
\& [+\-]? # match an optional sign |
| 2419 |
\& \ed+ # match a sequence of digits |
| 2420 |
\& ) |
| 2421 |
\& /x; |
| 2422 |
.Ve |
| 2423 |
.PP |
| 2424 |
Embedded modifiers can have two important advantages over the usual |
| 2425 |
modifiers. Embedded modifiers allow a custom set of modifiers for |
| 2426 |
\&\fIeach\fR regexp pattern. This is great for matching an array of regexps |
| 2427 |
that must have different modifiers: |
| 2428 |
.PP |
| 2429 |
.Vb 8 |
| 2430 |
\& $pattern[0] = \*(Aq(?i)doctor\*(Aq; |
| 2431 |
\& $pattern[1] = \*(AqJohnson\*(Aq; |
| 2432 |
\& ... |
| 2433 |
\& while (<>) { |
| 2434 |
\& foreach $patt (@pattern) { |
| 2435 |
\& print if /$patt/; |
| 2436 |
\& } |
| 2437 |
\& } |
| 2438 |
.Ve |
| 2439 |
.PP |
| 2440 |
The second advantage is that embedded modifiers (except \f(CW\*(C`/p\*(C'\fR, which |
| 2441 |
modifies the entire regexp) only affect the regexp |
| 2442 |
inside the group the embedded modifier is contained in. So grouping |
| 2443 |
can be used to localize the modifier's effects: |
| 2444 |
.PP |
| 2445 |
.Vb 1 |
| 2446 |
\& /Answer: ((?i)yes)/; # matches \*(AqAnswer: yes\*(Aq, \*(AqAnswer: YES\*(Aq, etc. |
| 2447 |
.Ve |
| 2448 |
.PP |
| 2449 |
Embedded modifiers can also turn off any modifiers already present |
| 2450 |
by using, \fIe.g.\fR, \f(CW\*(C`(?\-i)\*(C'\fR. Modifiers can also be combined into |
| 2451 |
a single expression, \fIe.g.\fR, \f(CW\*(C`(?s\-i)\*(C'\fR turns on single line mode and |
| 2452 |
turns off case insensitivity. |
| 2453 |
.PP |
| 2454 |
Embedded modifiers may also be added to a non-capturing grouping. |
| 2455 |
\&\f(CW\*(C`(?i\-m:regexp)\*(C'\fR is a non-capturing grouping that matches \f(CW\*(C`regexp\*(C'\fR |
| 2456 |
case insensitively and turns off multi-line mode. |
| 2457 |
.SS "Looking ahead and looking behind" |
| 2458 |
.IX Subsection "Looking ahead and looking behind" |
| 2459 |
This section concerns the lookahead and lookbehind assertions. First, |
| 2460 |
a little background. |
| 2461 |
.PP |
| 2462 |
In Perl regular expressions, most regexp elements "eat up" a certain |
| 2463 |
amount of string when they match. For instance, the regexp element |
| 2464 |
\&\f(CW\*(C`[abc]\*(C'\fR eats up one character of the string when it matches, in the |
| 2465 |
sense that Perl moves to the next character position in the string |
| 2466 |
after the match. There are some elements, however, that don't eat up |
| 2467 |
characters (advance the character position) if they match. The examples |
| 2468 |
we have seen so far are the anchors. The anchor \f(CW\*(Aq^\*(Aq\fR matches the |
| 2469 |
beginning of the line, but doesn't eat any characters. Similarly, the |
| 2470 |
word boundary anchor \f(CW\*(C`\eb\*(C'\fR matches wherever a character matching \f(CW\*(C`\ew\*(C'\fR |
| 2471 |
is next to a character that doesn't, but it doesn't eat up any |
| 2472 |
characters itself. Anchors are examples of \fIzero-width assertions\fR: |
| 2473 |
zero-width, because they consume |
| 2474 |
no characters, and assertions, because they test some property of the |
| 2475 |
string. In the context of our walk in the woods analogy to regexp |
| 2476 |
matching, most regexp elements move us along a trail, but anchors have |
| 2477 |
us stop a moment and check our surroundings. If the local environment |
| 2478 |
checks out, we can proceed forward. But if the local environment |
| 2479 |
doesn't satisfy us, we must backtrack. |
| 2480 |
.PP |
| 2481 |
Checking the environment entails either looking ahead on the trail, |
| 2482 |
looking behind, or both. \f(CW\*(Aq^\*(Aq\fR looks behind, to see that there are no |
| 2483 |
characters before. \f(CW\*(Aq$\*(Aq\fR looks ahead, to see that there are no |
| 2484 |
characters after. \f(CW\*(C`\eb\*(C'\fR looks both ahead and behind, to see if the |
| 2485 |
characters on either side differ in their "word-ness". |
| 2486 |
.PP |
| 2487 |
The lookahead and lookbehind assertions are generalizations of the |
| 2488 |
anchor concept. Lookahead and lookbehind are zero-width assertions |
| 2489 |
that let us specify which characters we want to test for. The |
| 2490 |
lookahead assertion is denoted by \f(CW\*(C`(?=regexp)\*(C'\fR or (starting in 5.32, |
| 2491 |
experimentally in 5.28) \f(CW\*(C`(*pla:regexp)\*(C'\fR or |
| 2492 |
\&\f(CW\*(C`(*positive_lookahead:regexp)\*(C'\fR; and the lookbehind assertion is denoted |
| 2493 |
by \f(CW\*(C`(?<=fixed\-regexp)\*(C'\fR or (starting in 5.32, experimentally in |
| 2494 |
5.28) \f(CW\*(C`(*plb:fixed\-regexp)\*(C'\fR or \f(CW\*(C`(*positive_lookbehind:fixed\-regexp)\*(C'\fR. |
| 2495 |
Some examples are |
| 2496 |
.PP |
| 2497 |
.Vb 8 |
| 2498 |
\& $x = "I catch the housecat \*(AqTom\-cat\*(Aq with catnip"; |
| 2499 |
\& $x =~ /cat(*pla:\es)/; # matches \*(Aqcat\*(Aq in \*(Aqhousecat\*(Aq |
| 2500 |
\& @catwords = ($x =~ /(?<=\es)cat\ew+/g); # matches, |
| 2501 |
\& # $catwords[0] = \*(Aqcatch\*(Aq |
| 2502 |
\& # $catwords[1] = \*(Aqcatnip\*(Aq |
| 2503 |
\& $x =~ /\ebcat\eb/; # matches \*(Aqcat\*(Aq in \*(AqTom\-cat\*(Aq |
| 2504 |
\& $x =~ /(?<=\es)cat(?=\es)/; # doesn\*(Aqt match; no isolated \*(Aqcat\*(Aq in |
| 2505 |
\& # middle of $x |
| 2506 |
.Ve |
| 2507 |
.PP |
| 2508 |
Note that the parentheses in these are |
| 2509 |
non-capturing, since these are zero-width assertions. Thus in the |
| 2510 |
second regexp, the substrings captured are those of the whole regexp |
| 2511 |
itself. Lookahead can match arbitrary regexps, but |
| 2512 |
lookbehind prior to 5.30 \f(CW\*(C`(?<=fixed\-regexp)\*(C'\fR only works for regexps |
| 2513 |
of fixed width, \fIi.e.\fR, a fixed number of characters long. Thus |
| 2514 |
\&\f(CW\*(C`(?<=(ab|bc))\*(C'\fR is fine, but \f(CW\*(C`(?<=(ab)*)\*(C'\fR prior to 5.30 is not. |
| 2515 |
.PP |
| 2516 |
The negated versions of the lookahead and lookbehind assertions are |
| 2517 |
denoted by \f(CW\*(C`(?!regexp)\*(C'\fR and \f(CW\*(C`(?<!fixed\-regexp)\*(C'\fR respectively. |
| 2518 |
Or, starting in 5.32 (experimentally in 5.28), \f(CW\*(C`(*nla:regexp)\*(C'\fR, |
| 2519 |
\&\f(CW\*(C`(*negative_lookahead:regexp)\*(C'\fR, \f(CW\*(C`(*nlb:regexp)\*(C'\fR, or |
| 2520 |
\&\f(CW\*(C`(*negative_lookbehind:regexp)\*(C'\fR. |
| 2521 |
They evaluate true if the regexps do \fInot\fR match: |
| 2522 |
.PP |
| 2523 |
.Vb 4 |
| 2524 |
\& $x = "foobar"; |
| 2525 |
\& $x =~ /foo(?!bar)/; # doesn\*(Aqt match, \*(Aqbar\*(Aq follows \*(Aqfoo\*(Aq |
| 2526 |
\& $x =~ /foo(?!baz)/; # matches, \*(Aqbaz\*(Aq doesn\*(Aqt follow \*(Aqfoo\*(Aq |
| 2527 |
\& $x =~ /(?<!\es)foo/; # matches, there is no \es before \*(Aqfoo\*(Aq |
| 2528 |
.Ve |
| 2529 |
.PP |
| 2530 |
Here is an example where a string containing blank-separated words, |
| 2531 |
numbers and single dashes is to be split into its components. |
| 2532 |
Using \f(CW\*(C`/\es+/\*(C'\fR alone won't work, because spaces are not required between |
| 2533 |
dashes, or a word or a dash. Additional places for a split are established |
| 2534 |
by looking ahead and behind: |
| 2535 |
.PP |
| 2536 |
.Vb 5 |
| 2537 |
\& $str = "one two \- \-\-6\-8"; |
| 2538 |
\& @toks = split / \es+ # a run of spaces |
| 2539 |
\& | (?<=\eS) (?=\-) # any non\-space followed by \*(Aq\-\*(Aq |
| 2540 |
\& | (?<=\-) (?=\eS) # a \*(Aq\-\*(Aq followed by any non\-space |
| 2541 |
\& /x, $str; # @toks = qw(one two \- \- \- 6 \- 8) |
| 2542 |
.Ve |
| 2543 |
.SS "Using independent subexpressions to prevent backtracking" |
| 2544 |
.IX Subsection "Using independent subexpressions to prevent backtracking" |
| 2545 |
\&\fIIndependent subexpressions\fR (or atomic subexpressions) are regular |
| 2546 |
expressions, in the context of a larger regular expression, that |
| 2547 |
function independently of the larger regular expression. That is, they |
| 2548 |
consume as much or as little of the string as they wish without regard |
| 2549 |
for the ability of the larger regexp to match. Independent |
| 2550 |
subexpressions are represented by |
| 2551 |
\&\f(CW\*(C`(?>regexp)\*(C'\fR or (starting in 5.32, experimentally in 5.28) |
| 2552 |
\&\f(CW\*(C`(*atomic:regexp)\*(C'\fR. We can illustrate their behavior by first |
| 2553 |
considering an ordinary regexp: |
| 2554 |
.PP |
| 2555 |
.Vb 2 |
| 2556 |
\& $x = "ab"; |
| 2557 |
\& $x =~ /a*ab/; # matches |
| 2558 |
.Ve |
| 2559 |
.PP |
| 2560 |
This obviously matches, but in the process of matching, the |
| 2561 |
subexpression \f(CW\*(C`a*\*(C'\fR first grabbed the \f(CW\*(Aqa\*(Aq\fR. Doing so, however, |
| 2562 |
wouldn't allow the whole regexp to match, so after backtracking, \f(CW\*(C`a*\*(C'\fR |
| 2563 |
eventually gave back the \f(CW\*(Aqa\*(Aq\fR and matched the empty string. Here, what |
| 2564 |
\&\f(CW\*(C`a*\*(C'\fR matched was \fIdependent\fR on what the rest of the regexp matched. |
| 2565 |
.PP |
| 2566 |
Contrast that with an independent subexpression: |
| 2567 |
.PP |
| 2568 |
.Vb 1 |
| 2569 |
\& $x =~ /(?>a*)ab/; # doesn\*(Aqt match! |
| 2570 |
.Ve |
| 2571 |
.PP |
| 2572 |
The independent subexpression \f(CW\*(C`(?>a*)\*(C'\fR doesn't care about the rest |
| 2573 |
of the regexp, so it sees an \f(CW\*(Aqa\*(Aq\fR and grabs it. Then the rest of the |
| 2574 |
regexp \f(CW\*(C`ab\*(C'\fR cannot match. Because \f(CW\*(C`(?>a*)\*(C'\fR is independent, there |
| 2575 |
is no backtracking and the independent subexpression does not give |
| 2576 |
up its \f(CW\*(Aqa\*(Aq\fR. Thus the match of the regexp as a whole fails. A similar |
| 2577 |
behavior occurs with completely independent regexps: |
| 2578 |
.PP |
| 2579 |
.Vb 3 |
| 2580 |
\& $x = "ab"; |
| 2581 |
\& $x =~ /a*/g; # matches, eats an \*(Aqa\*(Aq |
| 2582 |
\& $x =~ /\eGab/g; # doesn\*(Aqt match, no \*(Aqa\*(Aq available |
| 2583 |
.Ve |
| 2584 |
.PP |
| 2585 |
Here \f(CW\*(C`/g\*(C'\fR and \f(CW\*(C`\eG\*(C'\fR create a "tag team" handoff of the string from |
| 2586 |
one regexp to the other. Regexps with an independent subexpression are |
| 2587 |
much like this, with a handoff of the string to the independent |
| 2588 |
subexpression, and a handoff of the string back to the enclosing |
| 2589 |
regexp. |
| 2590 |
.PP |
| 2591 |
The ability of an independent subexpression to prevent backtracking |
| 2592 |
can be quite useful. Suppose we want to match a non-empty string |
| 2593 |
enclosed in parentheses up to two levels deep. Then the following |
| 2594 |
regexp matches: |
| 2595 |
.PP |
| 2596 |
.Vb 2 |
| 2597 |
\& $x = "abc(de(fg)h"; # unbalanced parentheses |
| 2598 |
\& $x =~ /\e( ( [ ^ () ]+ | \e( [ ^ () ]* \e) )+ \e)/xx; |
| 2599 |
.Ve |
| 2600 |
.PP |
| 2601 |
The regexp matches an open parenthesis, one or more copies of an |
| 2602 |
alternation, and a close parenthesis. The alternation is two-way, with |
| 2603 |
the first alternative \f(CW\*(C`[^()]+\*(C'\fR matching a substring with no |
| 2604 |
parentheses and the second alternative \f(CW\*(C`\e([^()]*\e)\*(C'\fR matching a |
| 2605 |
substring delimited by parentheses. The problem with this regexp is |
| 2606 |
that it is pathological: it has nested indeterminate quantifiers |
| 2607 |
of the form \f(CW\*(C`(a+|b)+\*(C'\fR. We discussed in Part 1 how nested quantifiers |
| 2608 |
like this could take an exponentially long time to execute if there |
| 2609 |
is no match possible. To prevent the exponential blowup, we need to |
| 2610 |
prevent useless backtracking at some point. This can be done by |
| 2611 |
enclosing the inner quantifier as an independent subexpression: |
| 2612 |
.PP |
| 2613 |
.Vb 1 |
| 2614 |
\& $x =~ /\e( ( (?> [ ^ () ]+ ) | \e([ ^ () ]* \e) )+ \e)/xx; |
| 2615 |
.Ve |
| 2616 |
.PP |
| 2617 |
Here, \f(CW\*(C`(?>[^()]+)\*(C'\fR breaks the degeneracy of string partitioning |
| 2618 |
by gobbling up as much of the string as possible and keeping it. Then |
| 2619 |
match failures fail much more quickly. |
| 2620 |
.SS "Conditional expressions" |
| 2621 |
.IX Subsection "Conditional expressions" |
| 2622 |
A \fIconditional expression\fR is a form of if-then-else statement |
| 2623 |
that allows one to choose which patterns are to be matched, based on |
| 2624 |
some condition. There are two types of conditional expression: |
| 2625 |
\&\f(CW\*(C`(?(\fR\f(CIcondition\fR\f(CW)\fR\f(CIyes\-regexp\fR\f(CW)\*(C'\fR and |
| 2626 |
\&\f(CW\*(C`(?(condition)\fR\f(CIyes\-regexp\fR\f(CW|\fR\f(CIno\-regexp\fR\f(CW)\*(C'\fR. |
| 2627 |
\&\f(CW\*(C`(?(\fR\f(CIcondition\fR\f(CW)\fR\f(CIyes\-regexp\fR\f(CW)\*(C'\fR is |
| 2628 |
like an \f(CW\*(Aqif\ ()\ {}\*(Aq\fR statement in Perl. If the \fIcondition\fR is true, |
| 2629 |
the \fIyes-regexp\fR will be matched. If the \fIcondition\fR is false, the |
| 2630 |
\&\fIyes-regexp\fR will be skipped and Perl will move onto the next regexp |
| 2631 |
element. The second form is like an \f(CW\*(Aqif\ ()\ {}\ else\ {}\*(Aq\fR statement |
| 2632 |
in Perl. If the \fIcondition\fR is true, the \fIyes-regexp\fR will be |
| 2633 |
matched, otherwise the \fIno-regexp\fR will be matched. |
| 2634 |
.PP |
| 2635 |
The \fIcondition\fR can have several forms. The first form is simply an |
| 2636 |
integer in parentheses \f(CW\*(C`(\fR\f(CIinteger\fR\f(CW)\*(C'\fR. It is true if the corresponding |
| 2637 |
backreference \f(CW\*(C`\e\fR\f(CIinteger\fR\f(CW\*(C'\fR matched earlier in the regexp. The same |
| 2638 |
thing can be done with a name associated with a capture group, written |
| 2639 |
as \f(CW\*(C`(<\fR\f(CIname\fR\f(CW>)\*(C'\fR or \f(CW\*(C`(\*(Aq\fR\f(CIname\fR\f(CW\*(Aq)\*(C'\fR. The second form is a bare |
| 2640 |
zero-width assertion \f(CW\*(C`(?...)\*(C'\fR, either a lookahead, a lookbehind, or a |
| 2641 |
code assertion (discussed in the next section). The third set of forms |
| 2642 |
provides tests that return true if the expression is executed within |
| 2643 |
a recursion (\f(CW\*(C`(R)\*(C'\fR) or is being called from some capturing group, |
| 2644 |
referenced either by number (\f(CW\*(C`(R1)\*(C'\fR, \f(CW\*(C`(R2)\*(C'\fR,...) or by name |
| 2645 |
(\f(CW\*(C`(R&\fR\f(CIname\fR\f(CW)\*(C'\fR). |
| 2646 |
.PP |
| 2647 |
The integer or name form of the \f(CW\*(C`condition\*(C'\fR allows us to choose, |
| 2648 |
with more flexibility, what to match based on what matched earlier in the |
| 2649 |
regexp. This searches for words of the form \f(CW"$x$x"\fR or \f(CW"$x$y$y$x"\fR: |
| 2650 |
.PP |
| 2651 |
.Vb 9 |
| 2652 |
\& % simple_grep \*(Aq^(\ew+)(\ew+)?(?(2)\eg2\eg1|\eg1)$\*(Aq /usr/dict/words |
| 2653 |
\& beriberi |
| 2654 |
\& coco |
| 2655 |
\& couscous |
| 2656 |
\& deed |
| 2657 |
\& ... |
| 2658 |
\& toot |
| 2659 |
\& toto |
| 2660 |
\& tutu |
| 2661 |
.Ve |
| 2662 |
.PP |
| 2663 |
The lookbehind \f(CW\*(C`condition\*(C'\fR allows, along with backreferences, |
| 2664 |
an earlier part of the match to influence a later part of the |
| 2665 |
match. For instance, |
| 2666 |
.PP |
| 2667 |
.Vb 1 |
| 2668 |
\& /[ATGC]+(?(?<=AA)G|C)$/; |
| 2669 |
.Ve |
| 2670 |
.PP |
| 2671 |
matches a DNA sequence such that it either ends in \f(CW\*(C`AAG\*(C'\fR, or some |
| 2672 |
other base pair combination and \f(CW\*(AqC\*(Aq\fR. Note that the form is |
| 2673 |
\&\f(CW\*(C`(?(?<=AA)G|C)\*(C'\fR and not \f(CW\*(C`(?((?<=AA))G|C)\*(C'\fR; for the |
| 2674 |
lookahead, lookbehind or code assertions, the parentheses around the |
| 2675 |
conditional are not needed. |
| 2676 |
.SS "Defining named patterns" |
| 2677 |
.IX Subsection "Defining named patterns" |
| 2678 |
Some regular expressions use identical subpatterns in several places. |
| 2679 |
Starting with Perl 5.10, it is possible to define named subpatterns in |
| 2680 |
a section of the pattern so that they can be called up by name |
| 2681 |
anywhere in the pattern. This syntactic pattern for this definition |
| 2682 |
group is \f(CW\*(C`(?(DEFINE)(?<\fR\f(CIname\fR\f(CW>\fR\f(CIpattern\fR\f(CW)...)\*(C'\fR. An insertion |
| 2683 |
of a named pattern is written as \f(CW\*(C`(?&\fR\f(CIname\fR\f(CW)\*(C'\fR. |
| 2684 |
.PP |
| 2685 |
The example below illustrates this feature using the pattern for |
| 2686 |
floating point numbers that was presented earlier on. The three |
| 2687 |
subpatterns that are used more than once are the optional sign, the |
| 2688 |
digit sequence for an integer and the decimal fraction. The \f(CW\*(C`DEFINE\*(C'\fR |
| 2689 |
group at the end of the pattern contains their definition. Notice |
| 2690 |
that the decimal fraction pattern is the first place where we can |
| 2691 |
reuse the integer pattern. |
| 2692 |
.PP |
| 2693 |
.Vb 8 |
| 2694 |
\& /^ (?&osg)\e * ( (?&int)(?&dec)? | (?&dec) ) |
| 2695 |
\& (?: [eE](?&osg)(?&int) )? |
| 2696 |
\& $ |
| 2697 |
\& (?(DEFINE) |
| 2698 |
\& (?<osg>[\-+]?) # optional sign |
| 2699 |
\& (?<int>\ed++) # integer |
| 2700 |
\& (?<dec>\e.(?&int)) # decimal fraction |
| 2701 |
\& )/x |
| 2702 |
.Ve |
| 2703 |
.SS "Recursive patterns" |
| 2704 |
.IX Subsection "Recursive patterns" |
| 2705 |
This feature (introduced in Perl 5.10) significantly extends the |
| 2706 |
power of Perl's pattern matching. By referring to some other |
| 2707 |
capture group anywhere in the pattern with the construct |
| 2708 |
\&\f(CW\*(C`(?\fR\f(CIgroup\-ref\fR\f(CW)\*(C'\fR, the \fIpattern\fR within the referenced group is used |
| 2709 |
as an independent subpattern in place of the group reference itself. |
| 2710 |
Because the group reference may be contained \fIwithin\fR the group it |
| 2711 |
refers to, it is now possible to apply pattern matching to tasks that |
| 2712 |
hitherto required a recursive parser. |
| 2713 |
.PP |
| 2714 |
To illustrate this feature, we'll design a pattern that matches if |
| 2715 |
a string contains a palindrome. (This is a word or a sentence that, |
| 2716 |
while ignoring spaces, interpunctuation and case, reads the same backwards |
| 2717 |
as forwards. We begin by observing that the empty string or a string |
| 2718 |
containing just one word character is a palindrome. Otherwise it must |
| 2719 |
have a word character up front and the same at its end, with another |
| 2720 |
palindrome in between. |
| 2721 |
.PP |
| 2722 |
.Vb 1 |
| 2723 |
\& /(?: (\ew) (?...Here be a palindrome...) \eg{ \-1 } | \ew? )/x |
| 2724 |
.Ve |
| 2725 |
.PP |
| 2726 |
Adding \f(CW\*(C`\eW*\*(C'\fR at either end to eliminate what is to be ignored, we already |
| 2727 |
have the full pattern: |
| 2728 |
.PP |
| 2729 |
.Vb 4 |
| 2730 |
\& my $pp = qr/^(\eW* (?: (\ew) (?1) \eg{\-1} | \ew? ) \eW*)$/ix; |
| 2731 |
\& for $s ( "saippuakauppias", "A man, a plan, a canal: Panama!" ){ |
| 2732 |
\& print "\*(Aq$s\*(Aq is a palindrome\en" if $s =~ /$pp/; |
| 2733 |
\& } |
| 2734 |
.Ve |
| 2735 |
.PP |
| 2736 |
In \f(CW\*(C`(?...)\*(C'\fR both absolute and relative backreferences may be used. |
| 2737 |
The entire pattern can be reinserted with \f(CW\*(C`(?R)\*(C'\fR or \f(CW\*(C`(?0)\*(C'\fR. |
| 2738 |
If you prefer to name your groups, you can use \f(CW\*(C`(?&\fR\f(CIname\fR\f(CW)\*(C'\fR to |
| 2739 |
recurse into that group. |
| 2740 |
.SS "A bit of magic: executing Perl code in a regular expression" |
| 2741 |
.IX Subsection "A bit of magic: executing Perl code in a regular expression" |
| 2742 |
Normally, regexps are a part of Perl expressions. |
| 2743 |
\&\fICode evaluation\fR expressions turn that around by allowing |
| 2744 |
arbitrary Perl code to be a part of a regexp. A code evaluation |
| 2745 |
expression is denoted \f(CW\*(C`(?{\fR\f(CIcode\fR\f(CW})\*(C'\fR, with \fIcode\fR a string of Perl |
| 2746 |
statements. |
| 2747 |
.PP |
| 2748 |
Code expressions are zero-width assertions, and the value they return |
| 2749 |
depends on their environment. There are two possibilities: either the |
| 2750 |
code expression is used as a conditional in a conditional expression |
| 2751 |
\&\f(CW\*(C`(?(\fR\f(CIcondition\fR\f(CW)...)\*(C'\fR, or it is not. If the code expression is a |
| 2752 |
conditional, the code is evaluated and the result (\fIi.e.\fR, the result of |
| 2753 |
the last statement) is used to determine truth or falsehood. If the |
| 2754 |
code expression is not used as a conditional, the assertion always |
| 2755 |
evaluates true and the result is put into the special variable |
| 2756 |
\&\f(CW$^R\fR. The variable \f(CW$^R\fR can then be used in code expressions later |
| 2757 |
in the regexp. Here are some silly examples: |
| 2758 |
.PP |
| 2759 |
.Vb 5 |
| 2760 |
\& $x = "abcdef"; |
| 2761 |
\& $x =~ /abc(?{print "Hi Mom!";})def/; # matches, |
| 2762 |
\& # prints \*(AqHi Mom!\*(Aq |
| 2763 |
\& $x =~ /aaa(?{print "Hi Mom!";})def/; # doesn\*(Aqt match, |
| 2764 |
\& # no \*(AqHi Mom!\*(Aq |
| 2765 |
.Ve |
| 2766 |
.PP |
| 2767 |
Pay careful attention to the next example: |
| 2768 |
.PP |
| 2769 |
.Vb 3 |
| 2770 |
\& $x =~ /abc(?{print "Hi Mom!";})ddd/; # doesn\*(Aqt match, |
| 2771 |
\& # no \*(AqHi Mom!\*(Aq |
| 2772 |
\& # but why not? |
| 2773 |
.Ve |
| 2774 |
.PP |
| 2775 |
At first glance, you'd think that it shouldn't print, because obviously |
| 2776 |
the \f(CW\*(C`ddd\*(C'\fR isn't going to match the target string. But look at this |
| 2777 |
example: |
| 2778 |
.PP |
| 2779 |
.Vb 2 |
| 2780 |
\& $x =~ /abc(?{print "Hi Mom!";})[dD]dd/; # doesn\*(Aqt match, |
| 2781 |
\& # but _does_ print |
| 2782 |
.Ve |
| 2783 |
.PP |
| 2784 |
Hmm. What happened here? If you've been following along, you know that |
| 2785 |
the above pattern should be effectively (almost) the same as the last one; |
| 2786 |
enclosing the \f(CW\*(Aqd\*(Aq\fR in a character class isn't going to change what it |
| 2787 |
matches. So why does the first not print while the second one does? |
| 2788 |
.PP |
| 2789 |
The answer lies in the optimizations the regexp engine makes. In the first |
| 2790 |
case, all the engine sees are plain old characters (aside from the |
| 2791 |
\&\f(CW\*(C`?{}\*(C'\fR construct). It's smart enough to realize that the string \f(CW\*(Aqddd\*(Aq\fR |
| 2792 |
doesn't occur in our target string before actually running the pattern |
| 2793 |
through. But in the second case, we've tricked it into thinking that our |
| 2794 |
pattern is more complicated. It takes a look, sees our |
| 2795 |
character class, and decides that it will have to actually run the |
| 2796 |
pattern to determine whether or not it matches, and in the process of |
| 2797 |
running it hits the print statement before it discovers that we don't |
| 2798 |
have a match. |
| 2799 |
.PP |
| 2800 |
To take a closer look at how the engine does optimizations, see the |
| 2801 |
section "Pragmas and debugging" below. |
| 2802 |
.PP |
| 2803 |
More fun with \f(CW\*(C`?{}\*(C'\fR: |
| 2804 |
.PP |
| 2805 |
.Vb 6 |
| 2806 |
\& $x =~ /(?{print "Hi Mom!";})/; # matches, |
| 2807 |
\& # prints \*(AqHi Mom!\*(Aq |
| 2808 |
\& $x =~ /(?{$c = 1;})(?{print "$c";})/; # matches, |
| 2809 |
\& # prints \*(Aq1\*(Aq |
| 2810 |
\& $x =~ /(?{$c = 1;})(?{print "$^R";})/; # matches, |
| 2811 |
\& # prints \*(Aq1\*(Aq |
| 2812 |
.Ve |
| 2813 |
.PP |
| 2814 |
The bit of magic mentioned in the section title occurs when the regexp |
| 2815 |
backtracks in the process of searching for a match. If the regexp |
| 2816 |
backtracks over a code expression and if the variables used within are |
| 2817 |
localized using \f(CW\*(C`local\*(C'\fR, the changes in the variables produced by the |
| 2818 |
code expression are undone! Thus, if we wanted to count how many times |
| 2819 |
a character got matched inside a group, we could use, \fIe.g.\fR, |
| 2820 |
.PP |
| 2821 |
.Vb 11 |
| 2822 |
\& $x = "aaaa"; |
| 2823 |
\& $count = 0; # initialize \*(Aqa\*(Aq count |
| 2824 |
\& $c = "bob"; # test if $c gets clobbered |
| 2825 |
\& $x =~ /(?{local $c = 0;}) # initialize count |
| 2826 |
\& ( a # match \*(Aqa\*(Aq |
| 2827 |
\& (?{local $c = $c + 1;}) # increment count |
| 2828 |
\& )* # do this any number of times, |
| 2829 |
\& aa # but match \*(Aqaa\*(Aq at the end |
| 2830 |
\& (?{$count = $c;}) # copy local $c var into $count |
| 2831 |
\& /x; |
| 2832 |
\& print "\*(Aqa\*(Aq count is $count, \e$c variable is \*(Aq$c\*(Aq\en"; |
| 2833 |
.Ve |
| 2834 |
.PP |
| 2835 |
This prints |
| 2836 |
.PP |
| 2837 |
.Vb 1 |
| 2838 |
\& \*(Aqa\*(Aq count is 2, $c variable is \*(Aqbob\*(Aq |
| 2839 |
.Ve |
| 2840 |
.PP |
| 2841 |
If we replace the \f(CW\*(C`\ (?{local\ $c\ =\ $c\ +\ 1;})\*(C'\fR with |
| 2842 |
\&\f(CW\*(C`\ (?{$c\ =\ $c\ +\ 1;})\*(C'\fR, the variable changes are \fInot\fR undone |
| 2843 |
during backtracking, and we get |
| 2844 |
.PP |
| 2845 |
.Vb 1 |
| 2846 |
\& \*(Aqa\*(Aq count is 4, $c variable is \*(Aqbob\*(Aq |
| 2847 |
.Ve |
| 2848 |
.PP |
| 2849 |
Note that only localized variable changes are undone. Other side |
| 2850 |
effects of code expression execution are permanent. Thus |
| 2851 |
.PP |
| 2852 |
.Vb 2 |
| 2853 |
\& $x = "aaaa"; |
| 2854 |
\& $x =~ /(a(?{print "Yow\en";}))*aa/; |
| 2855 |
.Ve |
| 2856 |
.PP |
| 2857 |
produces |
| 2858 |
.PP |
| 2859 |
.Vb 4 |
| 2860 |
\& Yow |
| 2861 |
\& Yow |
| 2862 |
\& Yow |
| 2863 |
\& Yow |
| 2864 |
.Ve |
| 2865 |
.PP |
| 2866 |
The result \f(CW$^R\fR is automatically localized, so that it will behave |
| 2867 |
properly in the presence of backtracking. |
| 2868 |
.PP |
| 2869 |
This example uses a code expression in a conditional to match a |
| 2870 |
definite article, either \f(CW\*(Aqthe\*(Aq\fR in English or \f(CW\*(Aqder|die|das\*(Aq\fR in |
| 2871 |
German: |
| 2872 |
.PP |
| 2873 |
.Vb 11 |
| 2874 |
\& $lang = \*(AqDE\*(Aq; # use German |
| 2875 |
\& ... |
| 2876 |
\& $text = "das"; |
| 2877 |
\& print "matched\en" |
| 2878 |
\& if $text =~ /(?(?{ |
| 2879 |
\& $lang eq \*(AqEN\*(Aq; # is the language English? |
| 2880 |
\& }) |
| 2881 |
\& the | # if so, then match \*(Aqthe\*(Aq |
| 2882 |
\& (der|die|das) # else, match \*(Aqder|die|das\*(Aq |
| 2883 |
\& ) |
| 2884 |
\& /xi; |
| 2885 |
.Ve |
| 2886 |
.PP |
| 2887 |
Note that the syntax here is \f(CW\*(C`(?(?{...})\fR\f(CIyes\-regexp\fR\f(CW|\fR\f(CIno\-regexp\fR\f(CW)\*(C'\fR, not |
| 2888 |
\&\f(CW\*(C`(?((?{...}))\fR\f(CIyes\-regexp\fR\f(CW|\fR\f(CIno\-regexp\fR\f(CW)\*(C'\fR. In other words, in the case of a |
| 2889 |
code expression, we don't need the extra parentheses around the |
| 2890 |
conditional. |
| 2891 |
.PP |
| 2892 |
If you try to use code expressions where the code text is contained within |
| 2893 |
an interpolated variable, rather than appearing literally in the pattern, |
| 2894 |
Perl may surprise you: |
| 2895 |
.PP |
| 2896 |
.Vb 5 |
| 2897 |
\& $bar = 5; |
| 2898 |
\& $pat = \*(Aq(?{ 1 })\*(Aq; |
| 2899 |
\& /foo(?{ $bar })bar/; # compiles ok, $bar not interpolated |
| 2900 |
\& /foo(?{ 1 })$bar/; # compiles ok, $bar interpolated |
| 2901 |
\& /foo${pat}bar/; # compile error! |
| 2902 |
\& |
| 2903 |
\& $pat = qr/(?{ $foo = 1 })/; # precompile code regexp |
| 2904 |
\& /foo${pat}bar/; # compiles ok |
| 2905 |
.Ve |
| 2906 |
.PP |
| 2907 |
If a regexp has a variable that interpolates a code expression, Perl |
| 2908 |
treats the regexp as an error. If the code expression is precompiled into |
| 2909 |
a variable, however, interpolating is ok. The question is, why is this an |
| 2910 |
error? |
| 2911 |
.PP |
| 2912 |
The reason is that variable interpolation and code expressions |
| 2913 |
together pose a security risk. The combination is dangerous because |
| 2914 |
many programmers who write search engines often take user input and |
| 2915 |
plug it directly into a regexp: |
| 2916 |
.PP |
| 2917 |
.Vb 3 |
| 2918 |
\& $regexp = <>; # read user\-supplied regexp |
| 2919 |
\& $chomp $regexp; # get rid of possible newline |
| 2920 |
\& $text =~ /$regexp/; # search $text for the $regexp |
| 2921 |
.Ve |
| 2922 |
.PP |
| 2923 |
If the \f(CW$regexp\fR variable contains a code expression, the user could |
| 2924 |
then execute arbitrary Perl code. For instance, some joker could |
| 2925 |
search for \f(CW\*(C`system(\*(Aqrm\ \-rf\ *\*(Aq);\*(C'\fR to erase your files. In this |
| 2926 |
sense, the combination of interpolation and code expressions \fItaints\fR |
| 2927 |
your regexp. So by default, using both interpolation and code |
| 2928 |
expressions in the same regexp is not allowed. If you're not |
| 2929 |
concerned about malicious users, it is possible to bypass this |
| 2930 |
security check by invoking \f(CW\*(C`use\ re\ \*(Aqeval\*(Aq\*(C'\fR: |
| 2931 |
.PP |
| 2932 |
.Vb 4 |
| 2933 |
\& use re \*(Aqeval\*(Aq; # throw caution out the door |
| 2934 |
\& $bar = 5; |
| 2935 |
\& $pat = \*(Aq(?{ 1 })\*(Aq; |
| 2936 |
\& /foo${pat}bar/; # compiles ok |
| 2937 |
.Ve |
| 2938 |
.PP |
| 2939 |
Another form of code expression is the \fIpattern code expression\fR. |
| 2940 |
The pattern code expression is like a regular code expression, except |
| 2941 |
that the result of the code evaluation is treated as a regular |
| 2942 |
expression and matched immediately. A simple example is |
| 2943 |
.PP |
| 2944 |
.Vb 4 |
| 2945 |
\& $length = 5; |
| 2946 |
\& $char = \*(Aqa\*(Aq; |
| 2947 |
\& $x = \*(Aqaaaaabb\*(Aq; |
| 2948 |
\& $x =~ /(??{$char x $length})/x; # matches, there are 5 of \*(Aqa\*(Aq |
| 2949 |
.Ve |
| 2950 |
.PP |
| 2951 |
This final example contains both ordinary and pattern code |
| 2952 |
expressions. It detects whether a binary string \f(CW1101010010001...\fR has a |
| 2953 |
Fibonacci spacing 0,1,1,2,3,5,... of the \f(CW\*(Aq1\*(Aq\fR's: |
| 2954 |
.PP |
| 2955 |
.Vb 12 |
| 2956 |
\& $x = "1101010010001000001"; |
| 2957 |
\& $z0 = \*(Aq\*(Aq; $z1 = \*(Aq0\*(Aq; # initial conditions |
| 2958 |
\& print "It is a Fibonacci sequence\en" |
| 2959 |
\& if $x =~ /^1 # match an initial \*(Aq1\*(Aq |
| 2960 |
\& (?: |
| 2961 |
\& ((??{ $z0 })) # match some \*(Aq0\*(Aq |
| 2962 |
\& 1 # and then a \*(Aq1\*(Aq |
| 2963 |
\& (?{ $z0 = $z1; $z1 .= $^N; }) |
| 2964 |
\& )+ # repeat as needed |
| 2965 |
\& $ # that is all there is |
| 2966 |
\& /x; |
| 2967 |
\& printf "Largest sequence matched was %d\en", length($z1)\-length($z0); |
| 2968 |
.Ve |
| 2969 |
.PP |
| 2970 |
Remember that \f(CW$^N\fR is set to whatever was matched by the last |
| 2971 |
completed capture group. This prints |
| 2972 |
.PP |
| 2973 |
.Vb 2 |
| 2974 |
\& It is a Fibonacci sequence |
| 2975 |
\& Largest sequence matched was 5 |
| 2976 |
.Ve |
| 2977 |
.PP |
| 2978 |
Ha! Try that with your garden variety regexp package... |
| 2979 |
.PP |
| 2980 |
Note that the variables \f(CW$z0\fR and \f(CW$z1\fR are not substituted when the |
| 2981 |
regexp is compiled, as happens for ordinary variables outside a code |
| 2982 |
expression. Rather, the whole code block is parsed as perl code at the |
| 2983 |
same time as perl is compiling the code containing the literal regexp |
| 2984 |
pattern. |
| 2985 |
.PP |
| 2986 |
This regexp without the \f(CW\*(C`/x\*(C'\fR modifier is |
| 2987 |
.PP |
| 2988 |
.Vb 1 |
| 2989 |
\& /^1(?:((??{ $z0 }))1(?{ $z0 = $z1; $z1 .= $^N; }))+$/ |
| 2990 |
.Ve |
| 2991 |
.PP |
| 2992 |
which shows that spaces are still possible in the code parts. Nevertheless, |
| 2993 |
when working with code and conditional expressions, the extended form of |
| 2994 |
regexps is almost necessary in creating and debugging regexps. |
| 2995 |
.SS "Backtracking control verbs" |
| 2996 |
.IX Subsection "Backtracking control verbs" |
| 2997 |
Perl 5.10 introduced a number of control verbs intended to provide |
| 2998 |
detailed control over the backtracking process, by directly influencing |
| 2999 |
the regexp engine and by providing monitoring techniques. See |
| 3000 |
"Special Backtracking Control Verbs" in perlre for a detailed |
| 3001 |
description. |
| 3002 |
.PP |
| 3003 |
Below is just one example, illustrating the control verb \f(CW\*(C`(*FAIL)\*(C'\fR, |
| 3004 |
which may be abbreviated as \f(CW\*(C`(*F)\*(C'\fR. If this is inserted in a regexp |
| 3005 |
it will cause it to fail, just as it would at some |
| 3006 |
mismatch between the pattern and the string. Processing |
| 3007 |
of the regexp continues as it would after any "normal" |
| 3008 |
failure, so that, for instance, the next position in the string or another |
| 3009 |
alternative will be tried. As failing to match doesn't preserve capture |
| 3010 |
groups or produce results, it may be necessary to use this in |
| 3011 |
combination with embedded code. |
| 3012 |
.PP |
| 3013 |
.Vb 4 |
| 3014 |
\& %count = (); |
| 3015 |
\& "supercalifragilisticexpialidocious" =~ |
| 3016 |
\& /([aeiou])(?{ $count{$1}++; })(*FAIL)/i; |
| 3017 |
\& printf "%3d \*(Aq%s\*(Aq\en", $count{$_}, $_ for (sort keys %count); |
| 3018 |
.Ve |
| 3019 |
.PP |
| 3020 |
The pattern begins with a class matching a subset of letters. Whenever |
| 3021 |
this matches, a statement like \f(CW\*(C`$count{\*(Aqa\*(Aq}++;\*(C'\fR is executed, incrementing |
| 3022 |
the letter's counter. Then \f(CW\*(C`(*FAIL)\*(C'\fR does what it says, and |
| 3023 |
the regexp engine proceeds according to the book: as long as the end of |
| 3024 |
the string hasn't been reached, the position is advanced before looking |
| 3025 |
for another vowel. Thus, match or no match makes no difference, and the |
| 3026 |
regexp engine proceeds until the entire string has been inspected. |
| 3027 |
(It's remarkable that an alternative solution using something like |
| 3028 |
.PP |
| 3029 |
.Vb 2 |
| 3030 |
\& $count{lc($_)}++ for split(\*(Aq\*(Aq, "supercalifragilisticexpialidocious"); |
| 3031 |
\& printf "%3d \*(Aq%s\*(Aq\en", $count2{$_}, $_ for ( qw{ a e i o u } ); |
| 3032 |
.Ve |
| 3033 |
.PP |
| 3034 |
is considerably slower.) |
| 3035 |
.SS "Pragmas and debugging" |
| 3036 |
.IX Subsection "Pragmas and debugging" |
| 3037 |
Speaking of debugging, there are several pragmas available to control |
| 3038 |
and debug regexps in Perl. We have already encountered one pragma in |
| 3039 |
the previous section, \f(CW\*(C`use\ re\ \*(Aqeval\*(Aq;\*(C'\fR, that allows variable |
| 3040 |
interpolation and code expressions to coexist in a regexp. The other |
| 3041 |
pragmas are |
| 3042 |
.PP |
| 3043 |
.Vb 3 |
| 3044 |
\& use re \*(Aqtaint\*(Aq; |
| 3045 |
\& $tainted = <>; |
| 3046 |
\& @parts = ($tainted =~ /(\ew+)\es+(\ew+)/; # @parts is now tainted |
| 3047 |
.Ve |
| 3048 |
.PP |
| 3049 |
The \f(CW\*(C`taint\*(C'\fR pragma causes any substrings from a match with a tainted |
| 3050 |
variable to be tainted as well, if your perl supports tainting |
| 3051 |
(see perlsec). This is not normally the case, as |
| 3052 |
regexps are often used to extract the safe bits from a tainted |
| 3053 |
variable. Use \f(CW\*(C`taint\*(C'\fR when you are not extracting safe bits, but are |
| 3054 |
performing some other processing. Both \f(CW\*(C`taint\*(C'\fR and \f(CW\*(C`eval\*(C'\fR pragmas |
| 3055 |
are lexically scoped, which means they are in effect only until |
| 3056 |
the end of the block enclosing the pragmas. |
| 3057 |
.PP |
| 3058 |
.Vb 2 |
| 3059 |
\& use re \*(Aq/m\*(Aq; # or any other flags |
| 3060 |
\& $multiline_string =~ /^foo/; # /m is implied |
| 3061 |
.Ve |
| 3062 |
.PP |
| 3063 |
The \f(CW\*(C`re \*(Aq/flags\*(Aq\*(C'\fR pragma (introduced in Perl |
| 3064 |
5.14) turns on the given regular expression flags |
| 3065 |
until the end of the lexical scope. See |
| 3066 |
"'/flags' mode" in re for more |
| 3067 |
detail. |
| 3068 |
.PP |
| 3069 |
.Vb 2 |
| 3070 |
\& use re \*(Aqdebug\*(Aq; |
| 3071 |
\& /^(.*)$/s; # output debugging info |
| 3072 |
\& |
| 3073 |
\& use re \*(Aqdebugcolor\*(Aq; |
| 3074 |
\& /^(.*)$/s; # output debugging info in living color |
| 3075 |
.Ve |
| 3076 |
.PP |
| 3077 |
The global \f(CW\*(C`debug\*(C'\fR and \f(CW\*(C`debugcolor\*(C'\fR pragmas allow one to get |
| 3078 |
detailed debugging info about regexp compilation and |
| 3079 |
execution. \f(CW\*(C`debugcolor\*(C'\fR is the same as debug, except the debugging |
| 3080 |
information is displayed in color on terminals that can display |
| 3081 |
termcap color sequences. Here is example output: |
| 3082 |
.PP |
| 3083 |
.Vb 10 |
| 3084 |
\& % perl \-e \*(Aquse re "debug"; "abc" =~ /a*b+c/;\*(Aq |
| 3085 |
\& Compiling REx \*(Aqa*b+c\*(Aq |
| 3086 |
\& size 9 first at 1 |
| 3087 |
\& 1: STAR(4) |
| 3088 |
\& 2: EXACT <a>(0) |
| 3089 |
\& 4: PLUS(7) |
| 3090 |
\& 5: EXACT <b>(0) |
| 3091 |
\& 7: EXACT <c>(9) |
| 3092 |
\& 9: END(0) |
| 3093 |
\& floating \*(Aqbc\*(Aq at 0..2147483647 (checking floating) minlen 2 |
| 3094 |
\& Guessing start of match, REx \*(Aqa*b+c\*(Aq against \*(Aqabc\*(Aq... |
| 3095 |
\& Found floating substr \*(Aqbc\*(Aq at offset 1... |
| 3096 |
\& Guessed: match at offset 0 |
| 3097 |
\& Matching REx \*(Aqa*b+c\*(Aq against \*(Aqabc\*(Aq |
| 3098 |
\& Setting an EVAL scope, savestack=3 |
| 3099 |
\& 0 <> <abc> | 1: STAR |
| 3100 |
\& EXACT <a> can match 1 times out of 32767... |
| 3101 |
\& Setting an EVAL scope, savestack=3 |
| 3102 |
\& 1 <a> <bc> | 4: PLUS |
| 3103 |
\& EXACT <b> can match 1 times out of 32767... |
| 3104 |
\& Setting an EVAL scope, savestack=3 |
| 3105 |
\& 2 <ab> <c> | 7: EXACT <c> |
| 3106 |
\& 3 <abc> <> | 9: END |
| 3107 |
\& Match successful! |
| 3108 |
\& Freeing REx: \*(Aqa*b+c\*(Aq |
| 3109 |
.Ve |
| 3110 |
.PP |
| 3111 |
If you have gotten this far into the tutorial, you can probably guess |
| 3112 |
what the different parts of the debugging output tell you. The first |
| 3113 |
part |
| 3114 |
.PP |
| 3115 |
.Vb 8 |
| 3116 |
\& Compiling REx \*(Aqa*b+c\*(Aq |
| 3117 |
\& size 9 first at 1 |
| 3118 |
\& 1: STAR(4) |
| 3119 |
\& 2: EXACT <a>(0) |
| 3120 |
\& 4: PLUS(7) |
| 3121 |
\& 5: EXACT <b>(0) |
| 3122 |
\& 7: EXACT <c>(9) |
| 3123 |
\& 9: END(0) |
| 3124 |
.Ve |
| 3125 |
.PP |
| 3126 |
describes the compilation stage. \f(CWSTAR(4)\fR means that there is a |
| 3127 |
starred object, in this case \f(CW\*(Aqa\*(Aq\fR, and if it matches, goto line 4, |
| 3128 |
\&\fIi.e.\fR, \f(CWPLUS(7)\fR. The middle lines describe some heuristics and |
| 3129 |
optimizations performed before a match: |
| 3130 |
.PP |
| 3131 |
.Vb 4 |
| 3132 |
\& floating \*(Aqbc\*(Aq at 0..2147483647 (checking floating) minlen 2 |
| 3133 |
\& Guessing start of match, REx \*(Aqa*b+c\*(Aq against \*(Aqabc\*(Aq... |
| 3134 |
\& Found floating substr \*(Aqbc\*(Aq at offset 1... |
| 3135 |
\& Guessed: match at offset 0 |
| 3136 |
.Ve |
| 3137 |
.PP |
| 3138 |
Then the match is executed and the remaining lines describe the |
| 3139 |
process: |
| 3140 |
.PP |
| 3141 |
.Vb 12 |
| 3142 |
\& Matching REx \*(Aqa*b+c\*(Aq against \*(Aqabc\*(Aq |
| 3143 |
\& Setting an EVAL scope, savestack=3 |
| 3144 |
\& 0 <> <abc> | 1: STAR |
| 3145 |
\& EXACT <a> can match 1 times out of 32767... |
| 3146 |
\& Setting an EVAL scope, savestack=3 |
| 3147 |
\& 1 <a> <bc> | 4: PLUS |
| 3148 |
\& EXACT <b> can match 1 times out of 32767... |
| 3149 |
\& Setting an EVAL scope, savestack=3 |
| 3150 |
\& 2 <ab> <c> | 7: EXACT <c> |
| 3151 |
\& 3 <abc> <> | 9: END |
| 3152 |
\& Match successful! |
| 3153 |
\& Freeing REx: \*(Aqa*b+c\*(Aq |
| 3154 |
.Ve |
| 3155 |
.PP |
| 3156 |
Each step is of the form \f(CW\*(C`n\ <x>\ <y>\*(C'\fR, with \f(CW\*(C`<x>\*(C'\fR the |
| 3157 |
part of the string matched and \f(CW\*(C`<y>\*(C'\fR the part not yet |
| 3158 |
matched. The \f(CW\*(C`|\ \ 1:\ \ STAR\*(C'\fR says that Perl is at line number 1 |
| 3159 |
in the compilation list above. See |
| 3160 |
"Debugging Regular Expressions" in perldebguts for much more detail. |
| 3161 |
.PP |
| 3162 |
An alternative method of debugging regexps is to embed \f(CW\*(C`print\*(C'\fR |
| 3163 |
statements within the regexp. This provides a blow-by-blow account of |
| 3164 |
the backtracking in an alternation: |
| 3165 |
.PP |
| 3166 |
.Vb 12 |
| 3167 |
\& "that this" =~ m@(?{print "Start at position ", pos, "\en";}) |
| 3168 |
\& t(?{print "t1\en";}) |
| 3169 |
\& h(?{print "h1\en";}) |
| 3170 |
\& i(?{print "i1\en";}) |
| 3171 |
\& s(?{print "s1\en";}) |
| 3172 |
\& | |
| 3173 |
\& t(?{print "t2\en";}) |
| 3174 |
\& h(?{print "h2\en";}) |
| 3175 |
\& a(?{print "a2\en";}) |
| 3176 |
\& t(?{print "t2\en";}) |
| 3177 |
\& (?{print "Done at position ", pos, "\en";}) |
| 3178 |
\& @x; |
| 3179 |
.Ve |
| 3180 |
.PP |
| 3181 |
prints |
| 3182 |
.PP |
| 3183 |
.Vb 8 |
| 3184 |
\& Start at position 0 |
| 3185 |
\& t1 |
| 3186 |
\& h1 |
| 3187 |
\& t2 |
| 3188 |
\& h2 |
| 3189 |
\& a2 |
| 3190 |
\& t2 |
| 3191 |
\& Done at position 4 |
| 3192 |
.Ve |
| 3193 |
.SH "SEE ALSO" |
| 3194 |
.IX Header "SEE ALSO" |
| 3195 |
This is just a tutorial. For the full story on Perl regular |
| 3196 |
expressions, see the perlre regular expressions reference page. |
| 3197 |
.PP |
| 3198 |
For more information on the matching \f(CW\*(C`m//\*(C'\fR and substitution \f(CW\*(C`s///\*(C'\fR |
| 3199 |
operators, see "Regexp Quote-Like Operators" in perlop. For |
| 3200 |
information on the \f(CW\*(C`split\*(C'\fR operation, see "split" in perlfunc. |
| 3201 |
.PP |
| 3202 |
For an excellent all-around resource on the care and feeding of |
| 3203 |
regular expressions, see the book \fIMastering Regular Expressions\fR by |
| 3204 |
Jeffrey Friedl (published by O'Reilly, ISBN 1556592\-257\-3). |
| 3205 |
.SH "AUTHOR AND COPYRIGHT" |
| 3206 |
.IX Header "AUTHOR AND COPYRIGHT" |
| 3207 |
Copyright (c) 2000 Mark Kvale. |
| 3208 |
All rights reserved. |
| 3209 |
Now maintained by Perl porters. |
| 3210 |
.PP |
| 3211 |
This document may be distributed under the same terms as Perl itself. |
| 3212 |
.SS Acknowledgments |
| 3213 |
.IX Subsection "Acknowledgments" |
| 3214 |
The inspiration for the stop codon DNA example came from the ZIP |
| 3215 |
code example in chapter 7 of \fIMastering Regular Expressions\fR. |
| 3216 |
.PP |
| 3217 |
The author would like to thank Jeff Pinyan, Andrew Johnson, Peter |
| 3218 |
Haworth, Ronald J Kimball, and Joe Smith for all their helpful |
| 3219 |
comments. |