| 1 |
NAME |
| 2 |
re::engine::RE2 - RE2 regex engine |
| 3 |
|
| 4 |
SYNOPSIS |
| 5 |
use re::engine::RE2; |
| 6 |
|
| 7 |
if ("Hello, world" =~ /Hello, (world)/) { |
| 8 |
print "Greetings, $1!"; |
| 9 |
} |
| 10 |
|
| 11 |
DESCRIPTION |
| 12 |
This module replaces perl's regex engine in a given lexical scope with |
| 13 |
RE2. |
| 14 |
|
| 15 |
RE2 is a primarily DFA based regexp engine from Google that is very fast |
| 16 |
at matching large amounts of text. However it does not support look |
| 17 |
behind and some other Perl regular expression features. See RE2's |
| 18 |
website <http://code.google.com/p/re2> for more information. |
| 19 |
|
| 20 |
Fallback to normal Perl regexp is implemented by this module. If RE2 is |
| 21 |
unable to compile a regexp it will use Perl instead, therefore features |
| 22 |
not implemented by RE2 don't suddenly stop working, they will just use |
| 23 |
Perl's regexp implementation. |
| 24 |
|
| 25 |
METHODS |
| 26 |
To access extra functionality of RE2 methods can be called on a compiled |
| 27 |
regular expression (i.e. a "qr//"). |
| 28 |
|
| 29 |
* "possible_match_range([length = 10])" |
| 30 |
|
| 31 |
Returns an array of two strings: where the expression will start |
| 32 |
matching and just after where it will finish matching. See RE2's |
| 33 |
documentation on PossibleMatchRange for further details. |
| 34 |
|
| 35 |
Example: |
| 36 |
|
| 37 |
my($min, $max) = qr/^(a|b)/->possible_match_range; |
| 38 |
is $min, 'a'; |
| 39 |
is $max, 'c';' |
| 40 |
|
| 41 |
* "named_captures()" |
| 42 |
|
| 43 |
Returns a hash of the name captures and index. |
| 44 |
|
| 45 |
Example: |
| 46 |
|
| 47 |
my $named_captures = qr/(?P<a>\w+) (?P<d>\w+)/->named_captures; |
| 48 |
is $named_captures->{a}, 1; |
| 49 |
is $named_captures->{d}, 2; |
| 50 |
|
| 51 |
* "number_of_capture_groups()" |
| 52 |
|
| 53 |
Return number of capture groups |
| 54 |
|
| 55 |
Example: |
| 56 |
|
| 57 |
my $captures = qr/(Hello), (world)/->number_of_capture_groups; |
| 58 |
is $captures, 2; |
| 59 |
|
| 60 |
PRAGMA OPTIONS |
| 61 |
Various options can be set by providing options to the "use" line. These |
| 62 |
will be pragma scoped. |
| 63 |
|
| 64 |
* "-max_mem => 1<<24" |
| 65 |
|
| 66 |
Configure RE2's memory limit. |
| 67 |
|
| 68 |
* "-strict => 1" |
| 69 |
|
| 70 |
Be strict, i.e. don't allow regexps that are not supported by RE2. |
| 71 |
|
| 72 |
* "-longest_match => 1" |
| 73 |
|
| 74 |
Match on the longest match in alternations. For example with this |
| 75 |
option set matching "abc" against "(a|abc)" will match "abc", |
| 76 |
without depending on order. |
| 77 |
|
| 78 |
* "-never_nl => 1" |
| 79 |
|
| 80 |
Never match a newline ("\n") even if the provided regexp contains |
| 81 |
it. |
| 82 |
|
| 83 |
PERFORMANCE |
| 84 |
Performance is really the primary reason for using RE2, so here's some |
| 85 |
benchmarks. Like any benchmark take them with a pinch of salt. |
| 86 |
|
| 87 |
Simple matching |
| 88 |
my $foo = "foo bar baz"; |
| 89 |
$foo =~ /foo/; |
| 90 |
$foo =~ /foox/; |
| 91 |
|
| 92 |
On this very simple match RE2 is actually slower: |
| 93 |
|
| 94 |
Rate re2 re |
| 95 |
re2 674634/s -- -76% |
| 96 |
re 2765739/s 310% -- |
| 97 |
|
| 98 |
URL matching |
| 99 |
Matching "m{([a-zA-Z][a-zA-Z0-9]*)://([^ /]+)(/[^ ]*)?|([^ @]+)@([^ |
| 100 |
@]+)}" against a several KB file: |
| 101 |
|
| 102 |
Rate re re2 |
| 103 |
re 35.2/s -- -99% |
| 104 |
re2 2511/s 7037% -- |
| 105 |
|
| 106 |
Many alternatives |
| 107 |
Matching a string against a regexp with 17,576 alternatives ("aaa .. |
| 108 |
zzz"). |
| 109 |
|
| 110 |
This uses trie matching on Perl (obviously RE2 does similar by default). |
| 111 |
|
| 112 |
$ perl misc/altern.pl |
| 113 |
Rate re re2 |
| 114 |
re 52631/s -- -91% |
| 115 |
re2 554938/s 954% -- |
| 116 |
|
| 117 |
NOTES |
| 118 |
* No support for "m//x" |
| 119 |
|
| 120 |
The "/x" modifier is not supported. (There's no particular reason |
| 121 |
for this, just RE2 itself doesn't support it). Fallback to Perl |
| 122 |
regexp will happen automatically if "//x" is used. |
| 123 |
|
| 124 |
* "re2/dfa.cc:447: DFA out of memory: prog size xxx mem yyy" |
| 125 |
|
| 126 |
If you attempt to compile a really large regular expression you may |
| 127 |
get this error. RE2 has an internal limit on memory consumption for |
| 128 |
the DFA state tables. By default this is 8 MiB. |
| 129 |
|
| 130 |
If you need to increase this size then use the max_mem parameter: |
| 131 |
|
| 132 |
use re::engine::RE2 -max_mem => 8<<23; # 64MiB |
| 133 |
|
| 134 |
* How do I tell if RE2 will be used? |
| 135 |
|
| 136 |
See if your regexp is matching quickly or slowly ;). |
| 137 |
|
| 138 |
Alternatively normal OO concepts apply and you may examine the |
| 139 |
object returned by "qr//": |
| 140 |
|
| 141 |
use re::engine::RE2; |
| 142 |
|
| 143 |
ok qr/foo/->isa("re::engine::RE2"); |
| 144 |
|
| 145 |
# Perl Regexp used instead |
| 146 |
ok not qr/(?<=foo)bar/->isa("re::engine::RE2"); |
| 147 |
|
| 148 |
If you wish to force RE2, use the "-strict" option. |
| 149 |
|
| 150 |
BUGS |
| 151 |
Known issues: |
| 152 |
|
| 153 |
* Unicode handling |
| 154 |
|
| 155 |
Currently the Unicode handling of re::engine::RE2 does not fully |
| 156 |
match Perl's behaviour. |
| 157 |
|
| 158 |
The UTF-8 flag of the regexp currently determines how the string is |
| 159 |
matched. This is obviously broken, so will be fixed at some point. |
| 160 |
|
| 161 |
* Final newline matching differs to Perl |
| 162 |
|
| 163 |
"\n" =~ /$/ |
| 164 |
|
| 165 |
The above is true in Perl, false in RE2. To work around the issue |
| 166 |
you can write "\n?\z" when you mean Perl's "$". |
| 167 |
|
| 168 |
Please report bugs or provide patches at |
| 169 |
<https://github.com/dgl/re-engine-RE2>. |
| 170 |
|
| 171 |
AUTHORS |
| 172 |
David Leadbeater <dgl[at]dgl[dot]cx> |
| 173 |
|
| 174 |
COPYRIGHT |
| 175 |
Copyright 2010 David Leadbeater. |
| 176 |
|
| 177 |
Based on re::engine::PCRE: |
| 178 |
|
| 179 |
Copyright 2007 Ævar Arnfjörð Bjarmason. |
| 180 |
|
| 181 |
The original version was copyright 2006 Audrey Tang <[email protected]> |
| 182 |
and Yves Orton. |
| 183 |
|
| 184 |
This program is free software; you can redistribute it and/or modify it |
| 185 |
under the same terms as Perl itself. |
| 186 |
|
| 187 |
(However the bundled copy of RE2 has a different copyright owner and is |
| 188 |
under a BSD-like license, see re2/LICENSE.) |
| 189 |
|