| 1 |
:mod:`!re` --- Regular expression operations |
| 2 |
============================================ |
| 3 |
|
| 4 |
.. module:: re |
| 5 |
:synopsis: Regular expression operations. |
| 6 |
|
| 7 |
.. moduleauthor:: Fredrik Lundh <[email protected]> |
| 8 |
.. sectionauthor:: Andrew M. Kuchling <[email protected]> |
| 9 |
|
| 10 |
**Source code:** :source:`Lib/re/` |
| 11 |
|
| 12 |
-------------- |
| 13 |
|
| 14 |
This module provides regular expression matching operations similar to |
| 15 |
those found in Perl. |
| 16 |
|
| 17 |
Both patterns and strings to be searched can be Unicode strings (:class:`str`) |
| 18 |
as well as 8-bit strings (:class:`bytes`). |
| 19 |
However, Unicode strings and 8-bit strings cannot be mixed: |
| 20 |
that is, you cannot match a Unicode string with a bytes pattern or |
| 21 |
vice-versa; similarly, when asking for a substitution, the replacement |
| 22 |
string must be of the same type as both the pattern and the search string. |
| 23 |
|
| 24 |
Regular expressions use the backslash character (``'\'``) to indicate |
| 25 |
special forms or to allow special characters to be used without invoking |
| 26 |
their special meaning. This collides with Python's usage of the same |
| 27 |
character for the same purpose in string literals; for example, to match |
| 28 |
a literal backslash, one might have to write ``'\\\\'`` as the pattern |
| 29 |
string, because the regular expression must be ``\\``, and each |
| 30 |
backslash must be expressed as ``\\`` inside a regular Python string |
| 31 |
literal. Also, please note that any invalid escape sequences in Python's |
| 32 |
usage of the backslash in string literals now generate a :exc:`SyntaxWarning` |
| 33 |
and in the future this will become a :exc:`SyntaxError`. This behaviour |
| 34 |
will happen even if it is a valid escape sequence for a regular expression. |
| 35 |
|
| 36 |
The solution is to use Python's raw string notation for regular expression |
| 37 |
patterns; backslashes are not handled in any special way in a string literal |
| 38 |
prefixed with ``'r'``. So ``r"\n"`` is a two-character string containing |
| 39 |
``'\'`` and ``'n'``, while ``"\n"`` is a one-character string containing a |
| 40 |
newline. Usually patterns will be expressed in Python code using this raw |
| 41 |
string notation. |
| 42 |
|
| 43 |
It is important to note that most regular expression operations are available as |
| 44 |
module-level functions and methods on |
| 45 |
:ref:`compiled regular expressions <re-objects>`. The functions are shortcuts |
| 46 |
that don't require you to compile a regex object first, but miss some |
| 47 |
fine-tuning parameters. |
| 48 |
|
| 49 |
.. seealso:: |
| 50 |
|
| 51 |
The third-party :pypi:`regex` module, |
| 52 |
which has an API compatible with the standard library :mod:`!re` module, |
| 53 |
but offers additional functionality and a more thorough Unicode support. |
| 54 |
|
| 55 |
|
| 56 |
.. _re-syntax: |
| 57 |
|
| 58 |
Regular Expression Syntax |
| 59 |
------------------------- |
| 60 |
|
| 61 |
A regular expression (or RE) specifies a set of strings that matches it; the |
| 62 |
functions in this module let you check if a particular string matches a given |
| 63 |
regular expression (or if a given regular expression matches a particular |
| 64 |
string, which comes down to the same thing). |
| 65 |
|
| 66 |
Regular expressions can be concatenated to form new regular expressions; if *A* |
| 67 |
and *B* are both regular expressions, then *AB* is also a regular expression. |
| 68 |
In general, if a string *p* matches *A* and another string *q* matches *B*, the |
| 69 |
string *pq* will match AB. This holds unless *A* or *B* contain low precedence |
| 70 |
operations; boundary conditions between *A* and *B*; or have numbered group |
| 71 |
references. Thus, complex expressions can easily be constructed from simpler |
| 72 |
primitive expressions like the ones described here. For details of the theory |
| 73 |
and implementation of regular expressions, consult the Friedl book [Frie09]_, |
| 74 |
or almost any textbook about compiler construction. |
| 75 |
|
| 76 |
A brief explanation of the format of regular expressions follows. For further |
| 77 |
information and a gentler presentation, consult the :ref:`regex-howto`. |
| 78 |
|
| 79 |
Regular expressions can contain both special and ordinary characters. Most |
| 80 |
ordinary characters, like ``'A'``, ``'a'``, or ``'0'``, are the simplest regular |
| 81 |
expressions; they simply match themselves. You can concatenate ordinary |
| 82 |
characters, so ``last`` matches the string ``'last'``. (In the rest of this |
| 83 |
section, we'll write RE's in ``this special style``, usually without quotes, and |
| 84 |
strings to be matched ``'in single quotes'``.) |
| 85 |
|
| 86 |
Some characters, like ``'|'`` or ``'('``, are special. Special |
| 87 |
characters either stand for classes of ordinary characters, or affect |
| 88 |
how the regular expressions around them are interpreted. |
| 89 |
|
| 90 |
Repetition operators or quantifiers (``*``, ``+``, ``?``, ``{m,n}``, etc) cannot be |
| 91 |
directly nested. This avoids ambiguity with the non-greedy modifier suffix |
| 92 |
``?``, and with other modifiers in other implementations. To apply a second |
| 93 |
repetition to an inner repetition, parentheses may be used. For example, |
| 94 |
the expression ``(?:a{6})*`` matches any multiple of six ``'a'`` characters. |
| 95 |
|
| 96 |
|
| 97 |
The special characters are: |
| 98 |
|
| 99 |
.. index:: single: . (dot); in regular expressions |
| 100 |
|
| 101 |
``.`` |
| 102 |
(Dot.) In the default mode, this matches any character except a newline. If |
| 103 |
the :const:`DOTALL` flag has been specified, this matches any character |
| 104 |
including a newline. ``(?s:.)`` matches any character regardless of flags. |
| 105 |
|
| 106 |
.. index:: single: ^ (caret); in regular expressions |
| 107 |
|
| 108 |
``^`` |
| 109 |
(Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also |
| 110 |
matches immediately after each newline. |
| 111 |
|
| 112 |
.. index:: single: $ (dollar); in regular expressions |
| 113 |
|
| 114 |
``$`` |
| 115 |
Matches the end of the string or just before the newline at the end of the |
| 116 |
string, and in :const:`MULTILINE` mode also matches before a newline. ``foo`` |
| 117 |
matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches |
| 118 |
only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'`` |
| 119 |
matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for |
| 120 |
a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before |
| 121 |
the newline, and one at the end of the string. |
| 122 |
|
| 123 |
.. index:: single: * (asterisk); in regular expressions |
| 124 |
|
| 125 |
``*`` |
| 126 |
Causes the resulting RE to match 0 or more repetitions of the preceding RE, as |
| 127 |
many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed |
| 128 |
by any number of 'b's. |
| 129 |
|
| 130 |
.. index:: single: + (plus); in regular expressions |
| 131 |
|
| 132 |
``+`` |
| 133 |
Causes the resulting RE to match 1 or more repetitions of the preceding RE. |
| 134 |
``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not |
| 135 |
match just 'a'. |
| 136 |
|
| 137 |
.. index:: single: ? (question mark); in regular expressions |
| 138 |
|
| 139 |
``?`` |
| 140 |
Causes the resulting RE to match 0 or 1 repetitions of the preceding RE. |
| 141 |
``ab?`` will match either 'a' or 'ab'. |
| 142 |
|
| 143 |
.. index:: |
| 144 |
single: *?; in regular expressions |
| 145 |
single: +?; in regular expressions |
| 146 |
single: ??; in regular expressions |
| 147 |
|
| 148 |
``*?``, ``+?``, ``??`` |
| 149 |
The ``'*'``, ``'+'``, and ``'?'`` quantifiers are all :dfn:`greedy`; they match |
| 150 |
as much text as possible. Sometimes this behaviour isn't desired; if the RE |
| 151 |
``<.*>`` is matched against ``'<a> b <c>'``, it will match the entire |
| 152 |
string, and not just ``'<a>'``. Adding ``?`` after the quantifier makes it |
| 153 |
perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few* |
| 154 |
characters as possible will be matched. Using the RE ``<.*?>`` will match |
| 155 |
only ``'<a>'``. |
| 156 |
|
| 157 |
.. index:: |
| 158 |
single: *+; in regular expressions |
| 159 |
single: ++; in regular expressions |
| 160 |
single: ?+; in regular expressions |
| 161 |
|
| 162 |
``*+``, ``++``, ``?+`` |
| 163 |
Like the ``'*'``, ``'+'``, and ``'?'`` quantifiers, those where ``'+'`` is |
| 164 |
appended also match as many times as possible. |
| 165 |
However, unlike the true greedy quantifiers, these do not allow |
| 166 |
back-tracking when the expression following it fails to match. |
| 167 |
These are known as :dfn:`possessive` quantifiers. |
| 168 |
For example, ``a*a`` will match ``'aaaa'`` because the ``a*`` will match |
| 169 |
all 4 ``'a'``\ s, but, when the final ``'a'`` is encountered, the |
| 170 |
expression is backtracked so that in the end the ``a*`` ends up matching |
| 171 |
3 ``'a'``\ s total, and the fourth ``'a'`` is matched by the final ``'a'``. |
| 172 |
However, when ``a*+a`` is used to match ``'aaaa'``, the ``a*+`` will |
| 173 |
match all 4 ``'a'``, but when the final ``'a'`` fails to find any more |
| 174 |
characters to match, the expression cannot be backtracked and will thus |
| 175 |
fail to match. |
| 176 |
``x*+``, ``x++`` and ``x?+`` are equivalent to ``(?>x*)``, ``(?>x+)`` |
| 177 |
and ``(?>x?)`` correspondingly. |
| 178 |
|
| 179 |
.. versionadded:: 3.11 |
| 180 |
|
| 181 |
.. index:: |
| 182 |
single: {} (curly brackets); in regular expressions |
| 183 |
|
| 184 |
``{m}`` |
| 185 |
Specifies that exactly *m* copies of the previous RE should be matched; fewer |
| 186 |
matches cause the entire RE not to match. For example, ``a{6}`` will match |
| 187 |
exactly six ``'a'`` characters, but not five. |
| 188 |
|
| 189 |
``{m,n}`` |
| 190 |
Causes the resulting RE to match from *m* to *n* repetitions of the preceding |
| 191 |
RE, attempting to match as many repetitions as possible. For example, |
| 192 |
``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a |
| 193 |
lower bound of zero, and omitting *n* specifies an infinite upper bound. As an |
| 194 |
example, ``a{4,}b`` will match ``'aaaab'`` or a thousand ``'a'`` characters |
| 195 |
followed by a ``'b'``, but not ``'aaab'``. The comma may not be omitted or the |
| 196 |
modifier would be confused with the previously described form. |
| 197 |
|
| 198 |
``{m,n}?`` |
| 199 |
Causes the resulting RE to match from *m* to *n* repetitions of the preceding |
| 200 |
RE, attempting to match as *few* repetitions as possible. This is the |
| 201 |
non-greedy version of the previous quantifier. For example, on the |
| 202 |
6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters, |
| 203 |
while ``a{3,5}?`` will only match 3 characters. |
| 204 |
|
| 205 |
``{m,n}+`` |
| 206 |
Causes the resulting RE to match from *m* to *n* repetitions of the |
| 207 |
preceding RE, attempting to match as many repetitions as possible |
| 208 |
*without* establishing any backtracking points. |
| 209 |
This is the possessive version of the quantifier above. |
| 210 |
For example, on the 6-character string ``'aaaaaa'``, ``a{3,5}+aa`` |
| 211 |
attempt to match 5 ``'a'`` characters, then, requiring 2 more ``'a'``\ s, |
| 212 |
will need more characters than available and thus fail, while |
| 213 |
``a{3,5}aa`` will match with ``a{3,5}`` capturing 5, then 4 ``'a'``\ s |
| 214 |
by backtracking and then the final 2 ``'a'``\ s are matched by the final |
| 215 |
``aa`` in the pattern. |
| 216 |
``x{m,n}+`` is equivalent to ``(?>x{m,n})``. |
| 217 |
|
| 218 |
.. versionadded:: 3.11 |
| 219 |
|
| 220 |
.. index:: single: \ (backslash); in regular expressions |
| 221 |
|
| 222 |
``\`` |
| 223 |
Either escapes special characters (permitting you to match characters like |
| 224 |
``'*'``, ``'?'``, and so forth), or signals a special sequence; special |
| 225 |
sequences are discussed below. |
| 226 |
|
| 227 |
If you're not using a raw string to express the pattern, remember that Python |
| 228 |
also uses the backslash as an escape sequence in string literals; if the escape |
| 229 |
sequence isn't recognized by Python's parser, the backslash and subsequent |
| 230 |
character are included in the resulting string. However, if Python would |
| 231 |
recognize the resulting sequence, the backslash should be repeated twice. This |
| 232 |
is complicated and hard to understand, so it's highly recommended that you use |
| 233 |
raw strings for all but the simplest expressions. |
| 234 |
|
| 235 |
.. index:: |
| 236 |
single: [] (square brackets); in regular expressions |
| 237 |
|
| 238 |
``[]`` |
| 239 |
Used to indicate a set of characters. In a set: |
| 240 |
|
| 241 |
* Characters can be listed individually, e.g. ``[amk]`` will match ``'a'``, |
| 242 |
``'m'``, or ``'k'``. |
| 243 |
|
| 244 |
.. index:: single: - (minus); in regular expressions |
| 245 |
|
| 246 |
* Ranges of characters can be indicated by giving two characters and separating |
| 247 |
them by a ``'-'``, for example ``[a-z]`` will match any lowercase ASCII letter, |
| 248 |
``[0-5][0-9]`` will match all the two-digits numbers from ``00`` to ``59``, and |
| 249 |
``[0-9A-Fa-f]`` will match any hexadecimal digit. If ``-`` is escaped (e.g. |
| 250 |
``[a\-z]``) or if it's placed as the first or last character |
| 251 |
(e.g. ``[-a]`` or ``[a-]``), it will match a literal ``'-'``. |
| 252 |
|
| 253 |
* Special characters except backslash lose their special meaning inside sets. |
| 254 |
For example, |
| 255 |
``[(+*)]`` will match any of the literal characters ``'('``, ``'+'``, |
| 256 |
``'*'``, or ``')'``. |
| 257 |
|
| 258 |
.. index:: single: \ (backslash); in regular expressions |
| 259 |
|
| 260 |
* Backslash either escapes characters which have special meaning in a set |
| 261 |
such as ``'-'``, ``']'``, ``'^'`` and ``'\\'`` itself or signals |
| 262 |
a special sequence which represents a single character such as |
| 263 |
``\xa0`` or ``\n`` or a character class such as ``\w`` or ``\S`` |
| 264 |
(defined below). |
| 265 |
Note that ``\b`` represents a single "backspace" character, |
| 266 |
not a word boundary as outside a set, and numeric escapes |
| 267 |
such as ``\1`` are always octal escapes, not group references. |
| 268 |
Special sequences which do not match a single character such as ``\A`` |
| 269 |
and ``\z`` are not allowed. |
| 270 |
|
| 271 |
.. index:: single: ^ (caret); in regular expressions |
| 272 |
|
| 273 |
* Characters that are not within a range can be matched by :dfn:`complementing` |
| 274 |
the set. If the first character of the set is ``'^'``, all the characters |
| 275 |
that are *not* in the set will be matched. For example, ``[^5]`` will match |
| 276 |
any character except ``'5'``, and ``[^^]`` will match any character except |
| 277 |
``'^'``. ``^`` has no special meaning if it's not the first character in |
| 278 |
the set. |
| 279 |
|
| 280 |
* To match a literal ``']'`` inside a set, precede it with a backslash, or |
| 281 |
place it at the beginning of the set. For example, both ``[()[\]{}]`` and |
| 282 |
``[]()[{}]`` will match a right bracket, as well as left bracket, braces, |
| 283 |
and parentheses. |
| 284 |
|
| 285 |
.. .. index:: single: --; in regular expressions |
| 286 |
.. .. index:: single: &&; in regular expressions |
| 287 |
.. .. index:: single: ~~; in regular expressions |
| 288 |
.. .. index:: single: ||; in regular expressions |
| 289 |
|
| 290 |
* Support of nested sets and set operations as in `Unicode Technical |
| 291 |
Standard #18`_ might be added in the future. This would change the |
| 292 |
syntax, so to facilitate this change a :exc:`FutureWarning` will be raised |
| 293 |
in ambiguous cases for the time being. |
| 294 |
That includes sets starting with a literal ``'['`` or containing literal |
| 295 |
character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To |
| 296 |
avoid a warning escape them with a backslash. |
| 297 |
|
| 298 |
.. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ |
| 299 |
|
| 300 |
.. versionchanged:: 3.7 |
| 301 |
:exc:`FutureWarning` is raised if a character set contains constructs |
| 302 |
that will change semantically in the future. |
| 303 |
|
| 304 |
.. index:: single: | (vertical bar); in regular expressions |
| 305 |
|
| 306 |
``|`` |
| 307 |
``A|B``, where *A* and *B* can be arbitrary REs, creates a regular expression that |
| 308 |
will match either *A* or *B*. An arbitrary number of REs can be separated by the |
| 309 |
``'|'`` in this way. This can be used inside groups (see below) as well. As |
| 310 |
the target string is scanned, REs separated by ``'|'`` are tried from left to |
| 311 |
right. When one pattern completely matches, that branch is accepted. This means |
| 312 |
that once *A* matches, *B* will not be tested further, even if it would |
| 313 |
produce a longer overall match. In other words, the ``'|'`` operator is never |
| 314 |
greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a |
| 315 |
character class, as in ``[|]``. |
| 316 |
|
| 317 |
.. index:: |
| 318 |
single: () (parentheses); in regular expressions |
| 319 |
|
| 320 |
``(...)`` |
| 321 |
Matches whatever regular expression is inside the parentheses, and indicates the |
| 322 |
start and end of a group; the contents of a group can be retrieved after a match |
| 323 |
has been performed, and can be matched later in the string with the ``\number`` |
| 324 |
special sequence, described below. To match the literals ``'('`` or ``')'``, |
| 325 |
use ``\(`` or ``\)``, or enclose them inside a character class: ``[(]``, ``[)]``. |
| 326 |
|
| 327 |
.. index:: single: (?; in regular expressions |
| 328 |
|
| 329 |
``(?...)`` |
| 330 |
This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful |
| 331 |
otherwise). The first character after the ``'?'`` determines what the meaning |
| 332 |
and further syntax of the construct is. Extensions usually do not create a new |
| 333 |
group; ``(?P<name>...)`` is the only exception to this rule. Following are the |
| 334 |
currently supported extensions. |
| 335 |
|
| 336 |
``(?aiLmsux)`` |
| 337 |
(One or more letters from the set |
| 338 |
``'a'``, ``'i'``, ``'L'``, ``'m'``, ``'s'``, ``'u'``, ``'x'``.) |
| 339 |
The group matches the empty string; |
| 340 |
the letters set the corresponding flags for the entire regular expression: |
| 341 |
|
| 342 |
* :const:`re.A` (ASCII-only matching) |
| 343 |
* :const:`re.I` (ignore case) |
| 344 |
* :const:`re.L` (locale dependent) |
| 345 |
* :const:`re.M` (multi-line) |
| 346 |
* :const:`re.S` (dot matches all) |
| 347 |
* :const:`re.U` (Unicode matching) |
| 348 |
* :const:`re.X` (verbose) |
| 349 |
|
| 350 |
(The flags are described in :ref:`contents-of-module-re`.) |
| 351 |
This is useful if you wish to include the flags as part of the |
| 352 |
regular expression, instead of passing a *flag* argument to the |
| 353 |
:func:`re.compile` function. |
| 354 |
Flags should be used first in the expression string. |
| 355 |
|
| 356 |
.. versionchanged:: 3.11 |
| 357 |
This construction can only be used at the start of the expression. |
| 358 |
|
| 359 |
.. index:: single: (?:; in regular expressions |
| 360 |
|
| 361 |
``(?:...)`` |
| 362 |
A non-capturing version of regular parentheses. Matches whatever regular |
| 363 |
expression is inside the parentheses, but the substring matched by the group |
| 364 |
*cannot* be retrieved after performing a match or referenced later in the |
| 365 |
pattern. |
| 366 |
|
| 367 |
``(?aiLmsux-imsx:...)`` |
| 368 |
(Zero or more letters from the set |
| 369 |
``'a'``, ``'i'``, ``'L'``, ``'m'``, ``'s'``, ``'u'``, ``'x'``, |
| 370 |
optionally followed by ``'-'`` followed by |
| 371 |
one or more letters from the ``'i'``, ``'m'``, ``'s'``, ``'x'``.) |
| 372 |
The letters set or remove the corresponding flags for the part of the expression: |
| 373 |
|
| 374 |
* :const:`re.A` (ASCII-only matching) |
| 375 |
* :const:`re.I` (ignore case) |
| 376 |
* :const:`re.L` (locale dependent) |
| 377 |
* :const:`re.M` (multi-line) |
| 378 |
* :const:`re.S` (dot matches all) |
| 379 |
* :const:`re.U` (Unicode matching) |
| 380 |
* :const:`re.X` (verbose) |
| 381 |
|
| 382 |
(The flags are described in :ref:`contents-of-module-re`.) |
| 383 |
|
| 384 |
The letters ``'a'``, ``'L'`` and ``'u'`` are mutually exclusive when used |
| 385 |
as inline flags, so they can't be combined or follow ``'-'``. Instead, |
| 386 |
when one of them appears in an inline group, it overrides the matching mode |
| 387 |
in the enclosing group. In Unicode patterns ``(?a:...)`` switches to |
| 388 |
ASCII-only matching, and ``(?u:...)`` switches to Unicode matching |
| 389 |
(default). In bytes patterns ``(?L:...)`` switches to locale dependent |
| 390 |
matching, and ``(?a:...)`` switches to ASCII-only matching (default). |
| 391 |
This override is only in effect for the narrow inline group, and the |
| 392 |
original matching mode is restored outside of the group. |
| 393 |
|
| 394 |
.. versionadded:: 3.6 |
| 395 |
|
| 396 |
.. versionchanged:: 3.7 |
| 397 |
The letters ``'a'``, ``'L'`` and ``'u'`` also can be used in a group. |
| 398 |
|
| 399 |
``(?>...)`` |
| 400 |
Attempts to match ``...`` as if it was a separate regular expression, and |
| 401 |
if successful, continues to match the rest of the pattern following it. |
| 402 |
If the subsequent pattern fails to match, the stack can only be unwound |
| 403 |
to a point *before* the ``(?>...)`` because once exited, the expression, |
| 404 |
known as an :dfn:`atomic group`, has thrown away all stack points within |
| 405 |
itself. |
| 406 |
Thus, ``(?>.*).`` would never match anything because first the ``.*`` |
| 407 |
would match all characters possible, then, having nothing left to match, |
| 408 |
the final ``.`` would fail to match. |
| 409 |
Since there are no stack points saved in the Atomic Group, and there is |
| 410 |
no stack point before it, the entire expression would thus fail to match. |
| 411 |
|
| 412 |
.. versionadded:: 3.11 |
| 413 |
|
| 414 |
.. index:: single: (?P<; in regular expressions |
| 415 |
|
| 416 |
``(?P<name>...)`` |
| 417 |
Similar to regular parentheses, but the substring matched by the group is |
| 418 |
accessible via the symbolic group name *name*. Group names must be valid |
| 419 |
Python identifiers, and in :class:`bytes` patterns they can only contain |
| 420 |
bytes in the ASCII range. Each group name must be defined only once within |
| 421 |
a regular expression. A symbolic group is also a numbered group, just as if |
| 422 |
the group were not named. |
| 423 |
|
| 424 |
Named groups can be referenced in three contexts. If the pattern is |
| 425 |
``(?P<quote>['"]).*?(?P=quote)`` (i.e. matching a string quoted with either |
| 426 |
single or double quotes): |
| 427 |
|
| 428 |
+---------------------------------------+----------------------------------+ |
| 429 |
| Context of reference to group "quote" | Ways to reference it | |
| 430 |
+=======================================+==================================+ |
| 431 |
| in the same pattern itself | * ``(?P=quote)`` (as shown) | |
| 432 |
| | * ``\1`` | |
| 433 |
+---------------------------------------+----------------------------------+ |
| 434 |
| when processing match object *m* | * ``m.group('quote')`` | |
| 435 |
| | * ``m.end('quote')`` (etc.) | |
| 436 |
+---------------------------------------+----------------------------------+ |
| 437 |
| in a string passed to the *repl* | * ``\g<quote>`` | |
| 438 |
| argument of ``re.sub()`` | * ``\g<1>`` | |
| 439 |
| | * ``\1`` | |
| 440 |
+---------------------------------------+----------------------------------+ |
| 441 |
|
| 442 |
.. versionchanged:: 3.12 |
| 443 |
In :class:`bytes` patterns, group *name* can only contain bytes |
| 444 |
in the ASCII range (``b'\x00'``-``b'\x7f'``). |
| 445 |
|
| 446 |
.. index:: single: (?P=; in regular expressions |
| 447 |
|
| 448 |
``(?P=name)`` |
| 449 |
A backreference to a named group; it matches whatever text was matched by the |
| 450 |
earlier group named *name*. |
| 451 |
|
| 452 |
.. index:: single: (?#; in regular expressions |
| 453 |
|
| 454 |
``(?#...)`` |
| 455 |
A comment; the contents of the parentheses are simply ignored. |
| 456 |
|
| 457 |
.. index:: single: (?=; in regular expressions |
| 458 |
|
| 459 |
``(?=...)`` |
| 460 |
Matches if ``...`` matches next, but doesn't consume any of the string. This is |
| 461 |
called a :dfn:`lookahead assertion`. For example, ``Isaac (?=Asimov)`` will match |
| 462 |
``'Isaac '`` only if it's followed by ``'Asimov'``. |
| 463 |
|
| 464 |
.. index:: single: (?!; in regular expressions |
| 465 |
|
| 466 |
``(?!...)`` |
| 467 |
Matches if ``...`` doesn't match next. This is a :dfn:`negative lookahead assertion`. |
| 468 |
For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not* |
| 469 |
followed by ``'Asimov'``. |
| 470 |
|
| 471 |
.. index:: single: (?<=; in regular expressions |
| 472 |
|
| 473 |
``(?<=...)`` |
| 474 |
Matches if the current position in the string is preceded by a match for ``...`` |
| 475 |
that ends at the current position. This is called a :dfn:`positive lookbehind |
| 476 |
assertion`. ``(?<=abc)def`` will find a match in ``'abcdef'``, since the |
| 477 |
lookbehind will back up 3 characters and check if the contained pattern matches. |
| 478 |
The contained pattern must only match strings of some fixed length, meaning that |
| 479 |
``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that |
| 480 |
patterns which start with positive lookbehind assertions will not match at the |
| 481 |
beginning of the string being searched; you will most likely want to use the |
| 482 |
:func:`search` function rather than the :func:`match` function: |
| 483 |
|
| 484 |
>>> import re |
| 485 |
>>> m = re.search('(?<=abc)def', 'abcdef') |
| 486 |
>>> m.group(0) |
| 487 |
'def' |
| 488 |
|
| 489 |
This example looks for a word following a hyphen: |
| 490 |
|
| 491 |
>>> m = re.search(r'(?<=-)\w+', 'spam-egg') |
| 492 |
>>> m.group(0) |
| 493 |
'egg' |
| 494 |
|
| 495 |
.. versionchanged:: 3.5 |
| 496 |
Added support for group references of fixed length. |
| 497 |
|
| 498 |
.. index:: single: (?<!; in regular expressions |
| 499 |
|
| 500 |
``(?<!...)`` |
| 501 |
Matches if the current position in the string is not preceded by a match for |
| 502 |
``...``. This is called a :dfn:`negative lookbehind assertion`. Similar to |
| 503 |
positive lookbehind assertions, the contained pattern must only match strings of |
| 504 |
some fixed length. Patterns which start with negative lookbehind assertions may |
| 505 |
match at the beginning of the string being searched. |
| 506 |
|
| 507 |
.. _re-conditional-expression: |
| 508 |
.. index:: single: (?(; in regular expressions |
| 509 |
|
| 510 |
``(?(id/name)yes-pattern|no-pattern)`` |
| 511 |
Will try to match with ``yes-pattern`` if the group with given *id* or |
| 512 |
*name* exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is |
| 513 |
optional and can be omitted. For example, |
| 514 |
``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>|$)`` is a poor email matching pattern, which |
| 515 |
will match with ``'<[email protected]>'`` as well as ``'[email protected]'``, but |
| 516 |
not with ``'<[email protected]'`` nor ``'[email protected]>'``. |
| 517 |
|
| 518 |
.. versionchanged:: 3.12 |
| 519 |
Group *id* can only contain ASCII digits. |
| 520 |
In :class:`bytes` patterns, group *name* can only contain bytes |
| 521 |
in the ASCII range (``b'\x00'``-``b'\x7f'``). |
| 522 |
|
| 523 |
|
| 524 |
.. _re-special-sequences: |
| 525 |
|
| 526 |
The special sequences consist of ``'\'`` and a character from the list below. |
| 527 |
If the ordinary character is not an ASCII digit or an ASCII letter, then the |
| 528 |
resulting RE will match the second character. For example, ``\$`` matches the |
| 529 |
character ``'$'``. |
| 530 |
|
| 531 |
.. index:: single: \ (backslash); in regular expressions |
| 532 |
|
| 533 |
``\number`` |
| 534 |
Matches the contents of the group of the same number. Groups are numbered |
| 535 |
starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``, |
| 536 |
but not ``'thethe'`` (note the space after the group). This special sequence |
| 537 |
can only be used to match one of the first 99 groups. If the first digit of |
| 538 |
*number* is 0, or *number* is 3 octal digits long, it will not be interpreted as |
| 539 |
a group match, but as the character with octal value *number*. Inside the |
| 540 |
``'['`` and ``']'`` of a character class, all numeric escapes are treated as |
| 541 |
characters. |
| 542 |
|
| 543 |
.. index:: single: \A; in regular expressions |
| 544 |
|
| 545 |
``\A`` |
| 546 |
Matches only at the start of the string. |
| 547 |
|
| 548 |
.. index:: single: \b; in regular expressions |
| 549 |
|
| 550 |
``\b`` |
| 551 |
Matches the empty string, but only at the beginning or end of a word. |
| 552 |
A word is defined as a sequence of word characters. |
| 553 |
Note that formally, ``\b`` is defined as the boundary |
| 554 |
between a ``\w`` and a ``\W`` character (or vice versa), |
| 555 |
or between ``\w`` and the beginning or end of the string. |
| 556 |
This means that ``r'\bat\b'`` matches ``'at'``, ``'at.'``, ``'(at)'``, |
| 557 |
and ``'as at ay'`` but not ``'attempt'`` or ``'atlas'``. |
| 558 |
|
| 559 |
The default word characters in Unicode (str) patterns |
| 560 |
are Unicode alphanumerics and the underscore, |
| 561 |
but this can be changed by using the :py:const:`~re.ASCII` flag. |
| 562 |
Word boundaries are determined by the current locale |
| 563 |
if the :py:const:`~re.LOCALE` flag is used. |
| 564 |
|
| 565 |
.. note:: |
| 566 |
|
| 567 |
Inside a character range, ``\b`` represents the backspace character, |
| 568 |
for compatibility with Python's string literals. |
| 569 |
|
| 570 |
.. index:: single: \B; in regular expressions |
| 571 |
|
| 572 |
``\B`` |
| 573 |
Matches the empty string, |
| 574 |
but only when it is *not* at the beginning or end of a word. |
| 575 |
This means that ``r'at\B'`` matches ``'athens'``, ``'atom'``, |
| 576 |
``'attorney'``, but not ``'at'``, ``'at.'``, or ``'at!'``. |
| 577 |
``\B`` is the opposite of ``\b``, |
| 578 |
so word characters in Unicode (str) patterns |
| 579 |
are Unicode alphanumerics or the underscore, |
| 580 |
although this can be changed by using the :py:const:`~re.ASCII` flag. |
| 581 |
Word boundaries are determined by the current locale |
| 582 |
if the :py:const:`~re.LOCALE` flag is used. |
| 583 |
|
| 584 |
.. versionchanged:: 3.14 |
| 585 |
``\B`` now matches empty input string. |
| 586 |
|
| 587 |
.. index:: single: \d; in regular expressions |
| 588 |
|
| 589 |
``\d`` |
| 590 |
For Unicode (str) patterns: |
| 591 |
Matches any Unicode decimal digit |
| 592 |
(that is, any character in Unicode character category `[Nd]`__). |
| 593 |
This includes ``[0-9]``, and also many other digit characters. |
| 594 |
|
| 595 |
Matches ``[0-9]`` if the :py:const:`~re.ASCII` flag is used. |
| 596 |
|
| 597 |
__ https://www.unicode.org/versions/Unicode15.0.0/ch04.pdf#G134153 |
| 598 |
|
| 599 |
For 8-bit (bytes) patterns: |
| 600 |
Matches any decimal digit in the ASCII character set; |
| 601 |
this is equivalent to ``[0-9]``. |
| 602 |
|
| 603 |
.. index:: single: \D; in regular expressions |
| 604 |
|
| 605 |
``\D`` |
| 606 |
Matches any character which is not a decimal digit. |
| 607 |
This is the opposite of ``\d``. |
| 608 |
|
| 609 |
Matches ``[^0-9]`` if the :py:const:`~re.ASCII` flag is used. |
| 610 |
|
| 611 |
.. index:: single: \s; in regular expressions |
| 612 |
|
| 613 |
``\s`` |
| 614 |
For Unicode (str) patterns: |
| 615 |
Matches Unicode whitespace characters (as defined by :py:meth:`str.isspace`). |
| 616 |
This includes ``[ \t\n\r\f\v]``, and also many other characters, for example the |
| 617 |
non-breaking spaces mandated by typography rules in many languages. |
| 618 |
|
| 619 |
Matches ``[ \t\n\r\f\v]`` if the :py:const:`~re.ASCII` flag is used. |
| 620 |
|
| 621 |
For 8-bit (bytes) patterns: |
| 622 |
Matches characters considered whitespace in the ASCII character set; |
| 623 |
this is equivalent to ``[ \t\n\r\f\v]``. |
| 624 |
|
| 625 |
.. index:: single: \S; in regular expressions |
| 626 |
|
| 627 |
``\S`` |
| 628 |
Matches any character which is not a whitespace character. This is |
| 629 |
the opposite of ``\s``. |
| 630 |
|
| 631 |
Matches ``[^ \t\n\r\f\v]`` if the :py:const:`~re.ASCII` flag is used. |
| 632 |
|
| 633 |
.. index:: single: \w; in regular expressions |
| 634 |
|
| 635 |
``\w`` |
| 636 |
For Unicode (str) patterns: |
| 637 |
Matches Unicode word characters; |
| 638 |
this includes all Unicode alphanumeric characters |
| 639 |
(as defined by :py:meth:`str.isalnum`), |
| 640 |
as well as the underscore (``_``). |
| 641 |
|
| 642 |
Matches ``[a-zA-Z0-9_]`` if the :py:const:`~re.ASCII` flag is used. |
| 643 |
|
| 644 |
For 8-bit (bytes) patterns: |
| 645 |
Matches characters considered alphanumeric in the ASCII character set; |
| 646 |
this is equivalent to ``[a-zA-Z0-9_]``. |
| 647 |
If the :py:const:`~re.LOCALE` flag is used, |
| 648 |
matches characters considered alphanumeric in the current locale and the underscore. |
| 649 |
|
| 650 |
.. index:: single: \W; in regular expressions |
| 651 |
|
| 652 |
``\W`` |
| 653 |
Matches any character which is not a word character. |
| 654 |
This is the opposite of ``\w``. |
| 655 |
By default, matches non-underscore (``_``) characters |
| 656 |
for which :py:meth:`str.isalnum` returns ``False``. |
| 657 |
|
| 658 |
Matches ``[^a-zA-Z0-9_]`` if the :py:const:`~re.ASCII` flag is used. |
| 659 |
|
| 660 |
If the :py:const:`~re.LOCALE` flag is used, |
| 661 |
matches characters which are neither alphanumeric in the current locale |
| 662 |
nor the underscore. |
| 663 |
|
| 664 |
.. index:: single: \z; in regular expressions |
| 665 |
single: \Z; in regular expressions |
| 666 |
|
| 667 |
``\z`` |
| 668 |
Matches only at the end of the string. |
| 669 |
|
| 670 |
.. versionadded:: 3.14 |
| 671 |
|
| 672 |
``\Z`` |
| 673 |
The same as ``\z``. For compatibility with old Python versions. |
| 674 |
|
| 675 |
.. index:: |
| 676 |
single: \a; in regular expressions |
| 677 |
single: \b; in regular expressions |
| 678 |
single: \f; in regular expressions |
| 679 |
single: \n; in regular expressions |
| 680 |
single: \N; in regular expressions |
| 681 |
single: \r; in regular expressions |
| 682 |
single: \t; in regular expressions |
| 683 |
single: \u; in regular expressions |
| 684 |
single: \U; in regular expressions |
| 685 |
single: \v; in regular expressions |
| 686 |
single: \x; in regular expressions |
| 687 |
single: \\; in regular expressions |
| 688 |
|
| 689 |
Most of the :ref:`escape sequences <escape-sequences>` supported by Python |
| 690 |
string literals are also accepted by the regular expression parser:: |
| 691 |
|
| 692 |
\a \b \f \n |
| 693 |
\N \r \t \u |
| 694 |
\U \v \x \\ |
| 695 |
|
| 696 |
(Note that ``\b`` is used to represent word boundaries, and means "backspace" |
| 697 |
only inside character classes.) |
| 698 |
|
| 699 |
``'\u'``, ``'\U'``, and ``'\N'`` escape sequences are |
| 700 |
only recognized in Unicode (str) patterns. |
| 701 |
In bytes patterns they are errors. |
| 702 |
Unknown escapes of ASCII letters are reserved |
| 703 |
for future use and treated as errors. |
| 704 |
|
| 705 |
Octal escapes are included in a limited form. If the first digit is a 0, or if |
| 706 |
there are three octal digits, it is considered an octal escape. Otherwise, it is |
| 707 |
a group reference. As for string literals, octal escapes are always at most |
| 708 |
three digits in length. |
| 709 |
|
| 710 |
.. versionchanged:: 3.3 |
| 711 |
The ``'\u'`` and ``'\U'`` escape sequences have been added. |
| 712 |
|
| 713 |
.. versionchanged:: 3.6 |
| 714 |
Unknown escapes consisting of ``'\'`` and an ASCII letter now are errors. |
| 715 |
|
| 716 |
.. versionchanged:: 3.8 |
| 717 |
The :samp:`'\\N\\{{name}\\}'` escape sequence has been added. As in string literals, |
| 718 |
it expands to the named Unicode character (e.g. ``'\N{EM DASH}'``). |
| 719 |
|
| 720 |
|
| 721 |
.. _contents-of-module-re: |
| 722 |
|
| 723 |
Module Contents |
| 724 |
--------------- |
| 725 |
|
| 726 |
The module defines several functions, constants, and an exception. Some of the |
| 727 |
functions are simplified versions of the full featured methods for compiled |
| 728 |
regular expressions. Most non-trivial applications always use the compiled |
| 729 |
form. |
| 730 |
|
| 731 |
|
| 732 |
Flags |
| 733 |
^^^^^ |
| 734 |
|
| 735 |
.. versionchanged:: 3.6 |
| 736 |
Flag constants are now instances of :class:`RegexFlag`, which is a subclass of |
| 737 |
:class:`enum.IntFlag`. |
| 738 |
|
| 739 |
|
| 740 |
.. class:: RegexFlag |
| 741 |
|
| 742 |
An :class:`enum.IntFlag` class containing the regex options listed below. |
| 743 |
|
| 744 |
.. versionadded:: 3.11 - added to ``__all__`` |
| 745 |
|
| 746 |
.. data:: A |
| 747 |
ASCII |
| 748 |
|
| 749 |
Make ``\w``, ``\W``, ``\b``, ``\B``, ``\d``, ``\D``, ``\s`` and ``\S`` |
| 750 |
perform ASCII-only matching instead of full Unicode matching. This is only |
| 751 |
meaningful for Unicode (str) patterns, and is ignored for bytes patterns. |
| 752 |
|
| 753 |
Corresponds to the inline flag ``(?a)``. |
| 754 |
|
| 755 |
.. note:: |
| 756 |
|
| 757 |
The :py:const:`~re.U` flag still exists for backward compatibility, |
| 758 |
but is redundant in Python 3 since |
| 759 |
matches are Unicode by default for ``str`` patterns, |
| 760 |
and Unicode matching isn't allowed for bytes patterns. |
| 761 |
:py:const:`~re.UNICODE` and the inline flag ``(?u)`` are similarly redundant. |
| 762 |
|
| 763 |
|
| 764 |
.. data:: DEBUG |
| 765 |
|
| 766 |
Display debug information about compiled expression. |
| 767 |
|
| 768 |
No corresponding inline flag. |
| 769 |
|
| 770 |
|
| 771 |
.. data:: I |
| 772 |
IGNORECASE |
| 773 |
|
| 774 |
Perform case-insensitive matching; |
| 775 |
expressions like ``[A-Z]`` will also match lowercase letters. |
| 776 |
Full Unicode matching (such as ``Ü`` matching ``ü``) |
| 777 |
also works unless the :py:const:`~re.ASCII` flag |
| 778 |
is used to disable non-ASCII matches. |
| 779 |
The current locale does not change the effect of this flag |
| 780 |
unless the :py:const:`~re.LOCALE` flag is also used. |
| 781 |
|
| 782 |
Corresponds to the inline flag ``(?i)``. |
| 783 |
|
| 784 |
Note that when the Unicode patterns ``[a-z]`` or ``[A-Z]`` are used in |
| 785 |
combination with the :const:`IGNORECASE` flag, they will match the 52 ASCII |
| 786 |
letters and 4 additional non-ASCII letters: 'İ' (U+0130, Latin capital |
| 787 |
letter I with dot above), 'ı' (U+0131, Latin small letter dotless i), |
| 788 |
'ſ' (U+017F, Latin small letter long s) and 'K' (U+212A, Kelvin sign). |
| 789 |
If the :py:const:`~re.ASCII` flag is used, only letters 'a' to 'z' |
| 790 |
and 'A' to 'Z' are matched. |
| 791 |
|
| 792 |
.. data:: L |
| 793 |
LOCALE |
| 794 |
|
| 795 |
Make ``\w``, ``\W``, ``\b``, ``\B`` and case-insensitive matching |
| 796 |
dependent on the current locale. |
| 797 |
This flag can be used only with bytes patterns. |
| 798 |
|
| 799 |
Corresponds to the inline flag ``(?L)``. |
| 800 |
|
| 801 |
.. warning:: |
| 802 |
|
| 803 |
This flag is discouraged; consider Unicode matching instead. |
| 804 |
The locale mechanism is very unreliable |
| 805 |
as it only handles one "culture" at a time |
| 806 |
and only works with 8-bit locales. |
| 807 |
Unicode matching is enabled by default for Unicode (str) patterns |
| 808 |
and it is able to handle different locales and languages. |
| 809 |
|
| 810 |
.. versionchanged:: 3.6 |
| 811 |
:py:const:`~re.LOCALE` can be used only with bytes patterns |
| 812 |
and is not compatible with :py:const:`~re.ASCII`. |
| 813 |
|
| 814 |
.. versionchanged:: 3.7 |
| 815 |
Compiled regular expression objects with the :py:const:`~re.LOCALE` flag |
| 816 |
no longer depend on the locale at compile time. |
| 817 |
Only the locale at matching time affects the result of matching. |
| 818 |
|
| 819 |
|
| 820 |
.. data:: M |
| 821 |
MULTILINE |
| 822 |
|
| 823 |
When specified, the pattern character ``'^'`` matches at the beginning of the |
| 824 |
string and at the beginning of each line (immediately following each newline); |
| 825 |
and the pattern character ``'$'`` matches at the end of the string and at the |
| 826 |
end of each line (immediately preceding each newline). By default, ``'^'`` |
| 827 |
matches only at the beginning of the string, and ``'$'`` only at the end of the |
| 828 |
string and immediately before the newline (if any) at the end of the string. |
| 829 |
|
| 830 |
Corresponds to the inline flag ``(?m)``. |
| 831 |
|
| 832 |
.. data:: NOFLAG |
| 833 |
|
| 834 |
Indicates no flag being applied, the value is ``0``. This flag may be used |
| 835 |
as a default value for a function keyword argument or as a base value that |
| 836 |
will be conditionally ORed with other flags. Example of use as a default |
| 837 |
value:: |
| 838 |
|
| 839 |
def myfunc(text, flag=re.NOFLAG): |
| 840 |
return re.match(text, flag) |
| 841 |
|
| 842 |
.. versionadded:: 3.11 |
| 843 |
|
| 844 |
.. data:: S |
| 845 |
DOTALL |
| 846 |
|
| 847 |
Make the ``'.'`` special character match any character at all, including a |
| 848 |
newline; without this flag, ``'.'`` will match anything *except* a newline. |
| 849 |
|
| 850 |
Corresponds to the inline flag ``(?s)``. |
| 851 |
|
| 852 |
|
| 853 |
.. data:: U |
| 854 |
UNICODE |
| 855 |
|
| 856 |
In Python 3, Unicode characters are matched by default |
| 857 |
for ``str`` patterns. |
| 858 |
This flag is therefore redundant with **no effect** |
| 859 |
and is only kept for backward compatibility. |
| 860 |
|
| 861 |
See :py:const:`~re.ASCII` to restrict matching to ASCII characters instead. |
| 862 |
|
| 863 |
.. data:: X |
| 864 |
VERBOSE |
| 865 |
|
| 866 |
.. index:: single: # (hash); in regular expressions |
| 867 |
|
| 868 |
This flag allows you to write regular expressions that look nicer and are |
| 869 |
more readable by allowing you to visually separate logical sections of the |
| 870 |
pattern and add comments. Whitespace within the pattern is ignored, except |
| 871 |
when in a character class, or when preceded by an unescaped backslash, |
| 872 |
or within tokens like ``*?``, ``(?:`` or ``(?P<...>``. For example, ``(? :`` |
| 873 |
and ``* ?`` are not allowed. |
| 874 |
When a line contains a ``#`` that is not in a character class and is not |
| 875 |
preceded by an unescaped backslash, all characters from the leftmost such |
| 876 |
``#`` through the end of the line are ignored. |
| 877 |
|
| 878 |
This means that the two following regular expression objects that match a |
| 879 |
decimal number are functionally equal:: |
| 880 |
|
| 881 |
a = re.compile(r"""\d + # the integral part |
| 882 |
\. # the decimal point |
| 883 |
\d * # some fractional digits""", re.X) |
| 884 |
b = re.compile(r"\d+\.\d*") |
| 885 |
|
| 886 |
Corresponds to the inline flag ``(?x)``. |
| 887 |
|
| 888 |
|
| 889 |
Functions |
| 890 |
^^^^^^^^^ |
| 891 |
|
| 892 |
.. function:: compile(pattern, flags=0) |
| 893 |
|
| 894 |
Compile a regular expression pattern into a :ref:`regular expression object |
| 895 |
<re-objects>`, which can be used for matching using its |
| 896 |
:func:`~Pattern.match`, :func:`~Pattern.search` and other methods, described |
| 897 |
below. |
| 898 |
|
| 899 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 900 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 901 |
(the ``|`` operator). |
| 902 |
|
| 903 |
The sequence :: |
| 904 |
|
| 905 |
prog = re.compile(pattern) |
| 906 |
result = prog.match(string) |
| 907 |
|
| 908 |
is equivalent to :: |
| 909 |
|
| 910 |
result = re.match(pattern, string) |
| 911 |
|
| 912 |
but using :func:`re.compile` and saving the resulting regular expression |
| 913 |
object for reuse is more efficient when the expression will be used several |
| 914 |
times in a single program. |
| 915 |
|
| 916 |
.. note:: |
| 917 |
|
| 918 |
The compiled versions of the most recent patterns passed to |
| 919 |
:func:`re.compile` and the module-level matching functions are cached, so |
| 920 |
programs that use only a few regular expressions at a time needn't worry |
| 921 |
about compiling regular expressions. |
| 922 |
|
| 923 |
|
| 924 |
.. function:: search(pattern, string, flags=0) |
| 925 |
|
| 926 |
Scan through *string* looking for the first location where the regular expression |
| 927 |
*pattern* produces a match, and return a corresponding :class:`~re.Match`. Return |
| 928 |
``None`` if no position in the string matches the pattern; note that this is |
| 929 |
different from finding a zero-length match at some point in the string. |
| 930 |
|
| 931 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 932 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 933 |
(the ``|`` operator). |
| 934 |
|
| 935 |
|
| 936 |
.. function:: match(pattern, string, flags=0) |
| 937 |
|
| 938 |
If zero or more characters at the beginning of *string* match the regular |
| 939 |
expression *pattern*, return a corresponding :class:`~re.Match`. Return |
| 940 |
``None`` if the string does not match the pattern; note that this is |
| 941 |
different from a zero-length match. |
| 942 |
|
| 943 |
Note that even in :const:`MULTILINE` mode, :func:`re.match` will only match |
| 944 |
at the beginning of the string and not at the beginning of each line. |
| 945 |
|
| 946 |
If you want to locate a match anywhere in *string*, use :func:`search` |
| 947 |
instead (see also :ref:`search-vs-match`). |
| 948 |
|
| 949 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 950 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 951 |
(the ``|`` operator). |
| 952 |
|
| 953 |
|
| 954 |
.. function:: fullmatch(pattern, string, flags=0) |
| 955 |
|
| 956 |
If the whole *string* matches the regular expression *pattern*, return a |
| 957 |
corresponding :class:`~re.Match`. Return ``None`` if the string does not match |
| 958 |
the pattern; note that this is different from a zero-length match. |
| 959 |
|
| 960 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 961 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 962 |
(the ``|`` operator). |
| 963 |
|
| 964 |
.. versionadded:: 3.4 |
| 965 |
|
| 966 |
|
| 967 |
.. function:: split(pattern, string, maxsplit=0, flags=0) |
| 968 |
|
| 969 |
Split *string* by the occurrences of *pattern*. If capturing parentheses are |
| 970 |
used in *pattern*, then the text of all groups in the pattern are also returned |
| 971 |
as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit* |
| 972 |
splits occur, and the remainder of the string is returned as the final element |
| 973 |
of the list. :: |
| 974 |
|
| 975 |
>>> re.split(r'\W+', 'Words, words, words.') |
| 976 |
['Words', 'words', 'words', ''] |
| 977 |
>>> re.split(r'(\W+)', 'Words, words, words.') |
| 978 |
['Words', ', ', 'words', ', ', 'words', '.', ''] |
| 979 |
>>> re.split(r'\W+', 'Words, words, words.', maxsplit=1) |
| 980 |
['Words', 'words, words.'] |
| 981 |
>>> re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE) |
| 982 |
['0', '3', '9'] |
| 983 |
|
| 984 |
If there are capturing groups in the separator and it matches at the start of |
| 985 |
the string, the result will start with an empty string. The same holds for |
| 986 |
the end of the string:: |
| 987 |
|
| 988 |
>>> re.split(r'(\W+)', '...words, words...') |
| 989 |
['', '...', 'words', ', ', 'words', '...', ''] |
| 990 |
|
| 991 |
That way, separator components are always found at the same relative |
| 992 |
indices within the result list. |
| 993 |
|
| 994 |
Adjacent empty matches are not possible, but an empty match can occur |
| 995 |
immediately after a non-empty match. |
| 996 |
|
| 997 |
.. code:: pycon |
| 998 |
|
| 999 |
>>> re.split(r'\b', 'Words, words, words.') |
| 1000 |
['', 'Words', ', ', 'words', ', ', 'words', '.'] |
| 1001 |
>>> re.split(r'\W*', '...words...') |
| 1002 |
['', '', 'w', 'o', 'r', 'd', 's', '', ''] |
| 1003 |
>>> re.split(r'(\W*)', '...words...') |
| 1004 |
['', '...', '', '', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '', '', ''] |
| 1005 |
|
| 1006 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 1007 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 1008 |
(the ``|`` operator). |
| 1009 |
|
| 1010 |
.. versionchanged:: 3.1 |
| 1011 |
Added the optional flags argument. |
| 1012 |
|
| 1013 |
.. versionchanged:: 3.7 |
| 1014 |
Added support of splitting on a pattern that could match an empty string. |
| 1015 |
|
| 1016 |
.. deprecated:: 3.13 |
| 1017 |
Passing *maxsplit* and *flags* as positional arguments is deprecated. |
| 1018 |
In future Python versions they will be |
| 1019 |
:ref:`keyword-only parameters <keyword-only_parameter>`. |
| 1020 |
|
| 1021 |
|
| 1022 |
.. function:: findall(pattern, string, flags=0) |
| 1023 |
|
| 1024 |
Return all non-overlapping matches of *pattern* in *string*, as a list of |
| 1025 |
strings or tuples. The *string* is scanned left-to-right, and matches |
| 1026 |
are returned in the order found. Empty matches are included in the result. |
| 1027 |
|
| 1028 |
The result depends on the number of capturing groups in the pattern. |
| 1029 |
If there are no groups, return a list of strings matching the whole |
| 1030 |
pattern. If there is exactly one group, return a list of strings |
| 1031 |
matching that group. If multiple groups are present, return a list |
| 1032 |
of tuples of strings matching the groups. Non-capturing groups do not |
| 1033 |
affect the form of the result. |
| 1034 |
|
| 1035 |
>>> re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest') |
| 1036 |
['foot', 'fell', 'fastest'] |
| 1037 |
>>> re.findall(r'(\w+)=(\d+)', 'set width=20 and height=10') |
| 1038 |
[('width', '20'), ('height', '10')] |
| 1039 |
|
| 1040 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 1041 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 1042 |
(the ``|`` operator). |
| 1043 |
|
| 1044 |
.. versionchanged:: 3.7 |
| 1045 |
Non-empty matches can now start just after a previous empty match. |
| 1046 |
|
| 1047 |
|
| 1048 |
.. function:: finditer(pattern, string, flags=0) |
| 1049 |
|
| 1050 |
Return an :term:`iterator` yielding :class:`~re.Match` objects over |
| 1051 |
all non-overlapping matches for the RE *pattern* in *string*. The *string* |
| 1052 |
is scanned left-to-right, and matches are returned in the order found. Empty |
| 1053 |
matches are included in the result. |
| 1054 |
|
| 1055 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 1056 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 1057 |
(the ``|`` operator). |
| 1058 |
|
| 1059 |
.. versionchanged:: 3.7 |
| 1060 |
Non-empty matches can now start just after a previous empty match. |
| 1061 |
|
| 1062 |
|
| 1063 |
.. function:: sub(pattern, repl, string, count=0, flags=0) |
| 1064 |
|
| 1065 |
Return the string obtained by replacing the leftmost non-overlapping occurrences |
| 1066 |
of *pattern* in *string* by the replacement *repl*. If the pattern isn't found, |
| 1067 |
*string* is returned unchanged. *repl* can be a string or a function; if it is |
| 1068 |
a string, any backslash escapes in it are processed. That is, ``\n`` is |
| 1069 |
converted to a single newline character, ``\r`` is converted to a carriage return, and |
| 1070 |
so forth. Unknown escapes of ASCII letters are reserved for future use and |
| 1071 |
treated as errors. Other unknown escapes such as ``\&`` are left alone. |
| 1072 |
Backreferences, such |
| 1073 |
as ``\6``, are replaced with the substring matched by group 6 in the pattern. |
| 1074 |
For example:: |
| 1075 |
|
| 1076 |
>>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):', |
| 1077 |
... r'static PyObject*\npy_\1(void)\n{', |
| 1078 |
... 'def myfunc():') |
| 1079 |
'static PyObject*\npy_myfunc(void)\n{' |
| 1080 |
|
| 1081 |
If *repl* is a function, it is called for every non-overlapping occurrence of |
| 1082 |
*pattern*. The function takes a single :class:`~re.Match` argument, and returns |
| 1083 |
the replacement string. For example:: |
| 1084 |
|
| 1085 |
>>> def dashrepl(matchobj): |
| 1086 |
... if matchobj.group(0) == '-': return ' ' |
| 1087 |
... else: return '-' |
| 1088 |
... |
| 1089 |
>>> re.sub('-{1,2}', dashrepl, 'pro----gram-files') |
| 1090 |
'pro--gram files' |
| 1091 |
>>> re.sub(r'\sAND\s', ' & ', 'Baked Beans And Spam', flags=re.IGNORECASE) |
| 1092 |
'Baked Beans & Spam' |
| 1093 |
|
| 1094 |
The pattern may be a string or a :class:`~re.Pattern`. |
| 1095 |
|
| 1096 |
The optional argument *count* is the maximum number of pattern occurrences to be |
| 1097 |
replaced; *count* must be a non-negative integer. If omitted or zero, all |
| 1098 |
occurrences will be replaced. |
| 1099 |
|
| 1100 |
Adjacent empty matches are not possible, but an empty match can occur |
| 1101 |
immediately after a non-empty match. |
| 1102 |
As a result, ``sub('x*', '-', 'abxd')`` returns ``'-a-b--d-'`` |
| 1103 |
instead of ``'-a-b-d-'``. |
| 1104 |
|
| 1105 |
.. index:: single: \g; in regular expressions |
| 1106 |
|
| 1107 |
In string-type *repl* arguments, in addition to the character escapes and |
| 1108 |
backreferences described above, |
| 1109 |
``\g<name>`` will use the substring matched by the group named ``name``, as |
| 1110 |
defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding |
| 1111 |
group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous |
| 1112 |
in a replacement such as ``\g<2>0``. ``\20`` would be interpreted as a |
| 1113 |
reference to group 20, not a reference to group 2 followed by the literal |
| 1114 |
character ``'0'``. The backreference ``\g<0>`` substitutes in the entire |
| 1115 |
substring matched by the RE. |
| 1116 |
|
| 1117 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 1118 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 1119 |
(the ``|`` operator). |
| 1120 |
|
| 1121 |
.. versionchanged:: 3.1 |
| 1122 |
Added the optional flags argument. |
| 1123 |
|
| 1124 |
.. versionchanged:: 3.5 |
| 1125 |
Unmatched groups are replaced with an empty string. |
| 1126 |
|
| 1127 |
.. versionchanged:: 3.6 |
| 1128 |
Unknown escapes in *pattern* consisting of ``'\'`` and an ASCII letter |
| 1129 |
now are errors. |
| 1130 |
|
| 1131 |
.. versionchanged:: 3.7 |
| 1132 |
Unknown escapes in *repl* consisting of ``'\'`` and an ASCII letter |
| 1133 |
now are errors. |
| 1134 |
An empty match can occur immediately after a non-empty match. |
| 1135 |
|
| 1136 |
.. versionchanged:: 3.12 |
| 1137 |
Group *id* can only contain ASCII digits. |
| 1138 |
In :class:`bytes` replacement strings, group *name* can only contain bytes |
| 1139 |
in the ASCII range (``b'\x00'``-``b'\x7f'``). |
| 1140 |
|
| 1141 |
.. deprecated:: 3.13 |
| 1142 |
Passing *count* and *flags* as positional arguments is deprecated. |
| 1143 |
In future Python versions they will be |
| 1144 |
:ref:`keyword-only parameters <keyword-only_parameter>`. |
| 1145 |
|
| 1146 |
|
| 1147 |
.. function:: subn(pattern, repl, string, count=0, flags=0) |
| 1148 |
|
| 1149 |
Perform the same operation as :func:`sub`, but return a tuple ``(new_string, |
| 1150 |
number_of_subs_made)``. |
| 1151 |
|
| 1152 |
The expression's behaviour can be modified by specifying a *flags* value. |
| 1153 |
Values can be any of the `flags`_ variables, combined using bitwise OR |
| 1154 |
(the ``|`` operator). |
| 1155 |
|
| 1156 |
|
| 1157 |
.. function:: escape(pattern) |
| 1158 |
|
| 1159 |
Escape special characters in *pattern*. |
| 1160 |
This is useful if you want to match an arbitrary literal string that may |
| 1161 |
have regular expression metacharacters in it. For example:: |
| 1162 |
|
| 1163 |
>>> print(re.escape('https://www.python.org')) |
| 1164 |
https://www\.python\.org |
| 1165 |
|
| 1166 |
>>> legal_chars = string.ascii_lowercase + string.digits + "!#$%&'*+-.^_`|~:" |
| 1167 |
>>> print('[%s]+' % re.escape(legal_chars)) |
| 1168 |
[abcdefghijklmnopqrstuvwxyz0123456789!\#\$%\&'\*\+\-\.\^_`\|\~:]+ |
| 1169 |
|
| 1170 |
>>> operators = ['+', '-', '*', '/', '**'] |
| 1171 |
>>> print('|'.join(map(re.escape, sorted(operators, reverse=True)))) |
| 1172 |
/|\-|\+|\*\*|\* |
| 1173 |
|
| 1174 |
This function must not be used for the replacement string in :func:`sub` |
| 1175 |
and :func:`subn`, only backslashes should be escaped. For example:: |
| 1176 |
|
| 1177 |
>>> digits_re = r'\d+' |
| 1178 |
>>> sample = '/usr/sbin/sendmail - 0 errors, 12 warnings' |
| 1179 |
>>> print(re.sub(digits_re, digits_re.replace('\\', r'\\'), sample)) |
| 1180 |
/usr/sbin/sendmail - \d+ errors, \d+ warnings |
| 1181 |
|
| 1182 |
.. versionchanged:: 3.3 |
| 1183 |
The ``'_'`` character is no longer escaped. |
| 1184 |
|
| 1185 |
.. versionchanged:: 3.7 |
| 1186 |
Only characters that can have special meaning in a regular expression |
| 1187 |
are escaped. As a result, ``'!'``, ``'"'``, ``'%'``, ``"'"``, ``','``, |
| 1188 |
``'/'``, ``':'``, ``';'``, ``'<'``, ``'='``, ``'>'``, ``'@'``, and |
| 1189 |
``"`"`` are no longer escaped. |
| 1190 |
|
| 1191 |
|
| 1192 |
.. function:: purge() |
| 1193 |
|
| 1194 |
Clear the regular expression cache. |
| 1195 |
|
| 1196 |
|
| 1197 |
Exceptions |
| 1198 |
^^^^^^^^^^ |
| 1199 |
|
| 1200 |
.. exception:: PatternError(msg, pattern=None, pos=None) |
| 1201 |
|
| 1202 |
Exception raised when a string passed to one of the functions here is not a |
| 1203 |
valid regular expression (for example, it might contain unmatched parentheses) |
| 1204 |
or when some other error occurs during compilation or matching. It is never an |
| 1205 |
error if a string contains no match for a pattern. The ``PatternError`` instance has |
| 1206 |
the following additional attributes: |
| 1207 |
|
| 1208 |
.. attribute:: msg |
| 1209 |
|
| 1210 |
The unformatted error message. |
| 1211 |
|
| 1212 |
.. attribute:: pattern |
| 1213 |
|
| 1214 |
The regular expression pattern. |
| 1215 |
|
| 1216 |
.. attribute:: pos |
| 1217 |
|
| 1218 |
The index in *pattern* where compilation failed (may be ``None``). |
| 1219 |
|
| 1220 |
.. attribute:: lineno |
| 1221 |
|
| 1222 |
The line corresponding to *pos* (may be ``None``). |
| 1223 |
|
| 1224 |
.. attribute:: colno |
| 1225 |
|
| 1226 |
The column corresponding to *pos* (may be ``None``). |
| 1227 |
|
| 1228 |
.. versionchanged:: 3.5 |
| 1229 |
Added additional attributes. |
| 1230 |
|
| 1231 |
.. versionchanged:: 3.13 |
| 1232 |
``PatternError`` was originally named ``error``; the latter is kept as an alias for |
| 1233 |
backward compatibility. |
| 1234 |
|
| 1235 |
.. _re-objects: |
| 1236 |
|
| 1237 |
Regular Expression Objects |
| 1238 |
-------------------------- |
| 1239 |
|
| 1240 |
.. class:: Pattern |
| 1241 |
|
| 1242 |
Compiled regular expression object returned by :func:`re.compile`. |
| 1243 |
|
| 1244 |
.. versionchanged:: 3.9 |
| 1245 |
:py:class:`re.Pattern` supports ``[]`` to indicate a Unicode (str) or bytes pattern. |
| 1246 |
See :ref:`types-genericalias`. |
| 1247 |
|
| 1248 |
.. method:: Pattern.search(string[, pos[, endpos]]) |
| 1249 |
|
| 1250 |
Scan through *string* looking for the first location where this regular |
| 1251 |
expression produces a match, and return a corresponding :class:`~re.Match`. |
| 1252 |
Return ``None`` if no position in the string matches the pattern; note that |
| 1253 |
this is different from finding a zero-length match at some point in the string. |
| 1254 |
|
| 1255 |
The optional second parameter *pos* gives an index in the string where the |
| 1256 |
search is to start; it defaults to ``0``. This is not completely equivalent to |
| 1257 |
slicing the string; the ``'^'`` pattern character matches at the real beginning |
| 1258 |
of the string and at positions just after a newline, but not necessarily at the |
| 1259 |
index where the search is to start. |
| 1260 |
|
| 1261 |
The optional parameter *endpos* limits how far the string will be searched; it |
| 1262 |
will be as if the string is *endpos* characters long, so only the characters |
| 1263 |
from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less |
| 1264 |
than *pos*, no match will be found; otherwise, if *rx* is a compiled regular |
| 1265 |
expression object, ``rx.search(string, 0, 50)`` is equivalent to |
| 1266 |
``rx.search(string[:50], 0)``. :: |
| 1267 |
|
| 1268 |
>>> pattern = re.compile("d") |
| 1269 |
>>> pattern.search("dog") # Match at index 0 |
| 1270 |
<re.Match object; span=(0, 1), match='d'> |
| 1271 |
>>> pattern.search("dog", 1) # No match; search doesn't include the "d" |
| 1272 |
|
| 1273 |
|
| 1274 |
.. method:: Pattern.match(string[, pos[, endpos]]) |
| 1275 |
|
| 1276 |
If zero or more characters at the *beginning* of *string* match this regular |
| 1277 |
expression, return a corresponding :class:`~re.Match`. Return ``None`` if the |
| 1278 |
string does not match the pattern; note that this is different from a |
| 1279 |
zero-length match. |
| 1280 |
|
| 1281 |
The optional *pos* and *endpos* parameters have the same meaning as for the |
| 1282 |
:meth:`~Pattern.search` method. :: |
| 1283 |
|
| 1284 |
>>> pattern = re.compile("o") |
| 1285 |
>>> pattern.match("dog") # No match as "o" is not at the start of "dog". |
| 1286 |
>>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog". |
| 1287 |
<re.Match object; span=(1, 2), match='o'> |
| 1288 |
|
| 1289 |
If you want to locate a match anywhere in *string*, use |
| 1290 |
:meth:`~Pattern.search` instead (see also :ref:`search-vs-match`). |
| 1291 |
|
| 1292 |
|
| 1293 |
.. method:: Pattern.fullmatch(string[, pos[, endpos]]) |
| 1294 |
|
| 1295 |
If the whole *string* matches this regular expression, return a corresponding |
| 1296 |
:class:`~re.Match`. Return ``None`` if the string does not match the pattern; |
| 1297 |
note that this is different from a zero-length match. |
| 1298 |
|
| 1299 |
The optional *pos* and *endpos* parameters have the same meaning as for the |
| 1300 |
:meth:`~Pattern.search` method. :: |
| 1301 |
|
| 1302 |
>>> pattern = re.compile("o[gh]") |
| 1303 |
>>> pattern.fullmatch("dog") # No match as "o" is not at the start of "dog". |
| 1304 |
>>> pattern.fullmatch("ogre") # No match as not the full string matches. |
| 1305 |
>>> pattern.fullmatch("doggie", 1, 3) # Matches within given limits. |
| 1306 |
<re.Match object; span=(1, 3), match='og'> |
| 1307 |
|
| 1308 |
.. versionadded:: 3.4 |
| 1309 |
|
| 1310 |
|
| 1311 |
.. method:: Pattern.split(string, maxsplit=0) |
| 1312 |
|
| 1313 |
Identical to the :func:`split` function, using the compiled pattern. |
| 1314 |
|
| 1315 |
|
| 1316 |
.. method:: Pattern.findall(string[, pos[, endpos]]) |
| 1317 |
|
| 1318 |
Similar to the :func:`findall` function, using the compiled pattern, but |
| 1319 |
also accepts optional *pos* and *endpos* parameters that limit the search |
| 1320 |
region like for :meth:`search`. |
| 1321 |
|
| 1322 |
|
| 1323 |
.. method:: Pattern.finditer(string[, pos[, endpos]]) |
| 1324 |
|
| 1325 |
Similar to the :func:`finditer` function, using the compiled pattern, but |
| 1326 |
also accepts optional *pos* and *endpos* parameters that limit the search |
| 1327 |
region like for :meth:`search`. |
| 1328 |
|
| 1329 |
|
| 1330 |
.. method:: Pattern.sub(repl, string, count=0) |
| 1331 |
|
| 1332 |
Identical to the :func:`sub` function, using the compiled pattern. |
| 1333 |
|
| 1334 |
|
| 1335 |
.. method:: Pattern.subn(repl, string, count=0) |
| 1336 |
|
| 1337 |
Identical to the :func:`subn` function, using the compiled pattern. |
| 1338 |
|
| 1339 |
|
| 1340 |
.. attribute:: Pattern.flags |
| 1341 |
|
| 1342 |
The regex matching flags. This is a combination of the flags given to |
| 1343 |
:func:`.compile`, any ``(?...)`` inline flags in the pattern, and implicit |
| 1344 |
flags such as :py:const:`~re.UNICODE` if the pattern is a Unicode string. |
| 1345 |
|
| 1346 |
|
| 1347 |
.. attribute:: Pattern.groups |
| 1348 |
|
| 1349 |
The number of capturing groups in the pattern. |
| 1350 |
|
| 1351 |
|
| 1352 |
.. attribute:: Pattern.groupindex |
| 1353 |
|
| 1354 |
A dictionary mapping any symbolic group names defined by ``(?P<id>)`` to group |
| 1355 |
numbers. The dictionary is empty if no symbolic groups were used in the |
| 1356 |
pattern. |
| 1357 |
|
| 1358 |
|
| 1359 |
.. attribute:: Pattern.pattern |
| 1360 |
|
| 1361 |
The pattern string from which the pattern object was compiled. |
| 1362 |
|
| 1363 |
|
| 1364 |
.. versionchanged:: 3.7 |
| 1365 |
Added support of :func:`copy.copy` and :func:`copy.deepcopy`. Compiled |
| 1366 |
regular expression objects are considered atomic. |
| 1367 |
|
| 1368 |
|
| 1369 |
.. _match-objects: |
| 1370 |
|
| 1371 |
Match Objects |
| 1372 |
------------- |
| 1373 |
|
| 1374 |
Match objects always have a boolean value of ``True``. |
| 1375 |
Since :meth:`~Pattern.match` and :meth:`~Pattern.search` return ``None`` |
| 1376 |
when there is no match, you can test whether there was a match with a simple |
| 1377 |
``if`` statement:: |
| 1378 |
|
| 1379 |
match = re.search(pattern, string) |
| 1380 |
if match: |
| 1381 |
process(match) |
| 1382 |
|
| 1383 |
.. class:: Match |
| 1384 |
|
| 1385 |
Match object returned by successful ``match``\ es and ``search``\ es. |
| 1386 |
|
| 1387 |
.. versionchanged:: 3.9 |
| 1388 |
:py:class:`re.Match` supports ``[]`` to indicate a Unicode (str) or bytes match. |
| 1389 |
See :ref:`types-genericalias`. |
| 1390 |
|
| 1391 |
.. method:: Match.expand(template) |
| 1392 |
|
| 1393 |
Return the string obtained by doing backslash substitution on the template |
| 1394 |
string *template*, as done by the :meth:`~Pattern.sub` method. |
| 1395 |
Escapes such as ``\n`` are converted to the appropriate characters, |
| 1396 |
and numeric backreferences (``\1``, ``\2``) and named backreferences |
| 1397 |
(``\g<1>``, ``\g<name>``) are replaced by the contents of the |
| 1398 |
corresponding group. The backreference ``\g<0>`` will be |
| 1399 |
replaced by the entire match. |
| 1400 |
|
| 1401 |
.. versionchanged:: 3.5 |
| 1402 |
Unmatched groups are replaced with an empty string. |
| 1403 |
|
| 1404 |
.. method:: Match.group([group1, ...]) |
| 1405 |
|
| 1406 |
Returns one or more subgroups of the match. If there is a single argument, the |
| 1407 |
result is a single string; if there are multiple arguments, the result is a |
| 1408 |
tuple with one item per argument. Without arguments, *group1* defaults to zero |
| 1409 |
(the whole match is returned). If a *groupN* argument is zero, the corresponding |
| 1410 |
return value is the entire matching string; if it is a positive integer, it is |
| 1411 |
the string matching the corresponding parenthesized group. If a group number is |
| 1412 |
negative or larger than the number of groups defined in the pattern, an |
| 1413 |
:exc:`IndexError` exception is raised. If a group is contained in a |
| 1414 |
part of the pattern that did not match, the corresponding result is ``None``. |
| 1415 |
If a group is contained in a part of the pattern that matched multiple times, |
| 1416 |
the last match is returned. :: |
| 1417 |
|
| 1418 |
>>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist") |
| 1419 |
>>> m.group(0) # The entire match |
| 1420 |
'Isaac Newton' |
| 1421 |
>>> m.group(1) # The first parenthesized subgroup. |
| 1422 |
'Isaac' |
| 1423 |
>>> m.group(2) # The second parenthesized subgroup. |
| 1424 |
'Newton' |
| 1425 |
>>> m.group(1, 2) # Multiple arguments give us a tuple. |
| 1426 |
('Isaac', 'Newton') |
| 1427 |
|
| 1428 |
If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN* |
| 1429 |
arguments may also be strings identifying groups by their group name. If a |
| 1430 |
string argument is not used as a group name in the pattern, an :exc:`IndexError` |
| 1431 |
exception is raised. |
| 1432 |
|
| 1433 |
A moderately complicated example:: |
| 1434 |
|
| 1435 |
>>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcolm Reynolds") |
| 1436 |
>>> m.group('first_name') |
| 1437 |
'Malcolm' |
| 1438 |
>>> m.group('last_name') |
| 1439 |
'Reynolds' |
| 1440 |
|
| 1441 |
Named groups can also be referred to by their index:: |
| 1442 |
|
| 1443 |
>>> m.group(1) |
| 1444 |
'Malcolm' |
| 1445 |
>>> m.group(2) |
| 1446 |
'Reynolds' |
| 1447 |
|
| 1448 |
If a group matches multiple times, only the last match is accessible:: |
| 1449 |
|
| 1450 |
>>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times. |
| 1451 |
>>> m.group(1) # Returns only the last match. |
| 1452 |
'c3' |
| 1453 |
|
| 1454 |
|
| 1455 |
.. method:: Match.__getitem__(g) |
| 1456 |
|
| 1457 |
This is identical to ``m.group(g)``. This allows easier access to |
| 1458 |
an individual group from a match:: |
| 1459 |
|
| 1460 |
>>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist") |
| 1461 |
>>> m[0] # The entire match |
| 1462 |
'Isaac Newton' |
| 1463 |
>>> m[1] # The first parenthesized subgroup. |
| 1464 |
'Isaac' |
| 1465 |
>>> m[2] # The second parenthesized subgroup. |
| 1466 |
'Newton' |
| 1467 |
|
| 1468 |
Named groups are supported as well:: |
| 1469 |
|
| 1470 |
>>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Isaac Newton") |
| 1471 |
>>> m['first_name'] |
| 1472 |
'Isaac' |
| 1473 |
>>> m['last_name'] |
| 1474 |
'Newton' |
| 1475 |
|
| 1476 |
.. versionadded:: 3.6 |
| 1477 |
|
| 1478 |
|
| 1479 |
.. method:: Match.groups(default=None) |
| 1480 |
|
| 1481 |
Return a tuple containing all the subgroups of the match, from 1 up to however |
| 1482 |
many groups are in the pattern. The *default* argument is used for groups that |
| 1483 |
did not participate in the match; it defaults to ``None``. |
| 1484 |
|
| 1485 |
For example:: |
| 1486 |
|
| 1487 |
>>> m = re.match(r"(\d+)\.(\d+)", "24.1632") |
| 1488 |
>>> m.groups() |
| 1489 |
('24', '1632') |
| 1490 |
|
| 1491 |
If we make the decimal place and everything after it optional, not all groups |
| 1492 |
might participate in the match. These groups will default to ``None`` unless |
| 1493 |
the *default* argument is given:: |
| 1494 |
|
| 1495 |
>>> m = re.match(r"(\d+)\.?(\d+)?", "24") |
| 1496 |
>>> m.groups() # Second group defaults to None. |
| 1497 |
('24', None) |
| 1498 |
>>> m.groups('0') # Now, the second group defaults to '0'. |
| 1499 |
('24', '0') |
| 1500 |
|
| 1501 |
|
| 1502 |
.. method:: Match.groupdict(default=None) |
| 1503 |
|
| 1504 |
Return a dictionary containing all the *named* subgroups of the match, keyed by |
| 1505 |
the subgroup name. The *default* argument is used for groups that did not |
| 1506 |
participate in the match; it defaults to ``None``. For example:: |
| 1507 |
|
| 1508 |
>>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcolm Reynolds") |
| 1509 |
>>> m.groupdict() |
| 1510 |
{'first_name': 'Malcolm', 'last_name': 'Reynolds'} |
| 1511 |
|
| 1512 |
|
| 1513 |
.. method:: Match.start([group]) |
| 1514 |
Match.end([group]) |
| 1515 |
|
| 1516 |
Return the indices of the start and end of the substring matched by *group*; |
| 1517 |
*group* defaults to zero (meaning the whole matched substring). Return ``-1`` if |
| 1518 |
*group* exists but did not contribute to the match. For a match object *m*, and |
| 1519 |
a group *g* that did contribute to the match, the substring matched by group *g* |
| 1520 |
(equivalent to ``m.group(g)``) is :: |
| 1521 |
|
| 1522 |
m.string[m.start(g):m.end(g)] |
| 1523 |
|
| 1524 |
Note that ``m.start(group)`` will equal ``m.end(group)`` if *group* matched a |
| 1525 |
null string. For example, after ``m = re.search('b(c?)', 'cba')``, |
| 1526 |
``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both |
| 1527 |
2, and ``m.start(2)`` raises an :exc:`IndexError` exception. |
| 1528 |
|
| 1529 |
An example that will remove *remove_this* from email addresses:: |
| 1530 |
|
| 1531 |
>>> email = "tony@tiremove_thisger.net" |
| 1532 |
>>> m = re.search("remove_this", email) |
| 1533 |
>>> email[:m.start()] + email[m.end():] |
| 1534 |
'[email protected]' |
| 1535 |
|
| 1536 |
|
| 1537 |
.. method:: Match.span([group]) |
| 1538 |
|
| 1539 |
For a match *m*, return the 2-tuple ``(m.start(group), m.end(group))``. Note |
| 1540 |
that if *group* did not contribute to the match, this is ``(-1, -1)``. |
| 1541 |
*group* defaults to zero, the entire match. |
| 1542 |
|
| 1543 |
|
| 1544 |
.. attribute:: Match.pos |
| 1545 |
|
| 1546 |
The value of *pos* which was passed to the :meth:`~Pattern.search` or |
| 1547 |
:meth:`~Pattern.match` method of a :ref:`regex object <re-objects>`. This is |
| 1548 |
the index into the string at which the RE engine started looking for a match. |
| 1549 |
|
| 1550 |
|
| 1551 |
.. attribute:: Match.endpos |
| 1552 |
|
| 1553 |
The value of *endpos* which was passed to the :meth:`~Pattern.search` or |
| 1554 |
:meth:`~Pattern.match` method of a :ref:`regex object <re-objects>`. This is |
| 1555 |
the index into the string beyond which the RE engine will not go. |
| 1556 |
|
| 1557 |
|
| 1558 |
.. attribute:: Match.lastindex |
| 1559 |
|
| 1560 |
The integer index of the last matched capturing group, or ``None`` if no group |
| 1561 |
was matched at all. For example, the expressions ``(a)b``, ``((a)(b))``, and |
| 1562 |
``((ab))`` will have ``lastindex == 1`` if applied to the string ``'ab'``, while |
| 1563 |
the expression ``(a)(b)`` will have ``lastindex == 2``, if applied to the same |
| 1564 |
string. |
| 1565 |
|
| 1566 |
|
| 1567 |
.. attribute:: Match.lastgroup |
| 1568 |
|
| 1569 |
The name of the last matched capturing group, or ``None`` if the group didn't |
| 1570 |
have a name, or if no group was matched at all. |
| 1571 |
|
| 1572 |
|
| 1573 |
.. attribute:: Match.re |
| 1574 |
|
| 1575 |
The :ref:`regular expression object <re-objects>` whose :meth:`~Pattern.match` or |
| 1576 |
:meth:`~Pattern.search` method produced this match instance. |
| 1577 |
|
| 1578 |
|
| 1579 |
.. attribute:: Match.string |
| 1580 |
|
| 1581 |
The string passed to :meth:`~Pattern.match` or :meth:`~Pattern.search`. |
| 1582 |
|
| 1583 |
|
| 1584 |
.. versionchanged:: 3.7 |
| 1585 |
Added support of :func:`copy.copy` and :func:`copy.deepcopy`. Match objects |
| 1586 |
are considered atomic. |
| 1587 |
|
| 1588 |
|
| 1589 |
.. _re-examples: |
| 1590 |
|
| 1591 |
Regular Expression Examples |
| 1592 |
--------------------------- |
| 1593 |
|
| 1594 |
|
| 1595 |
Checking for a Pair |
| 1596 |
^^^^^^^^^^^^^^^^^^^ |
| 1597 |
|
| 1598 |
In this example, we'll use the following helper function to display match |
| 1599 |
objects a little more gracefully:: |
| 1600 |
|
| 1601 |
def displaymatch(match): |
| 1602 |
if match is None: |
| 1603 |
return None |
| 1604 |
return '<Match: %r, groups=%r>' % (match.group(), match.groups()) |
| 1605 |
|
| 1606 |
Suppose you are writing a poker program where a player's hand is represented as |
| 1607 |
a 5-character string with each character representing a card, "a" for ace, "k" |
| 1608 |
for king, "q" for queen, "j" for jack, "t" for 10, and "2" through "9" |
| 1609 |
representing the card with that value. |
| 1610 |
|
| 1611 |
To see if a given string is a valid hand, one could do the following:: |
| 1612 |
|
| 1613 |
>>> valid = re.compile(r"^[a2-9tjqk]{5}$") |
| 1614 |
>>> displaymatch(valid.match("akt5q")) # Valid. |
| 1615 |
"<Match: 'akt5q', groups=()>" |
| 1616 |
>>> displaymatch(valid.match("akt5e")) # Invalid. |
| 1617 |
>>> displaymatch(valid.match("akt")) # Invalid. |
| 1618 |
>>> displaymatch(valid.match("727ak")) # Valid. |
| 1619 |
"<Match: '727ak', groups=()>" |
| 1620 |
|
| 1621 |
That last hand, ``"727ak"``, contained a pair, or two of the same valued cards. |
| 1622 |
To match this with a regular expression, one could use backreferences as such:: |
| 1623 |
|
| 1624 |
>>> pair = re.compile(r".*(.).*\1") |
| 1625 |
>>> displaymatch(pair.match("717ak")) # Pair of 7s. |
| 1626 |
"<Match: '717', groups=('7',)>" |
| 1627 |
>>> displaymatch(pair.match("718ak")) # No pairs. |
| 1628 |
>>> displaymatch(pair.match("354aa")) # Pair of aces. |
| 1629 |
"<Match: '354aa', groups=('a',)>" |
| 1630 |
|
| 1631 |
To find out what card the pair consists of, one could use the |
| 1632 |
:meth:`~Match.group` method of the match object in the following manner:: |
| 1633 |
|
| 1634 |
>>> pair = re.compile(r".*(.).*\1") |
| 1635 |
>>> pair.match("717ak").group(1) |
| 1636 |
'7' |
| 1637 |
|
| 1638 |
# Error because re.match() returns None, which doesn't have a group() method: |
| 1639 |
>>> pair.match("718ak").group(1) |
| 1640 |
Traceback (most recent call last): |
| 1641 |
File "<pyshell#23>", line 1, in <module> |
| 1642 |
re.match(r".*(.).*\1", "718ak").group(1) |
| 1643 |
AttributeError: 'NoneType' object has no attribute 'group' |
| 1644 |
|
| 1645 |
>>> pair.match("354aa").group(1) |
| 1646 |
'a' |
| 1647 |
|
| 1648 |
|
| 1649 |
Simulating scanf() |
| 1650 |
^^^^^^^^^^^^^^^^^^ |
| 1651 |
|
| 1652 |
.. index:: single: scanf (C function) |
| 1653 |
|
| 1654 |
Python does not currently have an equivalent to :c:func:`!scanf`. Regular |
| 1655 |
expressions are generally more powerful, though also more verbose, than |
| 1656 |
:c:func:`!scanf` format strings. The table below offers some more-or-less |
| 1657 |
equivalent mappings between :c:func:`!scanf` format tokens and regular |
| 1658 |
expressions. |
| 1659 |
|
| 1660 |
+--------------------------------+---------------------------------------------+ |
| 1661 |
| :c:func:`!scanf` Token | Regular Expression | |
| 1662 |
+================================+=============================================+ |
| 1663 |
| ``%c`` | ``.`` | |
| 1664 |
+--------------------------------+---------------------------------------------+ |
| 1665 |
| ``%5c`` | ``.{5}`` | |
| 1666 |
+--------------------------------+---------------------------------------------+ |
| 1667 |
| ``%d`` | ``[-+]?\d+`` | |
| 1668 |
+--------------------------------+---------------------------------------------+ |
| 1669 |
| ``%e``, ``%E``, ``%f``, ``%g`` | ``[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?`` | |
| 1670 |
+--------------------------------+---------------------------------------------+ |
| 1671 |
| ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` | |
| 1672 |
+--------------------------------+---------------------------------------------+ |
| 1673 |
| ``%o`` | ``[-+]?[0-7]+`` | |
| 1674 |
+--------------------------------+---------------------------------------------+ |
| 1675 |
| ``%s`` | ``\S+`` | |
| 1676 |
+--------------------------------+---------------------------------------------+ |
| 1677 |
| ``%u`` | ``\d+`` | |
| 1678 |
+--------------------------------+---------------------------------------------+ |
| 1679 |
| ``%x``, ``%X`` | ``[-+]?(0[xX])?[\dA-Fa-f]+`` | |
| 1680 |
+--------------------------------+---------------------------------------------+ |
| 1681 |
|
| 1682 |
To extract the filename and numbers from a string like :: |
| 1683 |
|
| 1684 |
/usr/sbin/sendmail - 0 errors, 4 warnings |
| 1685 |
|
| 1686 |
you would use a :c:func:`!scanf` format like :: |
| 1687 |
|
| 1688 |
%s - %d errors, %d warnings |
| 1689 |
|
| 1690 |
The equivalent regular expression would be :: |
| 1691 |
|
| 1692 |
(\S+) - (\d+) errors, (\d+) warnings |
| 1693 |
|
| 1694 |
|
| 1695 |
.. _search-vs-match: |
| 1696 |
|
| 1697 |
search() vs. match() |
| 1698 |
^^^^^^^^^^^^^^^^^^^^ |
| 1699 |
|
| 1700 |
.. sectionauthor:: Fred L. Drake, Jr. <[email protected]> |
| 1701 |
|
| 1702 |
Python offers different primitive operations based on regular expressions: |
| 1703 |
|
| 1704 |
+ :func:`re.match` checks for a match only at the beginning of the string |
| 1705 |
+ :func:`re.search` checks for a match anywhere in the string |
| 1706 |
(this is what Perl does by default) |
| 1707 |
+ :func:`re.fullmatch` checks for entire string to be a match |
| 1708 |
|
| 1709 |
|
| 1710 |
For example:: |
| 1711 |
|
| 1712 |
>>> re.match("c", "abcdef") # No match |
| 1713 |
>>> re.search("c", "abcdef") # Match |
| 1714 |
<re.Match object; span=(2, 3), match='c'> |
| 1715 |
>>> re.fullmatch("p.*n", "python") # Match |
| 1716 |
<re.Match object; span=(0, 6), match='python'> |
| 1717 |
>>> re.fullmatch("r.*n", "python") # No match |
| 1718 |
|
| 1719 |
Regular expressions beginning with ``'^'`` can be used with :func:`search` to |
| 1720 |
restrict the match at the beginning of the string:: |
| 1721 |
|
| 1722 |
>>> re.match("c", "abcdef") # No match |
| 1723 |
>>> re.search("^c", "abcdef") # No match |
| 1724 |
>>> re.search("^a", "abcdef") # Match |
| 1725 |
<re.Match object; span=(0, 1), match='a'> |
| 1726 |
|
| 1727 |
Note however that in :const:`MULTILINE` mode :func:`match` only matches at the |
| 1728 |
beginning of the string, whereas using :func:`search` with a regular expression |
| 1729 |
beginning with ``'^'`` will match at the beginning of each line. :: |
| 1730 |
|
| 1731 |
>>> re.match("X", "A\nB\nX", re.MULTILINE) # No match |
| 1732 |
>>> re.search("^X", "A\nB\nX", re.MULTILINE) # Match |
| 1733 |
<re.Match object; span=(4, 5), match='X'> |
| 1734 |
|
| 1735 |
|
| 1736 |
Making a Phonebook |
| 1737 |
^^^^^^^^^^^^^^^^^^ |
| 1738 |
|
| 1739 |
:func:`split` splits a string into a list delimited by the passed pattern. The |
| 1740 |
method is invaluable for converting textual data into data structures that can be |
| 1741 |
easily read and modified by Python as demonstrated in the following example that |
| 1742 |
creates a phonebook. |
| 1743 |
|
| 1744 |
First, here is the input. Normally it may come from a file, here we are using |
| 1745 |
triple-quoted string syntax |
| 1746 |
|
| 1747 |
.. doctest:: |
| 1748 |
|
| 1749 |
>>> text = """Ross McFluff: 834.345.1254 155 Elm Street |
| 1750 |
... |
| 1751 |
... Ronald Heathmore: 892.345.3428 436 Finley Avenue |
| 1752 |
... Frank Burger: 925.541.7625 662 South Dogwood Way |
| 1753 |
... |
| 1754 |
... |
| 1755 |
... Heather Albrecht: 548.326.4584 919 Park Place""" |
| 1756 |
|
| 1757 |
The entries are separated by one or more newlines. Now we convert the string |
| 1758 |
into a list with each nonempty line having its own entry: |
| 1759 |
|
| 1760 |
.. doctest:: |
| 1761 |
:options: +NORMALIZE_WHITESPACE |
| 1762 |
|
| 1763 |
>>> entries = re.split("\n+", text) |
| 1764 |
>>> entries |
| 1765 |
['Ross McFluff: 834.345.1254 155 Elm Street', |
| 1766 |
'Ronald Heathmore: 892.345.3428 436 Finley Avenue', |
| 1767 |
'Frank Burger: 925.541.7625 662 South Dogwood Way', |
| 1768 |
'Heather Albrecht: 548.326.4584 919 Park Place'] |
| 1769 |
|
| 1770 |
Finally, split each entry into a list with first name, last name, telephone |
| 1771 |
number, and address. We use the ``maxsplit`` parameter of :func:`split` |
| 1772 |
because the address has spaces, our splitting pattern, in it: |
| 1773 |
|
| 1774 |
.. doctest:: |
| 1775 |
:options: +NORMALIZE_WHITESPACE |
| 1776 |
|
| 1777 |
>>> [re.split(":? ", entry, maxsplit=3) for entry in entries] |
| 1778 |
[['Ross', 'McFluff', '834.345.1254', '155 Elm Street'], |
| 1779 |
['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'], |
| 1780 |
['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'], |
| 1781 |
['Heather', 'Albrecht', '548.326.4584', '919 Park Place']] |
| 1782 |
|
| 1783 |
The ``:?`` pattern matches the colon after the last name, so that it does not |
| 1784 |
occur in the result list. With a ``maxsplit`` of ``4``, we could separate the |
| 1785 |
house number from the street name: |
| 1786 |
|
| 1787 |
.. doctest:: |
| 1788 |
:options: +NORMALIZE_WHITESPACE |
| 1789 |
|
| 1790 |
>>> [re.split(":? ", entry, maxsplit=4) for entry in entries] |
| 1791 |
[['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'], |
| 1792 |
['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'], |
| 1793 |
['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'], |
| 1794 |
['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place']] |
| 1795 |
|
| 1796 |
|
| 1797 |
Text Munging |
| 1798 |
^^^^^^^^^^^^ |
| 1799 |
|
| 1800 |
:func:`sub` replaces every occurrence of a pattern with a string or the |
| 1801 |
result of a function. This example demonstrates using :func:`sub` with |
| 1802 |
a function to "munge" text, or randomize the order of all the characters |
| 1803 |
in each word of a sentence except for the first and last characters:: |
| 1804 |
|
| 1805 |
>>> def repl(m): |
| 1806 |
... inner_word = list(m.group(2)) |
| 1807 |
... random.shuffle(inner_word) |
| 1808 |
... return m.group(1) + "".join(inner_word) + m.group(3) |
| 1809 |
... |
| 1810 |
>>> text = "Professor Abdolmalek, please report your absences promptly." |
| 1811 |
>>> re.sub(r"(\w)(\w+)(\w)", repl, text) |
| 1812 |
'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.' |
| 1813 |
>>> re.sub(r"(\w)(\w+)(\w)", repl, text) |
| 1814 |
'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.' |
| 1815 |
|
| 1816 |
|
| 1817 |
Finding all Adverbs |
| 1818 |
^^^^^^^^^^^^^^^^^^^ |
| 1819 |
|
| 1820 |
:func:`findall` matches *all* occurrences of a pattern, not just the first |
| 1821 |
one as :func:`search` does. For example, if a writer wanted to |
| 1822 |
find all of the adverbs in some text, they might use :func:`findall` in |
| 1823 |
the following manner:: |
| 1824 |
|
| 1825 |
>>> text = "He was carefully disguised but captured quickly by police." |
| 1826 |
>>> re.findall(r"\w+ly\b", text) |
| 1827 |
['carefully', 'quickly'] |
| 1828 |
|
| 1829 |
|
| 1830 |
Finding all Adverbs and their Positions |
| 1831 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 1832 |
|
| 1833 |
If one wants more information about all matches of a pattern than the matched |
| 1834 |
text, :func:`finditer` is useful as it provides :class:`~re.Match` objects |
| 1835 |
instead of strings. Continuing with the previous example, if a writer wanted |
| 1836 |
to find all of the adverbs *and their positions* in some text, they would use |
| 1837 |
:func:`finditer` in the following manner:: |
| 1838 |
|
| 1839 |
>>> text = "He was carefully disguised but captured quickly by police." |
| 1840 |
>>> for m in re.finditer(r"\w+ly\b", text): |
| 1841 |
... print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0))) |
| 1842 |
07-16: carefully |
| 1843 |
40-47: quickly |
| 1844 |
|
| 1845 |
|
| 1846 |
Raw String Notation |
| 1847 |
^^^^^^^^^^^^^^^^^^^ |
| 1848 |
|
| 1849 |
Raw string notation (``r"text"``) keeps regular expressions sane. Without it, |
| 1850 |
every backslash (``'\'``) in a regular expression would have to be prefixed with |
| 1851 |
another one to escape it. For example, the two following lines of code are |
| 1852 |
functionally identical:: |
| 1853 |
|
| 1854 |
>>> re.match(r"\W(.)\1\W", " ff ") |
| 1855 |
<re.Match object; span=(0, 4), match=' ff '> |
| 1856 |
>>> re.match("\\W(.)\\1\\W", " ff ") |
| 1857 |
<re.Match object; span=(0, 4), match=' ff '> |
| 1858 |
|
| 1859 |
When one wants to match a literal backslash, it must be escaped in the regular |
| 1860 |
expression. With raw string notation, this means ``r"\\"``. Without raw string |
| 1861 |
notation, one must use ``"\\\\"``, making the following lines of code |
| 1862 |
functionally identical:: |
| 1863 |
|
| 1864 |
>>> re.match(r"\\", r"\\") |
| 1865 |
<re.Match object; span=(0, 1), match='\\'> |
| 1866 |
>>> re.match("\\\\", r"\\") |
| 1867 |
<re.Match object; span=(0, 1), match='\\'> |
| 1868 |
|
| 1869 |
|
| 1870 |
Writing a Tokenizer |
| 1871 |
^^^^^^^^^^^^^^^^^^^ |
| 1872 |
|
| 1873 |
A `tokenizer or scanner <https://en.wikipedia.org/wiki/Lexical_analysis>`_ |
| 1874 |
analyzes a string to categorize groups of characters. This is a useful first |
| 1875 |
step in writing a compiler or interpreter. |
| 1876 |
|
| 1877 |
The text categories are specified with regular expressions. The technique is |
| 1878 |
to combine those into a single master regular expression and to loop over |
| 1879 |
successive matches:: |
| 1880 |
|
| 1881 |
from typing import NamedTuple |
| 1882 |
import re |
| 1883 |
|
| 1884 |
class Token(NamedTuple): |
| 1885 |
type: str |
| 1886 |
value: int | float | str |
| 1887 |
line: int |
| 1888 |
column: int |
| 1889 |
|
| 1890 |
def tokenize(code): |
| 1891 |
keywords = {'IF', 'THEN', 'ENDIF', 'FOR', 'NEXT', 'GOSUB', 'RETURN'} |
| 1892 |
token_specification = [ |
| 1893 |
('NUMBER', r'\d+(\.\d*)?'), # Integer or decimal number |
| 1894 |
('ASSIGN', r':='), # Assignment operator |
| 1895 |
('END', r';'), # Statement terminator |
| 1896 |
('ID', r'[A-Za-z]+'), # Identifiers |
| 1897 |
('OP', r'[+\-*/]'), # Arithmetic operators |
| 1898 |
('NEWLINE', r'\n'), # Line endings |
| 1899 |
('SKIP', r'[ \t]+'), # Skip over spaces and tabs |
| 1900 |
('MISMATCH', r'.'), # Any other character |
| 1901 |
] |
| 1902 |
tok_regex = '|'.join('(?P<%s>%s)' % pair for pair in token_specification) |
| 1903 |
line_num = 1 |
| 1904 |
line_start = 0 |
| 1905 |
for mo in re.finditer(tok_regex, code): |
| 1906 |
kind = mo.lastgroup |
| 1907 |
value = mo.group() |
| 1908 |
column = mo.start() - line_start |
| 1909 |
if kind == 'NUMBER': |
| 1910 |
value = float(value) if '.' in value else int(value) |
| 1911 |
elif kind == 'ID' and value in keywords: |
| 1912 |
kind = value |
| 1913 |
elif kind == 'NEWLINE': |
| 1914 |
line_start = mo.end() |
| 1915 |
line_num += 1 |
| 1916 |
continue |
| 1917 |
elif kind == 'SKIP': |
| 1918 |
continue |
| 1919 |
elif kind == 'MISMATCH': |
| 1920 |
raise RuntimeError(f'{value!r} unexpected on line {line_num}') |
| 1921 |
yield Token(kind, value, line_num, column) |
| 1922 |
|
| 1923 |
statements = ''' |
| 1924 |
IF quantity THEN |
| 1925 |
total := total + price * quantity; |
| 1926 |
tax := price * 0.05; |
| 1927 |
ENDIF; |
| 1928 |
''' |
| 1929 |
|
| 1930 |
for token in tokenize(statements): |
| 1931 |
print(token) |
| 1932 |
|
| 1933 |
The tokenizer produces the following output:: |
| 1934 |
|
| 1935 |
Token(type='IF', value='IF', line=2, column=4) |
| 1936 |
Token(type='ID', value='quantity', line=2, column=7) |
| 1937 |
Token(type='THEN', value='THEN', line=2, column=16) |
| 1938 |
Token(type='ID', value='total', line=3, column=8) |
| 1939 |
Token(type='ASSIGN', value=':=', line=3, column=14) |
| 1940 |
Token(type='ID', value='total', line=3, column=17) |
| 1941 |
Token(type='OP', value='+', line=3, column=23) |
| 1942 |
Token(type='ID', value='price', line=3, column=25) |
| 1943 |
Token(type='OP', value='*', line=3, column=31) |
| 1944 |
Token(type='ID', value='quantity', line=3, column=33) |
| 1945 |
Token(type='END', value=';', line=3, column=41) |
| 1946 |
Token(type='ID', value='tax', line=4, column=8) |
| 1947 |
Token(type='ASSIGN', value=':=', line=4, column=12) |
| 1948 |
Token(type='ID', value='price', line=4, column=15) |
| 1949 |
Token(type='OP', value='*', line=4, column=21) |
| 1950 |
Token(type='NUMBER', value=0.05, line=4, column=23) |
| 1951 |
Token(type='END', value=';', line=4, column=27) |
| 1952 |
Token(type='ENDIF', value='ENDIF', line=5, column=4) |
| 1953 |
Token(type='END', value=';', line=5, column=9) |
| 1954 |
|
| 1955 |
|
| 1956 |
.. [Frie09] Friedl, Jeffrey. Mastering Regular Expressions. 3rd ed., O'Reilly |
| 1957 |
Media, 2009. The third edition of the book no longer covers Python at all, |
| 1958 |
but the first edition covered writing good regular expression patterns in |
| 1959 |
great detail. |