Debian Codegraph @

Doc/library/re.rst

raw
1 :mod:`!re` --- Regular expression operations
2 ============================================
3
4 .. module:: re
5 :synopsis: Regular expression operations.
6
7 .. moduleauthor:: Fredrik Lundh <[email protected]>
8 .. sectionauthor:: Andrew M. Kuchling <[email protected]>
9
10 **Source code:** :source:`Lib/re/`
11
12 --------------
13
14 This module provides regular expression matching operations similar to
15 those found in Perl.
16
17 Both patterns and strings to be searched can be Unicode strings (:class:`str`)
18 as well as 8-bit strings (:class:`bytes`).
19 However, Unicode strings and 8-bit strings cannot be mixed:
20 that is, you cannot match a Unicode string with a bytes pattern or
21 vice-versa; similarly, when asking for a substitution, the replacement
22 string must be of the same type as both the pattern and the search string.
23
24 Regular expressions use the backslash character (``'\'``) to indicate
25 special forms or to allow special characters to be used without invoking
26 their special meaning. This collides with Python's usage of the same
27 character for the same purpose in string literals; for example, to match
28 a literal backslash, one might have to write ``'\\\\'`` as the pattern
29 string, because the regular expression must be ``\\``, and each
30 backslash must be expressed as ``\\`` inside a regular Python string
31 literal. Also, please note that any invalid escape sequences in Python's
32 usage of the backslash in string literals now generate a :exc:`SyntaxWarning`
33 and in the future this will become a :exc:`SyntaxError`. This behaviour
34 will happen even if it is a valid escape sequence for a regular expression.
35
36 The solution is to use Python's raw string notation for regular expression
37 patterns; backslashes are not handled in any special way in a string literal
38 prefixed with ``'r'``. So ``r"\n"`` is a two-character string containing
39 ``'\'`` and ``'n'``, while ``"\n"`` is a one-character string containing a
40 newline. Usually patterns will be expressed in Python code using this raw
41 string notation.
42
43 It is important to note that most regular expression operations are available as
44 module-level functions and methods on
45 :ref:`compiled regular expressions <re-objects>`. The functions are shortcuts
46 that don't require you to compile a regex object first, but miss some
47 fine-tuning parameters.
48
49 .. seealso::
50
51 The third-party :pypi:`regex` module,
52 which has an API compatible with the standard library :mod:`!re` module,
53 but offers additional functionality and a more thorough Unicode support.
54
55
56 .. _re-syntax:
57
58 Regular Expression Syntax
59 -------------------------
60
61 A regular expression (or RE) specifies a set of strings that matches it; the
62 functions in this module let you check if a particular string matches a given
63 regular expression (or if a given regular expression matches a particular
64 string, which comes down to the same thing).
65
66 Regular expressions can be concatenated to form new regular expressions; if *A*
67 and *B* are both regular expressions, then *AB* is also a regular expression.
68 In general, if a string *p* matches *A* and another string *q* matches *B*, the
69 string *pq* will match AB. This holds unless *A* or *B* contain low precedence
70 operations; boundary conditions between *A* and *B*; or have numbered group
71 references. Thus, complex expressions can easily be constructed from simpler
72 primitive expressions like the ones described here. For details of the theory
73 and implementation of regular expressions, consult the Friedl book [Frie09]_,
74 or almost any textbook about compiler construction.
75
76 A brief explanation of the format of regular expressions follows. For further
77 information and a gentler presentation, consult the :ref:`regex-howto`.
78
79 Regular expressions can contain both special and ordinary characters. Most
80 ordinary characters, like ``'A'``, ``'a'``, or ``'0'``, are the simplest regular
81 expressions; they simply match themselves. You can concatenate ordinary
82 characters, so ``last`` matches the string ``'last'``. (In the rest of this
83 section, we'll write RE's in ``this special style``, usually without quotes, and
84 strings to be matched ``'in single quotes'``.)
85
86 Some characters, like ``'|'`` or ``'('``, are special. Special
87 characters either stand for classes of ordinary characters, or affect
88 how the regular expressions around them are interpreted.
89
90 Repetition operators or quantifiers (``*``, ``+``, ``?``, ``{m,n}``, etc) cannot be
91 directly nested. This avoids ambiguity with the non-greedy modifier suffix
92 ``?``, and with other modifiers in other implementations. To apply a second
93 repetition to an inner repetition, parentheses may be used. For example,
94 the expression ``(?:a{6})*`` matches any multiple of six ``'a'`` characters.
95
96
97 The special characters are:
98
99 .. index:: single: . (dot); in regular expressions
100
101 ``.``
102 (Dot.) In the default mode, this matches any character except a newline. If
103 the :const:`DOTALL` flag has been specified, this matches any character
104 including a newline. ``(?s:.)`` matches any character regardless of flags.
105
106 .. index:: single: ^ (caret); in regular expressions
107
108 ``^``
109 (Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also
110 matches immediately after each newline.
111
112 .. index:: single: $ (dollar); in regular expressions
113
114 ``$``
115 Matches the end of the string or just before the newline at the end of the
116 string, and in :const:`MULTILINE` mode also matches before a newline. ``foo``
117 matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches
118 only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
119 matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for
120 a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before
121 the newline, and one at the end of the string.
122
123 .. index:: single: * (asterisk); in regular expressions
124
125 ``*``
126 Causes the resulting RE to match 0 or more repetitions of the preceding RE, as
127 many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed
128 by any number of 'b's.
129
130 .. index:: single: + (plus); in regular expressions
131
132 ``+``
133 Causes the resulting RE to match 1 or more repetitions of the preceding RE.
134 ``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not
135 match just 'a'.
136
137 .. index:: single: ? (question mark); in regular expressions
138
139 ``?``
140 Causes the resulting RE to match 0 or 1 repetitions of the preceding RE.
141 ``ab?`` will match either 'a' or 'ab'.
142
143 .. index::
144 single: *?; in regular expressions
145 single: +?; in regular expressions
146 single: ??; in regular expressions
147
148 ``*?``, ``+?``, ``??``
149 The ``'*'``, ``'+'``, and ``'?'`` quantifiers are all :dfn:`greedy`; they match
150 as much text as possible. Sometimes this behaviour isn't desired; if the RE
151 ``<.*>`` is matched against ``'<a> b <c>'``, it will match the entire
152 string, and not just ``'<a>'``. Adding ``?`` after the quantifier makes it
153 perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few*
154 characters as possible will be matched. Using the RE ``<.*?>`` will match
155 only ``'<a>'``.
156
157 .. index::
158 single: *+; in regular expressions
159 single: ++; in regular expressions
160 single: ?+; in regular expressions
161
162 ``*+``, ``++``, ``?+``
163 Like the ``'*'``, ``'+'``, and ``'?'`` quantifiers, those where ``'+'`` is
164 appended also match as many times as possible.
165 However, unlike the true greedy quantifiers, these do not allow
166 back-tracking when the expression following it fails to match.
167 These are known as :dfn:`possessive` quantifiers.
168 For example, ``a*a`` will match ``'aaaa'`` because the ``a*`` will match
169 all 4 ``'a'``\ s, but, when the final ``'a'`` is encountered, the
170 expression is backtracked so that in the end the ``a*`` ends up matching
171 3 ``'a'``\ s total, and the fourth ``'a'`` is matched by the final ``'a'``.
172 However, when ``a*+a`` is used to match ``'aaaa'``, the ``a*+`` will
173 match all 4 ``'a'``, but when the final ``'a'`` fails to find any more
174 characters to match, the expression cannot be backtracked and will thus
175 fail to match.
176 ``x*+``, ``x++`` and ``x?+`` are equivalent to ``(?>x*)``, ``(?>x+)``
177 and ``(?>x?)`` correspondingly.
178
179 .. versionadded:: 3.11
180
181 .. index::
182 single: {} (curly brackets); in regular expressions
183
184 ``{m}``
185 Specifies that exactly *m* copies of the previous RE should be matched; fewer
186 matches cause the entire RE not to match. For example, ``a{6}`` will match
187 exactly six ``'a'`` characters, but not five.
188
189 ``{m,n}``
190 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
191 RE, attempting to match as many repetitions as possible. For example,
192 ``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a
193 lower bound of zero, and omitting *n* specifies an infinite upper bound. As an
194 example, ``a{4,}b`` will match ``'aaaab'`` or a thousand ``'a'`` characters
195 followed by a ``'b'``, but not ``'aaab'``. The comma may not be omitted or the
196 modifier would be confused with the previously described form.
197
198 ``{m,n}?``
199 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
200 RE, attempting to match as *few* repetitions as possible. This is the
201 non-greedy version of the previous quantifier. For example, on the
202 6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters,
203 while ``a{3,5}?`` will only match 3 characters.
204
205 ``{m,n}+``
206 Causes the resulting RE to match from *m* to *n* repetitions of the
207 preceding RE, attempting to match as many repetitions as possible
208 *without* establishing any backtracking points.
209 This is the possessive version of the quantifier above.
210 For example, on the 6-character string ``'aaaaaa'``, ``a{3,5}+aa``
211 attempt to match 5 ``'a'`` characters, then, requiring 2 more ``'a'``\ s,
212 will need more characters than available and thus fail, while
213 ``a{3,5}aa`` will match with ``a{3,5}`` capturing 5, then 4 ``'a'``\ s
214 by backtracking and then the final 2 ``'a'``\ s are matched by the final
215 ``aa`` in the pattern.
216 ``x{m,n}+`` is equivalent to ``(?>x{m,n})``.
217
218 .. versionadded:: 3.11
219
220 .. index:: single: \ (backslash); in regular expressions
221
222 ``\``
223 Either escapes special characters (permitting you to match characters like
224 ``'*'``, ``'?'``, and so forth), or signals a special sequence; special
225 sequences are discussed below.
226
227 If you're not using a raw string to express the pattern, remember that Python
228 also uses the backslash as an escape sequence in string literals; if the escape
229 sequence isn't recognized by Python's parser, the backslash and subsequent
230 character are included in the resulting string. However, if Python would
231 recognize the resulting sequence, the backslash should be repeated twice. This
232 is complicated and hard to understand, so it's highly recommended that you use
233 raw strings for all but the simplest expressions.
234
235 .. index::
236 single: [] (square brackets); in regular expressions
237
238 ``[]``
239 Used to indicate a set of characters. In a set:
240
241 * Characters can be listed individually, e.g. ``[amk]`` will match ``'a'``,
242 ``'m'``, or ``'k'``.
243
244 .. index:: single: - (minus); in regular expressions
245
246 * Ranges of characters can be indicated by giving two characters and separating
247 them by a ``'-'``, for example ``[a-z]`` will match any lowercase ASCII letter,
248 ``[0-5][0-9]`` will match all the two-digits numbers from ``00`` to ``59``, and
249 ``[0-9A-Fa-f]`` will match any hexadecimal digit. If ``-`` is escaped (e.g.
250 ``[a\-z]``) or if it's placed as the first or last character
251 (e.g. ``[-a]`` or ``[a-]``), it will match a literal ``'-'``.
252
253 * Special characters except backslash lose their special meaning inside sets.
254 For example,
255 ``[(+*)]`` will match any of the literal characters ``'('``, ``'+'``,
256 ``'*'``, or ``')'``.
257
258 .. index:: single: \ (backslash); in regular expressions
259
260 * Backslash either escapes characters which have special meaning in a set
261 such as ``'-'``, ``']'``, ``'^'`` and ``'\\'`` itself or signals
262 a special sequence which represents a single character such as
263 ``\xa0`` or ``\n`` or a character class such as ``\w`` or ``\S``
264 (defined below).
265 Note that ``\b`` represents a single "backspace" character,
266 not a word boundary as outside a set, and numeric escapes
267 such as ``\1`` are always octal escapes, not group references.
268 Special sequences which do not match a single character such as ``\A``
269 and ``\z`` are not allowed.
270
271 .. index:: single: ^ (caret); in regular expressions
272
273 * Characters that are not within a range can be matched by :dfn:`complementing`
274 the set. If the first character of the set is ``'^'``, all the characters
275 that are *not* in the set will be matched. For example, ``[^5]`` will match
276 any character except ``'5'``, and ``[^^]`` will match any character except
277 ``'^'``. ``^`` has no special meaning if it's not the first character in
278 the set.
279
280 * To match a literal ``']'`` inside a set, precede it with a backslash, or
281 place it at the beginning of the set. For example, both ``[()[\]{}]`` and
282 ``[]()[{}]`` will match a right bracket, as well as left bracket, braces,
283 and parentheses.
284
285 .. .. index:: single: --; in regular expressions
286 .. .. index:: single: &&; in regular expressions
287 .. .. index:: single: ~~; in regular expressions
288 .. .. index:: single: ||; in regular expressions
289
290 * Support of nested sets and set operations as in `Unicode Technical
291 Standard #18`_ might be added in the future. This would change the
292 syntax, so to facilitate this change a :exc:`FutureWarning` will be raised
293 in ambiguous cases for the time being.
294 That includes sets starting with a literal ``'['`` or containing literal
295 character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To
296 avoid a warning escape them with a backslash.
297
298 .. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/
299
300 .. versionchanged:: 3.7
301 :exc:`FutureWarning` is raised if a character set contains constructs
302 that will change semantically in the future.
303
304 .. index:: single: | (vertical bar); in regular expressions
305
306 ``|``
307 ``A|B``, where *A* and *B* can be arbitrary REs, creates a regular expression that
308 will match either *A* or *B*. An arbitrary number of REs can be separated by the
309 ``'|'`` in this way. This can be used inside groups (see below) as well. As
310 the target string is scanned, REs separated by ``'|'`` are tried from left to
311 right. When one pattern completely matches, that branch is accepted. This means
312 that once *A* matches, *B* will not be tested further, even if it would
313 produce a longer overall match. In other words, the ``'|'`` operator is never
314 greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a
315 character class, as in ``[|]``.
316
317 .. index::
318 single: () (parentheses); in regular expressions
319
320 ``(...)``
321 Matches whatever regular expression is inside the parentheses, and indicates the
322 start and end of a group; the contents of a group can be retrieved after a match
323 has been performed, and can be matched later in the string with the ``\number``
324 special sequence, described below. To match the literals ``'('`` or ``')'``,
325 use ``\(`` or ``\)``, or enclose them inside a character class: ``[(]``, ``[)]``.
326
327 .. index:: single: (?; in regular expressions
328
329 ``(?...)``
330 This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful
331 otherwise). The first character after the ``'?'`` determines what the meaning
332 and further syntax of the construct is. Extensions usually do not create a new
333 group; ``(?P<name>...)`` is the only exception to this rule. Following are the
334 currently supported extensions.
335
336 ``(?aiLmsux)``
337 (One or more letters from the set
338 ``'a'``, ``'i'``, ``'L'``, ``'m'``, ``'s'``, ``'u'``, ``'x'``.)
339 The group matches the empty string;
340 the letters set the corresponding flags for the entire regular expression:
341
342 * :const:`re.A` (ASCII-only matching)
343 * :const:`re.I` (ignore case)
344 * :const:`re.L` (locale dependent)
345 * :const:`re.M` (multi-line)
346 * :const:`re.S` (dot matches all)
347 * :const:`re.U` (Unicode matching)
348 * :const:`re.X` (verbose)
349
350 (The flags are described in :ref:`contents-of-module-re`.)
351 This is useful if you wish to include the flags as part of the
352 regular expression, instead of passing a *flag* argument to the
353 :func:`re.compile` function.
354 Flags should be used first in the expression string.
355
356 .. versionchanged:: 3.11
357 This construction can only be used at the start of the expression.
358
359 .. index:: single: (?:; in regular expressions
360
361 ``(?:...)``
362 A non-capturing version of regular parentheses. Matches whatever regular
363 expression is inside the parentheses, but the substring matched by the group
364 *cannot* be retrieved after performing a match or referenced later in the
365 pattern.
366
367 ``(?aiLmsux-imsx:...)``
368 (Zero or more letters from the set
369 ``'a'``, ``'i'``, ``'L'``, ``'m'``, ``'s'``, ``'u'``, ``'x'``,
370 optionally followed by ``'-'`` followed by
371 one or more letters from the ``'i'``, ``'m'``, ``'s'``, ``'x'``.)
372 The letters set or remove the corresponding flags for the part of the expression:
373
374 * :const:`re.A` (ASCII-only matching)
375 * :const:`re.I` (ignore case)
376 * :const:`re.L` (locale dependent)
377 * :const:`re.M` (multi-line)
378 * :const:`re.S` (dot matches all)
379 * :const:`re.U` (Unicode matching)
380 * :const:`re.X` (verbose)
381
382 (The flags are described in :ref:`contents-of-module-re`.)
383
384 The letters ``'a'``, ``'L'`` and ``'u'`` are mutually exclusive when used
385 as inline flags, so they can't be combined or follow ``'-'``. Instead,
386 when one of them appears in an inline group, it overrides the matching mode
387 in the enclosing group. In Unicode patterns ``(?a:...)`` switches to
388 ASCII-only matching, and ``(?u:...)`` switches to Unicode matching
389 (default). In bytes patterns ``(?L:...)`` switches to locale dependent
390 matching, and ``(?a:...)`` switches to ASCII-only matching (default).
391 This override is only in effect for the narrow inline group, and the
392 original matching mode is restored outside of the group.
393
394 .. versionadded:: 3.6
395
396 .. versionchanged:: 3.7
397 The letters ``'a'``, ``'L'`` and ``'u'`` also can be used in a group.
398
399 ``(?>...)``
400 Attempts to match ``...`` as if it was a separate regular expression, and
401 if successful, continues to match the rest of the pattern following it.
402 If the subsequent pattern fails to match, the stack can only be unwound
403 to a point *before* the ``(?>...)`` because once exited, the expression,
404 known as an :dfn:`atomic group`, has thrown away all stack points within
405 itself.
406 Thus, ``(?>.*).`` would never match anything because first the ``.*``
407 would match all characters possible, then, having nothing left to match,
408 the final ``.`` would fail to match.
409 Since there are no stack points saved in the Atomic Group, and there is
410 no stack point before it, the entire expression would thus fail to match.
411
412 .. versionadded:: 3.11
413
414 .. index:: single: (?P<; in regular expressions
415
416 ``(?P<name>...)``
417 Similar to regular parentheses, but the substring matched by the group is
418 accessible via the symbolic group name *name*. Group names must be valid
419 Python identifiers, and in :class:`bytes` patterns they can only contain
420 bytes in the ASCII range. Each group name must be defined only once within
421 a regular expression. A symbolic group is also a numbered group, just as if
422 the group were not named.
423
424 Named groups can be referenced in three contexts. If the pattern is
425 ``(?P<quote>['"]).*?(?P=quote)`` (i.e. matching a string quoted with either
426 single or double quotes):
427
428 +---------------------------------------+----------------------------------+
429 | Context of reference to group "quote" | Ways to reference it |
430 +=======================================+==================================+
431 | in the same pattern itself | * ``(?P=quote)`` (as shown) |
432 | | * ``\1`` |
433 +---------------------------------------+----------------------------------+
434 | when processing match object *m* | * ``m.group('quote')`` |
435 | | * ``m.end('quote')`` (etc.) |
436 +---------------------------------------+----------------------------------+
437 | in a string passed to the *repl* | * ``\g<quote>`` |
438 | argument of ``re.sub()`` | * ``\g<1>`` |
439 | | * ``\1`` |
440 +---------------------------------------+----------------------------------+
441
442 .. versionchanged:: 3.12
443 In :class:`bytes` patterns, group *name* can only contain bytes
444 in the ASCII range (``b'\x00'``-``b'\x7f'``).
445
446 .. index:: single: (?P=; in regular expressions
447
448 ``(?P=name)``
449 A backreference to a named group; it matches whatever text was matched by the
450 earlier group named *name*.
451
452 .. index:: single: (?#; in regular expressions
453
454 ``(?#...)``
455 A comment; the contents of the parentheses are simply ignored.
456
457 .. index:: single: (?=; in regular expressions
458
459 ``(?=...)``
460 Matches if ``...`` matches next, but doesn't consume any of the string. This is
461 called a :dfn:`lookahead assertion`. For example, ``Isaac (?=Asimov)`` will match
462 ``'Isaac '`` only if it's followed by ``'Asimov'``.
463
464 .. index:: single: (?!; in regular expressions
465
466 ``(?!...)``
467 Matches if ``...`` doesn't match next. This is a :dfn:`negative lookahead assertion`.
468 For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not*
469 followed by ``'Asimov'``.
470
471 .. index:: single: (?<=; in regular expressions
472
473 ``(?<=...)``
474 Matches if the current position in the string is preceded by a match for ``...``
475 that ends at the current position. This is called a :dfn:`positive lookbehind
476 assertion`. ``(?<=abc)def`` will find a match in ``'abcdef'``, since the
477 lookbehind will back up 3 characters and check if the contained pattern matches.
478 The contained pattern must only match strings of some fixed length, meaning that
479 ``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that
480 patterns which start with positive lookbehind assertions will not match at the
481 beginning of the string being searched; you will most likely want to use the
482 :func:`search` function rather than the :func:`match` function:
483
484 >>> import re
485 >>> m = re.search('(?<=abc)def', 'abcdef')
486 >>> m.group(0)
487 'def'
488
489 This example looks for a word following a hyphen:
490
491 >>> m = re.search(r'(?<=-)\w+', 'spam-egg')
492 >>> m.group(0)
493 'egg'
494
495 .. versionchanged:: 3.5
496 Added support for group references of fixed length.
497
498 .. index:: single: (?<!; in regular expressions
499
500 ``(?<!...)``
501 Matches if the current position in the string is not preceded by a match for
502 ``...``. This is called a :dfn:`negative lookbehind assertion`. Similar to
503 positive lookbehind assertions, the contained pattern must only match strings of
504 some fixed length. Patterns which start with negative lookbehind assertions may
505 match at the beginning of the string being searched.
506
507 .. _re-conditional-expression:
508 .. index:: single: (?(; in regular expressions
509
510 ``(?(id/name)yes-pattern|no-pattern)``
511 Will try to match with ``yes-pattern`` if the group with given *id* or
512 *name* exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is
513 optional and can be omitted. For example,
514 ``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>|$)`` is a poor email matching pattern, which
515 will match with ``'<[email protected]>'`` as well as ``'[email protected]'``, but
516 not with ``'<[email protected]'`` nor ``'[email protected]>'``.
517
518 .. versionchanged:: 3.12
519 Group *id* can only contain ASCII digits.
520 In :class:`bytes` patterns, group *name* can only contain bytes
521 in the ASCII range (``b'\x00'``-``b'\x7f'``).
522
523
524 .. _re-special-sequences:
525
526 The special sequences consist of ``'\'`` and a character from the list below.
527 If the ordinary character is not an ASCII digit or an ASCII letter, then the
528 resulting RE will match the second character. For example, ``\$`` matches the
529 character ``'$'``.
530
531 .. index:: single: \ (backslash); in regular expressions
532
533 ``\number``
534 Matches the contents of the group of the same number. Groups are numbered
535 starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``,
536 but not ``'thethe'`` (note the space after the group). This special sequence
537 can only be used to match one of the first 99 groups. If the first digit of
538 *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as
539 a group match, but as the character with octal value *number*. Inside the
540 ``'['`` and ``']'`` of a character class, all numeric escapes are treated as
541 characters.
542
543 .. index:: single: \A; in regular expressions
544
545 ``\A``
546 Matches only at the start of the string.
547
548 .. index:: single: \b; in regular expressions
549
550 ``\b``
551 Matches the empty string, but only at the beginning or end of a word.
552 A word is defined as a sequence of word characters.
553 Note that formally, ``\b`` is defined as the boundary
554 between a ``\w`` and a ``\W`` character (or vice versa),
555 or between ``\w`` and the beginning or end of the string.
556 This means that ``r'\bat\b'`` matches ``'at'``, ``'at.'``, ``'(at)'``,
557 and ``'as at ay'`` but not ``'attempt'`` or ``'atlas'``.
558
559 The default word characters in Unicode (str) patterns
560 are Unicode alphanumerics and the underscore,
561 but this can be changed by using the :py:const:`~re.ASCII` flag.
562 Word boundaries are determined by the current locale
563 if the :py:const:`~re.LOCALE` flag is used.
564
565 .. note::
566
567 Inside a character range, ``\b`` represents the backspace character,
568 for compatibility with Python's string literals.
569
570 .. index:: single: \B; in regular expressions
571
572 ``\B``
573 Matches the empty string,
574 but only when it is *not* at the beginning or end of a word.
575 This means that ``r'at\B'`` matches ``'athens'``, ``'atom'``,
576 ``'attorney'``, but not ``'at'``, ``'at.'``, or ``'at!'``.
577 ``\B`` is the opposite of ``\b``,
578 so word characters in Unicode (str) patterns
579 are Unicode alphanumerics or the underscore,
580 although this can be changed by using the :py:const:`~re.ASCII` flag.
581 Word boundaries are determined by the current locale
582 if the :py:const:`~re.LOCALE` flag is used.
583
584 .. versionchanged:: 3.14
585 ``\B`` now matches empty input string.
586
587 .. index:: single: \d; in regular expressions
588
589 ``\d``
590 For Unicode (str) patterns:
591 Matches any Unicode decimal digit
592 (that is, any character in Unicode character category `[Nd]`__).
593 This includes ``[0-9]``, and also many other digit characters.
594
595 Matches ``[0-9]`` if the :py:const:`~re.ASCII` flag is used.
596
597 __ https://www.unicode.org/versions/Unicode15.0.0/ch04.pdf#G134153
598
599 For 8-bit (bytes) patterns:
600 Matches any decimal digit in the ASCII character set;
601 this is equivalent to ``[0-9]``.
602
603 .. index:: single: \D; in regular expressions
604
605 ``\D``
606 Matches any character which is not a decimal digit.
607 This is the opposite of ``\d``.
608
609 Matches ``[^0-9]`` if the :py:const:`~re.ASCII` flag is used.
610
611 .. index:: single: \s; in regular expressions
612
613 ``\s``
614 For Unicode (str) patterns:
615 Matches Unicode whitespace characters (as defined by :py:meth:`str.isspace`).
616 This includes ``[ \t\n\r\f\v]``, and also many other characters, for example the
617 non-breaking spaces mandated by typography rules in many languages.
618
619 Matches ``[ \t\n\r\f\v]`` if the :py:const:`~re.ASCII` flag is used.
620
621 For 8-bit (bytes) patterns:
622 Matches characters considered whitespace in the ASCII character set;
623 this is equivalent to ``[ \t\n\r\f\v]``.
624
625 .. index:: single: \S; in regular expressions
626
627 ``\S``
628 Matches any character which is not a whitespace character. This is
629 the opposite of ``\s``.
630
631 Matches ``[^ \t\n\r\f\v]`` if the :py:const:`~re.ASCII` flag is used.
632
633 .. index:: single: \w; in regular expressions
634
635 ``\w``
636 For Unicode (str) patterns:
637 Matches Unicode word characters;
638 this includes all Unicode alphanumeric characters
639 (as defined by :py:meth:`str.isalnum`),
640 as well as the underscore (``_``).
641
642 Matches ``[a-zA-Z0-9_]`` if the :py:const:`~re.ASCII` flag is used.
643
644 For 8-bit (bytes) patterns:
645 Matches characters considered alphanumeric in the ASCII character set;
646 this is equivalent to ``[a-zA-Z0-9_]``.
647 If the :py:const:`~re.LOCALE` flag is used,
648 matches characters considered alphanumeric in the current locale and the underscore.
649
650 .. index:: single: \W; in regular expressions
651
652 ``\W``
653 Matches any character which is not a word character.
654 This is the opposite of ``\w``.
655 By default, matches non-underscore (``_``) characters
656 for which :py:meth:`str.isalnum` returns ``False``.
657
658 Matches ``[^a-zA-Z0-9_]`` if the :py:const:`~re.ASCII` flag is used.
659
660 If the :py:const:`~re.LOCALE` flag is used,
661 matches characters which are neither alphanumeric in the current locale
662 nor the underscore.
663
664 .. index:: single: \z; in regular expressions
665 single: \Z; in regular expressions
666
667 ``\z``
668 Matches only at the end of the string.
669
670 .. versionadded:: 3.14
671
672 ``\Z``
673 The same as ``\z``. For compatibility with old Python versions.
674
675 .. index::
676 single: \a; in regular expressions
677 single: \b; in regular expressions
678 single: \f; in regular expressions
679 single: \n; in regular expressions
680 single: \N; in regular expressions
681 single: \r; in regular expressions
682 single: \t; in regular expressions
683 single: \u; in regular expressions
684 single: \U; in regular expressions
685 single: \v; in regular expressions
686 single: \x; in regular expressions
687 single: \\; in regular expressions
688
689 Most of the :ref:`escape sequences <escape-sequences>` supported by Python
690 string literals are also accepted by the regular expression parser::
691
692 \a \b \f \n
693 \N \r \t \u
694 \U \v \x \\
695
696 (Note that ``\b`` is used to represent word boundaries, and means "backspace"
697 only inside character classes.)
698
699 ``'\u'``, ``'\U'``, and ``'\N'`` escape sequences are
700 only recognized in Unicode (str) patterns.
701 In bytes patterns they are errors.
702 Unknown escapes of ASCII letters are reserved
703 for future use and treated as errors.
704
705 Octal escapes are included in a limited form. If the first digit is a 0, or if
706 there are three octal digits, it is considered an octal escape. Otherwise, it is
707 a group reference. As for string literals, octal escapes are always at most
708 three digits in length.
709
710 .. versionchanged:: 3.3
711 The ``'\u'`` and ``'\U'`` escape sequences have been added.
712
713 .. versionchanged:: 3.6
714 Unknown escapes consisting of ``'\'`` and an ASCII letter now are errors.
715
716 .. versionchanged:: 3.8
717 The :samp:`'\\N\\{{name}\\}'` escape sequence has been added. As in string literals,
718 it expands to the named Unicode character (e.g. ``'\N{EM DASH}'``).
719
720
721 .. _contents-of-module-re:
722
723 Module Contents
724 ---------------
725
726 The module defines several functions, constants, and an exception. Some of the
727 functions are simplified versions of the full featured methods for compiled
728 regular expressions. Most non-trivial applications always use the compiled
729 form.
730
731
732 Flags
733 ^^^^^
734
735 .. versionchanged:: 3.6
736 Flag constants are now instances of :class:`RegexFlag`, which is a subclass of
737 :class:`enum.IntFlag`.
738
739
740 .. class:: RegexFlag
741
742 An :class:`enum.IntFlag` class containing the regex options listed below.
743
744 .. versionadded:: 3.11 - added to ``__all__``
745
746 .. data:: A
747 ASCII
748
749 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\d``, ``\D``, ``\s`` and ``\S``
750 perform ASCII-only matching instead of full Unicode matching. This is only
751 meaningful for Unicode (str) patterns, and is ignored for bytes patterns.
752
753 Corresponds to the inline flag ``(?a)``.
754
755 .. note::
756
757 The :py:const:`~re.U` flag still exists for backward compatibility,
758 but is redundant in Python 3 since
759 matches are Unicode by default for ``str`` patterns,
760 and Unicode matching isn't allowed for bytes patterns.
761 :py:const:`~re.UNICODE` and the inline flag ``(?u)`` are similarly redundant.
762
763
764 .. data:: DEBUG
765
766 Display debug information about compiled expression.
767
768 No corresponding inline flag.
769
770
771 .. data:: I
772 IGNORECASE
773
774 Perform case-insensitive matching;
775 expressions like ``[A-Z]`` will also match lowercase letters.
776 Full Unicode matching (such as ``Ü`` matching ``ü``)
777 also works unless the :py:const:`~re.ASCII` flag
778 is used to disable non-ASCII matches.
779 The current locale does not change the effect of this flag
780 unless the :py:const:`~re.LOCALE` flag is also used.
781
782 Corresponds to the inline flag ``(?i)``.
783
784 Note that when the Unicode patterns ``[a-z]`` or ``[A-Z]`` are used in
785 combination with the :const:`IGNORECASE` flag, they will match the 52 ASCII
786 letters and 4 additional non-ASCII letters: 'İ' (U+0130, Latin capital
787 letter I with dot above), 'ı' (U+0131, Latin small letter dotless i),
788 'ſ' (U+017F, Latin small letter long s) and 'K' (U+212A, Kelvin sign).
789 If the :py:const:`~re.ASCII` flag is used, only letters 'a' to 'z'
790 and 'A' to 'Z' are matched.
791
792 .. data:: L
793 LOCALE
794
795 Make ``\w``, ``\W``, ``\b``, ``\B`` and case-insensitive matching
796 dependent on the current locale.
797 This flag can be used only with bytes patterns.
798
799 Corresponds to the inline flag ``(?L)``.
800
801 .. warning::
802
803 This flag is discouraged; consider Unicode matching instead.
804 The locale mechanism is very unreliable
805 as it only handles one "culture" at a time
806 and only works with 8-bit locales.
807 Unicode matching is enabled by default for Unicode (str) patterns
808 and it is able to handle different locales and languages.
809
810 .. versionchanged:: 3.6
811 :py:const:`~re.LOCALE` can be used only with bytes patterns
812 and is not compatible with :py:const:`~re.ASCII`.
813
814 .. versionchanged:: 3.7
815 Compiled regular expression objects with the :py:const:`~re.LOCALE` flag
816 no longer depend on the locale at compile time.
817 Only the locale at matching time affects the result of matching.
818
819
820 .. data:: M
821 MULTILINE
822
823 When specified, the pattern character ``'^'`` matches at the beginning of the
824 string and at the beginning of each line (immediately following each newline);
825 and the pattern character ``'$'`` matches at the end of the string and at the
826 end of each line (immediately preceding each newline). By default, ``'^'``
827 matches only at the beginning of the string, and ``'$'`` only at the end of the
828 string and immediately before the newline (if any) at the end of the string.
829
830 Corresponds to the inline flag ``(?m)``.
831
832 .. data:: NOFLAG
833
834 Indicates no flag being applied, the value is ``0``. This flag may be used
835 as a default value for a function keyword argument or as a base value that
836 will be conditionally ORed with other flags. Example of use as a default
837 value::
838
839 def myfunc(text, flag=re.NOFLAG):
840 return re.match(text, flag)
841
842 .. versionadded:: 3.11
843
844 .. data:: S
845 DOTALL
846
847 Make the ``'.'`` special character match any character at all, including a
848 newline; without this flag, ``'.'`` will match anything *except* a newline.
849
850 Corresponds to the inline flag ``(?s)``.
851
852
853 .. data:: U
854 UNICODE
855
856 In Python 3, Unicode characters are matched by default
857 for ``str`` patterns.
858 This flag is therefore redundant with **no effect**
859 and is only kept for backward compatibility.
860
861 See :py:const:`~re.ASCII` to restrict matching to ASCII characters instead.
862
863 .. data:: X
864 VERBOSE
865
866 .. index:: single: # (hash); in regular expressions
867
868 This flag allows you to write regular expressions that look nicer and are
869 more readable by allowing you to visually separate logical sections of the
870 pattern and add comments. Whitespace within the pattern is ignored, except
871 when in a character class, or when preceded by an unescaped backslash,
872 or within tokens like ``*?``, ``(?:`` or ``(?P<...>``. For example, ``(? :``
873 and ``* ?`` are not allowed.
874 When a line contains a ``#`` that is not in a character class and is not
875 preceded by an unescaped backslash, all characters from the leftmost such
876 ``#`` through the end of the line are ignored.
877
878 This means that the two following regular expression objects that match a
879 decimal number are functionally equal::
880
881 a = re.compile(r"""\d + # the integral part
882 \. # the decimal point
883 \d * # some fractional digits""", re.X)
884 b = re.compile(r"\d+\.\d*")
885
886 Corresponds to the inline flag ``(?x)``.
887
888
889 Functions
890 ^^^^^^^^^
891
892 .. function:: compile(pattern, flags=0)
893
894 Compile a regular expression pattern into a :ref:`regular expression object
895 <re-objects>`, which can be used for matching using its
896 :func:`~Pattern.match`, :func:`~Pattern.search` and other methods, described
897 below.
898
899 The expression's behaviour can be modified by specifying a *flags* value.
900 Values can be any of the `flags`_ variables, combined using bitwise OR
901 (the ``|`` operator).
902
903 The sequence ::
904
905 prog = re.compile(pattern)
906 result = prog.match(string)
907
908 is equivalent to ::
909
910 result = re.match(pattern, string)
911
912 but using :func:`re.compile` and saving the resulting regular expression
913 object for reuse is more efficient when the expression will be used several
914 times in a single program.
915
916 .. note::
917
918 The compiled versions of the most recent patterns passed to
919 :func:`re.compile` and the module-level matching functions are cached, so
920 programs that use only a few regular expressions at a time needn't worry
921 about compiling regular expressions.
922
923
924 .. function:: search(pattern, string, flags=0)
925
926 Scan through *string* looking for the first location where the regular expression
927 *pattern* produces a match, and return a corresponding :class:`~re.Match`. Return
928 ``None`` if no position in the string matches the pattern; note that this is
929 different from finding a zero-length match at some point in the string.
930
931 The expression's behaviour can be modified by specifying a *flags* value.
932 Values can be any of the `flags`_ variables, combined using bitwise OR
933 (the ``|`` operator).
934
935
936 .. function:: match(pattern, string, flags=0)
937
938 If zero or more characters at the beginning of *string* match the regular
939 expression *pattern*, return a corresponding :class:`~re.Match`. Return
940 ``None`` if the string does not match the pattern; note that this is
941 different from a zero-length match.
942
943 Note that even in :const:`MULTILINE` mode, :func:`re.match` will only match
944 at the beginning of the string and not at the beginning of each line.
945
946 If you want to locate a match anywhere in *string*, use :func:`search`
947 instead (see also :ref:`search-vs-match`).
948
949 The expression's behaviour can be modified by specifying a *flags* value.
950 Values can be any of the `flags`_ variables, combined using bitwise OR
951 (the ``|`` operator).
952
953
954 .. function:: fullmatch(pattern, string, flags=0)
955
956 If the whole *string* matches the regular expression *pattern*, return a
957 corresponding :class:`~re.Match`. Return ``None`` if the string does not match
958 the pattern; note that this is different from a zero-length match.
959
960 The expression's behaviour can be modified by specifying a *flags* value.
961 Values can be any of the `flags`_ variables, combined using bitwise OR
962 (the ``|`` operator).
963
964 .. versionadded:: 3.4
965
966
967 .. function:: split(pattern, string, maxsplit=0, flags=0)
968
969 Split *string* by the occurrences of *pattern*. If capturing parentheses are
970 used in *pattern*, then the text of all groups in the pattern are also returned
971 as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit*
972 splits occur, and the remainder of the string is returned as the final element
973 of the list. ::
974
975 >>> re.split(r'\W+', 'Words, words, words.')
976 ['Words', 'words', 'words', '']
977 >>> re.split(r'(\W+)', 'Words, words, words.')
978 ['Words', ', ', 'words', ', ', 'words', '.', '']
979 >>> re.split(r'\W+', 'Words, words, words.', maxsplit=1)
980 ['Words', 'words, words.']
981 >>> re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE)
982 ['0', '3', '9']
983
984 If there are capturing groups in the separator and it matches at the start of
985 the string, the result will start with an empty string. The same holds for
986 the end of the string::
987
988 >>> re.split(r'(\W+)', '...words, words...')
989 ['', '...', 'words', ', ', 'words', '...', '']
990
991 That way, separator components are always found at the same relative
992 indices within the result list.
993
994 Adjacent empty matches are not possible, but an empty match can occur
995 immediately after a non-empty match.
996
997 .. code:: pycon
998
999 >>> re.split(r'\b', 'Words, words, words.')
1000 ['', 'Words', ', ', 'words', ', ', 'words', '.']
1001 >>> re.split(r'\W*', '...words...')
1002 ['', '', 'w', 'o', 'r', 'd', 's', '', '']
1003 >>> re.split(r'(\W*)', '...words...')
1004 ['', '...', '', '', 'w', '', 'o', '', 'r', '', 'd', '', 's', '...', '', '', '']
1005
1006 The expression's behaviour can be modified by specifying a *flags* value.
1007 Values can be any of the `flags`_ variables, combined using bitwise OR
1008 (the ``|`` operator).
1009
1010 .. versionchanged:: 3.1
1011 Added the optional flags argument.
1012
1013 .. versionchanged:: 3.7
1014 Added support of splitting on a pattern that could match an empty string.
1015
1016 .. deprecated:: 3.13
1017 Passing *maxsplit* and *flags* as positional arguments is deprecated.
1018 In future Python versions they will be
1019 :ref:`keyword-only parameters <keyword-only_parameter>`.
1020
1021
1022 .. function:: findall(pattern, string, flags=0)
1023
1024 Return all non-overlapping matches of *pattern* in *string*, as a list of
1025 strings or tuples. The *string* is scanned left-to-right, and matches
1026 are returned in the order found. Empty matches are included in the result.
1027
1028 The result depends on the number of capturing groups in the pattern.
1029 If there are no groups, return a list of strings matching the whole
1030 pattern. If there is exactly one group, return a list of strings
1031 matching that group. If multiple groups are present, return a list
1032 of tuples of strings matching the groups. Non-capturing groups do not
1033 affect the form of the result.
1034
1035 >>> re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest')
1036 ['foot', 'fell', 'fastest']
1037 >>> re.findall(r'(\w+)=(\d+)', 'set width=20 and height=10')
1038 [('width', '20'), ('height', '10')]
1039
1040 The expression's behaviour can be modified by specifying a *flags* value.
1041 Values can be any of the `flags`_ variables, combined using bitwise OR
1042 (the ``|`` operator).
1043
1044 .. versionchanged:: 3.7
1045 Non-empty matches can now start just after a previous empty match.
1046
1047
1048 .. function:: finditer(pattern, string, flags=0)
1049
1050 Return an :term:`iterator` yielding :class:`~re.Match` objects over
1051 all non-overlapping matches for the RE *pattern* in *string*. The *string*
1052 is scanned left-to-right, and matches are returned in the order found. Empty
1053 matches are included in the result.
1054
1055 The expression's behaviour can be modified by specifying a *flags* value.
1056 Values can be any of the `flags`_ variables, combined using bitwise OR
1057 (the ``|`` operator).
1058
1059 .. versionchanged:: 3.7
1060 Non-empty matches can now start just after a previous empty match.
1061
1062
1063 .. function:: sub(pattern, repl, string, count=0, flags=0)
1064
1065 Return the string obtained by replacing the leftmost non-overlapping occurrences
1066 of *pattern* in *string* by the replacement *repl*. If the pattern isn't found,
1067 *string* is returned unchanged. *repl* can be a string or a function; if it is
1068 a string, any backslash escapes in it are processed. That is, ``\n`` is
1069 converted to a single newline character, ``\r`` is converted to a carriage return, and
1070 so forth. Unknown escapes of ASCII letters are reserved for future use and
1071 treated as errors. Other unknown escapes such as ``\&`` are left alone.
1072 Backreferences, such
1073 as ``\6``, are replaced with the substring matched by group 6 in the pattern.
1074 For example::
1075
1076 >>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
1077 ... r'static PyObject*\npy_\1(void)\n{',
1078 ... 'def myfunc():')
1079 'static PyObject*\npy_myfunc(void)\n{'
1080
1081 If *repl* is a function, it is called for every non-overlapping occurrence of
1082 *pattern*. The function takes a single :class:`~re.Match` argument, and returns
1083 the replacement string. For example::
1084
1085 >>> def dashrepl(matchobj):
1086 ... if matchobj.group(0) == '-': return ' '
1087 ... else: return '-'
1088 ...
1089 >>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
1090 'pro--gram files'
1091 >>> re.sub(r'\sAND\s', ' & ', 'Baked Beans And Spam', flags=re.IGNORECASE)
1092 'Baked Beans & Spam'
1093
1094 The pattern may be a string or a :class:`~re.Pattern`.
1095
1096 The optional argument *count* is the maximum number of pattern occurrences to be
1097 replaced; *count* must be a non-negative integer. If omitted or zero, all
1098 occurrences will be replaced.
1099
1100 Adjacent empty matches are not possible, but an empty match can occur
1101 immediately after a non-empty match.
1102 As a result, ``sub('x*', '-', 'abxd')`` returns ``'-a-b--d-'``
1103 instead of ``'-a-b-d-'``.
1104
1105 .. index:: single: \g; in regular expressions
1106
1107 In string-type *repl* arguments, in addition to the character escapes and
1108 backreferences described above,
1109 ``\g<name>`` will use the substring matched by the group named ``name``, as
1110 defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding
1111 group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous
1112 in a replacement such as ``\g<2>0``. ``\20`` would be interpreted as a
1113 reference to group 20, not a reference to group 2 followed by the literal
1114 character ``'0'``. The backreference ``\g<0>`` substitutes in the entire
1115 substring matched by the RE.
1116
1117 The expression's behaviour can be modified by specifying a *flags* value.
1118 Values can be any of the `flags`_ variables, combined using bitwise OR
1119 (the ``|`` operator).
1120
1121 .. versionchanged:: 3.1
1122 Added the optional flags argument.
1123
1124 .. versionchanged:: 3.5
1125 Unmatched groups are replaced with an empty string.
1126
1127 .. versionchanged:: 3.6
1128 Unknown escapes in *pattern* consisting of ``'\'`` and an ASCII letter
1129 now are errors.
1130
1131 .. versionchanged:: 3.7
1132 Unknown escapes in *repl* consisting of ``'\'`` and an ASCII letter
1133 now are errors.
1134 An empty match can occur immediately after a non-empty match.
1135
1136 .. versionchanged:: 3.12
1137 Group *id* can only contain ASCII digits.
1138 In :class:`bytes` replacement strings, group *name* can only contain bytes
1139 in the ASCII range (``b'\x00'``-``b'\x7f'``).
1140
1141 .. deprecated:: 3.13
1142 Passing *count* and *flags* as positional arguments is deprecated.
1143 In future Python versions they will be
1144 :ref:`keyword-only parameters <keyword-only_parameter>`.
1145
1146
1147 .. function:: subn(pattern, repl, string, count=0, flags=0)
1148
1149 Perform the same operation as :func:`sub`, but return a tuple ``(new_string,
1150 number_of_subs_made)``.
1151
1152 The expression's behaviour can be modified by specifying a *flags* value.
1153 Values can be any of the `flags`_ variables, combined using bitwise OR
1154 (the ``|`` operator).
1155
1156
1157 .. function:: escape(pattern)
1158
1159 Escape special characters in *pattern*.
1160 This is useful if you want to match an arbitrary literal string that may
1161 have regular expression metacharacters in it. For example::
1162
1163 >>> print(re.escape('https://www.python.org'))
1164 https://www\.python\.org
1165
1166 >>> legal_chars = string.ascii_lowercase + string.digits + "!#$%&'*+-.^_`|~:"
1167 >>> print('[%s]+' % re.escape(legal_chars))
1168 [abcdefghijklmnopqrstuvwxyz0123456789!\#\$%\&'\*\+\-\.\^_`\|\~:]+
1169
1170 >>> operators = ['+', '-', '*', '/', '**']
1171 >>> print('|'.join(map(re.escape, sorted(operators, reverse=True))))
1172 /|\-|\+|\*\*|\*
1173
1174 This function must not be used for the replacement string in :func:`sub`
1175 and :func:`subn`, only backslashes should be escaped. For example::
1176
1177 >>> digits_re = r'\d+'
1178 >>> sample = '/usr/sbin/sendmail - 0 errors, 12 warnings'
1179 >>> print(re.sub(digits_re, digits_re.replace('\\', r'\\'), sample))
1180 /usr/sbin/sendmail - \d+ errors, \d+ warnings
1181
1182 .. versionchanged:: 3.3
1183 The ``'_'`` character is no longer escaped.
1184
1185 .. versionchanged:: 3.7
1186 Only characters that can have special meaning in a regular expression
1187 are escaped. As a result, ``'!'``, ``'"'``, ``'%'``, ``"'"``, ``','``,
1188 ``'/'``, ``':'``, ``';'``, ``'<'``, ``'='``, ``'>'``, ``'@'``, and
1189 ``"`"`` are no longer escaped.
1190
1191
1192 .. function:: purge()
1193
1194 Clear the regular expression cache.
1195
1196
1197 Exceptions
1198 ^^^^^^^^^^
1199
1200 .. exception:: PatternError(msg, pattern=None, pos=None)
1201
1202 Exception raised when a string passed to one of the functions here is not a
1203 valid regular expression (for example, it might contain unmatched parentheses)
1204 or when some other error occurs during compilation or matching. It is never an
1205 error if a string contains no match for a pattern. The ``PatternError`` instance has
1206 the following additional attributes:
1207
1208 .. attribute:: msg
1209
1210 The unformatted error message.
1211
1212 .. attribute:: pattern
1213
1214 The regular expression pattern.
1215
1216 .. attribute:: pos
1217
1218 The index in *pattern* where compilation failed (may be ``None``).
1219
1220 .. attribute:: lineno
1221
1222 The line corresponding to *pos* (may be ``None``).
1223
1224 .. attribute:: colno
1225
1226 The column corresponding to *pos* (may be ``None``).
1227
1228 .. versionchanged:: 3.5
1229 Added additional attributes.
1230
1231 .. versionchanged:: 3.13
1232 ``PatternError`` was originally named ``error``; the latter is kept as an alias for
1233 backward compatibility.
1234
1235 .. _re-objects:
1236
1237 Regular Expression Objects
1238 --------------------------
1239
1240 .. class:: Pattern
1241
1242 Compiled regular expression object returned by :func:`re.compile`.
1243
1244 .. versionchanged:: 3.9
1245 :py:class:`re.Pattern` supports ``[]`` to indicate a Unicode (str) or bytes pattern.
1246 See :ref:`types-genericalias`.
1247
1248 .. method:: Pattern.search(string[, pos[, endpos]])
1249
1250 Scan through *string* looking for the first location where this regular
1251 expression produces a match, and return a corresponding :class:`~re.Match`.
1252 Return ``None`` if no position in the string matches the pattern; note that
1253 this is different from finding a zero-length match at some point in the string.
1254
1255 The optional second parameter *pos* gives an index in the string where the
1256 search is to start; it defaults to ``0``. This is not completely equivalent to
1257 slicing the string; the ``'^'`` pattern character matches at the real beginning
1258 of the string and at positions just after a newline, but not necessarily at the
1259 index where the search is to start.
1260
1261 The optional parameter *endpos* limits how far the string will be searched; it
1262 will be as if the string is *endpos* characters long, so only the characters
1263 from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
1264 than *pos*, no match will be found; otherwise, if *rx* is a compiled regular
1265 expression object, ``rx.search(string, 0, 50)`` is equivalent to
1266 ``rx.search(string[:50], 0)``. ::
1267
1268 >>> pattern = re.compile("d")
1269 >>> pattern.search("dog") # Match at index 0
1270 <re.Match object; span=(0, 1), match='d'>
1271 >>> pattern.search("dog", 1) # No match; search doesn't include the "d"
1272
1273
1274 .. method:: Pattern.match(string[, pos[, endpos]])
1275
1276 If zero or more characters at the *beginning* of *string* match this regular
1277 expression, return a corresponding :class:`~re.Match`. Return ``None`` if the
1278 string does not match the pattern; note that this is different from a
1279 zero-length match.
1280
1281 The optional *pos* and *endpos* parameters have the same meaning as for the
1282 :meth:`~Pattern.search` method. ::
1283
1284 >>> pattern = re.compile("o")
1285 >>> pattern.match("dog") # No match as "o" is not at the start of "dog".
1286 >>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".
1287 <re.Match object; span=(1, 2), match='o'>
1288
1289 If you want to locate a match anywhere in *string*, use
1290 :meth:`~Pattern.search` instead (see also :ref:`search-vs-match`).
1291
1292
1293 .. method:: Pattern.fullmatch(string[, pos[, endpos]])
1294
1295 If the whole *string* matches this regular expression, return a corresponding
1296 :class:`~re.Match`. Return ``None`` if the string does not match the pattern;
1297 note that this is different from a zero-length match.
1298
1299 The optional *pos* and *endpos* parameters have the same meaning as for the
1300 :meth:`~Pattern.search` method. ::
1301
1302 >>> pattern = re.compile("o[gh]")
1303 >>> pattern.fullmatch("dog") # No match as "o" is not at the start of "dog".
1304 >>> pattern.fullmatch("ogre") # No match as not the full string matches.
1305 >>> pattern.fullmatch("doggie", 1, 3) # Matches within given limits.
1306 <re.Match object; span=(1, 3), match='og'>
1307
1308 .. versionadded:: 3.4
1309
1310
1311 .. method:: Pattern.split(string, maxsplit=0)
1312
1313 Identical to the :func:`split` function, using the compiled pattern.
1314
1315
1316 .. method:: Pattern.findall(string[, pos[, endpos]])
1317
1318 Similar to the :func:`findall` function, using the compiled pattern, but
1319 also accepts optional *pos* and *endpos* parameters that limit the search
1320 region like for :meth:`search`.
1321
1322
1323 .. method:: Pattern.finditer(string[, pos[, endpos]])
1324
1325 Similar to the :func:`finditer` function, using the compiled pattern, but
1326 also accepts optional *pos* and *endpos* parameters that limit the search
1327 region like for :meth:`search`.
1328
1329
1330 .. method:: Pattern.sub(repl, string, count=0)
1331
1332 Identical to the :func:`sub` function, using the compiled pattern.
1333
1334
1335 .. method:: Pattern.subn(repl, string, count=0)
1336
1337 Identical to the :func:`subn` function, using the compiled pattern.
1338
1339
1340 .. attribute:: Pattern.flags
1341
1342 The regex matching flags. This is a combination of the flags given to
1343 :func:`.compile`, any ``(?...)`` inline flags in the pattern, and implicit
1344 flags such as :py:const:`~re.UNICODE` if the pattern is a Unicode string.
1345
1346
1347 .. attribute:: Pattern.groups
1348
1349 The number of capturing groups in the pattern.
1350
1351
1352 .. attribute:: Pattern.groupindex
1353
1354 A dictionary mapping any symbolic group names defined by ``(?P<id>)`` to group
1355 numbers. The dictionary is empty if no symbolic groups were used in the
1356 pattern.
1357
1358
1359 .. attribute:: Pattern.pattern
1360
1361 The pattern string from which the pattern object was compiled.
1362
1363
1364 .. versionchanged:: 3.7
1365 Added support of :func:`copy.copy` and :func:`copy.deepcopy`. Compiled
1366 regular expression objects are considered atomic.
1367
1368
1369 .. _match-objects:
1370
1371 Match Objects
1372 -------------
1373
1374 Match objects always have a boolean value of ``True``.
1375 Since :meth:`~Pattern.match` and :meth:`~Pattern.search` return ``None``
1376 when there is no match, you can test whether there was a match with a simple
1377 ``if`` statement::
1378
1379 match = re.search(pattern, string)
1380 if match:
1381 process(match)
1382
1383 .. class:: Match
1384
1385 Match object returned by successful ``match``\ es and ``search``\ es.
1386
1387 .. versionchanged:: 3.9
1388 :py:class:`re.Match` supports ``[]`` to indicate a Unicode (str) or bytes match.
1389 See :ref:`types-genericalias`.
1390
1391 .. method:: Match.expand(template)
1392
1393 Return the string obtained by doing backslash substitution on the template
1394 string *template*, as done by the :meth:`~Pattern.sub` method.
1395 Escapes such as ``\n`` are converted to the appropriate characters,
1396 and numeric backreferences (``\1``, ``\2``) and named backreferences
1397 (``\g<1>``, ``\g<name>``) are replaced by the contents of the
1398 corresponding group. The backreference ``\g<0>`` will be
1399 replaced by the entire match.
1400
1401 .. versionchanged:: 3.5
1402 Unmatched groups are replaced with an empty string.
1403
1404 .. method:: Match.group([group1, ...])
1405
1406 Returns one or more subgroups of the match. If there is a single argument, the
1407 result is a single string; if there are multiple arguments, the result is a
1408 tuple with one item per argument. Without arguments, *group1* defaults to zero
1409 (the whole match is returned). If a *groupN* argument is zero, the corresponding
1410 return value is the entire matching string; if it is a positive integer, it is
1411 the string matching the corresponding parenthesized group. If a group number is
1412 negative or larger than the number of groups defined in the pattern, an
1413 :exc:`IndexError` exception is raised. If a group is contained in a
1414 part of the pattern that did not match, the corresponding result is ``None``.
1415 If a group is contained in a part of the pattern that matched multiple times,
1416 the last match is returned. ::
1417
1418 >>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
1419 >>> m.group(0) # The entire match
1420 'Isaac Newton'
1421 >>> m.group(1) # The first parenthesized subgroup.
1422 'Isaac'
1423 >>> m.group(2) # The second parenthesized subgroup.
1424 'Newton'
1425 >>> m.group(1, 2) # Multiple arguments give us a tuple.
1426 ('Isaac', 'Newton')
1427
1428 If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN*
1429 arguments may also be strings identifying groups by their group name. If a
1430 string argument is not used as a group name in the pattern, an :exc:`IndexError`
1431 exception is raised.
1432
1433 A moderately complicated example::
1434
1435 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcolm Reynolds")
1436 >>> m.group('first_name')
1437 'Malcolm'
1438 >>> m.group('last_name')
1439 'Reynolds'
1440
1441 Named groups can also be referred to by their index::
1442
1443 >>> m.group(1)
1444 'Malcolm'
1445 >>> m.group(2)
1446 'Reynolds'
1447
1448 If a group matches multiple times, only the last match is accessible::
1449
1450 >>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times.
1451 >>> m.group(1) # Returns only the last match.
1452 'c3'
1453
1454
1455 .. method:: Match.__getitem__(g)
1456
1457 This is identical to ``m.group(g)``. This allows easier access to
1458 an individual group from a match::
1459
1460 >>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
1461 >>> m[0] # The entire match
1462 'Isaac Newton'
1463 >>> m[1] # The first parenthesized subgroup.
1464 'Isaac'
1465 >>> m[2] # The second parenthesized subgroup.
1466 'Newton'
1467
1468 Named groups are supported as well::
1469
1470 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Isaac Newton")
1471 >>> m['first_name']
1472 'Isaac'
1473 >>> m['last_name']
1474 'Newton'
1475
1476 .. versionadded:: 3.6
1477
1478
1479 .. method:: Match.groups(default=None)
1480
1481 Return a tuple containing all the subgroups of the match, from 1 up to however
1482 many groups are in the pattern. The *default* argument is used for groups that
1483 did not participate in the match; it defaults to ``None``.
1484
1485 For example::
1486
1487 >>> m = re.match(r"(\d+)\.(\d+)", "24.1632")
1488 >>> m.groups()
1489 ('24', '1632')
1490
1491 If we make the decimal place and everything after it optional, not all groups
1492 might participate in the match. These groups will default to ``None`` unless
1493 the *default* argument is given::
1494
1495 >>> m = re.match(r"(\d+)\.?(\d+)?", "24")
1496 >>> m.groups() # Second group defaults to None.
1497 ('24', None)
1498 >>> m.groups('0') # Now, the second group defaults to '0'.
1499 ('24', '0')
1500
1501
1502 .. method:: Match.groupdict(default=None)
1503
1504 Return a dictionary containing all the *named* subgroups of the match, keyed by
1505 the subgroup name. The *default* argument is used for groups that did not
1506 participate in the match; it defaults to ``None``. For example::
1507
1508 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcolm Reynolds")
1509 >>> m.groupdict()
1510 {'first_name': 'Malcolm', 'last_name': 'Reynolds'}
1511
1512
1513 .. method:: Match.start([group])
1514 Match.end([group])
1515
1516 Return the indices of the start and end of the substring matched by *group*;
1517 *group* defaults to zero (meaning the whole matched substring). Return ``-1`` if
1518 *group* exists but did not contribute to the match. For a match object *m*, and
1519 a group *g* that did contribute to the match, the substring matched by group *g*
1520 (equivalent to ``m.group(g)``) is ::
1521
1522 m.string[m.start(g):m.end(g)]
1523
1524 Note that ``m.start(group)`` will equal ``m.end(group)`` if *group* matched a
1525 null string. For example, after ``m = re.search('b(c?)', 'cba')``,
1526 ``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both
1527 2, and ``m.start(2)`` raises an :exc:`IndexError` exception.
1528
1529 An example that will remove *remove_this* from email addresses::
1530
1531 >>> email = "tony@tiremove_thisger.net"
1532 >>> m = re.search("remove_this", email)
1533 >>> email[:m.start()] + email[m.end():]
1534 '[email protected]'
1535
1536
1537 .. method:: Match.span([group])
1538
1539 For a match *m*, return the 2-tuple ``(m.start(group), m.end(group))``. Note
1540 that if *group* did not contribute to the match, this is ``(-1, -1)``.
1541 *group* defaults to zero, the entire match.
1542
1543
1544 .. attribute:: Match.pos
1545
1546 The value of *pos* which was passed to the :meth:`~Pattern.search` or
1547 :meth:`~Pattern.match` method of a :ref:`regex object <re-objects>`. This is
1548 the index into the string at which the RE engine started looking for a match.
1549
1550
1551 .. attribute:: Match.endpos
1552
1553 The value of *endpos* which was passed to the :meth:`~Pattern.search` or
1554 :meth:`~Pattern.match` method of a :ref:`regex object <re-objects>`. This is
1555 the index into the string beyond which the RE engine will not go.
1556
1557
1558 .. attribute:: Match.lastindex
1559
1560 The integer index of the last matched capturing group, or ``None`` if no group
1561 was matched at all. For example, the expressions ``(a)b``, ``((a)(b))``, and
1562 ``((ab))`` will have ``lastindex == 1`` if applied to the string ``'ab'``, while
1563 the expression ``(a)(b)`` will have ``lastindex == 2``, if applied to the same
1564 string.
1565
1566
1567 .. attribute:: Match.lastgroup
1568
1569 The name of the last matched capturing group, or ``None`` if the group didn't
1570 have a name, or if no group was matched at all.
1571
1572
1573 .. attribute:: Match.re
1574
1575 The :ref:`regular expression object <re-objects>` whose :meth:`~Pattern.match` or
1576 :meth:`~Pattern.search` method produced this match instance.
1577
1578
1579 .. attribute:: Match.string
1580
1581 The string passed to :meth:`~Pattern.match` or :meth:`~Pattern.search`.
1582
1583
1584 .. versionchanged:: 3.7
1585 Added support of :func:`copy.copy` and :func:`copy.deepcopy`. Match objects
1586 are considered atomic.
1587
1588
1589 .. _re-examples:
1590
1591 Regular Expression Examples
1592 ---------------------------
1593
1594
1595 Checking for a Pair
1596 ^^^^^^^^^^^^^^^^^^^
1597
1598 In this example, we'll use the following helper function to display match
1599 objects a little more gracefully::
1600
1601 def displaymatch(match):
1602 if match is None:
1603 return None
1604 return '<Match: %r, groups=%r>' % (match.group(), match.groups())
1605
1606 Suppose you are writing a poker program where a player's hand is represented as
1607 a 5-character string with each character representing a card, "a" for ace, "k"
1608 for king, "q" for queen, "j" for jack, "t" for 10, and "2" through "9"
1609 representing the card with that value.
1610
1611 To see if a given string is a valid hand, one could do the following::
1612
1613 >>> valid = re.compile(r"^[a2-9tjqk]{5}$")
1614 >>> displaymatch(valid.match("akt5q")) # Valid.
1615 "<Match: 'akt5q', groups=()>"
1616 >>> displaymatch(valid.match("akt5e")) # Invalid.
1617 >>> displaymatch(valid.match("akt")) # Invalid.
1618 >>> displaymatch(valid.match("727ak")) # Valid.
1619 "<Match: '727ak', groups=()>"
1620
1621 That last hand, ``"727ak"``, contained a pair, or two of the same valued cards.
1622 To match this with a regular expression, one could use backreferences as such::
1623
1624 >>> pair = re.compile(r".*(.).*\1")
1625 >>> displaymatch(pair.match("717ak")) # Pair of 7s.
1626 "<Match: '717', groups=('7',)>"
1627 >>> displaymatch(pair.match("718ak")) # No pairs.
1628 >>> displaymatch(pair.match("354aa")) # Pair of aces.
1629 "<Match: '354aa', groups=('a',)>"
1630
1631 To find out what card the pair consists of, one could use the
1632 :meth:`~Match.group` method of the match object in the following manner::
1633
1634 >>> pair = re.compile(r".*(.).*\1")
1635 >>> pair.match("717ak").group(1)
1636 '7'
1637
1638 # Error because re.match() returns None, which doesn't have a group() method:
1639 >>> pair.match("718ak").group(1)
1640 Traceback (most recent call last):
1641 File "<pyshell#23>", line 1, in <module>
1642 re.match(r".*(.).*\1", "718ak").group(1)
1643 AttributeError: 'NoneType' object has no attribute 'group'
1644
1645 >>> pair.match("354aa").group(1)
1646 'a'
1647
1648
1649 Simulating scanf()
1650 ^^^^^^^^^^^^^^^^^^
1651
1652 .. index:: single: scanf (C function)
1653
1654 Python does not currently have an equivalent to :c:func:`!scanf`. Regular
1655 expressions are generally more powerful, though also more verbose, than
1656 :c:func:`!scanf` format strings. The table below offers some more-or-less
1657 equivalent mappings between :c:func:`!scanf` format tokens and regular
1658 expressions.
1659
1660 +--------------------------------+---------------------------------------------+
1661 | :c:func:`!scanf` Token | Regular Expression |
1662 +================================+=============================================+
1663 | ``%c`` | ``.`` |
1664 +--------------------------------+---------------------------------------------+
1665 | ``%5c`` | ``.{5}`` |
1666 +--------------------------------+---------------------------------------------+
1667 | ``%d`` | ``[-+]?\d+`` |
1668 +--------------------------------+---------------------------------------------+
1669 | ``%e``, ``%E``, ``%f``, ``%g`` | ``[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?`` |
1670 +--------------------------------+---------------------------------------------+
1671 | ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` |
1672 +--------------------------------+---------------------------------------------+
1673 | ``%o`` | ``[-+]?[0-7]+`` |
1674 +--------------------------------+---------------------------------------------+
1675 | ``%s`` | ``\S+`` |
1676 +--------------------------------+---------------------------------------------+
1677 | ``%u`` | ``\d+`` |
1678 +--------------------------------+---------------------------------------------+
1679 | ``%x``, ``%X`` | ``[-+]?(0[xX])?[\dA-Fa-f]+`` |
1680 +--------------------------------+---------------------------------------------+
1681
1682 To extract the filename and numbers from a string like ::
1683
1684 /usr/sbin/sendmail - 0 errors, 4 warnings
1685
1686 you would use a :c:func:`!scanf` format like ::
1687
1688 %s - %d errors, %d warnings
1689
1690 The equivalent regular expression would be ::
1691
1692 (\S+) - (\d+) errors, (\d+) warnings
1693
1694
1695 .. _search-vs-match:
1696
1697 search() vs. match()
1698 ^^^^^^^^^^^^^^^^^^^^
1699
1700 .. sectionauthor:: Fred L. Drake, Jr. <[email protected]>
1701
1702 Python offers different primitive operations based on regular expressions:
1703
1704 + :func:`re.match` checks for a match only at the beginning of the string
1705 + :func:`re.search` checks for a match anywhere in the string
1706 (this is what Perl does by default)
1707 + :func:`re.fullmatch` checks for entire string to be a match
1708
1709
1710 For example::
1711
1712 >>> re.match("c", "abcdef") # No match
1713 >>> re.search("c", "abcdef") # Match
1714 <re.Match object; span=(2, 3), match='c'>
1715 >>> re.fullmatch("p.*n", "python") # Match
1716 <re.Match object; span=(0, 6), match='python'>
1717 >>> re.fullmatch("r.*n", "python") # No match
1718
1719 Regular expressions beginning with ``'^'`` can be used with :func:`search` to
1720 restrict the match at the beginning of the string::
1721
1722 >>> re.match("c", "abcdef") # No match
1723 >>> re.search("^c", "abcdef") # No match
1724 >>> re.search("^a", "abcdef") # Match
1725 <re.Match object; span=(0, 1), match='a'>
1726
1727 Note however that in :const:`MULTILINE` mode :func:`match` only matches at the
1728 beginning of the string, whereas using :func:`search` with a regular expression
1729 beginning with ``'^'`` will match at the beginning of each line. ::
1730
1731 >>> re.match("X", "A\nB\nX", re.MULTILINE) # No match
1732 >>> re.search("^X", "A\nB\nX", re.MULTILINE) # Match
1733 <re.Match object; span=(4, 5), match='X'>
1734
1735
1736 Making a Phonebook
1737 ^^^^^^^^^^^^^^^^^^
1738
1739 :func:`split` splits a string into a list delimited by the passed pattern. The
1740 method is invaluable for converting textual data into data structures that can be
1741 easily read and modified by Python as demonstrated in the following example that
1742 creates a phonebook.
1743
1744 First, here is the input. Normally it may come from a file, here we are using
1745 triple-quoted string syntax
1746
1747 .. doctest::
1748
1749 >>> text = """Ross McFluff: 834.345.1254 155 Elm Street
1750 ...
1751 ... Ronald Heathmore: 892.345.3428 436 Finley Avenue
1752 ... Frank Burger: 925.541.7625 662 South Dogwood Way
1753 ...
1754 ...
1755 ... Heather Albrecht: 548.326.4584 919 Park Place"""
1756
1757 The entries are separated by one or more newlines. Now we convert the string
1758 into a list with each nonempty line having its own entry:
1759
1760 .. doctest::
1761 :options: +NORMALIZE_WHITESPACE
1762
1763 >>> entries = re.split("\n+", text)
1764 >>> entries
1765 ['Ross McFluff: 834.345.1254 155 Elm Street',
1766 'Ronald Heathmore: 892.345.3428 436 Finley Avenue',
1767 'Frank Burger: 925.541.7625 662 South Dogwood Way',
1768 'Heather Albrecht: 548.326.4584 919 Park Place']
1769
1770 Finally, split each entry into a list with first name, last name, telephone
1771 number, and address. We use the ``maxsplit`` parameter of :func:`split`
1772 because the address has spaces, our splitting pattern, in it:
1773
1774 .. doctest::
1775 :options: +NORMALIZE_WHITESPACE
1776
1777 >>> [re.split(":? ", entry, maxsplit=3) for entry in entries]
1778 [['Ross', 'McFluff', '834.345.1254', '155 Elm Street'],
1779 ['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'],
1780 ['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'],
1781 ['Heather', 'Albrecht', '548.326.4584', '919 Park Place']]
1782
1783 The ``:?`` pattern matches the colon after the last name, so that it does not
1784 occur in the result list. With a ``maxsplit`` of ``4``, we could separate the
1785 house number from the street name:
1786
1787 .. doctest::
1788 :options: +NORMALIZE_WHITESPACE
1789
1790 >>> [re.split(":? ", entry, maxsplit=4) for entry in entries]
1791 [['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'],
1792 ['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'],
1793 ['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'],
1794 ['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place']]
1795
1796
1797 Text Munging
1798 ^^^^^^^^^^^^
1799
1800 :func:`sub` replaces every occurrence of a pattern with a string or the
1801 result of a function. This example demonstrates using :func:`sub` with
1802 a function to "munge" text, or randomize the order of all the characters
1803 in each word of a sentence except for the first and last characters::
1804
1805 >>> def repl(m):
1806 ... inner_word = list(m.group(2))
1807 ... random.shuffle(inner_word)
1808 ... return m.group(1) + "".join(inner_word) + m.group(3)
1809 ...
1810 >>> text = "Professor Abdolmalek, please report your absences promptly."
1811 >>> re.sub(r"(\w)(\w+)(\w)", repl, text)
1812 'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.'
1813 >>> re.sub(r"(\w)(\w+)(\w)", repl, text)
1814 'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.'
1815
1816
1817 Finding all Adverbs
1818 ^^^^^^^^^^^^^^^^^^^
1819
1820 :func:`findall` matches *all* occurrences of a pattern, not just the first
1821 one as :func:`search` does. For example, if a writer wanted to
1822 find all of the adverbs in some text, they might use :func:`findall` in
1823 the following manner::
1824
1825 >>> text = "He was carefully disguised but captured quickly by police."
1826 >>> re.findall(r"\w+ly\b", text)
1827 ['carefully', 'quickly']
1828
1829
1830 Finding all Adverbs and their Positions
1831 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1832
1833 If one wants more information about all matches of a pattern than the matched
1834 text, :func:`finditer` is useful as it provides :class:`~re.Match` objects
1835 instead of strings. Continuing with the previous example, if a writer wanted
1836 to find all of the adverbs *and their positions* in some text, they would use
1837 :func:`finditer` in the following manner::
1838
1839 >>> text = "He was carefully disguised but captured quickly by police."
1840 >>> for m in re.finditer(r"\w+ly\b", text):
1841 ... print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0)))
1842 07-16: carefully
1843 40-47: quickly
1844
1845
1846 Raw String Notation
1847 ^^^^^^^^^^^^^^^^^^^
1848
1849 Raw string notation (``r"text"``) keeps regular expressions sane. Without it,
1850 every backslash (``'\'``) in a regular expression would have to be prefixed with
1851 another one to escape it. For example, the two following lines of code are
1852 functionally identical::
1853
1854 >>> re.match(r"\W(.)\1\W", " ff ")
1855 <re.Match object; span=(0, 4), match=' ff '>
1856 >>> re.match("\\W(.)\\1\\W", " ff ")
1857 <re.Match object; span=(0, 4), match=' ff '>
1858
1859 When one wants to match a literal backslash, it must be escaped in the regular
1860 expression. With raw string notation, this means ``r"\\"``. Without raw string
1861 notation, one must use ``"\\\\"``, making the following lines of code
1862 functionally identical::
1863
1864 >>> re.match(r"\\", r"\\")
1865 <re.Match object; span=(0, 1), match='\\'>
1866 >>> re.match("\\\\", r"\\")
1867 <re.Match object; span=(0, 1), match='\\'>
1868
1869
1870 Writing a Tokenizer
1871 ^^^^^^^^^^^^^^^^^^^
1872
1873 A `tokenizer or scanner <https://en.wikipedia.org/wiki/Lexical_analysis>`_
1874 analyzes a string to categorize groups of characters. This is a useful first
1875 step in writing a compiler or interpreter.
1876
1877 The text categories are specified with regular expressions. The technique is
1878 to combine those into a single master regular expression and to loop over
1879 successive matches::
1880
1881 from typing import NamedTuple
1882 import re
1883
1884 class Token(NamedTuple):
1885 type: str
1886 value: int | float | str
1887 line: int
1888 column: int
1889
1890 def tokenize(code):
1891 keywords = {'IF', 'THEN', 'ENDIF', 'FOR', 'NEXT', 'GOSUB', 'RETURN'}
1892 token_specification = [
1893 ('NUMBER', r'\d+(\.\d*)?'), # Integer or decimal number
1894 ('ASSIGN', r':='), # Assignment operator
1895 ('END', r';'), # Statement terminator
1896 ('ID', r'[A-Za-z]+'), # Identifiers
1897 ('OP', r'[+\-*/]'), # Arithmetic operators
1898 ('NEWLINE', r'\n'), # Line endings
1899 ('SKIP', r'[ \t]+'), # Skip over spaces and tabs
1900 ('MISMATCH', r'.'), # Any other character
1901 ]
1902 tok_regex = '|'.join('(?P<%s>%s)' % pair for pair in token_specification)
1903 line_num = 1
1904 line_start = 0
1905 for mo in re.finditer(tok_regex, code):
1906 kind = mo.lastgroup
1907 value = mo.group()
1908 column = mo.start() - line_start
1909 if kind == 'NUMBER':
1910 value = float(value) if '.' in value else int(value)
1911 elif kind == 'ID' and value in keywords:
1912 kind = value
1913 elif kind == 'NEWLINE':
1914 line_start = mo.end()
1915 line_num += 1
1916 continue
1917 elif kind == 'SKIP':
1918 continue
1919 elif kind == 'MISMATCH':
1920 raise RuntimeError(f'{value!r} unexpected on line {line_num}')
1921 yield Token(kind, value, line_num, column)
1922
1923 statements = '''
1924 IF quantity THEN
1925 total := total + price * quantity;
1926 tax := price * 0.05;
1927 ENDIF;
1928 '''
1929
1930 for token in tokenize(statements):
1931 print(token)
1932
1933 The tokenizer produces the following output::
1934
1935 Token(type='IF', value='IF', line=2, column=4)
1936 Token(type='ID', value='quantity', line=2, column=7)
1937 Token(type='THEN', value='THEN', line=2, column=16)
1938 Token(type='ID', value='total', line=3, column=8)
1939 Token(type='ASSIGN', value=':=', line=3, column=14)
1940 Token(type='ID', value='total', line=3, column=17)
1941 Token(type='OP', value='+', line=3, column=23)
1942 Token(type='ID', value='price', line=3, column=25)
1943 Token(type='OP', value='*', line=3, column=31)
1944 Token(type='ID', value='quantity', line=3, column=33)
1945 Token(type='END', value=';', line=3, column=41)
1946 Token(type='ID', value='tax', line=4, column=8)
1947 Token(type='ASSIGN', value=':=', line=4, column=12)
1948 Token(type='ID', value='price', line=4, column=15)
1949 Token(type='OP', value='*', line=4, column=21)
1950 Token(type='NUMBER', value=0.05, line=4, column=23)
1951 Token(type='END', value=';', line=4, column=27)
1952 Token(type='ENDIF', value='ENDIF', line=5, column=4)
1953 Token(type='END', value=';', line=5, column=9)
1954
1955
1956 .. [Frie09] Friedl, Jeffrey. Mastering Regular Expressions. 3rd ed., O'Reilly
1957 Media, 2009. The third edition of the book no longer covers Python at all,
1958 but the first edition covered writing good regular expression patterns in
1959 great detail.