| 1 |
|
| 2 |
.. _lexical: |
| 3 |
|
| 4 |
**************** |
| 5 |
Lexical analysis |
| 6 |
**************** |
| 7 |
|
| 8 |
.. index:: lexical analysis, parser, token |
| 9 |
|
| 10 |
A Python program is read by a *parser*. Input to the parser is a stream of |
| 11 |
:term:`tokens <token>`, generated by the *lexical analyzer* (also known as |
| 12 |
the *tokenizer*). |
| 13 |
This chapter describes how the lexical analyzer produces these tokens. |
| 14 |
|
| 15 |
The lexical analyzer determines the program text's :ref:`encoding <encodings>` |
| 16 |
(UTF-8 by default), and decodes the text into |
| 17 |
:ref:`source characters <lexical-source-character>`. |
| 18 |
If the text cannot be decoded, a :exc:`SyntaxError` is raised. |
| 19 |
|
| 20 |
Next, the lexical analyzer uses the source characters to generate a stream of tokens. |
| 21 |
The type of a generated token generally depends on the next source character to |
| 22 |
be processed. Similarly, other special behavior of the analyzer depends on |
| 23 |
the first source character that hasn't yet been processed. |
| 24 |
The following table gives a quick summary of these source characters, |
| 25 |
with links to sections that contain more information. |
| 26 |
|
| 27 |
.. list-table:: |
| 28 |
:header-rows: 1 |
| 29 |
|
| 30 |
* - Character |
| 31 |
- Next token (or other relevant documentation) |
| 32 |
|
| 33 |
* - * space |
| 34 |
* tab |
| 35 |
* formfeed |
| 36 |
- * :ref:`Whitespace <whitespace>` |
| 37 |
|
| 38 |
* - * CR, LF |
| 39 |
- * :ref:`New line <line-structure>` |
| 40 |
* :ref:`Indentation <indentation>` |
| 41 |
|
| 42 |
* - * backslash (``\``) |
| 43 |
- * :ref:`Explicit line joining <explicit-joining>` |
| 44 |
* (Also significant in :ref:`string escape sequences <escape-sequences>`) |
| 45 |
|
| 46 |
* - * hash (``#``) |
| 47 |
- * :ref:`Comment <comments>` |
| 48 |
|
| 49 |
* - * quote (``'``, ``"``) |
| 50 |
- * :ref:`String literal <strings>` |
| 51 |
|
| 52 |
* - * ASCII letter (``a``-``z``, ``A``-``Z``) |
| 53 |
* non-ASCII character |
| 54 |
- * :ref:`Name <identifiers>` |
| 55 |
* Prefixed :ref:`string or bytes literal <strings>` |
| 56 |
|
| 57 |
* - * underscore (``_``) |
| 58 |
- * :ref:`Name <identifiers>` |
| 59 |
* (Can also be part of :ref:`numeric literals <numbers>`) |
| 60 |
|
| 61 |
* - * number (``0``-``9``) |
| 62 |
- * :ref:`Numeric literal <numbers>` |
| 63 |
|
| 64 |
* - * dot (``.``) |
| 65 |
- * :ref:`Numeric literal <numbers>` |
| 66 |
* :ref:`Operator <operators>` |
| 67 |
|
| 68 |
* - * question mark (``?``) |
| 69 |
* dollar (``$``) |
| 70 |
* |
| 71 |
.. (the following uses zero-width space characters to render |
| 72 |
.. a literal backquote) |
| 73 |
|
| 74 |
backquote (`````) |
| 75 |
* control character |
| 76 |
- * Error (outside string literals and comments) |
| 77 |
|
| 78 |
* - * other printing character |
| 79 |
- * :ref:`Operator or delimiter <operators>` |
| 80 |
|
| 81 |
* - * end of file |
| 82 |
- * :ref:`End marker <endmarker-token>` |
| 83 |
|
| 84 |
|
| 85 |
.. _line-structure: |
| 86 |
|
| 87 |
Line structure |
| 88 |
============== |
| 89 |
|
| 90 |
.. index:: line structure |
| 91 |
|
| 92 |
A Python program is divided into a number of *logical lines*. |
| 93 |
|
| 94 |
|
| 95 |
.. _logical-lines: |
| 96 |
|
| 97 |
Logical lines |
| 98 |
------------- |
| 99 |
|
| 100 |
.. index:: logical line, physical line, line joining, NEWLINE token |
| 101 |
|
| 102 |
The end of a logical line is represented by the token :data:`~token.NEWLINE`. |
| 103 |
Statements cannot cross logical line boundaries except where :data:`!NEWLINE` |
| 104 |
is allowed by the syntax (e.g., between statements in compound statements). |
| 105 |
A logical line is constructed from one or more *physical lines* by following |
| 106 |
the :ref:`explicit <explicit-joining>` or :ref:`implicit <implicit-joining>` |
| 107 |
*line joining* rules. |
| 108 |
|
| 109 |
|
| 110 |
.. _physical-lines: |
| 111 |
|
| 112 |
Physical lines |
| 113 |
-------------- |
| 114 |
|
| 115 |
A physical line is a sequence of characters terminated by one the following |
| 116 |
end-of-line sequences: |
| 117 |
|
| 118 |
* the Unix form using ASCII LF (linefeed), |
| 119 |
* the Windows form using the ASCII sequence CR LF (return followed by linefeed), |
| 120 |
* the '`Classic Mac OS`__' form using the ASCII CR (return) character. |
| 121 |
|
| 122 |
__ https://en.wikipedia.org/wiki/Classic_Mac_OS |
| 123 |
|
| 124 |
Regardless of platform, each of these sequences is replaced by a single |
| 125 |
ASCII LF (linefeed) character. |
| 126 |
(This is done even inside :ref:`string literals <strings>`.) |
| 127 |
Each line can use any of the sequences; they do not need to be consistent |
| 128 |
within a file. |
| 129 |
|
| 130 |
The end of input also serves as an implicit terminator for the final |
| 131 |
physical line. |
| 132 |
|
| 133 |
Formally: |
| 134 |
|
| 135 |
.. grammar-snippet:: |
| 136 |
:group: python-grammar |
| 137 |
|
| 138 |
newline: <ASCII LF> | <ASCII CR> <ASCII LF> | <ASCII CR> |
| 139 |
|
| 140 |
|
| 141 |
.. _comments: |
| 142 |
|
| 143 |
Comments |
| 144 |
-------- |
| 145 |
|
| 146 |
.. index:: comment, hash character |
| 147 |
single: # (hash); comment |
| 148 |
|
| 149 |
A comment starts with a hash character (``#``) that is not part of a string |
| 150 |
literal, and ends at the end of the physical line. A comment signifies the end |
| 151 |
of the logical line unless the implicit line joining rules are invoked. Comments |
| 152 |
are ignored by the syntax. |
| 153 |
|
| 154 |
|
| 155 |
.. _encodings: |
| 156 |
|
| 157 |
Encoding declarations |
| 158 |
--------------------- |
| 159 |
|
| 160 |
.. index:: source character set, encoding declarations (source file) |
| 161 |
single: # (hash); source encoding declaration |
| 162 |
|
| 163 |
If a comment in the first or second line of the Python script matches the |
| 164 |
regular expression ``coding[=:]\s*([-\w.]+)``, this comment is processed as an |
| 165 |
encoding declaration; the first group of this expression names the encoding of |
| 166 |
the source code file. The encoding declaration must appear on a line of its |
| 167 |
own. If it is the second line, the first line must also be a comment-only line. |
| 168 |
The recommended forms of an encoding expression are :: |
| 169 |
|
| 170 |
# -*- coding: <encoding-name> -*- |
| 171 |
|
| 172 |
which is recognized also by GNU Emacs, and :: |
| 173 |
|
| 174 |
# vim:fileencoding=<encoding-name> |
| 175 |
|
| 176 |
which is recognized by Bram Moolenaar's VIM. |
| 177 |
|
| 178 |
If no encoding declaration is found, the default encoding is UTF-8. If the |
| 179 |
implicit or explicit encoding of a file is UTF-8, an initial UTF-8 byte-order |
| 180 |
mark (``b'\xef\xbb\xbf'``) is ignored rather than being a syntax error. |
| 181 |
|
| 182 |
If an encoding is declared, the encoding name must be recognized by Python |
| 183 |
(see :ref:`standard-encodings`). The |
| 184 |
encoding is used for all lexical analysis, including string literals, comments |
| 185 |
and identifiers. |
| 186 |
|
| 187 |
.. _lexical-source-character: |
| 188 |
|
| 189 |
All lexical analysis, including string literals, comments |
| 190 |
and identifiers, works on Unicode text decoded using the source encoding. |
| 191 |
Any Unicode code point, except the NUL control character, can appear in |
| 192 |
Python source. |
| 193 |
|
| 194 |
.. grammar-snippet:: |
| 195 |
:group: python-grammar |
| 196 |
|
| 197 |
source_character: <any Unicode code point, except NUL> |
| 198 |
|
| 199 |
|
| 200 |
.. _explicit-joining: |
| 201 |
|
| 202 |
Explicit line joining |
| 203 |
--------------------- |
| 204 |
|
| 205 |
.. index:: physical line, line joining, line continuation, backslash character |
| 206 |
|
| 207 |
Two or more physical lines may be joined into logical lines using backslash |
| 208 |
characters (``\``), as follows: when a physical line ends in a backslash that is |
| 209 |
not part of a string literal or comment, it is joined with the following forming |
| 210 |
a single logical line, deleting the backslash and the following end-of-line |
| 211 |
character. For example:: |
| 212 |
|
| 213 |
if 1900 < year < 2100 and 1 <= month <= 12 \ |
| 214 |
and 1 <= day <= 31 and 0 <= hour < 24 \ |
| 215 |
and 0 <= minute < 60 and 0 <= second < 60: # Looks like a valid date |
| 216 |
return 1 |
| 217 |
|
| 218 |
A line ending in a backslash cannot carry a comment. A backslash does not |
| 219 |
continue a comment. A backslash does not continue a token except for string |
| 220 |
literals (i.e., tokens other than string literals cannot be split across |
| 221 |
physical lines using a backslash). A backslash is illegal elsewhere on a line |
| 222 |
outside a string literal. |
| 223 |
|
| 224 |
|
| 225 |
.. _implicit-joining: |
| 226 |
|
| 227 |
Implicit line joining |
| 228 |
--------------------- |
| 229 |
|
| 230 |
Expressions in parentheses, square brackets or curly braces can be split over |
| 231 |
more than one physical line without using backslashes. For example:: |
| 232 |
|
| 233 |
month_names = ['Januari', 'Februari', 'Maart', # These are the |
| 234 |
'April', 'Mei', 'Juni', # Dutch names |
| 235 |
'Juli', 'Augustus', 'September', # for the months |
| 236 |
'Oktober', 'November', 'December'] # of the year |
| 237 |
|
| 238 |
Implicitly continued lines can carry comments. The indentation of the |
| 239 |
continuation lines is not important. Blank continuation lines are allowed. |
| 240 |
There is no NEWLINE token between implicit continuation lines. Implicitly |
| 241 |
continued lines can also occur within triple-quoted strings (see below); in that |
| 242 |
case they cannot carry comments. |
| 243 |
|
| 244 |
|
| 245 |
.. _blank-lines: |
| 246 |
|
| 247 |
Blank lines |
| 248 |
----------- |
| 249 |
|
| 250 |
.. index:: single: blank line |
| 251 |
|
| 252 |
A logical line that contains only spaces, tabs, formfeeds and possibly a |
| 253 |
comment, is ignored (i.e., no :data:`~token.NEWLINE` token is generated). |
| 254 |
During interactive input of statements, handling of a blank line may differ |
| 255 |
depending on the implementation of the read-eval-print loop. |
| 256 |
In the standard interactive interpreter, an entirely blank logical line (that |
| 257 |
is, one containing not even whitespace or a comment) terminates a multi-line |
| 258 |
statement. |
| 259 |
|
| 260 |
|
| 261 |
.. _indentation: |
| 262 |
|
| 263 |
Indentation |
| 264 |
----------- |
| 265 |
|
| 266 |
.. index:: indentation, leading whitespace, space, tab, grouping, statement grouping |
| 267 |
|
| 268 |
Leading whitespace (spaces and tabs) at the beginning of a logical line is used |
| 269 |
to compute the indentation level of the line, which in turn is used to determine |
| 270 |
the grouping of statements. |
| 271 |
|
| 272 |
Tabs are replaced (from left to right) by one to eight spaces such that the |
| 273 |
total number of characters up to and including the replacement is a multiple of |
| 274 |
eight (this is intended to be the same rule as used by Unix). The total number |
| 275 |
of spaces preceding the first non-blank character then determines the line's |
| 276 |
indentation. Indentation cannot be split over multiple physical lines using |
| 277 |
backslashes; the whitespace up to the first backslash determines the |
| 278 |
indentation. |
| 279 |
|
| 280 |
Indentation is rejected as inconsistent if a source file mixes tabs and spaces |
| 281 |
in a way that makes the meaning dependent on the worth of a tab in spaces; a |
| 282 |
:exc:`TabError` is raised in that case. |
| 283 |
|
| 284 |
**Cross-platform compatibility note:** because of the nature of text editors on |
| 285 |
non-UNIX platforms, it is unwise to use a mixture of spaces and tabs for the |
| 286 |
indentation in a single source file. It should also be noted that different |
| 287 |
platforms may explicitly limit the maximum indentation level. |
| 288 |
|
| 289 |
A formfeed character may be present at the start of the line; it will be ignored |
| 290 |
for the indentation calculations above. Formfeed characters occurring elsewhere |
| 291 |
in the leading whitespace have an undefined effect (for instance, they may reset |
| 292 |
the space count to zero). |
| 293 |
|
| 294 |
.. index:: INDENT token, DEDENT token |
| 295 |
|
| 296 |
The indentation levels of consecutive lines are used to generate |
| 297 |
:data:`~token.INDENT` and :data:`~token.DEDENT` tokens, using a stack, |
| 298 |
as follows. |
| 299 |
|
| 300 |
Before the first line of the file is read, a single zero is pushed on the stack; |
| 301 |
this will never be popped off again. The numbers pushed on the stack will |
| 302 |
always be strictly increasing from bottom to top. At the beginning of each |
| 303 |
logical line, the line's indentation level is compared to the top of the stack. |
| 304 |
If it is equal, nothing happens. If it is larger, it is pushed on the stack, and |
| 305 |
one :data:`!INDENT` token is generated. If it is smaller, it *must* be one of the |
| 306 |
numbers occurring on the stack; all numbers on the stack that are larger are |
| 307 |
popped off, and for each number popped off a :data:`!DEDENT` token is generated. |
| 308 |
At the end of the file, a :data:`!DEDENT` token is generated for each number |
| 309 |
remaining on the stack that is larger than zero. |
| 310 |
|
| 311 |
Here is an example of a correctly (though confusingly) indented piece of Python |
| 312 |
code:: |
| 313 |
|
| 314 |
def perm(l): |
| 315 |
# Compute the list of all permutations of l |
| 316 |
if len(l) <= 1: |
| 317 |
return [l] |
| 318 |
r = [] |
| 319 |
for i in range(len(l)): |
| 320 |
s = l[:i] + l[i+1:] |
| 321 |
p = perm(s) |
| 322 |
for x in p: |
| 323 |
r.append(l[i:i+1] + x) |
| 324 |
return r |
| 325 |
|
| 326 |
The following example shows various indentation errors:: |
| 327 |
|
| 328 |
def perm(l): # error: first line indented |
| 329 |
for i in range(len(l)): # error: not indented |
| 330 |
s = l[:i] + l[i+1:] |
| 331 |
p = perm(l[:i] + l[i+1:]) # error: unexpected indent |
| 332 |
for x in p: |
| 333 |
r.append(l[i:i+1] + x) |
| 334 |
return r # error: inconsistent dedent |
| 335 |
|
| 336 |
(Actually, the first three errors are detected by the parser; only the last |
| 337 |
error is found by the lexical analyzer --- the indentation of ``return r`` does |
| 338 |
not match a level popped off the stack.) |
| 339 |
|
| 340 |
|
| 341 |
.. _whitespace: |
| 342 |
|
| 343 |
Whitespace between tokens |
| 344 |
------------------------- |
| 345 |
|
| 346 |
Except at the beginning of a logical line or in string literals, the whitespace |
| 347 |
characters space, tab and formfeed can be used interchangeably to separate |
| 348 |
tokens: |
| 349 |
|
| 350 |
.. grammar-snippet:: |
| 351 |
:group: python-grammar |
| 352 |
|
| 353 |
whitespace: ' ' | tab | formfeed |
| 354 |
|
| 355 |
|
| 356 |
Whitespace is needed between two tokens only if their concatenation |
| 357 |
could otherwise be interpreted as a different token. For example, ``ab`` is one |
| 358 |
token, but ``a b`` is two tokens. However, ``+a`` and ``+ a`` both produce |
| 359 |
two tokens, ``+`` and ``a``, as ``+a`` is not a valid token. |
| 360 |
|
| 361 |
|
| 362 |
.. _endmarker-token: |
| 363 |
|
| 364 |
End marker |
| 365 |
---------- |
| 366 |
|
| 367 |
At the end of non-interactive input, the lexical analyzer generates an |
| 368 |
:data:`~token.ENDMARKER` token. |
| 369 |
|
| 370 |
|
| 371 |
.. _other-tokens: |
| 372 |
|
| 373 |
Other tokens |
| 374 |
============ |
| 375 |
|
| 376 |
Besides :data:`~token.NEWLINE`, :data:`~token.INDENT` and :data:`~token.DEDENT`, |
| 377 |
the following categories of tokens exist: |
| 378 |
*identifiers* and *keywords* (:data:`~token.NAME`), *literals* (such as |
| 379 |
:data:`~token.NUMBER` and :data:`~token.STRING`), and other symbols |
| 380 |
(*operators* and *delimiters*, :data:`~token.OP`). |
| 381 |
Whitespace characters (other than logical line terminators, discussed earlier) |
| 382 |
are not tokens, but serve to delimit tokens. |
| 383 |
Where ambiguity exists, a token comprises the longest possible string that |
| 384 |
forms a legal token, when read from left to right. |
| 385 |
|
| 386 |
|
| 387 |
.. _identifiers: |
| 388 |
|
| 389 |
Names (identifiers and keywords) |
| 390 |
================================ |
| 391 |
|
| 392 |
.. index:: identifier, name |
| 393 |
|
| 394 |
:data:`~token.NAME` tokens represent *identifiers*, *keywords*, and |
| 395 |
*soft keywords*. |
| 396 |
|
| 397 |
Names are composed of the following characters: |
| 398 |
|
| 399 |
* uppercase and lowercase letters (``A-Z`` and ``a-z``), |
| 400 |
* the underscore (``_``), |
| 401 |
* digits (``0`` through ``9``), which cannot appear as the first character, and |
| 402 |
* non-ASCII characters. Valid names may only contain "letter-like" and |
| 403 |
"digit-like" characters; see :ref:`lexical-names-nonascii` for details. |
| 404 |
|
| 405 |
Names must contain at least one character, but have no upper length limit. |
| 406 |
Case is significant. |
| 407 |
|
| 408 |
Formally, names are described by the following lexical definitions: |
| 409 |
|
| 410 |
.. grammar-snippet:: |
| 411 |
:group: python-grammar |
| 412 |
|
| 413 |
NAME: `name_start` `name_continue`* |
| 414 |
name_start: "a"..."z" | "A"..."Z" | "_" | <non-ASCII character> |
| 415 |
name_continue: name_start | "0"..."9" |
| 416 |
identifier: <`NAME`, except keywords> |
| 417 |
|
| 418 |
Note that not all names matched by this grammar are valid; see |
| 419 |
:ref:`lexical-names-nonascii` for details. |
| 420 |
|
| 421 |
|
| 422 |
.. _keywords: |
| 423 |
|
| 424 |
Keywords |
| 425 |
-------- |
| 426 |
|
| 427 |
.. index:: |
| 428 |
single: keyword |
| 429 |
single: reserved word |
| 430 |
|
| 431 |
The following names are used as reserved words, or *keywords* of the |
| 432 |
language, and cannot be used as ordinary identifiers. They must be spelled |
| 433 |
exactly as written here: |
| 434 |
|
| 435 |
.. sourcecode:: text |
| 436 |
|
| 437 |
False await else import pass |
| 438 |
None break except in raise |
| 439 |
True class finally is return |
| 440 |
and continue for lambda try |
| 441 |
as def from nonlocal while |
| 442 |
assert del global not with |
| 443 |
async elif if or yield |
| 444 |
|
| 445 |
|
| 446 |
.. _soft-keywords: |
| 447 |
|
| 448 |
Soft Keywords |
| 449 |
------------- |
| 450 |
|
| 451 |
.. index:: soft keyword, keyword |
| 452 |
|
| 453 |
.. versionadded:: 3.10 |
| 454 |
|
| 455 |
Some names are only reserved under specific contexts. These are known as |
| 456 |
*soft keywords*: |
| 457 |
|
| 458 |
- ``match``, ``case``, and ``_``, when used in the :keyword:`match` statement. |
| 459 |
- ``type``, when used in the :keyword:`type` statement. |
| 460 |
|
| 461 |
These syntactically act as keywords in their specific contexts, |
| 462 |
but this distinction is done at the parser level, not when tokenizing. |
| 463 |
|
| 464 |
As soft keywords, their use in the grammar is possible while still |
| 465 |
preserving compatibility with existing code that uses these names as |
| 466 |
identifier names. |
| 467 |
|
| 468 |
.. versionchanged:: 3.12 |
| 469 |
``type`` is now a soft keyword. |
| 470 |
|
| 471 |
.. index:: |
| 472 |
single: _, identifiers |
| 473 |
single: __, identifiers |
| 474 |
.. _id-classes: |
| 475 |
|
| 476 |
Reserved classes of identifiers |
| 477 |
------------------------------- |
| 478 |
|
| 479 |
Certain classes of identifiers (besides keywords) have special meanings. These |
| 480 |
classes are identified by the patterns of leading and trailing underscore |
| 481 |
characters: |
| 482 |
|
| 483 |
``_*`` |
| 484 |
Not imported by ``from module import *``. |
| 485 |
|
| 486 |
``_`` |
| 487 |
In a ``case`` pattern within a :keyword:`match` statement, ``_`` is a |
| 488 |
:ref:`soft keyword <soft-keywords>` that denotes a |
| 489 |
:ref:`wildcard <wildcard-patterns>`. |
| 490 |
|
| 491 |
Separately, the interactive interpreter makes the result of the last evaluation |
| 492 |
available in the variable ``_``. |
| 493 |
(It is stored in the :mod:`builtins` module, alongside built-in |
| 494 |
functions like ``print``.) |
| 495 |
|
| 496 |
Elsewhere, ``_`` is a regular identifier. It is often used to name |
| 497 |
"special" items, but it is not special to Python itself. |
| 498 |
|
| 499 |
.. note:: |
| 500 |
|
| 501 |
The name ``_`` is often used in conjunction with internationalization; |
| 502 |
refer to the documentation for the :mod:`gettext` module for more |
| 503 |
information on this convention. |
| 504 |
|
| 505 |
It is also commonly used for unused variables. |
| 506 |
|
| 507 |
``__*__`` |
| 508 |
System-defined names, informally known as "dunder" names. These names are |
| 509 |
defined by the interpreter and its implementation (including the standard library). |
| 510 |
Current system names are discussed in the :ref:`specialnames` section and elsewhere. |
| 511 |
More will likely be defined in future versions of Python. *Any* use of ``__*__`` names, |
| 512 |
in any context, that does not follow explicitly documented use, is subject to |
| 513 |
breakage without warning. |
| 514 |
|
| 515 |
``__*`` |
| 516 |
Class-private names. Names in this category, when used within the context of a |
| 517 |
class definition, are re-written to use a mangled form to help avoid name |
| 518 |
clashes between "private" attributes of base and derived classes. See section |
| 519 |
:ref:`atom-identifiers`. |
| 520 |
|
| 521 |
|
| 522 |
.. _lexical-names-nonascii: |
| 523 |
|
| 524 |
Non-ASCII characters in names |
| 525 |
----------------------------- |
| 526 |
|
| 527 |
Names that contain non-ASCII characters need additional normalization |
| 528 |
and validation beyond the rules and grammar explained |
| 529 |
:ref:`above <identifiers>`. |
| 530 |
For example, ``ř_1``, ``蛇``, or ``साँप`` are valid names, but ``r〰2``, |
| 531 |
``€``, or ``🐍`` are not. |
| 532 |
|
| 533 |
This section explains the exact rules. |
| 534 |
|
| 535 |
All names are converted into the `normalization form`_ NFKC while parsing. |
| 536 |
This means that, for example, some typographic variants of characters are |
| 537 |
converted to their "basic" form. For example, ``fiⁿₐˡᵢᶻₐᵗᵢᵒₙ`` normalizes to |
| 538 |
``finalization``, so Python treats them as the same name:: |
| 539 |
|
| 540 |
>>> fiⁿₐˡᵢᶻₐᵗᵢᵒₙ = 3 |
| 541 |
>>> finalization |
| 542 |
3 |
| 543 |
|
| 544 |
.. note:: |
| 545 |
|
| 546 |
Normalization is done at the lexical level only. |
| 547 |
Run-time functions that take names as *strings* generally do not normalize |
| 548 |
their arguments. |
| 549 |
For example, the variable defined above is accessible at run time in the |
| 550 |
:func:`globals` dictionary as ``globals()["finalization"]`` but not |
| 551 |
``globals()["fiⁿₐˡᵢᶻₐᵗᵢᵒₙ"]``. |
| 552 |
|
| 553 |
Similarly to how ASCII-only names must contain only letters, digits and |
| 554 |
the underscore, and cannot start with a digit, a valid name must |
| 555 |
start with a character in the "letter-like" set ``xid_start``, |
| 556 |
and the remaining characters must be in the "letter- and digit-like" set |
| 557 |
``xid_continue``. |
| 558 |
|
| 559 |
These sets are based on the *XID_Start* and *XID_Continue* sets as defined by the |
| 560 |
Unicode standard annex `UAX-31`_. |
| 561 |
Python's ``xid_start`` additionally includes the underscore (``_``). |
| 562 |
Note that Python does not necessarily conform to `UAX-31`_. |
| 563 |
|
| 564 |
A non-normative listing of characters in the *XID_Start* and *XID_Continue* |
| 565 |
sets as defined by Unicode is available in the `DerivedCoreProperties.txt`_ |
| 566 |
file in the Unicode Character Database. |
| 567 |
For reference, the construction rules for the ``xid_*`` sets are given below. |
| 568 |
|
| 569 |
The set ``id_start`` is defined as the union of: |
| 570 |
|
| 571 |
* Unicode category ``<Lu>`` - uppercase letters (includes ``A`` to ``Z``) |
| 572 |
* Unicode category ``<Ll>`` - lowercase letters (includes ``a`` to ``z``) |
| 573 |
* Unicode category ``<Lt>`` - titlecase letters |
| 574 |
* Unicode category ``<Lm>`` - modifier letters |
| 575 |
* Unicode category ``<Lo>`` - other letters |
| 576 |
* Unicode category ``<Nl>`` - letter numbers |
| 577 |
* {``"_"``} - the underscore |
| 578 |
* ``<Other_ID_Start>`` - an explicit set of characters in `PropList.txt`_ |
| 579 |
to support backwards compatibility |
| 580 |
|
| 581 |
The set ``xid_start`` then closes this set under NFKC normalization, by |
| 582 |
removing all characters whose normalization is not of the form |
| 583 |
``id_start id_continue*``. |
| 584 |
|
| 585 |
The set ``id_continue`` is defined as the union of: |
| 586 |
|
| 587 |
* ``id_start`` (see above) |
| 588 |
* Unicode category ``<Nd>`` - decimal numbers (includes ``0`` to ``9``) |
| 589 |
* Unicode category ``<Pc>`` - connector punctuations |
| 590 |
* Unicode category ``<Mn>`` - nonspacing marks |
| 591 |
* Unicode category ``<Mc>`` - spacing combining marks |
| 592 |
* ``<Other_ID_Continue>`` - another explicit set of characters in |
| 593 |
`PropList.txt`_ to support backwards compatibility |
| 594 |
|
| 595 |
Again, ``xid_continue`` closes this set under NFKC normalization. |
| 596 |
|
| 597 |
Unicode categories use the version of the Unicode Character Database as |
| 598 |
included in the :mod:`unicodedata` module. |
| 599 |
|
| 600 |
.. _UAX-31: https://www.unicode.org/reports/tr31/ |
| 601 |
.. _PropList.txt: https://www.unicode.org/Public/16.0.0/ucd/PropList.txt |
| 602 |
.. _DerivedCoreProperties.txt: https://www.unicode.org/Public/16.0.0/ucd/DerivedCoreProperties.txt |
| 603 |
.. _normalization form: https://www.unicode.org/reports/tr15/#Norm_Forms |
| 604 |
|
| 605 |
.. seealso:: |
| 606 |
|
| 607 |
* :pep:`3131` -- Supporting Non-ASCII Identifiers |
| 608 |
* :pep:`672` -- Unicode-related Security Considerations for Python |
| 609 |
|
| 610 |
|
| 611 |
.. _literals: |
| 612 |
|
| 613 |
Literals |
| 614 |
======== |
| 615 |
|
| 616 |
.. index:: literal, constant |
| 617 |
|
| 618 |
Literals are notations for constant values of some built-in types. |
| 619 |
|
| 620 |
In terms of lexical analysis, Python has :ref:`string, bytes <strings>` |
| 621 |
and :ref:`numeric <numbers>` literals. |
| 622 |
|
| 623 |
Other "literals" are lexically denoted using :ref:`keywords <keywords>` |
| 624 |
(``None``, ``True``, ``False``) and the special |
| 625 |
:ref:`ellipsis token <lexical-ellipsis>` (``...``). |
| 626 |
|
| 627 |
|
| 628 |
.. index:: string literal, bytes literal, ASCII |
| 629 |
single: ' (single quote); string literal |
| 630 |
single: " (double quote); string literal |
| 631 |
.. _strings: |
| 632 |
|
| 633 |
String and Bytes literals |
| 634 |
========================= |
| 635 |
|
| 636 |
String literals are text enclosed in single quotes (``'``) or double |
| 637 |
quotes (``"``). For example: |
| 638 |
|
| 639 |
.. code-block:: python |
| 640 |
|
| 641 |
"spam" |
| 642 |
'eggs' |
| 643 |
|
| 644 |
The quote used to start the literal also terminates it, so a string literal |
| 645 |
can only contain the other quote (except with escape sequences, see below). |
| 646 |
For example: |
| 647 |
|
| 648 |
.. code-block:: python |
| 649 |
|
| 650 |
'Say "Hello", please.' |
| 651 |
"Don't do that!" |
| 652 |
|
| 653 |
Except for this limitation, the choice of quote character (``'`` or ``"``) |
| 654 |
does not affect how the literal is parsed. |
| 655 |
|
| 656 |
Inside a string literal, the backslash (``\``) character introduces an |
| 657 |
:dfn:`escape sequence`, which has special meaning depending on the character |
| 658 |
after the backslash. |
| 659 |
For example, ``\"`` denotes the double quote character, and does *not* end |
| 660 |
the string: |
| 661 |
|
| 662 |
.. code-block:: pycon |
| 663 |
|
| 664 |
>>> print("Say \"Hello\" to everyone!") |
| 665 |
Say "Hello" to everyone! |
| 666 |
|
| 667 |
See :ref:`escape sequences <escape-sequences>` below for a full list of such |
| 668 |
sequences, and more details. |
| 669 |
|
| 670 |
|
| 671 |
.. index:: triple-quoted string |
| 672 |
single: """; string literal |
| 673 |
single: '''; string literal |
| 674 |
|
| 675 |
Triple-quoted strings |
| 676 |
--------------------- |
| 677 |
|
| 678 |
Strings can also be enclosed in matching groups of three single or double |
| 679 |
quotes. |
| 680 |
These are generally referred to as :dfn:`triple-quoted strings`:: |
| 681 |
|
| 682 |
"""This is a triple-quoted string.""" |
| 683 |
|
| 684 |
In triple-quoted literals, unescaped quotes are allowed (and are |
| 685 |
retained), except that three unescaped quotes in a row terminate the literal, |
| 686 |
if they are of the same kind (``'`` or ``"``) used at the start:: |
| 687 |
|
| 688 |
"""This string has "quotes" inside.""" |
| 689 |
|
| 690 |
Unescaped newlines are also allowed and retained:: |
| 691 |
|
| 692 |
'''This triple-quoted string |
| 693 |
continues on the next line.''' |
| 694 |
|
| 695 |
|
| 696 |
.. index:: |
| 697 |
single: u'; string literal |
| 698 |
single: u"; string literal |
| 699 |
|
| 700 |
String prefixes |
| 701 |
--------------- |
| 702 |
|
| 703 |
String literals can have an optional :dfn:`prefix` that influences how the |
| 704 |
content of the literal is parsed, for example: |
| 705 |
|
| 706 |
.. code-block:: python |
| 707 |
|
| 708 |
b"data" |
| 709 |
f'{result=}' |
| 710 |
|
| 711 |
The allowed prefixes are: |
| 712 |
|
| 713 |
* ``b``: :ref:`Bytes literal <bytes-literal>` |
| 714 |
* ``r``: :ref:`Raw string <raw-strings>` |
| 715 |
* ``f``: :ref:`Formatted string literal <f-strings>` ("f-string") |
| 716 |
* ``t``: :ref:`Template string literal <t-strings>` ("t-string") |
| 717 |
* ``u``: No effect (allowed for backwards compatibility) |
| 718 |
|
| 719 |
See the linked sections for details on each type. |
| 720 |
|
| 721 |
Prefixes are case-insensitive (for example, '``B``' works the same as '``b``'). |
| 722 |
The '``r``' prefix can be combined with '``f``', '``t``' or '``b``', so '``fr``', |
| 723 |
'``rf``', '``tr``', '``rt``', '``br``', and '``rb``' are also valid prefixes. |
| 724 |
|
| 725 |
.. versionadded:: 3.3 |
| 726 |
The ``'rb'`` prefix of raw bytes literals has been added as a synonym |
| 727 |
of ``'br'``. |
| 728 |
|
| 729 |
Support for the unicode legacy literal (``u'value'``) was reintroduced |
| 730 |
to simplify the maintenance of dual Python 2.x and 3.x codebases. |
| 731 |
See :pep:`414` for more information. |
| 732 |
|
| 733 |
|
| 734 |
Formal grammar |
| 735 |
-------------- |
| 736 |
|
| 737 |
String literals, except :ref:`"f-strings" <f-strings>` and |
| 738 |
:ref:`"t-strings" <t-strings>`, are described by the |
| 739 |
following lexical definitions. |
| 740 |
|
| 741 |
These definitions use :ref:`negative lookaheads <lexical-lookaheads>` (``!``) |
| 742 |
to indicate that an ending quote ends the literal. |
| 743 |
|
| 744 |
.. grammar-snippet:: |
| 745 |
:group: python-grammar |
| 746 |
|
| 747 |
STRING: [`stringprefix`] (`stringcontent`) |
| 748 |
stringprefix: <("r" | "u" | "b" | "br" | "rb"), case-insensitive> |
| 749 |
stringcontent: |
| 750 |
| "'''" ( !"'''" `longstringitem`)* "'''" |
| 751 |
| '"""' ( !'"""' `longstringitem`)* '"""' |
| 752 |
| "'" ( !"'" `stringitem`)* "'" |
| 753 |
| '"' ( !'"' `stringitem`)* '"' |
| 754 |
stringitem: `stringchar` | `stringescapeseq` |
| 755 |
stringchar: <any `source_character`, except backslash and newline> |
| 756 |
longstringitem: `stringitem` | newline |
| 757 |
stringescapeseq: "\" <any `source_character`> |
| 758 |
|
| 759 |
Note that as in all lexical definitions, whitespace is significant. |
| 760 |
In particular, the prefix (if any) must be immediately followed by the starting |
| 761 |
quote. |
| 762 |
|
| 763 |
.. index:: physical line, escape sequence, Standard C, C |
| 764 |
single: \ (backslash); escape sequence |
| 765 |
single: \\; escape sequence |
| 766 |
single: \a; escape sequence |
| 767 |
single: \b; escape sequence |
| 768 |
single: \f; escape sequence |
| 769 |
single: \n; escape sequence |
| 770 |
single: \r; escape sequence |
| 771 |
single: \t; escape sequence |
| 772 |
single: \v; escape sequence |
| 773 |
single: \x; escape sequence |
| 774 |
single: \N; escape sequence |
| 775 |
single: \u; escape sequence |
| 776 |
single: \U; escape sequence |
| 777 |
|
| 778 |
.. _escape-sequences: |
| 779 |
|
| 780 |
Escape sequences |
| 781 |
---------------- |
| 782 |
|
| 783 |
Unless an '``r``' or '``R``' prefix is present, escape sequences in string and |
| 784 |
bytes literals are interpreted according to rules similar to those used by |
| 785 |
Standard C. The recognized escape sequences are: |
| 786 |
|
| 787 |
.. list-table:: |
| 788 |
:widths: auto |
| 789 |
:header-rows: 1 |
| 790 |
|
| 791 |
* * Escape Sequence |
| 792 |
* Meaning |
| 793 |
* * ``\``\ <newline> |
| 794 |
* :ref:`string-escape-ignore` |
| 795 |
* * ``\\`` |
| 796 |
* :ref:`Backslash <string-escape-escaped-char>` |
| 797 |
* * ``\'`` |
| 798 |
* :ref:`Single quote <string-escape-escaped-char>` |
| 799 |
* * ``\"`` |
| 800 |
* :ref:`Double quote <string-escape-escaped-char>` |
| 801 |
* * ``\a`` |
| 802 |
* ASCII Bell (BEL) |
| 803 |
* * ``\b`` |
| 804 |
* ASCII Backspace (BS) |
| 805 |
* * ``\f`` |
| 806 |
* ASCII Formfeed (FF) |
| 807 |
* * ``\n`` |
| 808 |
* ASCII Linefeed (LF) |
| 809 |
* * ``\r`` |
| 810 |
* ASCII Carriage Return (CR) |
| 811 |
* * ``\t`` |
| 812 |
* ASCII Horizontal Tab (TAB) |
| 813 |
* * ``\v`` |
| 814 |
* ASCII Vertical Tab (VT) |
| 815 |
* * :samp:`\\\\{ooo}` |
| 816 |
* :ref:`string-escape-oct` |
| 817 |
* * :samp:`\\x{hh}` |
| 818 |
* :ref:`string-escape-hex` |
| 819 |
* * :samp:`\\N\\{{name}\\}` |
| 820 |
* :ref:`string-escape-named` |
| 821 |
* * :samp:`\\u{xxxx}` |
| 822 |
* :ref:`Hexadecimal Unicode character <string-escape-long-hex>` |
| 823 |
* * :samp:`\\U{xxxxxxxx}` |
| 824 |
* :ref:`Hexadecimal Unicode character <string-escape-long-hex>` |
| 825 |
|
| 826 |
.. _string-escape-ignore: |
| 827 |
|
| 828 |
Ignored end of line |
| 829 |
^^^^^^^^^^^^^^^^^^^ |
| 830 |
|
| 831 |
A backslash can be added at the end of a line to ignore the newline:: |
| 832 |
|
| 833 |
>>> 'This string will not include \ |
| 834 |
... backslashes or newline characters.' |
| 835 |
'This string will not include backslashes or newline characters.' |
| 836 |
|
| 837 |
The same result can be achieved using :ref:`triple-quoted strings <strings>`, |
| 838 |
or parentheses and :ref:`string literal concatenation <string-concatenation>`. |
| 839 |
|
| 840 |
.. _string-escape-escaped-char: |
| 841 |
|
| 842 |
Escaped characters |
| 843 |
^^^^^^^^^^^^^^^^^^ |
| 844 |
|
| 845 |
To include a backslash in a non-:ref:`raw <raw-strings>` Python string |
| 846 |
literal, it must be doubled. The ``\\`` escape sequence denotes a single |
| 847 |
backslash character:: |
| 848 |
|
| 849 |
>>> print('C:\\Program Files') |
| 850 |
C:\Program Files |
| 851 |
|
| 852 |
Similarly, the ``\'`` and ``\"`` sequences denote the single and double |
| 853 |
quote character, respectively:: |
| 854 |
|
| 855 |
>>> print('\' and \"') |
| 856 |
' and " |
| 857 |
|
| 858 |
.. _string-escape-oct: |
| 859 |
|
| 860 |
Octal character |
| 861 |
^^^^^^^^^^^^^^^ |
| 862 |
|
| 863 |
The sequence :samp:`\\\\{ooo}` denotes a *character* with the octal (base 8) |
| 864 |
value *ooo*:: |
| 865 |
|
| 866 |
>>> '\120' |
| 867 |
'P' |
| 868 |
|
| 869 |
Up to three octal digits (0 through 7) are accepted. |
| 870 |
|
| 871 |
In a bytes literal, *character* means a *byte* with the given value. |
| 872 |
In a string literal, it means a Unicode character with the given value. |
| 873 |
|
| 874 |
.. versionchanged:: 3.11 |
| 875 |
Octal escapes with value larger than ``0o377`` (255) produce a |
| 876 |
:exc:`DeprecationWarning`. |
| 877 |
|
| 878 |
.. versionchanged:: 3.12 |
| 879 |
Octal escapes with value larger than ``0o377`` (255) produce a |
| 880 |
:exc:`SyntaxWarning`. |
| 881 |
In a future Python version they will raise a :exc:`SyntaxError`. |
| 882 |
|
| 883 |
.. _string-escape-hex: |
| 884 |
|
| 885 |
Hexadecimal character |
| 886 |
^^^^^^^^^^^^^^^^^^^^^ |
| 887 |
|
| 888 |
The sequence :samp:`\\x{hh}` denotes a *character* with the hex (base 16) |
| 889 |
value *hh*:: |
| 890 |
|
| 891 |
>>> '\x50' |
| 892 |
'P' |
| 893 |
|
| 894 |
Unlike in Standard C, exactly two hex digits are required. |
| 895 |
|
| 896 |
In a bytes literal, *character* means a *byte* with the given value. |
| 897 |
In a string literal, it means a Unicode character with the given value. |
| 898 |
|
| 899 |
.. _string-escape-named: |
| 900 |
|
| 901 |
Named Unicode character |
| 902 |
^^^^^^^^^^^^^^^^^^^^^^^ |
| 903 |
|
| 904 |
The sequence :samp:`\\N\\{{name}\\}` denotes a Unicode character |
| 905 |
with the given *name*:: |
| 906 |
|
| 907 |
>>> '\N{LATIN CAPITAL LETTER P}' |
| 908 |
'P' |
| 909 |
>>> '\N{SNAKE}' |
| 910 |
'🐍' |
| 911 |
|
| 912 |
This sequence cannot appear in :ref:`bytes literals <bytes-literal>`. |
| 913 |
|
| 914 |
.. versionchanged:: 3.3 |
| 915 |
Support for `name aliases <https://www.unicode.org/Public/16.0.0/ucd/NameAliases.txt>`__ |
| 916 |
has been added. |
| 917 |
|
| 918 |
.. _string-escape-long-hex: |
| 919 |
|
| 920 |
Hexadecimal Unicode characters |
| 921 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 922 |
|
| 923 |
These sequences :samp:`\\u{xxxx}` and :samp:`\\U{xxxxxxxx}` denote the |
| 924 |
Unicode character with the given hex (base 16) value. |
| 925 |
Exactly four digits are required for ``\u``; exactly eight digits are |
| 926 |
required for ``\U``. |
| 927 |
The latter can encode any Unicode character. |
| 928 |
|
| 929 |
.. code-block:: pycon |
| 930 |
|
| 931 |
>>> '\u1234' |
| 932 |
'ሴ' |
| 933 |
>>> '\U0001f40d' |
| 934 |
'🐍' |
| 935 |
|
| 936 |
These sequences cannot appear in :ref:`bytes literals <bytes-literal>`. |
| 937 |
|
| 938 |
|
| 939 |
.. index:: unrecognized escape sequence |
| 940 |
|
| 941 |
Unrecognized escape sequences |
| 942 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 943 |
|
| 944 |
Unlike in Standard C, all unrecognized escape sequences are left in the string |
| 945 |
unchanged, that is, *the backslash is left in the result*:: |
| 946 |
|
| 947 |
>>> print('\q') |
| 948 |
\q |
| 949 |
>>> list('\q') |
| 950 |
['\\', 'q'] |
| 951 |
|
| 952 |
Note that for bytes literals, the escape sequences only recognized in string |
| 953 |
literals (``\N...``, ``\u...``, ``\U...``) fall into the category of |
| 954 |
unrecognized escapes. |
| 955 |
|
| 956 |
.. versionchanged:: 3.6 |
| 957 |
Unrecognized escape sequences produce a :exc:`DeprecationWarning`. |
| 958 |
|
| 959 |
.. versionchanged:: 3.12 |
| 960 |
Unrecognized escape sequences produce a :exc:`SyntaxWarning`. |
| 961 |
In a future Python version they will raise a :exc:`SyntaxError`. |
| 962 |
|
| 963 |
|
| 964 |
.. index:: |
| 965 |
single: b'; bytes literal |
| 966 |
single: b"; bytes literal |
| 967 |
|
| 968 |
|
| 969 |
.. _bytes-literal: |
| 970 |
|
| 971 |
Bytes literals |
| 972 |
-------------- |
| 973 |
|
| 974 |
:dfn:`Bytes literals` are always prefixed with '``b``' or '``B``'; they produce an |
| 975 |
instance of the :class:`bytes` type instead of the :class:`str` type. |
| 976 |
They may only contain ASCII characters; bytes with a numeric value of 128 |
| 977 |
or greater must be expressed with escape sequences (typically |
| 978 |
:ref:`string-escape-hex` or :ref:`string-escape-oct`): |
| 979 |
|
| 980 |
.. code-block:: pycon |
| 981 |
|
| 982 |
>>> b'\x89PNG\r\n\x1a\n' |
| 983 |
b'\x89PNG\r\n\x1a\n' |
| 984 |
>>> list(b'\x89PNG\r\n\x1a\n') |
| 985 |
[137, 80, 78, 71, 13, 10, 26, 10] |
| 986 |
|
| 987 |
Similarly, a zero byte must be expressed using an escape sequence (typically |
| 988 |
``\0`` or ``\x00``). |
| 989 |
|
| 990 |
|
| 991 |
.. index:: |
| 992 |
single: r'; raw string literal |
| 993 |
single: r"; raw string literal |
| 994 |
|
| 995 |
.. _raw-strings: |
| 996 |
|
| 997 |
Raw string literals |
| 998 |
------------------- |
| 999 |
|
| 1000 |
Both string and bytes literals may optionally be prefixed with a letter '``r``' |
| 1001 |
or '``R``'; such constructs are called :dfn:`raw string literals` |
| 1002 |
and :dfn:`raw bytes literals` respectively and treat backslashes as |
| 1003 |
literal characters. |
| 1004 |
As a result, in raw string literals, :ref:`escape sequences <escape-sequences>` |
| 1005 |
are not treated specially: |
| 1006 |
|
| 1007 |
.. code-block:: pycon |
| 1008 |
|
| 1009 |
>>> r'\d{4}-\d{2}-\d{2}' |
| 1010 |
'\\d{4}-\\d{2}-\\d{2}' |
| 1011 |
|
| 1012 |
Even in a raw literal, quotes can be escaped with a backslash, but the |
| 1013 |
backslash remains in the result; for example, ``r"\""`` is a valid string |
| 1014 |
literal consisting of two characters: a backslash and a double quote; ``r"\"`` |
| 1015 |
is not a valid string literal (even a raw string cannot end in an odd number of |
| 1016 |
backslashes). Specifically, *a raw literal cannot end in a single backslash* |
| 1017 |
(since the backslash would escape the following quote character). Note also |
| 1018 |
that a single backslash followed by a newline is interpreted as those two |
| 1019 |
characters as part of the literal, *not* as a line continuation. |
| 1020 |
|
| 1021 |
|
| 1022 |
.. index:: |
| 1023 |
single: formatted string literal |
| 1024 |
single: interpolated string literal |
| 1025 |
single: string; formatted literal |
| 1026 |
single: string; interpolated literal |
| 1027 |
single: f-string |
| 1028 |
single: fstring |
| 1029 |
single: f'; formatted string literal |
| 1030 |
single: f"; formatted string literal |
| 1031 |
single: {} (curly brackets); in formatted string literal |
| 1032 |
single: ! (exclamation); in formatted string literal |
| 1033 |
single: : (colon); in formatted string literal |
| 1034 |
single: = (equals); for help in debugging using string literals |
| 1035 |
|
| 1036 |
.. _f-strings: |
| 1037 |
.. _formatted-string-literals: |
| 1038 |
|
| 1039 |
f-strings |
| 1040 |
--------- |
| 1041 |
|
| 1042 |
.. versionadded:: 3.6 |
| 1043 |
.. versionchanged:: 3.7 |
| 1044 |
The :keyword:`await` and :keyword:`async for` can be used in expressions |
| 1045 |
within f-strings. |
| 1046 |
.. versionchanged:: 3.8 |
| 1047 |
Added the debug specifier (``=``) |
| 1048 |
.. versionchanged:: 3.12 |
| 1049 |
Many restrictions on expressions within f-strings have been removed. |
| 1050 |
Notably, nested strings, comments, and backslashes are now permitted. |
| 1051 |
|
| 1052 |
A :dfn:`formatted string literal` or :dfn:`f-string` is a string literal |
| 1053 |
that is prefixed with '``f``' or '``F``'. |
| 1054 |
Unlike other string literals, f-strings do not have a constant value. |
| 1055 |
They may contain *replacement fields* delimited by curly braces ``{}``. |
| 1056 |
Replacement fields contain expressions which are evaluated at run time. |
| 1057 |
For example:: |
| 1058 |
|
| 1059 |
>>> who = 'nobody' |
| 1060 |
>>> nationality = 'Spanish' |
| 1061 |
>>> f'{who.title()} expects the {nationality} Inquisition!' |
| 1062 |
'Nobody expects the Spanish Inquisition!' |
| 1063 |
|
| 1064 |
Any doubled curly braces (``{{`` or ``}}``) outside replacement fields |
| 1065 |
are replaced with the corresponding single curly brace:: |
| 1066 |
|
| 1067 |
>>> print(f'{{...}}') |
| 1068 |
{...} |
| 1069 |
|
| 1070 |
Other characters outside replacement fields are treated like in ordinary |
| 1071 |
string literals. |
| 1072 |
This means that escape sequences are decoded (except when a literal is |
| 1073 |
also marked as a raw string), and newlines are possible in triple-quoted |
| 1074 |
f-strings:: |
| 1075 |
|
| 1076 |
>>> name = 'Galahad' |
| 1077 |
>>> favorite_color = 'blue' |
| 1078 |
>>> print(f'{name}:\t{favorite_color}') |
| 1079 |
Galahad: blue |
| 1080 |
>>> print(rf"C:\Users\{name}") |
| 1081 |
C:\Users\Galahad |
| 1082 |
>>> print(f'''Three shall be the number of the counting |
| 1083 |
... and the number of the counting shall be three.''') |
| 1084 |
Three shall be the number of the counting |
| 1085 |
and the number of the counting shall be three. |
| 1086 |
|
| 1087 |
Expressions in formatted string literals are treated like regular |
| 1088 |
Python expressions. |
| 1089 |
Each expression is evaluated in the context where the formatted string literal |
| 1090 |
appears, in order from left to right. |
| 1091 |
An empty expression is not allowed, and both :keyword:`lambda` and |
| 1092 |
assignment expressions ``:=`` must be surrounded by explicit parentheses:: |
| 1093 |
|
| 1094 |
>>> f'{(half := 1/2)}, {half * 42}' |
| 1095 |
'0.5, 21.0' |
| 1096 |
|
| 1097 |
Reusing the outer f-string quoting type inside a replacement field is |
| 1098 |
permitted:: |
| 1099 |
|
| 1100 |
>>> a = dict(x=2) |
| 1101 |
>>> f"abc {a["x"]} def" |
| 1102 |
'abc 2 def' |
| 1103 |
|
| 1104 |
Backslashes are also allowed in replacement fields and are evaluated the same |
| 1105 |
way as in any other context:: |
| 1106 |
|
| 1107 |
>>> a = ["a", "b", "c"] |
| 1108 |
>>> print(f"List a contains:\n{"\n".join(a)}") |
| 1109 |
List a contains: |
| 1110 |
a |
| 1111 |
b |
| 1112 |
c |
| 1113 |
|
| 1114 |
It is possible to nest f-strings:: |
| 1115 |
|
| 1116 |
>>> name = 'world' |
| 1117 |
>>> f'Repeated:{f' hello {name}' * 3}' |
| 1118 |
'Repeated: hello world hello world hello world' |
| 1119 |
|
| 1120 |
Portable Python programs should not use more than 5 levels of nesting. |
| 1121 |
|
| 1122 |
.. impl-detail:: |
| 1123 |
|
| 1124 |
CPython does not limit nesting of f-strings. |
| 1125 |
|
| 1126 |
Replacement expressions can contain newlines in both single-quoted and |
| 1127 |
triple-quoted f-strings and they can contain comments. |
| 1128 |
Everything that comes after a ``#`` inside a replacement field |
| 1129 |
is a comment (even closing braces and quotes). |
| 1130 |
This means that replacement fields with comments must be closed in a |
| 1131 |
different line: |
| 1132 |
|
| 1133 |
.. code-block:: text |
| 1134 |
|
| 1135 |
>>> a = 2 |
| 1136 |
>>> f"abc{a # This comment }" continues until the end of the line |
| 1137 |
... + 3}" |
| 1138 |
'abc5' |
| 1139 |
|
| 1140 |
After the expression, replacement fields may optionally contain: |
| 1141 |
|
| 1142 |
* a *debug specifier* -- an equal sign (``=``), optionally surrounded by |
| 1143 |
whitespace on one or both sides; |
| 1144 |
* a *conversion specifier* -- ``!s``, ``!r`` or ``!a``; and/or |
| 1145 |
* a *format specifier* prefixed with a colon (``:``). |
| 1146 |
|
| 1147 |
See the :ref:`Standard Library section on f-strings <stdtypes-fstrings>` |
| 1148 |
for details on how these fields are evaluated. |
| 1149 |
|
| 1150 |
As that section explains, *format specifiers* are passed as the second argument |
| 1151 |
to the :func:`format` function to format a replacement field value. |
| 1152 |
For example, they can be used to specify a field width and padding characters |
| 1153 |
using the :ref:`Format Specification Mini-Language <formatspec>`:: |
| 1154 |
|
| 1155 |
>>> number = 14.3 |
| 1156 |
>>> f'{number:20.7f}' |
| 1157 |
' 14.3000000' |
| 1158 |
|
| 1159 |
Top-level format specifiers may include nested replacement fields:: |
| 1160 |
|
| 1161 |
>>> field_size = 20 |
| 1162 |
>>> precision = 7 |
| 1163 |
>>> f'{number:{field_size}.{precision}f}' |
| 1164 |
' 14.3000000' |
| 1165 |
|
| 1166 |
These nested fields may include their own conversion fields and |
| 1167 |
:ref:`format specifiers <formatspec>`:: |
| 1168 |
|
| 1169 |
>>> number = 3 |
| 1170 |
>>> f'{number:{field_size}}' |
| 1171 |
' 3' |
| 1172 |
>>> f'{number:{field_size:05}}' |
| 1173 |
'00000000000000000003' |
| 1174 |
|
| 1175 |
However, these nested fields may not include more deeply nested replacement |
| 1176 |
fields. |
| 1177 |
|
| 1178 |
Formatted string literals cannot be used as :term:`docstrings <docstring>`, |
| 1179 |
even if they do not include expressions:: |
| 1180 |
|
| 1181 |
>>> def foo(): |
| 1182 |
... f"Not a docstring" |
| 1183 |
... |
| 1184 |
>>> print(foo.__doc__) |
| 1185 |
None |
| 1186 |
|
| 1187 |
.. seealso:: |
| 1188 |
|
| 1189 |
* :pep:`498` -- Literal String Interpolation |
| 1190 |
* :pep:`701` -- Syntactic formalization of f-strings |
| 1191 |
* :meth:`str.format`, which uses a related format string mechanism. |
| 1192 |
|
| 1193 |
|
| 1194 |
.. _t-strings: |
| 1195 |
.. _template-string-literals: |
| 1196 |
|
| 1197 |
t-strings |
| 1198 |
--------- |
| 1199 |
|
| 1200 |
.. versionadded:: 3.14 |
| 1201 |
|
| 1202 |
A :dfn:`template string literal` or :dfn:`t-string` is a string literal |
| 1203 |
that is prefixed with '``t``' or '``T``'. |
| 1204 |
These strings follow the same syntax rules as |
| 1205 |
:ref:`formatted string literals <f-strings>`. |
| 1206 |
For differences in evaluation rules, see the |
| 1207 |
:ref:`Standard Library section on t-strings <stdtypes-tstrings>` |
| 1208 |
|
| 1209 |
|
| 1210 |
Formal grammar for f-strings |
| 1211 |
---------------------------- |
| 1212 |
|
| 1213 |
F-strings are handled partly by the :term:`lexical analyzer`, which produces the |
| 1214 |
tokens :py:data:`~token.FSTRING_START`, :py:data:`~token.FSTRING_MIDDLE` |
| 1215 |
and :py:data:`~token.FSTRING_END`, and partly by the parser, which handles |
| 1216 |
expressions in the replacement field. |
| 1217 |
The exact way the work is split is a CPython implementation detail. |
| 1218 |
|
| 1219 |
Correspondingly, the f-string grammar is a mix of |
| 1220 |
:ref:`lexical and syntactic definitions <notation-lexical-vs-syntactic>`. |
| 1221 |
|
| 1222 |
Whitespace is significant in these situations: |
| 1223 |
|
| 1224 |
* There may be no whitespace in :py:data:`~token.FSTRING_START` (between |
| 1225 |
the prefix and quote). |
| 1226 |
* Whitespace in :py:data:`~token.FSTRING_MIDDLE` is part of the literal |
| 1227 |
string contents. |
| 1228 |
* In ``fstring_replacement_field``, if ``f_debug_specifier`` is present, |
| 1229 |
all whitespace after the opening brace until the ``f_debug_specifier``, |
| 1230 |
as well as whitespace immediately following ``f_debug_specifier``, |
| 1231 |
is retained as part of the expression. |
| 1232 |
|
| 1233 |
.. impl-detail:: |
| 1234 |
|
| 1235 |
The expression is not handled in the tokenization phase; it is |
| 1236 |
retrieved from the source code using locations of the ``{`` token |
| 1237 |
and the token after ``=``. |
| 1238 |
|
| 1239 |
|
| 1240 |
The ``FSTRING_MIDDLE`` definition uses |
| 1241 |
:ref:`negative lookaheads <lexical-lookaheads>` (``!``) |
| 1242 |
to indicate special characters (backslash, newline, ``{``, ``}``) and |
| 1243 |
sequences (``f_quote``). |
| 1244 |
|
| 1245 |
.. grammar-snippet:: |
| 1246 |
:group: python-grammar |
| 1247 |
|
| 1248 |
fstring: `FSTRING_START` `fstring_middle`* `FSTRING_END` |
| 1249 |
|
| 1250 |
FSTRING_START: `fstringprefix` ("'" | '"' | "'''" | '"""') |
| 1251 |
FSTRING_END: `f_quote` |
| 1252 |
fstringprefix: <("f" | "fr" | "rf"), case-insensitive> |
| 1253 |
f_debug_specifier: '=' |
| 1254 |
f_quote: <the quote character(s) used in FSTRING_START> |
| 1255 |
|
| 1256 |
fstring_middle: |
| 1257 |
| `fstring_replacement_field` |
| 1258 |
| `FSTRING_MIDDLE` |
| 1259 |
FSTRING_MIDDLE: |
| 1260 |
| (!"\" !`newline` !'{' !'}' !`f_quote`) `source_character` |
| 1261 |
| `stringescapeseq` |
| 1262 |
| "{{" |
| 1263 |
| "}}" |
| 1264 |
| <newline, in triple-quoted f-strings only> |
| 1265 |
fstring_replacement_field: |
| 1266 |
| '{' `f_expression` [`f_debug_specifier`] [`fstring_conversion`] |
| 1267 |
[`fstring_full_format_spec`] '}' |
| 1268 |
fstring_conversion: |
| 1269 |
| "!" ("s" | "r" | "a") |
| 1270 |
fstring_full_format_spec: |
| 1271 |
| ':' `fstring_format_spec`* |
| 1272 |
fstring_format_spec: |
| 1273 |
| `FSTRING_MIDDLE` |
| 1274 |
| `fstring_replacement_field` |
| 1275 |
f_expression: |
| 1276 |
| ','.(`conditional_expression` | "*" `or_expr`)+ [","] |
| 1277 |
| `yield_expression` |
| 1278 |
|
| 1279 |
.. note:: |
| 1280 |
|
| 1281 |
In the above grammar snippet, the ``f_quote`` and ``FSTRING_MIDDLE`` rules |
| 1282 |
are context-sensitive -- they depend on the contents of ``FSTRING_START`` |
| 1283 |
of the nearest enclosing ``fstring``. |
| 1284 |
|
| 1285 |
Constructing a more traditional formal grammar from this template is left |
| 1286 |
as an exercise for the reader. |
| 1287 |
|
| 1288 |
The grammar for t-strings is identical to the one for f-strings, with *t* |
| 1289 |
instead of *f* at the beginning of rule and token names and in the prefix. |
| 1290 |
|
| 1291 |
.. grammar-snippet:: |
| 1292 |
:group: python-grammar |
| 1293 |
|
| 1294 |
tstring: TSTRING_START tstring_middle* TSTRING_END |
| 1295 |
|
| 1296 |
<rest of the t-string grammar is omitted; see above> |
| 1297 |
|
| 1298 |
|
| 1299 |
.. _numbers: |
| 1300 |
|
| 1301 |
Numeric literals |
| 1302 |
================ |
| 1303 |
|
| 1304 |
.. index:: number, numeric literal, integer literal |
| 1305 |
floating-point literal, hexadecimal literal |
| 1306 |
octal literal, binary literal, decimal literal, imaginary literal, complex literal |
| 1307 |
|
| 1308 |
:data:`~token.NUMBER` tokens represent numeric literals, of which there are |
| 1309 |
three types: integers, floating-point numbers, and imaginary numbers. |
| 1310 |
|
| 1311 |
.. grammar-snippet:: |
| 1312 |
:group: python-grammar |
| 1313 |
|
| 1314 |
NUMBER: `integer` | `floatnumber` | `imagnumber` |
| 1315 |
|
| 1316 |
The numeric value of a numeric literal is the same as if it were passed as a |
| 1317 |
string to the :class:`int`, :class:`float` or :class:`complex` class |
| 1318 |
constructor, respectively. |
| 1319 |
Note that not all valid inputs for those constructors are also valid literals. |
| 1320 |
|
| 1321 |
Numeric literals do not include a sign; a phrase like ``-1`` is |
| 1322 |
actually an expression composed of the unary operator '``-``' and the literal |
| 1323 |
``1``. |
| 1324 |
|
| 1325 |
|
| 1326 |
.. index:: |
| 1327 |
single: 0b; integer literal |
| 1328 |
single: 0o; integer literal |
| 1329 |
single: 0x; integer literal |
| 1330 |
single: _ (underscore); in numeric literal |
| 1331 |
|
| 1332 |
.. _integers: |
| 1333 |
|
| 1334 |
Integer literals |
| 1335 |
---------------- |
| 1336 |
|
| 1337 |
Integer literals denote whole numbers. For example:: |
| 1338 |
|
| 1339 |
7 |
| 1340 |
3 |
| 1341 |
2147483647 |
| 1342 |
|
| 1343 |
There is no limit for the length of integer literals apart from what can be |
| 1344 |
stored in available memory:: |
| 1345 |
|
| 1346 |
7922816251426433759354395033679228162514264337593543950336 |
| 1347 |
|
| 1348 |
Underscores can be used to group digits for enhanced readability, |
| 1349 |
and are ignored for determining the numeric value of the literal. |
| 1350 |
For example, the following literals are equivalent:: |
| 1351 |
|
| 1352 |
100_000_000_000 |
| 1353 |
100000000000 |
| 1354 |
1_00_00_00_00_000 |
| 1355 |
|
| 1356 |
Underscores can only occur between digits. |
| 1357 |
For example, ``_123``, ``321_``, and ``123__321`` are *not* valid literals. |
| 1358 |
|
| 1359 |
Integers can be specified in binary (base 2), octal (base 8), or hexadecimal |
| 1360 |
(base 16) using the prefixes ``0b``, ``0o`` and ``0x``, respectively. |
| 1361 |
Hexadecimal digits 10 through 15 are represented by letters ``A``-``F``, |
| 1362 |
case-insensitive. For example:: |
| 1363 |
|
| 1364 |
0b100110111 |
| 1365 |
0b_1110_0101 |
| 1366 |
0o177 |
| 1367 |
0o377 |
| 1368 |
0xdeadbeef |
| 1369 |
0xDead_Beef |
| 1370 |
|
| 1371 |
An underscore can follow the base specifier. |
| 1372 |
For example, ``0x_1f`` is a valid literal, but ``0_x1f`` and ``0x__1f`` are |
| 1373 |
not. |
| 1374 |
|
| 1375 |
Leading zeros in a non-zero decimal number are not allowed. |
| 1376 |
For example, ``0123`` is not a valid literal. |
| 1377 |
This is for disambiguation with C-style octal literals, which Python used |
| 1378 |
before version 3.0. |
| 1379 |
|
| 1380 |
Formally, integer literals are described by the following lexical definitions: |
| 1381 |
|
| 1382 |
.. grammar-snippet:: |
| 1383 |
:group: python-grammar |
| 1384 |
|
| 1385 |
integer: `decinteger` | `bininteger` | `octinteger` | `hexinteger` | `zerointeger` |
| 1386 |
decinteger: `nonzerodigit` (["_"] `digit`)* |
| 1387 |
bininteger: "0" ("b" | "B") (["_"] `bindigit`)+ |
| 1388 |
octinteger: "0" ("o" | "O") (["_"] `octdigit`)+ |
| 1389 |
hexinteger: "0" ("x" | "X") (["_"] `hexdigit`)+ |
| 1390 |
zerointeger: "0"+ (["_"] "0")* |
| 1391 |
nonzerodigit: "1"..."9" |
| 1392 |
digit: "0"..."9" |
| 1393 |
bindigit: "0" | "1" |
| 1394 |
octdigit: "0"..."7" |
| 1395 |
hexdigit: `digit` | "a"..."f" | "A"..."F" |
| 1396 |
|
| 1397 |
.. versionchanged:: 3.6 |
| 1398 |
Underscores are now allowed for grouping purposes in literals. |
| 1399 |
|
| 1400 |
|
| 1401 |
.. index:: |
| 1402 |
single: . (dot); in numeric literal |
| 1403 |
single: e; in numeric literal |
| 1404 |
single: _ (underscore); in numeric literal |
| 1405 |
.. _floating: |
| 1406 |
|
| 1407 |
Floating-point literals |
| 1408 |
----------------------- |
| 1409 |
|
| 1410 |
Floating-point (float) literals, such as ``3.14`` or ``1.5``, denote |
| 1411 |
:ref:`approximations of real numbers <datamodel-float>`. |
| 1412 |
|
| 1413 |
They consist of *integer* and *fraction* parts, each composed of decimal digits. |
| 1414 |
The parts are separated by a decimal point, ``.``:: |
| 1415 |
|
| 1416 |
2.71828 |
| 1417 |
4.0 |
| 1418 |
|
| 1419 |
Unlike in integer literals, leading zeros are allowed. |
| 1420 |
For example, ``077.010`` is legal, and denotes the same number as ``77.01``. |
| 1421 |
|
| 1422 |
As in integer literals, single underscores may occur between digits to help |
| 1423 |
readability:: |
| 1424 |
|
| 1425 |
96_485.332_123 |
| 1426 |
3.14_15_93 |
| 1427 |
|
| 1428 |
Either of these parts, but not both, can be empty. For example:: |
| 1429 |
|
| 1430 |
10. # (equivalent to 10.0) |
| 1431 |
.001 # (equivalent to 0.001) |
| 1432 |
|
| 1433 |
Optionally, the integer and fraction may be followed by an *exponent*: |
| 1434 |
the letter ``e`` or ``E``, followed by an optional sign, ``+`` or ``-``, |
| 1435 |
and a number in the same format as the integer and fraction parts. |
| 1436 |
The ``e`` or ``E`` represents "times ten raised to the power of":: |
| 1437 |
|
| 1438 |
1.0e3 # (represents 1.0×10³, or 1000.0) |
| 1439 |
1.166e-5 # (represents 1.166×10⁻⁵, or 0.00001166) |
| 1440 |
6.02214076e+23 # (represents 6.02214076×10²³, or 602214076000000000000000.) |
| 1441 |
|
| 1442 |
In floats with only integer and exponent parts, the decimal point may be |
| 1443 |
omitted:: |
| 1444 |
|
| 1445 |
1e3 # (equivalent to 1.e3 and 1.0e3) |
| 1446 |
0e0 # (equivalent to 0.) |
| 1447 |
|
| 1448 |
Formally, floating-point literals are described by the following |
| 1449 |
lexical definitions: |
| 1450 |
|
| 1451 |
.. grammar-snippet:: |
| 1452 |
:group: python-grammar |
| 1453 |
|
| 1454 |
floatnumber: |
| 1455 |
| `digitpart` "." [`digitpart`] [`exponent`] |
| 1456 |
| "." `digitpart` [`exponent`] |
| 1457 |
| `digitpart` `exponent` |
| 1458 |
digitpart: `digit` (["_"] `digit`)* |
| 1459 |
exponent: ("e" | "E") ["+" | "-"] `digitpart` |
| 1460 |
|
| 1461 |
.. versionchanged:: 3.6 |
| 1462 |
Underscores are now allowed for grouping purposes in literals. |
| 1463 |
|
| 1464 |
|
| 1465 |
.. index:: |
| 1466 |
single: j; in numeric literal |
| 1467 |
.. _imaginary: |
| 1468 |
|
| 1469 |
Imaginary literals |
| 1470 |
------------------ |
| 1471 |
|
| 1472 |
Python has :ref:`complex number <typesnumeric>` objects, but no complex |
| 1473 |
literals. |
| 1474 |
Instead, *imaginary literals* denote complex numbers with a zero |
| 1475 |
real part. |
| 1476 |
|
| 1477 |
For example, in math, the complex number 3+4.2\ *i* is written |
| 1478 |
as the real number 3 added to the imaginary number 4.2\ *i*. |
| 1479 |
Python uses a similar syntax, except the imaginary unit is written as ``j`` |
| 1480 |
rather than *i*:: |
| 1481 |
|
| 1482 |
3+4.2j |
| 1483 |
|
| 1484 |
This is an expression composed |
| 1485 |
of the :ref:`integer literal <integers>` ``3``, |
| 1486 |
the :ref:`operator <operators>` '``+``', |
| 1487 |
and the :ref:`imaginary literal <imaginary>` ``4.2j``. |
| 1488 |
Since these are three separate tokens, whitespace is allowed between them:: |
| 1489 |
|
| 1490 |
3 + 4.2j |
| 1491 |
|
| 1492 |
No whitespace is allowed *within* each token. |
| 1493 |
In particular, the ``j`` suffix, may not be separated from the number |
| 1494 |
before it. |
| 1495 |
|
| 1496 |
The number before the ``j`` has the same syntax as a floating-point literal. |
| 1497 |
Thus, the following are valid imaginary literals:: |
| 1498 |
|
| 1499 |
4.2j |
| 1500 |
3.14j |
| 1501 |
10.j |
| 1502 |
.001j |
| 1503 |
1e100j |
| 1504 |
3.14e-10j |
| 1505 |
3.14_15_93j |
| 1506 |
|
| 1507 |
Unlike in a floating-point literal the decimal point can be omitted if the |
| 1508 |
imaginary number only has an integer part. |
| 1509 |
The number is still evaluated as a floating-point number, not an integer:: |
| 1510 |
|
| 1511 |
10j |
| 1512 |
0j |
| 1513 |
1000000000000000000000000j # equivalent to 1e+24j |
| 1514 |
|
| 1515 |
The ``j`` suffix is case-insensitive. |
| 1516 |
That means you can use ``J`` instead:: |
| 1517 |
|
| 1518 |
3.14J # equivalent to 3.14j |
| 1519 |
|
| 1520 |
Formally, imaginary literals are described by the following lexical definition: |
| 1521 |
|
| 1522 |
.. grammar-snippet:: |
| 1523 |
:group: python-grammar |
| 1524 |
|
| 1525 |
imagnumber: (`floatnumber` | `digitpart`) ("j" | "J") |
| 1526 |
|
| 1527 |
|
| 1528 |
.. _delimiters: |
| 1529 |
.. _operators: |
| 1530 |
.. _lexical-ellipsis: |
| 1531 |
|
| 1532 |
Operators and delimiters |
| 1533 |
======================== |
| 1534 |
|
| 1535 |
.. index:: |
| 1536 |
single: operators |
| 1537 |
single: delimiters |
| 1538 |
|
| 1539 |
The following grammar defines :dfn:`operator` and :dfn:`delimiter` tokens, |
| 1540 |
that is, the generic :data:`~token.OP` token type. |
| 1541 |
A :ref:`list of these tokens and their names <token_operators_delimiters>` |
| 1542 |
is also available in the :mod:`!token` module documentation. |
| 1543 |
|
| 1544 |
.. grammar-snippet:: |
| 1545 |
:group: python-grammar |
| 1546 |
|
| 1547 |
OP: |
| 1548 |
| assignment_operator |
| 1549 |
| bitwise_operator |
| 1550 |
| comparison_operator |
| 1551 |
| enclosing_delimiter |
| 1552 |
| other_delimiter |
| 1553 |
| arithmetic_operator |
| 1554 |
| "..." |
| 1555 |
| other_op |
| 1556 |
|
| 1557 |
assignment_operator: "+=" | "-=" | "*=" | "**=" | "/=" | "//=" | "%=" | |
| 1558 |
"&=" | "|=" | "^=" | "<<=" | ">>=" | "@=" | ":=" |
| 1559 |
bitwise_operator: "&" | "|" | "^" | "~" | "<<" | ">>" |
| 1560 |
comparison_operator: "<=" | ">=" | "<" | ">" | "==" | "!=" |
| 1561 |
enclosing_delimiter: "(" | ")" | "[" | "]" | "{" | "}" |
| 1562 |
other_delimiter: "," | ":" | "!" | ";" | "=" | "->" |
| 1563 |
arithmetic_operator: "+" | "-" | "**" | "*" | "//" | "/" | "%" |
| 1564 |
other_op: "." | "@" |
| 1565 |
|
| 1566 |
.. note:: |
| 1567 |
|
| 1568 |
Generally, *operators* are used to combine :ref:`expressions <expressions>`, |
| 1569 |
while *delimiters* serve other purposes. |
| 1570 |
However, there is no clear, formal distinction between the two categories. |
| 1571 |
|
| 1572 |
Some tokens can serve as either operators or delimiters, depending on usage. |
| 1573 |
For example, ``*`` is both the multiplication operator and a delimiter used |
| 1574 |
for sequence unpacking, and ``@`` is both the matrix multiplication and |
| 1575 |
a delimiter that introduces decorators. |
| 1576 |
|
| 1577 |
For some tokens, the distinction is unclear. |
| 1578 |
For example, some people consider ``.``, ``(``, and ``)`` to be delimiters, while others |
| 1579 |
see the :py:func:`getattr` operator and the function call operator(s). |
| 1580 |
|
| 1581 |
Some of Python's operators, like ``and``, ``or``, and ``not in``, use |
| 1582 |
:ref:`keyword <keywords>` tokens rather than "symbols" (operator tokens). |
| 1583 |
|
| 1584 |
A sequence of three consecutive periods (``...``) has a special |
| 1585 |
meaning as an :py:data:`Ellipsis` literal. |
| 1586 |
|