| 1 |
|
| 2 |
.. _introduction: |
| 3 |
|
| 4 |
************ |
| 5 |
Introduction |
| 6 |
************ |
| 7 |
|
| 8 |
This reference manual describes the Python programming language. It is not |
| 9 |
intended as a tutorial. |
| 10 |
|
| 11 |
While I am trying to be as precise as possible, I chose to use English rather |
| 12 |
than formal specifications for everything except syntax and lexical analysis. |
| 13 |
This should make the document more understandable to the average reader, but |
| 14 |
will leave room for ambiguities. Consequently, if you were coming from Mars and |
| 15 |
tried to re-implement Python from this document alone, you might have to guess |
| 16 |
things and in fact you would probably end up implementing quite a different |
| 17 |
language. On the other hand, if you are using Python and wonder what the precise |
| 18 |
rules about a particular area of the language are, you should definitely be able |
| 19 |
to find them here. If you would like to see a more formal definition of the |
| 20 |
language, maybe you could volunteer your time --- or invent a cloning machine |
| 21 |
:-). |
| 22 |
|
| 23 |
It is dangerous to add too many implementation details to a language reference |
| 24 |
document --- the implementation may change, and other implementations of the |
| 25 |
same language may work differently. On the other hand, CPython is the one |
| 26 |
Python implementation in widespread use (although alternate implementations |
| 27 |
continue to gain support), and its particular quirks are sometimes worth being |
| 28 |
mentioned, especially where the implementation imposes additional limitations. |
| 29 |
Therefore, you'll find short "implementation notes" sprinkled throughout the |
| 30 |
text. |
| 31 |
|
| 32 |
Every Python implementation comes with a number of built-in and standard |
| 33 |
modules. These are documented in :ref:`library-index`. A few built-in modules |
| 34 |
are mentioned when they interact in a significant way with the language |
| 35 |
definition. |
| 36 |
|
| 37 |
|
| 38 |
.. _implementations: |
| 39 |
|
| 40 |
Alternate Implementations |
| 41 |
========================= |
| 42 |
|
| 43 |
Though there is one Python implementation which is by far the most popular, |
| 44 |
there are some alternate implementations which are of particular interest to |
| 45 |
different audiences. |
| 46 |
|
| 47 |
Known implementations include: |
| 48 |
|
| 49 |
CPython |
| 50 |
This is the original and most-maintained implementation of Python, written in C. |
| 51 |
New language features generally appear here first. |
| 52 |
|
| 53 |
Jython |
| 54 |
Python implemented in Java. This implementation can be used as a scripting |
| 55 |
language for Java applications, or can be used to create applications using the |
| 56 |
Java class libraries. It is also often used to create tests for Java libraries. |
| 57 |
More information can be found at `the Jython website <https://www.jython.org/>`_. |
| 58 |
|
| 59 |
Python for .NET |
| 60 |
This implementation actually uses the CPython implementation, but is a managed |
| 61 |
.NET application and makes .NET libraries available. It was created by Brian |
| 62 |
Lloyd. For more information, see the `Python for .NET home page |
| 63 |
<https://pythonnet.github.io/>`_. |
| 64 |
|
| 65 |
IronPython |
| 66 |
An alternate Python for .NET. Unlike Python.NET, this is a complete Python |
| 67 |
implementation that generates IL, and compiles Python code directly to .NET |
| 68 |
assemblies. It was created by Jim Hugunin, the original creator of Jython. For |
| 69 |
more information, see `the IronPython website <https://ironpython.net/>`_. |
| 70 |
|
| 71 |
PyPy |
| 72 |
An implementation of Python written completely in Python. It supports several |
| 73 |
advanced features not found in other implementations like stackless support |
| 74 |
and a Just in Time compiler. One of the goals of the project is to encourage |
| 75 |
experimentation with the language itself by making it easier to modify the |
| 76 |
interpreter (since it is written in Python). Additional information is |
| 77 |
available on `the PyPy project's home page <https://pypy.org/>`_. |
| 78 |
|
| 79 |
Each of these implementations varies in some way from the language as documented |
| 80 |
in this manual, or introduces specific information beyond what's covered in the |
| 81 |
standard Python documentation. Please refer to the implementation-specific |
| 82 |
documentation to determine what else you need to know about the specific |
| 83 |
implementation you're using. |
| 84 |
|
| 85 |
|
| 86 |
.. _notation: |
| 87 |
|
| 88 |
Notation |
| 89 |
======== |
| 90 |
|
| 91 |
.. index:: BNF, grammar, syntax, notation |
| 92 |
|
| 93 |
The descriptions of lexical analysis and syntax use a grammar notation that |
| 94 |
is a mixture of |
| 95 |
`EBNF <https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form>`_ |
| 96 |
and `PEG <https://en.wikipedia.org/wiki/Parsing_expression_grammar>`_. |
| 97 |
For example: |
| 98 |
|
| 99 |
.. grammar-snippet:: |
| 100 |
:group: notation |
| 101 |
|
| 102 |
name: `letter` (`letter` | `digit` | "_")* |
| 103 |
letter: "a"..."z" | "A"..."Z" |
| 104 |
digit: "0"..."9" |
| 105 |
|
| 106 |
In this example, the first line says that a ``name`` is a ``letter`` followed |
| 107 |
by a sequence of zero or more ``letter``\ s, ``digit``\ s, and underscores. |
| 108 |
A ``letter`` in turn is any of the single characters ``'a'`` through |
| 109 |
``'z'`` and ``A`` through ``Z``; a ``digit`` is a single character from ``0`` |
| 110 |
to ``9``. |
| 111 |
|
| 112 |
Each rule begins with a name (which identifies the rule that's being defined) |
| 113 |
followed by a colon, ``:``. |
| 114 |
The definition to the right of the colon uses the following syntax elements: |
| 115 |
|
| 116 |
* ``name``: A name refers to another rule. |
| 117 |
Where possible, it is a link to the rule's definition. |
| 118 |
|
| 119 |
* ``TOKEN``: An uppercase name refers to a :term:`token`. |
| 120 |
For the purposes of grammar definitions, tokens are the same as rules. |
| 121 |
|
| 122 |
* ``"text"``, ``'text'``: Text in single or double quotes must match literally |
| 123 |
(without the quotes). The type of quote is chosen according to the meaning |
| 124 |
of ``text``: |
| 125 |
|
| 126 |
* ``'if'``: A name in single quotes denotes a :ref:`keyword <keywords>`. |
| 127 |
* ``"case"``: A name in double quotes denotes a |
| 128 |
:ref:`soft-keyword <soft-keywords>`. |
| 129 |
* ``'@'``: A non-letter symbol in single quotes denotes an |
| 130 |
:py:data:`~token.OP` token, that is, a :ref:`delimiter <delimiters>` or |
| 131 |
:ref:`operator <operators>`. |
| 132 |
|
| 133 |
* ``e1 e2``: Items separated only by whitespace denote a sequence. |
| 134 |
Here, ``e1`` must be followed by ``e2``. |
| 135 |
* ``e1 | e2``: A vertical bar is used to separate alternatives. |
| 136 |
It denotes PEG's "ordered choice": if ``e1`` matches, ``e2`` is |
| 137 |
not considered. |
| 138 |
In traditional PEG grammars, this is written as a slash, ``/``, rather than |
| 139 |
a vertical bar. |
| 140 |
See :pep:`617` for more background and details. |
| 141 |
* ``e*``: A star means zero or more repetitions of the preceding item. |
| 142 |
* ``e+``: Likewise, a plus means one or more repetitions. |
| 143 |
* ``[e]``: A phrase enclosed in square brackets means zero or |
| 144 |
one occurrences. In other words, the enclosed phrase is optional. |
| 145 |
* ``e?``: A question mark has exactly the same meaning as square brackets: |
| 146 |
the preceding item is optional. |
| 147 |
* ``(e)``: Parentheses are used for grouping. |
| 148 |
|
| 149 |
The following notation is only used in |
| 150 |
:ref:`lexical definitions <notation-lexical-vs-syntactic>`. |
| 151 |
|
| 152 |
* ``"a"..."z"``: Two literal characters separated by three dots mean a choice |
| 153 |
of any single character in the given (inclusive) range of ASCII characters. |
| 154 |
* ``<...>``: A phrase between angular brackets gives an informal description |
| 155 |
of the matched symbol (for example, ``<any ASCII character except "\">``), |
| 156 |
or an abbreviation that is defined in nearby text (for example, ``<Lu>``). |
| 157 |
|
| 158 |
.. _lexical-lookaheads: |
| 159 |
|
| 160 |
Some definitions also use *lookaheads*, which indicate that an element |
| 161 |
must (or must not) match at a given position, but without consuming any input: |
| 162 |
|
| 163 |
* ``&e``: a positive lookahead (that is, ``e`` is required to match) |
| 164 |
* ``!e``: a negative lookahead (that is, ``e`` is required *not* to match) |
| 165 |
|
| 166 |
The unary operators (``*``, ``+``, ``?``) bind as tightly as possible; |
| 167 |
the vertical bar (``|``) binds most loosely. |
| 168 |
|
| 169 |
White space is only meaningful to separate tokens. |
| 170 |
|
| 171 |
Rules are normally contained on a single line, but rules that are too long |
| 172 |
may be wrapped: |
| 173 |
|
| 174 |
.. grammar-snippet:: |
| 175 |
:group: notation |
| 176 |
|
| 177 |
literal: stringliteral | bytesliteral |
| 178 |
| integer | floatnumber | imagnumber |
| 179 |
|
| 180 |
Alternatively, rules may be formatted with the first line ending at the colon, |
| 181 |
and each alternative beginning with a vertical bar on a new line. |
| 182 |
For example: |
| 183 |
|
| 184 |
|
| 185 |
.. grammar-snippet:: |
| 186 |
:group: notation-alt |
| 187 |
|
| 188 |
literal: |
| 189 |
| stringliteral |
| 190 |
| bytesliteral |
| 191 |
| integer |
| 192 |
| floatnumber |
| 193 |
| imagnumber |
| 194 |
|
| 195 |
This does *not* mean that there is an empty first alternative. |
| 196 |
|
| 197 |
.. index:: lexical definitions |
| 198 |
|
| 199 |
.. _notation-lexical-vs-syntactic: |
| 200 |
|
| 201 |
Lexical and Syntactic definitions |
| 202 |
--------------------------------- |
| 203 |
|
| 204 |
There is some difference between *lexical* and *syntactic* analysis: |
| 205 |
the :term:`lexical analyzer` operates on the individual characters of the |
| 206 |
input source, while the *parser* (syntactic analyzer) operates on the stream |
| 207 |
of :term:`tokens <token>` generated by the lexical analysis. |
| 208 |
However, in some cases the exact boundary between the two phases is a |
| 209 |
CPython implementation detail. |
| 210 |
|
| 211 |
The practical difference between the two is that in *lexical* definitions, |
| 212 |
all whitespace is significant. |
| 213 |
The lexical analyzer :ref:`discards <whitespace>` all whitespace that is not |
| 214 |
converted to tokens like :data:`token.INDENT` or :data:`~token.NEWLINE`. |
| 215 |
*Syntactic* definitions then use these tokens, rather than source characters. |
| 216 |
|
| 217 |
This documentation uses the same BNF grammar for both styles of definitions. |
| 218 |
All uses of BNF in the next chapter (:ref:`lexical`) are lexical definitions; |
| 219 |
uses in subsequent chapters are syntactic definitions. |