Debian Codegraph @

Doc/reference/introduction.rst

raw
1
2 .. _introduction:
3
4 ************
5 Introduction
6 ************
7
8 This reference manual describes the Python programming language. It is not
9 intended as a tutorial.
10
11 While I am trying to be as precise as possible, I chose to use English rather
12 than formal specifications for everything except syntax and lexical analysis.
13 This should make the document more understandable to the average reader, but
14 will leave room for ambiguities. Consequently, if you were coming from Mars and
15 tried to re-implement Python from this document alone, you might have to guess
16 things and in fact you would probably end up implementing quite a different
17 language. On the other hand, if you are using Python and wonder what the precise
18 rules about a particular area of the language are, you should definitely be able
19 to find them here. If you would like to see a more formal definition of the
20 language, maybe you could volunteer your time --- or invent a cloning machine
21 :-).
22
23 It is dangerous to add too many implementation details to a language reference
24 document --- the implementation may change, and other implementations of the
25 same language may work differently. On the other hand, CPython is the one
26 Python implementation in widespread use (although alternate implementations
27 continue to gain support), and its particular quirks are sometimes worth being
28 mentioned, especially where the implementation imposes additional limitations.
29 Therefore, you'll find short "implementation notes" sprinkled throughout the
30 text.
31
32 Every Python implementation comes with a number of built-in and standard
33 modules. These are documented in :ref:`library-index`. A few built-in modules
34 are mentioned when they interact in a significant way with the language
35 definition.
36
37
38 .. _implementations:
39
40 Alternate Implementations
41 =========================
42
43 Though there is one Python implementation which is by far the most popular,
44 there are some alternate implementations which are of particular interest to
45 different audiences.
46
47 Known implementations include:
48
49 CPython
50 This is the original and most-maintained implementation of Python, written in C.
51 New language features generally appear here first.
52
53 Jython
54 Python implemented in Java. This implementation can be used as a scripting
55 language for Java applications, or can be used to create applications using the
56 Java class libraries. It is also often used to create tests for Java libraries.
57 More information can be found at `the Jython website <https://www.jython.org/>`_.
58
59 Python for .NET
60 This implementation actually uses the CPython implementation, but is a managed
61 .NET application and makes .NET libraries available. It was created by Brian
62 Lloyd. For more information, see the `Python for .NET home page
63 <https://pythonnet.github.io/>`_.
64
65 IronPython
66 An alternate Python for .NET. Unlike Python.NET, this is a complete Python
67 implementation that generates IL, and compiles Python code directly to .NET
68 assemblies. It was created by Jim Hugunin, the original creator of Jython. For
69 more information, see `the IronPython website <https://ironpython.net/>`_.
70
71 PyPy
72 An implementation of Python written completely in Python. It supports several
73 advanced features not found in other implementations like stackless support
74 and a Just in Time compiler. One of the goals of the project is to encourage
75 experimentation with the language itself by making it easier to modify the
76 interpreter (since it is written in Python). Additional information is
77 available on `the PyPy project's home page <https://pypy.org/>`_.
78
79 Each of these implementations varies in some way from the language as documented
80 in this manual, or introduces specific information beyond what's covered in the
81 standard Python documentation. Please refer to the implementation-specific
82 documentation to determine what else you need to know about the specific
83 implementation you're using.
84
85
86 .. _notation:
87
88 Notation
89 ========
90
91 .. index:: BNF, grammar, syntax, notation
92
93 The descriptions of lexical analysis and syntax use a grammar notation that
94 is a mixture of
95 `EBNF <https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form>`_
96 and `PEG <https://en.wikipedia.org/wiki/Parsing_expression_grammar>`_.
97 For example:
98
99 .. grammar-snippet::
100 :group: notation
101
102 name: `letter` (`letter` | `digit` | "_")*
103 letter: "a"..."z" | "A"..."Z"
104 digit: "0"..."9"
105
106 In this example, the first line says that a ``name`` is a ``letter`` followed
107 by a sequence of zero or more ``letter``\ s, ``digit``\ s, and underscores.
108 A ``letter`` in turn is any of the single characters ``'a'`` through
109 ``'z'`` and ``A`` through ``Z``; a ``digit`` is a single character from ``0``
110 to ``9``.
111
112 Each rule begins with a name (which identifies the rule that's being defined)
113 followed by a colon, ``:``.
114 The definition to the right of the colon uses the following syntax elements:
115
116 * ``name``: A name refers to another rule.
117 Where possible, it is a link to the rule's definition.
118
119 * ``TOKEN``: An uppercase name refers to a :term:`token`.
120 For the purposes of grammar definitions, tokens are the same as rules.
121
122 * ``"text"``, ``'text'``: Text in single or double quotes must match literally
123 (without the quotes). The type of quote is chosen according to the meaning
124 of ``text``:
125
126 * ``'if'``: A name in single quotes denotes a :ref:`keyword <keywords>`.
127 * ``"case"``: A name in double quotes denotes a
128 :ref:`soft-keyword <soft-keywords>`.
129 * ``'@'``: A non-letter symbol in single quotes denotes an
130 :py:data:`~token.OP` token, that is, a :ref:`delimiter <delimiters>` or
131 :ref:`operator <operators>`.
132
133 * ``e1 e2``: Items separated only by whitespace denote a sequence.
134 Here, ``e1`` must be followed by ``e2``.
135 * ``e1 | e2``: A vertical bar is used to separate alternatives.
136 It denotes PEG's "ordered choice": if ``e1`` matches, ``e2`` is
137 not considered.
138 In traditional PEG grammars, this is written as a slash, ``/``, rather than
139 a vertical bar.
140 See :pep:`617` for more background and details.
141 * ``e*``: A star means zero or more repetitions of the preceding item.
142 * ``e+``: Likewise, a plus means one or more repetitions.
143 * ``[e]``: A phrase enclosed in square brackets means zero or
144 one occurrences. In other words, the enclosed phrase is optional.
145 * ``e?``: A question mark has exactly the same meaning as square brackets:
146 the preceding item is optional.
147 * ``(e)``: Parentheses are used for grouping.
148
149 The following notation is only used in
150 :ref:`lexical definitions <notation-lexical-vs-syntactic>`.
151
152 * ``"a"..."z"``: Two literal characters separated by three dots mean a choice
153 of any single character in the given (inclusive) range of ASCII characters.
154 * ``<...>``: A phrase between angular brackets gives an informal description
155 of the matched symbol (for example, ``<any ASCII character except "\">``),
156 or an abbreviation that is defined in nearby text (for example, ``<Lu>``).
157
158 .. _lexical-lookaheads:
159
160 Some definitions also use *lookaheads*, which indicate that an element
161 must (or must not) match at a given position, but without consuming any input:
162
163 * ``&e``: a positive lookahead (that is, ``e`` is required to match)
164 * ``!e``: a negative lookahead (that is, ``e`` is required *not* to match)
165
166 The unary operators (``*``, ``+``, ``?``) bind as tightly as possible;
167 the vertical bar (``|``) binds most loosely.
168
169 White space is only meaningful to separate tokens.
170
171 Rules are normally contained on a single line, but rules that are too long
172 may be wrapped:
173
174 .. grammar-snippet::
175 :group: notation
176
177 literal: stringliteral | bytesliteral
178 | integer | floatnumber | imagnumber
179
180 Alternatively, rules may be formatted with the first line ending at the colon,
181 and each alternative beginning with a vertical bar on a new line.
182 For example:
183
184
185 .. grammar-snippet::
186 :group: notation-alt
187
188 literal:
189 | stringliteral
190 | bytesliteral
191 | integer
192 | floatnumber
193 | imagnumber
194
195 This does *not* mean that there is an empty first alternative.
196
197 .. index:: lexical definitions
198
199 .. _notation-lexical-vs-syntactic:
200
201 Lexical and Syntactic definitions
202 ---------------------------------
203
204 There is some difference between *lexical* and *syntactic* analysis:
205 the :term:`lexical analyzer` operates on the individual characters of the
206 input source, while the *parser* (syntactic analyzer) operates on the stream
207 of :term:`tokens <token>` generated by the lexical analysis.
208 However, in some cases the exact boundary between the two phases is a
209 CPython implementation detail.
210
211 The practical difference between the two is that in *lexical* definitions,
212 all whitespace is significant.
213 The lexical analyzer :ref:`discards <whitespace>` all whitespace that is not
214 converted to tokens like :data:`token.INDENT` or :data:`~token.NEWLINE`.
215 *Syntactic* definitions then use these tokens, rather than source characters.
216
217 This documentation uses the same BNF grammar for both styles of definitions.
218 All uses of BNF in the next chapter (:ref:`lexical`) are lexical definitions;
219 uses in subsequent chapters are syntactic definitions.