| 1 |
====================== |
| 2 |
Design and History FAQ |
| 3 |
====================== |
| 4 |
|
| 5 |
.. only:: html |
| 6 |
|
| 7 |
.. contents:: |
| 8 |
|
| 9 |
|
| 10 |
Why does Python use indentation for grouping of statements? |
| 11 |
----------------------------------------------------------- |
| 12 |
|
| 13 |
Guido van Rossum believes that using indentation for grouping is extremely |
| 14 |
elegant and contributes a lot to the clarity of the average Python program. |
| 15 |
Most people learn to love this feature after a while. |
| 16 |
|
| 17 |
Since there are no begin/end brackets there cannot be a disagreement between |
| 18 |
grouping perceived by the parser and the human reader. Occasionally C |
| 19 |
programmers will encounter a fragment of code like this:: |
| 20 |
|
| 21 |
if (x <= y) |
| 22 |
x++; |
| 23 |
y--; |
| 24 |
z++; |
| 25 |
|
| 26 |
Only the ``x++`` statement is executed if the condition is true, but the |
| 27 |
indentation leads many to believe otherwise. Even experienced C programmers will |
| 28 |
sometimes stare at it a long time wondering as to why ``y`` is being decremented even |
| 29 |
for ``x > y``. |
| 30 |
|
| 31 |
Because there are no begin/end brackets, Python is much less prone to |
| 32 |
coding-style conflicts. In C there are many different ways to place the braces. |
| 33 |
After becoming used to reading and writing code using a particular style, |
| 34 |
it is normal to feel somewhat uneasy when reading (or being required to write) |
| 35 |
in a different one. |
| 36 |
|
| 37 |
|
| 38 |
Many coding styles place begin/end brackets on a line by themselves. This makes |
| 39 |
programs considerably longer and wastes valuable screen space, making it harder |
| 40 |
to get a good overview of a program. Ideally, a function should fit on one |
| 41 |
screen (say, 20--30 lines). 20 lines of Python can do a lot more work than 20 |
| 42 |
lines of C. This is not solely due to the lack of begin/end brackets -- the |
| 43 |
lack of declarations and the high-level data types are also responsible -- but |
| 44 |
the indentation-based syntax certainly helps. |
| 45 |
|
| 46 |
|
| 47 |
Why am I getting strange results with simple arithmetic operations? |
| 48 |
------------------------------------------------------------------- |
| 49 |
|
| 50 |
See the next question. |
| 51 |
|
| 52 |
|
| 53 |
Why are floating-point calculations so inaccurate? |
| 54 |
-------------------------------------------------- |
| 55 |
|
| 56 |
Users are often surprised by results like this:: |
| 57 |
|
| 58 |
>>> 1.2 - 1.0 |
| 59 |
0.19999999999999996 |
| 60 |
|
| 61 |
and think it is a bug in Python. It's not. This has little to do with Python, |
| 62 |
and much more to do with how the underlying platform handles floating-point |
| 63 |
numbers. |
| 64 |
|
| 65 |
The :class:`float` type in CPython uses a C ``double`` for storage. A |
| 66 |
:class:`float` object's value is stored in binary floating-point with a fixed |
| 67 |
precision (typically 53 bits) and Python uses C operations, which in turn rely |
| 68 |
on the hardware implementation in the processor, to perform floating-point |
| 69 |
operations. This means that as far as floating-point operations are concerned, |
| 70 |
Python behaves like many popular languages including C and Java. |
| 71 |
|
| 72 |
Many numbers that can be written easily in decimal notation cannot be expressed |
| 73 |
exactly in binary floating point. For example, after:: |
| 74 |
|
| 75 |
>>> x = 1.2 |
| 76 |
|
| 77 |
the value stored for ``x`` is a (very good) approximation to the decimal value |
| 78 |
``1.2``, but is not exactly equal to it. On a typical machine, the actual |
| 79 |
stored value is:: |
| 80 |
|
| 81 |
1.0011001100110011001100110011001100110011001100110011 (binary) |
| 82 |
|
| 83 |
which is exactly:: |
| 84 |
|
| 85 |
1.1999999999999999555910790149937383830547332763671875 (decimal) |
| 86 |
|
| 87 |
The typical precision of 53 bits provides Python floats with 15--16 |
| 88 |
decimal digits of accuracy. |
| 89 |
|
| 90 |
For a fuller explanation, please see the :ref:`floating-point arithmetic |
| 91 |
<tut-fp-issues>` chapter in the Python tutorial. |
| 92 |
|
| 93 |
|
| 94 |
Why are Python strings immutable? |
| 95 |
--------------------------------- |
| 96 |
|
| 97 |
There are several advantages. |
| 98 |
|
| 99 |
One is performance: knowing that a string is immutable means we can allocate |
| 100 |
space for it at creation time, and the storage requirements are fixed and |
| 101 |
unchanging. This is also one of the reasons for the distinction between tuples |
| 102 |
and lists. |
| 103 |
|
| 104 |
Another advantage is that strings in Python are considered as "elemental" as |
| 105 |
numbers. No amount of activity will change the value 8 to anything else, and in |
| 106 |
Python, no amount of activity will change the string "eight" to anything else. |
| 107 |
|
| 108 |
|
| 109 |
.. _why-self: |
| 110 |
|
| 111 |
Why must 'self' be used explicitly in method definitions and calls? |
| 112 |
------------------------------------------------------------------- |
| 113 |
|
| 114 |
The idea was borrowed from Modula-3. It turns out to be very useful, for a |
| 115 |
variety of reasons. |
| 116 |
|
| 117 |
First, it's more obvious that you are using a method or instance attribute |
| 118 |
instead of a local variable. Reading ``self.x`` or ``self.meth()`` makes it |
| 119 |
absolutely clear that an instance variable or method is used even if you don't |
| 120 |
know the class definition by heart. In C++, you can sort of tell by the lack of |
| 121 |
a local variable declaration (assuming globals are rare or easily recognizable) |
| 122 |
-- but in Python, there are no local variable declarations, so you'd have to |
| 123 |
look up the class definition to be sure. Some C++ and Java coding standards |
| 124 |
call for instance attributes to have an ``m_`` prefix, so this explicitness is |
| 125 |
still useful in those languages, too. |
| 126 |
|
| 127 |
Second, it means that no special syntax is necessary if you want to explicitly |
| 128 |
reference or call the method from a particular class. In C++, if you want to |
| 129 |
use a method from a base class which is overridden in a derived class, you have |
| 130 |
to use the ``::`` operator -- in Python you can write |
| 131 |
``baseclass.methodname(self, <argument list>)``. This is particularly useful |
| 132 |
for :meth:`~object.__init__` methods, and in general in cases where a derived class |
| 133 |
method wants to extend the base class method of the same name and thus has to |
| 134 |
call the base class method somehow. |
| 135 |
|
| 136 |
Finally, for instance variables it solves a syntactic problem with assignment: |
| 137 |
since local variables in Python are (by definition!) those variables to which a |
| 138 |
value is assigned in a function body (and that aren't explicitly declared |
| 139 |
global), there has to be some way to tell the interpreter that an assignment was |
| 140 |
meant to assign to an instance variable instead of to a local variable, and it |
| 141 |
should preferably be syntactic (for efficiency reasons). C++ does this through |
| 142 |
declarations, but Python doesn't have declarations and it would be a pity having |
| 143 |
to introduce them just for this purpose. Using the explicit ``self.var`` solves |
| 144 |
this nicely. Similarly, for using instance variables, having to write |
| 145 |
``self.var`` means that references to unqualified names inside a method don't |
| 146 |
have to search the instance's directories. To put it another way, local |
| 147 |
variables and instance variables live in two different namespaces, and you need |
| 148 |
to tell Python which namespace to use. |
| 149 |
|
| 150 |
|
| 151 |
.. _why-can-t-i-use-an-assignment-in-an-expression: |
| 152 |
|
| 153 |
Why can't I use an assignment in an expression? |
| 154 |
----------------------------------------------- |
| 155 |
|
| 156 |
Starting in Python 3.8, you can! |
| 157 |
|
| 158 |
Assignment expressions using the walrus operator ``:=`` assign a variable in an |
| 159 |
expression:: |
| 160 |
|
| 161 |
while chunk := fp.read(200): |
| 162 |
print(chunk) |
| 163 |
|
| 164 |
See :pep:`572` for more information. |
| 165 |
|
| 166 |
|
| 167 |
|
| 168 |
Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))? |
| 169 |
---------------------------------------------------------------------------------------------------------------- |
| 170 |
|
| 171 |
As Guido said: |
| 172 |
|
| 173 |
(a) For some operations, prefix notation just reads better than |
| 174 |
postfix -- prefix (and infix!) operations have a long tradition in |
| 175 |
mathematics which likes notations where the visuals help the |
| 176 |
mathematician thinking about a problem. Compare the easy with which we |
| 177 |
rewrite a formula like x*(a+b) into x*a + x*b to the clumsiness of |
| 178 |
doing the same thing using a raw OO notation. |
| 179 |
|
| 180 |
(b) When I read code that says len(x) I *know* that it is asking for |
| 181 |
the length of something. This tells me two things: the result is an |
| 182 |
integer, and the argument is some kind of container. To the contrary, |
| 183 |
when I read x.len(), I have to already know that x is some kind of |
| 184 |
container implementing an interface or inheriting from a class that |
| 185 |
has a standard len(). Witness the confusion we occasionally have when |
| 186 |
a class that is not implementing a mapping has a get() or keys() |
| 187 |
method, or something that isn't a file has a write() method. |
| 188 |
|
| 189 |
-- https://mail.python.org/pipermail/python-3000/2006-November/004643.html |
| 190 |
|
| 191 |
|
| 192 |
Why is join() a string method instead of a list or tuple method? |
| 193 |
---------------------------------------------------------------- |
| 194 |
|
| 195 |
Strings became much more like other standard types starting in Python 1.6, when |
| 196 |
methods were added which give the same functionality that has always been |
| 197 |
available using the functions of the string module. Most of these new methods |
| 198 |
have been widely accepted, but the one which appears to make some programmers |
| 199 |
feel uncomfortable is:: |
| 200 |
|
| 201 |
", ".join(['1', '2', '4', '8', '16']) |
| 202 |
|
| 203 |
which gives the result:: |
| 204 |
|
| 205 |
"1, 2, 4, 8, 16" |
| 206 |
|
| 207 |
There are two common arguments against this usage. |
| 208 |
|
| 209 |
The first runs along the lines of: "It looks really ugly using a method of a |
| 210 |
string literal (string constant)", to which the answer is that it might, but a |
| 211 |
string literal is just a fixed value. If the methods are to be allowed on names |
| 212 |
bound to strings there is no logical reason to make them unavailable on |
| 213 |
literals. |
| 214 |
|
| 215 |
The second objection is typically cast as: "I am really telling a sequence to |
| 216 |
join its members together with a string constant". Sadly, you aren't. For some |
| 217 |
reason there seems to be much less difficulty with having :meth:`~str.split` as |
| 218 |
a string method, since in that case it is easy to see that :: |
| 219 |
|
| 220 |
"1, 2, 4, 8, 16".split(", ") |
| 221 |
|
| 222 |
is an instruction to a string literal to return the substrings delimited by the |
| 223 |
given separator (or, by default, arbitrary runs of white space). |
| 224 |
|
| 225 |
:meth:`~str.join` is a string method because in using it you are telling the |
| 226 |
separator string to iterate over a sequence of strings and insert itself between |
| 227 |
adjacent elements. This method can be used with any argument which obeys the |
| 228 |
rules for sequence objects, including any new classes you might define yourself. |
| 229 |
Similar methods exist for bytes and bytearray objects. |
| 230 |
|
| 231 |
|
| 232 |
How fast are exceptions? |
| 233 |
------------------------ |
| 234 |
|
| 235 |
A :keyword:`try`/:keyword:`except` block is extremely efficient if no exceptions |
| 236 |
are raised. Actually |
| 237 |
catching an exception is expensive. In versions of Python prior to 2.0 it was |
| 238 |
common to use this idiom:: |
| 239 |
|
| 240 |
try: |
| 241 |
value = mydict[key] |
| 242 |
except KeyError: |
| 243 |
mydict[key] = getvalue(key) |
| 244 |
value = mydict[key] |
| 245 |
|
| 246 |
This only made sense when you expected the dict to have the key almost all the |
| 247 |
time. If that wasn't the case, you coded it like this:: |
| 248 |
|
| 249 |
if key in mydict: |
| 250 |
value = mydict[key] |
| 251 |
else: |
| 252 |
value = mydict[key] = getvalue(key) |
| 253 |
|
| 254 |
For this specific case, you could also use ``value = dict.setdefault(key, |
| 255 |
getvalue(key))``, but only if the ``getvalue()`` call is cheap enough because it |
| 256 |
is evaluated in all cases. |
| 257 |
|
| 258 |
|
| 259 |
Why isn't there a switch or case statement in Python? |
| 260 |
----------------------------------------------------- |
| 261 |
|
| 262 |
In general, structured switch statements execute one block of code |
| 263 |
when an expression has a particular value or set of values. |
| 264 |
Since Python 3.10 one can easily match literal values, or constants |
| 265 |
within a namespace, with a ``match ... case`` statement. |
| 266 |
An older alternative is a sequence of ``if... elif... elif... else``. |
| 267 |
|
| 268 |
For cases where you need to choose from a very large number of possibilities, |
| 269 |
you can create a dictionary mapping case values to functions to call. For |
| 270 |
example:: |
| 271 |
|
| 272 |
functions = {'a': function_1, |
| 273 |
'b': function_2, |
| 274 |
'c': self.method_1} |
| 275 |
|
| 276 |
func = functions[value] |
| 277 |
func() |
| 278 |
|
| 279 |
For calling methods on objects, you can simplify yet further by using the |
| 280 |
:func:`getattr` built-in to retrieve methods with a particular name:: |
| 281 |
|
| 282 |
class MyVisitor: |
| 283 |
def visit_a(self): |
| 284 |
... |
| 285 |
|
| 286 |
def dispatch(self, value): |
| 287 |
method_name = 'visit_' + str(value) |
| 288 |
method = getattr(self, method_name) |
| 289 |
method() |
| 290 |
|
| 291 |
It's suggested that you use a prefix for the method names, such as ``visit_`` in |
| 292 |
this example. Without such a prefix, if values are coming from an untrusted |
| 293 |
source, an attacker would be able to call any method on your object. |
| 294 |
|
| 295 |
Imitating switch with fallthrough, as with C's switch-case-default, |
| 296 |
is possible, much harder, and less needed. |
| 297 |
|
| 298 |
|
| 299 |
Can't you emulate threads in the interpreter instead of relying on an OS-specific thread implementation? |
| 300 |
-------------------------------------------------------------------------------------------------------- |
| 301 |
|
| 302 |
Answer 1: Unfortunately, the interpreter pushes at least one C stack frame for |
| 303 |
each Python stack frame. Also, extensions can call back into Python at almost |
| 304 |
random moments. Therefore, a complete threads implementation requires thread |
| 305 |
support for C. |
| 306 |
|
| 307 |
Answer 2: Fortunately, there is `Stackless Python <https://github.com/stackless-dev/stackless/wiki>`_, |
| 308 |
which has a completely redesigned interpreter loop that avoids the C stack. |
| 309 |
|
| 310 |
|
| 311 |
Why can't lambda expressions contain statements? |
| 312 |
------------------------------------------------ |
| 313 |
|
| 314 |
Python lambda expressions cannot contain statements because Python's syntactic |
| 315 |
framework can't handle statements nested inside expressions. However, in |
| 316 |
Python, this is not a serious problem. Unlike lambda forms in other languages, |
| 317 |
where they add functionality, Python lambdas are only a shorthand notation if |
| 318 |
you're too lazy to define a function. |
| 319 |
|
| 320 |
Functions are already first class objects in Python, and can be declared in a |
| 321 |
local scope. Therefore the only advantage of using a lambda instead of a |
| 322 |
locally defined function is that you don't need to invent a name for the |
| 323 |
function -- but that's just a local variable to which the function object (which |
| 324 |
is exactly the same type of object that a lambda expression yields) is assigned! |
| 325 |
|
| 326 |
|
| 327 |
Can Python be compiled to machine code, C or some other language? |
| 328 |
----------------------------------------------------------------- |
| 329 |
|
| 330 |
`Cython <https://cython.org/>`_ compiles a modified version of Python with |
| 331 |
optional annotations into C extensions. `Nuitka <https://nuitka.net/>`_ is |
| 332 |
an up-and-coming compiler of Python into C++ code, aiming to support the full |
| 333 |
Python language. |
| 334 |
|
| 335 |
|
| 336 |
How does Python manage memory? |
| 337 |
------------------------------ |
| 338 |
|
| 339 |
The details of Python memory management depend on the implementation. The |
| 340 |
standard implementation of Python, :term:`CPython`, uses reference counting to |
| 341 |
detect inaccessible objects, and another mechanism to collect reference cycles, |
| 342 |
periodically executing a cycle detection algorithm which looks for inaccessible |
| 343 |
cycles and deletes the objects involved. The :mod:`gc` module provides functions |
| 344 |
to perform a garbage collection, obtain debugging statistics, and tune the |
| 345 |
collector's parameters. |
| 346 |
|
| 347 |
Other implementations (such as `Jython <https://www.jython.org>`_ or |
| 348 |
`PyPy <https://pypy.org>`_), however, can rely on a different mechanism |
| 349 |
such as a full-blown garbage collector. This difference can cause some |
| 350 |
subtle porting problems if your Python code depends on the behavior of the |
| 351 |
reference counting implementation. |
| 352 |
|
| 353 |
In some Python implementations, the following code (which is fine in CPython) |
| 354 |
will probably run out of file descriptors:: |
| 355 |
|
| 356 |
for file in very_long_list_of_files: |
| 357 |
f = open(file) |
| 358 |
c = f.read(1) |
| 359 |
|
| 360 |
Indeed, using CPython's reference counting and destructor scheme, each new |
| 361 |
assignment to ``f`` closes the previous file. With a traditional GC, however, |
| 362 |
those file objects will only get collected (and closed) at varying and possibly |
| 363 |
long intervals. |
| 364 |
|
| 365 |
If you want to write code that will work with any Python implementation, |
| 366 |
you should explicitly close the file or use the :keyword:`with` statement; |
| 367 |
this will work regardless of memory management scheme:: |
| 368 |
|
| 369 |
for file in very_long_list_of_files: |
| 370 |
with open(file) as f: |
| 371 |
c = f.read(1) |
| 372 |
|
| 373 |
|
| 374 |
Why doesn't CPython use a more traditional garbage collection scheme? |
| 375 |
--------------------------------------------------------------------- |
| 376 |
|
| 377 |
For one thing, this is not a C standard feature and hence it's not portable. |
| 378 |
(Yes, we know about the Boehm GC library. It has bits of assembler code for |
| 379 |
*most* common platforms, not for all of them, and although it is mostly |
| 380 |
transparent, it isn't completely transparent; patches are required to get |
| 381 |
Python to work with it.) |
| 382 |
|
| 383 |
Traditional GC also becomes a problem when Python is embedded into other |
| 384 |
applications. While in a standalone Python it's fine to replace the standard |
| 385 |
``malloc()`` and ``free()`` with versions provided by the GC library, an application |
| 386 |
embedding Python may want to have its *own* substitute for ``malloc()`` and ``free()``, |
| 387 |
and may not want Python's. Right now, CPython works with anything that |
| 388 |
implements ``malloc()`` and ``free()`` properly. |
| 389 |
|
| 390 |
|
| 391 |
Why isn't all memory freed when CPython exits? |
| 392 |
---------------------------------------------- |
| 393 |
|
| 394 |
Objects referenced from the global namespaces of Python modules are not always |
| 395 |
deallocated when Python exits. This may happen if there are circular |
| 396 |
references. There are also certain bits of memory that are allocated by the C |
| 397 |
library that are impossible to free (e.g. a tool like Purify will complain about |
| 398 |
these). Python is, however, aggressive about cleaning up memory on exit and |
| 399 |
does try to destroy every single object. |
| 400 |
|
| 401 |
If you want to force Python to delete certain things on deallocation use the |
| 402 |
:mod:`atexit` module to run a function that will force those deletions. |
| 403 |
|
| 404 |
|
| 405 |
Why are there separate tuple and list data types? |
| 406 |
------------------------------------------------- |
| 407 |
|
| 408 |
Lists and tuples, while similar in many respects, are generally used in |
| 409 |
fundamentally different ways. Tuples can be thought of as being similar to |
| 410 |
Pascal ``records`` or C ``structs``; they're small collections of related data which may |
| 411 |
be of different types which are operated on as a group. For example, a |
| 412 |
Cartesian coordinate is appropriately represented as a tuple of two or three |
| 413 |
numbers. |
| 414 |
|
| 415 |
Lists, on the other hand, are more like arrays in other languages. They tend to |
| 416 |
hold a varying number of objects all of which have the same type and which are |
| 417 |
operated on one-by-one. For example, :func:`os.listdir('.') <os.listdir>` |
| 418 |
returns a list of |
| 419 |
strings representing the files in the current directory. Functions which |
| 420 |
operate on this output would generally not break if you added another file or |
| 421 |
two to the directory. |
| 422 |
|
| 423 |
Tuples are immutable, meaning that once a tuple has been created, you can't |
| 424 |
replace any of its elements with a new value. Lists are mutable, meaning that |
| 425 |
you can always change a list's elements. Only immutable elements can be used as |
| 426 |
dictionary keys, and hence only tuples and not lists can be used as keys. |
| 427 |
|
| 428 |
|
| 429 |
How are lists implemented in CPython? |
| 430 |
------------------------------------- |
| 431 |
|
| 432 |
CPython's lists are really variable-length arrays, not Lisp-style linked lists. |
| 433 |
The implementation uses a contiguous array of references to other objects, and |
| 434 |
keeps a pointer to this array and the array's length in a list head structure. |
| 435 |
|
| 436 |
This makes indexing a list ``a[i]`` an operation whose cost is independent of |
| 437 |
the size of the list or the value of the index. |
| 438 |
|
| 439 |
When items are appended or inserted, the array of references is resized. Some |
| 440 |
cleverness is applied to improve the performance of appending items repeatedly; |
| 441 |
when the array must be grown, some extra space is allocated so the next few |
| 442 |
times don't require an actual resize. |
| 443 |
|
| 444 |
|
| 445 |
How are dictionaries implemented in CPython? |
| 446 |
-------------------------------------------- |
| 447 |
|
| 448 |
CPython's dictionaries are implemented as resizable hash tables. Compared to |
| 449 |
B-trees, this gives better performance for lookup (the most common operation by |
| 450 |
far) under most circumstances, and the implementation is simpler. |
| 451 |
|
| 452 |
Dictionaries work by computing a hash code for each key stored in the dictionary |
| 453 |
using the :func:`hash` built-in function. The hash code varies widely depending |
| 454 |
on the key and a per-process seed; for example, ``'Python'`` could hash to |
| 455 |
``-539294296`` while ``'python'``, a string that differs by a single bit, could hash |
| 456 |
to ``1142331976``. The hash code is then used to calculate a location in an |
| 457 |
internal array where the value will be stored. Assuming that you're storing |
| 458 |
keys that all have different hash values, this means that dictionaries take |
| 459 |
constant time -- *O*\ (1), in Big-O notation -- to retrieve a key. |
| 460 |
|
| 461 |
|
| 462 |
Why must dictionary keys be immutable? |
| 463 |
-------------------------------------- |
| 464 |
|
| 465 |
The hash table implementation of dictionaries uses a hash value calculated from |
| 466 |
the key value to find the key. If the key were a mutable object, its value |
| 467 |
could change, and thus its hash could also change. But since whoever changes |
| 468 |
the key object can't tell that it was being used as a dictionary key, it can't |
| 469 |
move the entry around in the dictionary. Then, when you try to look up the same |
| 470 |
object in the dictionary it won't be found because its hash value is different. |
| 471 |
If you tried to look up the old value it wouldn't be found either, because the |
| 472 |
value of the object found in that hash bin would be different. |
| 473 |
|
| 474 |
If you want a dictionary indexed with a list, simply convert the list to a tuple |
| 475 |
first; the function ``tuple(L)`` creates a tuple with the same entries as the |
| 476 |
list ``L``. Tuples are immutable and can therefore be used as dictionary keys. |
| 477 |
|
| 478 |
Some unacceptable solutions that have been proposed: |
| 479 |
|
| 480 |
- Hash lists by their address (object ID). This doesn't work because if you |
| 481 |
construct a new list with the same value it won't be found; e.g.:: |
| 482 |
|
| 483 |
mydict = {[1, 2]: '12'} |
| 484 |
print(mydict[[1, 2]]) |
| 485 |
|
| 486 |
would raise a :exc:`KeyError` exception because the id of the ``[1, 2]`` used in the |
| 487 |
second line differs from that in the first line. In other words, dictionary |
| 488 |
keys should be compared using ``==``, not using :keyword:`is`. |
| 489 |
|
| 490 |
- Make a copy when using a list as a key. This doesn't work because the list, |
| 491 |
being a mutable object, could contain a reference to itself, and then the |
| 492 |
copying code would run into an infinite loop. |
| 493 |
|
| 494 |
- Allow lists as keys but tell the user not to modify them. This would allow a |
| 495 |
class of hard-to-track bugs in programs when you forgot or modified a list by |
| 496 |
accident. It also invalidates an important invariant of dictionaries: every |
| 497 |
value in ``d.keys()`` is usable as a key of the dictionary. |
| 498 |
|
| 499 |
- Mark lists as read-only once they are used as a dictionary key. The problem |
| 500 |
is that it's not just the top-level object that could change its value; you |
| 501 |
could use a tuple containing a list as a key. Entering anything as a key into |
| 502 |
a dictionary would require marking all objects reachable from there as |
| 503 |
read-only -- and again, self-referential objects could cause an infinite loop. |
| 504 |
|
| 505 |
There is a trick to get around this if you need to, but use it at your own risk: |
| 506 |
You can wrap a mutable structure inside a class instance which has both a |
| 507 |
:meth:`~object.__eq__` and a :meth:`~object.__hash__` method. |
| 508 |
You must then make sure that the |
| 509 |
hash value for all such wrapper objects that reside in a dictionary (or other |
| 510 |
hash based structure), remain fixed while the object is in the dictionary (or |
| 511 |
other structure). :: |
| 512 |
|
| 513 |
class ListWrapper: |
| 514 |
def __init__(self, the_list): |
| 515 |
self.the_list = the_list |
| 516 |
|
| 517 |
def __eq__(self, other): |
| 518 |
return self.the_list == other.the_list |
| 519 |
|
| 520 |
def __hash__(self): |
| 521 |
l = self.the_list |
| 522 |
result = 98767 - len(l)*555 |
| 523 |
for i, el in enumerate(l): |
| 524 |
try: |
| 525 |
result = result + (hash(el) % 9999999) * 1001 + i |
| 526 |
except Exception: |
| 527 |
result = (result % 7777777) + i * 333 |
| 528 |
return result |
| 529 |
|
| 530 |
Note that the hash computation is complicated by the possibility that some |
| 531 |
members of the list may be unhashable and also by the possibility of arithmetic |
| 532 |
overflow. |
| 533 |
|
| 534 |
Furthermore it must always be the case that if ``o1 == o2`` (ie ``o1.__eq__(o2) |
| 535 |
is True``) then ``hash(o1) == hash(o2)`` (ie, ``o1.__hash__() == o2.__hash__()``), |
| 536 |
regardless of whether the object is in a dictionary or not. If you fail to meet |
| 537 |
these restrictions dictionaries and other hash based structures will misbehave. |
| 538 |
|
| 539 |
In the case of :class:`!ListWrapper`, whenever the wrapper object is in a dictionary the |
| 540 |
wrapped list must not change to avoid anomalies. Don't do this unless you are |
| 541 |
prepared to think hard about the requirements and the consequences of not |
| 542 |
meeting them correctly. Consider yourself warned. |
| 543 |
|
| 544 |
|
| 545 |
Why doesn't list.sort() return the sorted list? |
| 546 |
----------------------------------------------- |
| 547 |
|
| 548 |
In situations where performance matters, making a copy of the list just to sort |
| 549 |
it would be wasteful. Therefore, :meth:`list.sort` sorts the list in place. In |
| 550 |
order to remind you of that fact, it does not return the sorted list. This way, |
| 551 |
you won't be fooled into accidentally overwriting a list when you need a sorted |
| 552 |
copy but also need to keep the unsorted version around. |
| 553 |
|
| 554 |
If you want to return a new list, use the built-in :func:`sorted` function |
| 555 |
instead. This function creates a new list from a provided iterable, sorts |
| 556 |
it and returns it. For example, here's how to iterate over the keys of a |
| 557 |
dictionary in sorted order:: |
| 558 |
|
| 559 |
for key in sorted(mydict): |
| 560 |
... # do whatever with mydict[key]... |
| 561 |
|
| 562 |
|
| 563 |
How do you specify and enforce an interface spec in Python? |
| 564 |
----------------------------------------------------------- |
| 565 |
|
| 566 |
An interface specification for a module as provided by languages such as C++ and |
| 567 |
Java describes the prototypes for the methods and functions of the module. Many |
| 568 |
feel that compile-time enforcement of interface specifications helps in the |
| 569 |
construction of large programs. |
| 570 |
|
| 571 |
Python 2.6 adds an :mod:`abc` module that lets you define Abstract Base Classes |
| 572 |
(ABCs). You can then use :func:`isinstance` and :func:`issubclass` to check |
| 573 |
whether an instance or a class implements a particular ABC. The |
| 574 |
:mod:`collections.abc` module defines a set of useful ABCs such as |
| 575 |
:class:`~collections.abc.Iterable`, :class:`~collections.abc.Container`, and |
| 576 |
:class:`~collections.abc.MutableMapping`. |
| 577 |
|
| 578 |
For Python, many of the advantages of interface specifications can be obtained |
| 579 |
by an appropriate test discipline for components. |
| 580 |
|
| 581 |
A good test suite for a module can both provide a regression test and serve as a |
| 582 |
module interface specification and a set of examples. Many Python modules can |
| 583 |
be run as a script to provide a simple "self test." Even modules which use |
| 584 |
complex external interfaces can often be tested in isolation using trivial |
| 585 |
"stub" emulations of the external interface. The :mod:`doctest` and |
| 586 |
:mod:`unittest` modules or third-party test frameworks can be used to construct |
| 587 |
exhaustive test suites that exercise every line of code in a module. |
| 588 |
|
| 589 |
An appropriate testing discipline can help build large complex applications in |
| 590 |
Python as well as having interface specifications would. In fact, it can be |
| 591 |
better because an interface specification cannot test certain properties of a |
| 592 |
program. For example, the :meth:`list.append` method is expected to add new elements |
| 593 |
to the end of some internal list; an interface specification cannot test that |
| 594 |
your :meth:`list.append` implementation will actually do this correctly, but it's |
| 595 |
trivial to check this property in a test suite. |
| 596 |
|
| 597 |
Writing test suites is very helpful, and you might want to design your code to |
| 598 |
make it easily tested. One increasingly popular technique, test-driven |
| 599 |
development, calls for writing parts of the test suite first, before you write |
| 600 |
any of the actual code. Of course Python allows you to be sloppy and not write |
| 601 |
test cases at all. |
| 602 |
|
| 603 |
|
| 604 |
Why is there no goto? |
| 605 |
--------------------- |
| 606 |
|
| 607 |
In the 1970s people realized that unrestricted goto could lead |
| 608 |
to messy "spaghetti" code that was hard to understand and revise. |
| 609 |
In a high-level language, it is also unneeded as long as there |
| 610 |
are ways to branch (in Python, with :keyword:`if` statements and :keyword:`or`, |
| 611 |
:keyword:`and`, and :keyword:`if`/:keyword:`else` expressions) and loop (with :keyword:`while` |
| 612 |
and :keyword:`for` statements, possibly containing :keyword:`continue` and :keyword:`break`). |
| 613 |
|
| 614 |
One can also use exceptions to provide a "structured goto" |
| 615 |
that works even across |
| 616 |
function calls. Many feel that exceptions can conveniently emulate all |
| 617 |
reasonable uses of the ``go`` or ``goto`` constructs of C, Fortran, and other |
| 618 |
languages. For example:: |
| 619 |
|
| 620 |
class label(Exception): pass # declare a label |
| 621 |
|
| 622 |
try: |
| 623 |
... |
| 624 |
if condition: raise label() # goto label |
| 625 |
... |
| 626 |
except label: # where to goto |
| 627 |
pass |
| 628 |
... |
| 629 |
|
| 630 |
This doesn't allow you to jump into the middle of a loop, but that's usually |
| 631 |
considered an abuse of ``goto`` anyway. Use sparingly. |
| 632 |
|
| 633 |
|
| 634 |
Why can't raw strings (r-strings) end with a backslash? |
| 635 |
------------------------------------------------------- |
| 636 |
|
| 637 |
More precisely, they can't end with an odd number of backslashes: the unpaired |
| 638 |
backslash at the end escapes the closing quote character, leaving an |
| 639 |
unterminated string. |
| 640 |
|
| 641 |
Raw strings were designed to ease creating input for processors (chiefly regular |
| 642 |
expression engines) that want to do their own backslash escape processing. Such |
| 643 |
processors consider an unmatched trailing backslash to be an error anyway, so |
| 644 |
raw strings disallow that. In return, they allow you to pass on the string |
| 645 |
quote character by escaping it with a backslash. These rules work well when |
| 646 |
r-strings are used for their intended purpose. |
| 647 |
|
| 648 |
If you're trying to build Windows pathnames, note that all Windows system calls |
| 649 |
accept forward slashes too:: |
| 650 |
|
| 651 |
f = open("/mydir/file.txt") # works fine! |
| 652 |
|
| 653 |
If you're trying to build a pathname for a DOS command, try e.g. one of :: |
| 654 |
|
| 655 |
dir = r"\this\is\my\dos\dir" "\\" |
| 656 |
dir = r"\this\is\my\dos\dir\ "[:-1] |
| 657 |
dir = "\\this\\is\\my\\dos\\dir\\" |
| 658 |
|
| 659 |
|
| 660 |
Why doesn't Python have a "with" statement for attribute assignments? |
| 661 |
--------------------------------------------------------------------- |
| 662 |
|
| 663 |
Python has a :keyword:`with` statement that wraps the execution of a block, calling code |
| 664 |
on the entrance and exit from the block. Some languages have a construct that |
| 665 |
looks like this:: |
| 666 |
|
| 667 |
with obj: |
| 668 |
a = 1 # equivalent to obj.a = 1 |
| 669 |
total = total + 1 # obj.total = obj.total + 1 |
| 670 |
|
| 671 |
In Python, such a construct would be ambiguous. |
| 672 |
|
| 673 |
Other languages, such as Object Pascal, Delphi, and C++, use static types, so |
| 674 |
it's possible to know, in an unambiguous way, what member is being assigned |
| 675 |
to. This is the main point of static typing -- the compiler *always* knows the |
| 676 |
scope of every variable at compile time. |
| 677 |
|
| 678 |
Python uses dynamic types. It is impossible to know in advance which attribute |
| 679 |
will be referenced at runtime. Member attributes may be added or removed from |
| 680 |
objects on the fly. This makes it impossible to know, from a simple reading, |
| 681 |
what attribute is being referenced: a local one, a global one, or a member |
| 682 |
attribute? |
| 683 |
|
| 684 |
For instance, take the following incomplete snippet:: |
| 685 |
|
| 686 |
def foo(a): |
| 687 |
with a: |
| 688 |
print(x) |
| 689 |
|
| 690 |
The snippet assumes that ``a`` must have a member attribute called ``x``. However, |
| 691 |
there is nothing in Python that tells the interpreter this. What should happen |
| 692 |
if ``a`` is, let us say, an integer? If there is a global variable named ``x``, |
| 693 |
will it be used inside the :keyword:`with` block? As you see, the dynamic nature of Python |
| 694 |
makes such choices much harder. |
| 695 |
|
| 696 |
The primary benefit of :keyword:`with` and similar language features (reduction of code |
| 697 |
volume) can, however, easily be achieved in Python by assignment. Instead of:: |
| 698 |
|
| 699 |
function(args).mydict[index][index].a = 21 |
| 700 |
function(args).mydict[index][index].b = 42 |
| 701 |
function(args).mydict[index][index].c = 63 |
| 702 |
|
| 703 |
write this:: |
| 704 |
|
| 705 |
ref = function(args).mydict[index][index] |
| 706 |
ref.a = 21 |
| 707 |
ref.b = 42 |
| 708 |
ref.c = 63 |
| 709 |
|
| 710 |
This also has the side-effect of increasing execution speed because name |
| 711 |
bindings are resolved at run-time in Python, and the second version only needs |
| 712 |
to perform the resolution once. |
| 713 |
|
| 714 |
Similar proposals that would introduce syntax to further reduce code volume, |
| 715 |
such as using a 'leading dot', have been rejected in favour of explicitness (see |
| 716 |
https://mail.python.org/pipermail/python-ideas/2016-May/040070.html). |
| 717 |
|
| 718 |
|
| 719 |
Why don't generators support the with statement? |
| 720 |
------------------------------------------------ |
| 721 |
|
| 722 |
For technical reasons, a generator used directly as a context manager |
| 723 |
would not work correctly. When, as is most common, a generator is used as |
| 724 |
an iterator run to completion, no closing is needed. When it is, wrap |
| 725 |
it as :func:`contextlib.closing(generator) <contextlib.closing>` |
| 726 |
in the :keyword:`with` statement. |
| 727 |
|
| 728 |
|
| 729 |
Why are colons required for the if/while/def/class statements? |
| 730 |
-------------------------------------------------------------- |
| 731 |
|
| 732 |
The colon is required primarily to enhance readability (one of the results of |
| 733 |
the experimental ABC language). Consider this:: |
| 734 |
|
| 735 |
if a == b |
| 736 |
print(a) |
| 737 |
|
| 738 |
versus :: |
| 739 |
|
| 740 |
if a == b: |
| 741 |
print(a) |
| 742 |
|
| 743 |
Notice how the second one is slightly easier to read. Notice further how a |
| 744 |
colon sets off the example in this FAQ answer; it's a standard usage in English. |
| 745 |
|
| 746 |
Another minor reason is that the colon makes it easier for editors with syntax |
| 747 |
highlighting; they can look for colons to decide when indentation needs to be |
| 748 |
increased instead of having to do a more elaborate parsing of the program text. |
| 749 |
|
| 750 |
|
| 751 |
Why does Python allow commas at the end of lists and tuples? |
| 752 |
------------------------------------------------------------ |
| 753 |
|
| 754 |
Python lets you add a trailing comma at the end of lists, tuples, and |
| 755 |
dictionaries:: |
| 756 |
|
| 757 |
[1, 2, 3,] |
| 758 |
('a', 'b', 'c',) |
| 759 |
d = { |
| 760 |
"A": [1, 5], |
| 761 |
"B": [6, 7], # last trailing comma is optional but good style |
| 762 |
} |
| 763 |
|
| 764 |
|
| 765 |
There are several reasons to allow this. |
| 766 |
|
| 767 |
When you have a literal value for a list, tuple, or dictionary spread across |
| 768 |
multiple lines, it's easier to add more elements because you don't have to |
| 769 |
remember to add a comma to the previous line. The lines can also be reordered |
| 770 |
without creating a syntax error. |
| 771 |
|
| 772 |
Accidentally omitting the comma can lead to errors that are hard to diagnose. |
| 773 |
For example:: |
| 774 |
|
| 775 |
x = [ |
| 776 |
"fee", |
| 777 |
"fie" |
| 778 |
"foo", |
| 779 |
"fum" |
| 780 |
] |
| 781 |
|
| 782 |
This list looks like it has four elements, but it actually contains three: |
| 783 |
"fee", "fiefoo" and "fum". Always adding the comma avoids this source of error. |
| 784 |
|
| 785 |
Allowing the trailing comma may also make programmatic code generation easier. |