Debian Codegraph @

Doc/faq/design.rst

raw
1 ======================
2 Design and History FAQ
3 ======================
4
5 .. only:: html
6
7 .. contents::
8
9
10 Why does Python use indentation for grouping of statements?
11 -----------------------------------------------------------
12
13 Guido van Rossum believes that using indentation for grouping is extremely
14 elegant and contributes a lot to the clarity of the average Python program.
15 Most people learn to love this feature after a while.
16
17 Since there are no begin/end brackets there cannot be a disagreement between
18 grouping perceived by the parser and the human reader. Occasionally C
19 programmers will encounter a fragment of code like this::
20
21 if (x <= y)
22 x++;
23 y--;
24 z++;
25
26 Only the ``x++`` statement is executed if the condition is true, but the
27 indentation leads many to believe otherwise. Even experienced C programmers will
28 sometimes stare at it a long time wondering as to why ``y`` is being decremented even
29 for ``x > y``.
30
31 Because there are no begin/end brackets, Python is much less prone to
32 coding-style conflicts. In C there are many different ways to place the braces.
33 After becoming used to reading and writing code using a particular style,
34 it is normal to feel somewhat uneasy when reading (or being required to write)
35 in a different one.
36
37
38 Many coding styles place begin/end brackets on a line by themselves. This makes
39 programs considerably longer and wastes valuable screen space, making it harder
40 to get a good overview of a program. Ideally, a function should fit on one
41 screen (say, 20--30 lines). 20 lines of Python can do a lot more work than 20
42 lines of C. This is not solely due to the lack of begin/end brackets -- the
43 lack of declarations and the high-level data types are also responsible -- but
44 the indentation-based syntax certainly helps.
45
46
47 Why am I getting strange results with simple arithmetic operations?
48 -------------------------------------------------------------------
49
50 See the next question.
51
52
53 Why are floating-point calculations so inaccurate?
54 --------------------------------------------------
55
56 Users are often surprised by results like this::
57
58 >>> 1.2 - 1.0
59 0.19999999999999996
60
61 and think it is a bug in Python. It's not. This has little to do with Python,
62 and much more to do with how the underlying platform handles floating-point
63 numbers.
64
65 The :class:`float` type in CPython uses a C ``double`` for storage. A
66 :class:`float` object's value is stored in binary floating-point with a fixed
67 precision (typically 53 bits) and Python uses C operations, which in turn rely
68 on the hardware implementation in the processor, to perform floating-point
69 operations. This means that as far as floating-point operations are concerned,
70 Python behaves like many popular languages including C and Java.
71
72 Many numbers that can be written easily in decimal notation cannot be expressed
73 exactly in binary floating point. For example, after::
74
75 >>> x = 1.2
76
77 the value stored for ``x`` is a (very good) approximation to the decimal value
78 ``1.2``, but is not exactly equal to it. On a typical machine, the actual
79 stored value is::
80
81 1.0011001100110011001100110011001100110011001100110011 (binary)
82
83 which is exactly::
84
85 1.1999999999999999555910790149937383830547332763671875 (decimal)
86
87 The typical precision of 53 bits provides Python floats with 15--16
88 decimal digits of accuracy.
89
90 For a fuller explanation, please see the :ref:`floating-point arithmetic
91 <tut-fp-issues>` chapter in the Python tutorial.
92
93
94 Why are Python strings immutable?
95 ---------------------------------
96
97 There are several advantages.
98
99 One is performance: knowing that a string is immutable means we can allocate
100 space for it at creation time, and the storage requirements are fixed and
101 unchanging. This is also one of the reasons for the distinction between tuples
102 and lists.
103
104 Another advantage is that strings in Python are considered as "elemental" as
105 numbers. No amount of activity will change the value 8 to anything else, and in
106 Python, no amount of activity will change the string "eight" to anything else.
107
108
109 .. _why-self:
110
111 Why must 'self' be used explicitly in method definitions and calls?
112 -------------------------------------------------------------------
113
114 The idea was borrowed from Modula-3. It turns out to be very useful, for a
115 variety of reasons.
116
117 First, it's more obvious that you are using a method or instance attribute
118 instead of a local variable. Reading ``self.x`` or ``self.meth()`` makes it
119 absolutely clear that an instance variable or method is used even if you don't
120 know the class definition by heart. In C++, you can sort of tell by the lack of
121 a local variable declaration (assuming globals are rare or easily recognizable)
122 -- but in Python, there are no local variable declarations, so you'd have to
123 look up the class definition to be sure. Some C++ and Java coding standards
124 call for instance attributes to have an ``m_`` prefix, so this explicitness is
125 still useful in those languages, too.
126
127 Second, it means that no special syntax is necessary if you want to explicitly
128 reference or call the method from a particular class. In C++, if you want to
129 use a method from a base class which is overridden in a derived class, you have
130 to use the ``::`` operator -- in Python you can write
131 ``baseclass.methodname(self, <argument list>)``. This is particularly useful
132 for :meth:`~object.__init__` methods, and in general in cases where a derived class
133 method wants to extend the base class method of the same name and thus has to
134 call the base class method somehow.
135
136 Finally, for instance variables it solves a syntactic problem with assignment:
137 since local variables in Python are (by definition!) those variables to which a
138 value is assigned in a function body (and that aren't explicitly declared
139 global), there has to be some way to tell the interpreter that an assignment was
140 meant to assign to an instance variable instead of to a local variable, and it
141 should preferably be syntactic (for efficiency reasons). C++ does this through
142 declarations, but Python doesn't have declarations and it would be a pity having
143 to introduce them just for this purpose. Using the explicit ``self.var`` solves
144 this nicely. Similarly, for using instance variables, having to write
145 ``self.var`` means that references to unqualified names inside a method don't
146 have to search the instance's directories. To put it another way, local
147 variables and instance variables live in two different namespaces, and you need
148 to tell Python which namespace to use.
149
150
151 .. _why-can-t-i-use-an-assignment-in-an-expression:
152
153 Why can't I use an assignment in an expression?
154 -----------------------------------------------
155
156 Starting in Python 3.8, you can!
157
158 Assignment expressions using the walrus operator ``:=`` assign a variable in an
159 expression::
160
161 while chunk := fp.read(200):
162 print(chunk)
163
164 See :pep:`572` for more information.
165
166
167
168 Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))?
169 ----------------------------------------------------------------------------------------------------------------
170
171 As Guido said:
172
173 (a) For some operations, prefix notation just reads better than
174 postfix -- prefix (and infix!) operations have a long tradition in
175 mathematics which likes notations where the visuals help the
176 mathematician thinking about a problem. Compare the easy with which we
177 rewrite a formula like x*(a+b) into x*a + x*b to the clumsiness of
178 doing the same thing using a raw OO notation.
179
180 (b) When I read code that says len(x) I *know* that it is asking for
181 the length of something. This tells me two things: the result is an
182 integer, and the argument is some kind of container. To the contrary,
183 when I read x.len(), I have to already know that x is some kind of
184 container implementing an interface or inheriting from a class that
185 has a standard len(). Witness the confusion we occasionally have when
186 a class that is not implementing a mapping has a get() or keys()
187 method, or something that isn't a file has a write() method.
188
189 -- https://mail.python.org/pipermail/python-3000/2006-November/004643.html
190
191
192 Why is join() a string method instead of a list or tuple method?
193 ----------------------------------------------------------------
194
195 Strings became much more like other standard types starting in Python 1.6, when
196 methods were added which give the same functionality that has always been
197 available using the functions of the string module. Most of these new methods
198 have been widely accepted, but the one which appears to make some programmers
199 feel uncomfortable is::
200
201 ", ".join(['1', '2', '4', '8', '16'])
202
203 which gives the result::
204
205 "1, 2, 4, 8, 16"
206
207 There are two common arguments against this usage.
208
209 The first runs along the lines of: "It looks really ugly using a method of a
210 string literal (string constant)", to which the answer is that it might, but a
211 string literal is just a fixed value. If the methods are to be allowed on names
212 bound to strings there is no logical reason to make them unavailable on
213 literals.
214
215 The second objection is typically cast as: "I am really telling a sequence to
216 join its members together with a string constant". Sadly, you aren't. For some
217 reason there seems to be much less difficulty with having :meth:`~str.split` as
218 a string method, since in that case it is easy to see that ::
219
220 "1, 2, 4, 8, 16".split(", ")
221
222 is an instruction to a string literal to return the substrings delimited by the
223 given separator (or, by default, arbitrary runs of white space).
224
225 :meth:`~str.join` is a string method because in using it you are telling the
226 separator string to iterate over a sequence of strings and insert itself between
227 adjacent elements. This method can be used with any argument which obeys the
228 rules for sequence objects, including any new classes you might define yourself.
229 Similar methods exist for bytes and bytearray objects.
230
231
232 How fast are exceptions?
233 ------------------------
234
235 A :keyword:`try`/:keyword:`except` block is extremely efficient if no exceptions
236 are raised. Actually
237 catching an exception is expensive. In versions of Python prior to 2.0 it was
238 common to use this idiom::
239
240 try:
241 value = mydict[key]
242 except KeyError:
243 mydict[key] = getvalue(key)
244 value = mydict[key]
245
246 This only made sense when you expected the dict to have the key almost all the
247 time. If that wasn't the case, you coded it like this::
248
249 if key in mydict:
250 value = mydict[key]
251 else:
252 value = mydict[key] = getvalue(key)
253
254 For this specific case, you could also use ``value = dict.setdefault(key,
255 getvalue(key))``, but only if the ``getvalue()`` call is cheap enough because it
256 is evaluated in all cases.
257
258
259 Why isn't there a switch or case statement in Python?
260 -----------------------------------------------------
261
262 In general, structured switch statements execute one block of code
263 when an expression has a particular value or set of values.
264 Since Python 3.10 one can easily match literal values, or constants
265 within a namespace, with a ``match ... case`` statement.
266 An older alternative is a sequence of ``if... elif... elif... else``.
267
268 For cases where you need to choose from a very large number of possibilities,
269 you can create a dictionary mapping case values to functions to call. For
270 example::
271
272 functions = {'a': function_1,
273 'b': function_2,
274 'c': self.method_1}
275
276 func = functions[value]
277 func()
278
279 For calling methods on objects, you can simplify yet further by using the
280 :func:`getattr` built-in to retrieve methods with a particular name::
281
282 class MyVisitor:
283 def visit_a(self):
284 ...
285
286 def dispatch(self, value):
287 method_name = 'visit_' + str(value)
288 method = getattr(self, method_name)
289 method()
290
291 It's suggested that you use a prefix for the method names, such as ``visit_`` in
292 this example. Without such a prefix, if values are coming from an untrusted
293 source, an attacker would be able to call any method on your object.
294
295 Imitating switch with fallthrough, as with C's switch-case-default,
296 is possible, much harder, and less needed.
297
298
299 Can't you emulate threads in the interpreter instead of relying on an OS-specific thread implementation?
300 --------------------------------------------------------------------------------------------------------
301
302 Answer 1: Unfortunately, the interpreter pushes at least one C stack frame for
303 each Python stack frame. Also, extensions can call back into Python at almost
304 random moments. Therefore, a complete threads implementation requires thread
305 support for C.
306
307 Answer 2: Fortunately, there is `Stackless Python <https://github.com/stackless-dev/stackless/wiki>`_,
308 which has a completely redesigned interpreter loop that avoids the C stack.
309
310
311 Why can't lambda expressions contain statements?
312 ------------------------------------------------
313
314 Python lambda expressions cannot contain statements because Python's syntactic
315 framework can't handle statements nested inside expressions. However, in
316 Python, this is not a serious problem. Unlike lambda forms in other languages,
317 where they add functionality, Python lambdas are only a shorthand notation if
318 you're too lazy to define a function.
319
320 Functions are already first class objects in Python, and can be declared in a
321 local scope. Therefore the only advantage of using a lambda instead of a
322 locally defined function is that you don't need to invent a name for the
323 function -- but that's just a local variable to which the function object (which
324 is exactly the same type of object that a lambda expression yields) is assigned!
325
326
327 Can Python be compiled to machine code, C or some other language?
328 -----------------------------------------------------------------
329
330 `Cython <https://cython.org/>`_ compiles a modified version of Python with
331 optional annotations into C extensions. `Nuitka <https://nuitka.net/>`_ is
332 an up-and-coming compiler of Python into C++ code, aiming to support the full
333 Python language.
334
335
336 How does Python manage memory?
337 ------------------------------
338
339 The details of Python memory management depend on the implementation. The
340 standard implementation of Python, :term:`CPython`, uses reference counting to
341 detect inaccessible objects, and another mechanism to collect reference cycles,
342 periodically executing a cycle detection algorithm which looks for inaccessible
343 cycles and deletes the objects involved. The :mod:`gc` module provides functions
344 to perform a garbage collection, obtain debugging statistics, and tune the
345 collector's parameters.
346
347 Other implementations (such as `Jython <https://www.jython.org>`_ or
348 `PyPy <https://pypy.org>`_), however, can rely on a different mechanism
349 such as a full-blown garbage collector. This difference can cause some
350 subtle porting problems if your Python code depends on the behavior of the
351 reference counting implementation.
352
353 In some Python implementations, the following code (which is fine in CPython)
354 will probably run out of file descriptors::
355
356 for file in very_long_list_of_files:
357 f = open(file)
358 c = f.read(1)
359
360 Indeed, using CPython's reference counting and destructor scheme, each new
361 assignment to ``f`` closes the previous file. With a traditional GC, however,
362 those file objects will only get collected (and closed) at varying and possibly
363 long intervals.
364
365 If you want to write code that will work with any Python implementation,
366 you should explicitly close the file or use the :keyword:`with` statement;
367 this will work regardless of memory management scheme::
368
369 for file in very_long_list_of_files:
370 with open(file) as f:
371 c = f.read(1)
372
373
374 Why doesn't CPython use a more traditional garbage collection scheme?
375 ---------------------------------------------------------------------
376
377 For one thing, this is not a C standard feature and hence it's not portable.
378 (Yes, we know about the Boehm GC library. It has bits of assembler code for
379 *most* common platforms, not for all of them, and although it is mostly
380 transparent, it isn't completely transparent; patches are required to get
381 Python to work with it.)
382
383 Traditional GC also becomes a problem when Python is embedded into other
384 applications. While in a standalone Python it's fine to replace the standard
385 ``malloc()`` and ``free()`` with versions provided by the GC library, an application
386 embedding Python may want to have its *own* substitute for ``malloc()`` and ``free()``,
387 and may not want Python's. Right now, CPython works with anything that
388 implements ``malloc()`` and ``free()`` properly.
389
390
391 Why isn't all memory freed when CPython exits?
392 ----------------------------------------------
393
394 Objects referenced from the global namespaces of Python modules are not always
395 deallocated when Python exits. This may happen if there are circular
396 references. There are also certain bits of memory that are allocated by the C
397 library that are impossible to free (e.g. a tool like Purify will complain about
398 these). Python is, however, aggressive about cleaning up memory on exit and
399 does try to destroy every single object.
400
401 If you want to force Python to delete certain things on deallocation use the
402 :mod:`atexit` module to run a function that will force those deletions.
403
404
405 Why are there separate tuple and list data types?
406 -------------------------------------------------
407
408 Lists and tuples, while similar in many respects, are generally used in
409 fundamentally different ways. Tuples can be thought of as being similar to
410 Pascal ``records`` or C ``structs``; they're small collections of related data which may
411 be of different types which are operated on as a group. For example, a
412 Cartesian coordinate is appropriately represented as a tuple of two or three
413 numbers.
414
415 Lists, on the other hand, are more like arrays in other languages. They tend to
416 hold a varying number of objects all of which have the same type and which are
417 operated on one-by-one. For example, :func:`os.listdir('.') <os.listdir>`
418 returns a list of
419 strings representing the files in the current directory. Functions which
420 operate on this output would generally not break if you added another file or
421 two to the directory.
422
423 Tuples are immutable, meaning that once a tuple has been created, you can't
424 replace any of its elements with a new value. Lists are mutable, meaning that
425 you can always change a list's elements. Only immutable elements can be used as
426 dictionary keys, and hence only tuples and not lists can be used as keys.
427
428
429 How are lists implemented in CPython?
430 -------------------------------------
431
432 CPython's lists are really variable-length arrays, not Lisp-style linked lists.
433 The implementation uses a contiguous array of references to other objects, and
434 keeps a pointer to this array and the array's length in a list head structure.
435
436 This makes indexing a list ``a[i]`` an operation whose cost is independent of
437 the size of the list or the value of the index.
438
439 When items are appended or inserted, the array of references is resized. Some
440 cleverness is applied to improve the performance of appending items repeatedly;
441 when the array must be grown, some extra space is allocated so the next few
442 times don't require an actual resize.
443
444
445 How are dictionaries implemented in CPython?
446 --------------------------------------------
447
448 CPython's dictionaries are implemented as resizable hash tables. Compared to
449 B-trees, this gives better performance for lookup (the most common operation by
450 far) under most circumstances, and the implementation is simpler.
451
452 Dictionaries work by computing a hash code for each key stored in the dictionary
453 using the :func:`hash` built-in function. The hash code varies widely depending
454 on the key and a per-process seed; for example, ``'Python'`` could hash to
455 ``-539294296`` while ``'python'``, a string that differs by a single bit, could hash
456 to ``1142331976``. The hash code is then used to calculate a location in an
457 internal array where the value will be stored. Assuming that you're storing
458 keys that all have different hash values, this means that dictionaries take
459 constant time -- *O*\ (1), in Big-O notation -- to retrieve a key.
460
461
462 Why must dictionary keys be immutable?
463 --------------------------------------
464
465 The hash table implementation of dictionaries uses a hash value calculated from
466 the key value to find the key. If the key were a mutable object, its value
467 could change, and thus its hash could also change. But since whoever changes
468 the key object can't tell that it was being used as a dictionary key, it can't
469 move the entry around in the dictionary. Then, when you try to look up the same
470 object in the dictionary it won't be found because its hash value is different.
471 If you tried to look up the old value it wouldn't be found either, because the
472 value of the object found in that hash bin would be different.
473
474 If you want a dictionary indexed with a list, simply convert the list to a tuple
475 first; the function ``tuple(L)`` creates a tuple with the same entries as the
476 list ``L``. Tuples are immutable and can therefore be used as dictionary keys.
477
478 Some unacceptable solutions that have been proposed:
479
480 - Hash lists by their address (object ID). This doesn't work because if you
481 construct a new list with the same value it won't be found; e.g.::
482
483 mydict = {[1, 2]: '12'}
484 print(mydict[[1, 2]])
485
486 would raise a :exc:`KeyError` exception because the id of the ``[1, 2]`` used in the
487 second line differs from that in the first line. In other words, dictionary
488 keys should be compared using ``==``, not using :keyword:`is`.
489
490 - Make a copy when using a list as a key. This doesn't work because the list,
491 being a mutable object, could contain a reference to itself, and then the
492 copying code would run into an infinite loop.
493
494 - Allow lists as keys but tell the user not to modify them. This would allow a
495 class of hard-to-track bugs in programs when you forgot or modified a list by
496 accident. It also invalidates an important invariant of dictionaries: every
497 value in ``d.keys()`` is usable as a key of the dictionary.
498
499 - Mark lists as read-only once they are used as a dictionary key. The problem
500 is that it's not just the top-level object that could change its value; you
501 could use a tuple containing a list as a key. Entering anything as a key into
502 a dictionary would require marking all objects reachable from there as
503 read-only -- and again, self-referential objects could cause an infinite loop.
504
505 There is a trick to get around this if you need to, but use it at your own risk:
506 You can wrap a mutable structure inside a class instance which has both a
507 :meth:`~object.__eq__` and a :meth:`~object.__hash__` method.
508 You must then make sure that the
509 hash value for all such wrapper objects that reside in a dictionary (or other
510 hash based structure), remain fixed while the object is in the dictionary (or
511 other structure). ::
512
513 class ListWrapper:
514 def __init__(self, the_list):
515 self.the_list = the_list
516
517 def __eq__(self, other):
518 return self.the_list == other.the_list
519
520 def __hash__(self):
521 l = self.the_list
522 result = 98767 - len(l)*555
523 for i, el in enumerate(l):
524 try:
525 result = result + (hash(el) % 9999999) * 1001 + i
526 except Exception:
527 result = (result % 7777777) + i * 333
528 return result
529
530 Note that the hash computation is complicated by the possibility that some
531 members of the list may be unhashable and also by the possibility of arithmetic
532 overflow.
533
534 Furthermore it must always be the case that if ``o1 == o2`` (ie ``o1.__eq__(o2)
535 is True``) then ``hash(o1) == hash(o2)`` (ie, ``o1.__hash__() == o2.__hash__()``),
536 regardless of whether the object is in a dictionary or not. If you fail to meet
537 these restrictions dictionaries and other hash based structures will misbehave.
538
539 In the case of :class:`!ListWrapper`, whenever the wrapper object is in a dictionary the
540 wrapped list must not change to avoid anomalies. Don't do this unless you are
541 prepared to think hard about the requirements and the consequences of not
542 meeting them correctly. Consider yourself warned.
543
544
545 Why doesn't list.sort() return the sorted list?
546 -----------------------------------------------
547
548 In situations where performance matters, making a copy of the list just to sort
549 it would be wasteful. Therefore, :meth:`list.sort` sorts the list in place. In
550 order to remind you of that fact, it does not return the sorted list. This way,
551 you won't be fooled into accidentally overwriting a list when you need a sorted
552 copy but also need to keep the unsorted version around.
553
554 If you want to return a new list, use the built-in :func:`sorted` function
555 instead. This function creates a new list from a provided iterable, sorts
556 it and returns it. For example, here's how to iterate over the keys of a
557 dictionary in sorted order::
558
559 for key in sorted(mydict):
560 ... # do whatever with mydict[key]...
561
562
563 How do you specify and enforce an interface spec in Python?
564 -----------------------------------------------------------
565
566 An interface specification for a module as provided by languages such as C++ and
567 Java describes the prototypes for the methods and functions of the module. Many
568 feel that compile-time enforcement of interface specifications helps in the
569 construction of large programs.
570
571 Python 2.6 adds an :mod:`abc` module that lets you define Abstract Base Classes
572 (ABCs). You can then use :func:`isinstance` and :func:`issubclass` to check
573 whether an instance or a class implements a particular ABC. The
574 :mod:`collections.abc` module defines a set of useful ABCs such as
575 :class:`~collections.abc.Iterable`, :class:`~collections.abc.Container`, and
576 :class:`~collections.abc.MutableMapping`.
577
578 For Python, many of the advantages of interface specifications can be obtained
579 by an appropriate test discipline for components.
580
581 A good test suite for a module can both provide a regression test and serve as a
582 module interface specification and a set of examples. Many Python modules can
583 be run as a script to provide a simple "self test." Even modules which use
584 complex external interfaces can often be tested in isolation using trivial
585 "stub" emulations of the external interface. The :mod:`doctest` and
586 :mod:`unittest` modules or third-party test frameworks can be used to construct
587 exhaustive test suites that exercise every line of code in a module.
588
589 An appropriate testing discipline can help build large complex applications in
590 Python as well as having interface specifications would. In fact, it can be
591 better because an interface specification cannot test certain properties of a
592 program. For example, the :meth:`list.append` method is expected to add new elements
593 to the end of some internal list; an interface specification cannot test that
594 your :meth:`list.append` implementation will actually do this correctly, but it's
595 trivial to check this property in a test suite.
596
597 Writing test suites is very helpful, and you might want to design your code to
598 make it easily tested. One increasingly popular technique, test-driven
599 development, calls for writing parts of the test suite first, before you write
600 any of the actual code. Of course Python allows you to be sloppy and not write
601 test cases at all.
602
603
604 Why is there no goto?
605 ---------------------
606
607 In the 1970s people realized that unrestricted goto could lead
608 to messy "spaghetti" code that was hard to understand and revise.
609 In a high-level language, it is also unneeded as long as there
610 are ways to branch (in Python, with :keyword:`if` statements and :keyword:`or`,
611 :keyword:`and`, and :keyword:`if`/:keyword:`else` expressions) and loop (with :keyword:`while`
612 and :keyword:`for` statements, possibly containing :keyword:`continue` and :keyword:`break`).
613
614 One can also use exceptions to provide a "structured goto"
615 that works even across
616 function calls. Many feel that exceptions can conveniently emulate all
617 reasonable uses of the ``go`` or ``goto`` constructs of C, Fortran, and other
618 languages. For example::
619
620 class label(Exception): pass # declare a label
621
622 try:
623 ...
624 if condition: raise label() # goto label
625 ...
626 except label: # where to goto
627 pass
628 ...
629
630 This doesn't allow you to jump into the middle of a loop, but that's usually
631 considered an abuse of ``goto`` anyway. Use sparingly.
632
633
634 Why can't raw strings (r-strings) end with a backslash?
635 -------------------------------------------------------
636
637 More precisely, they can't end with an odd number of backslashes: the unpaired
638 backslash at the end escapes the closing quote character, leaving an
639 unterminated string.
640
641 Raw strings were designed to ease creating input for processors (chiefly regular
642 expression engines) that want to do their own backslash escape processing. Such
643 processors consider an unmatched trailing backslash to be an error anyway, so
644 raw strings disallow that. In return, they allow you to pass on the string
645 quote character by escaping it with a backslash. These rules work well when
646 r-strings are used for their intended purpose.
647
648 If you're trying to build Windows pathnames, note that all Windows system calls
649 accept forward slashes too::
650
651 f = open("/mydir/file.txt") # works fine!
652
653 If you're trying to build a pathname for a DOS command, try e.g. one of ::
654
655 dir = r"\this\is\my\dos\dir" "\\"
656 dir = r"\this\is\my\dos\dir\ "[:-1]
657 dir = "\\this\\is\\my\\dos\\dir\\"
658
659
660 Why doesn't Python have a "with" statement for attribute assignments?
661 ---------------------------------------------------------------------
662
663 Python has a :keyword:`with` statement that wraps the execution of a block, calling code
664 on the entrance and exit from the block. Some languages have a construct that
665 looks like this::
666
667 with obj:
668 a = 1 # equivalent to obj.a = 1
669 total = total + 1 # obj.total = obj.total + 1
670
671 In Python, such a construct would be ambiguous.
672
673 Other languages, such as Object Pascal, Delphi, and C++, use static types, so
674 it's possible to know, in an unambiguous way, what member is being assigned
675 to. This is the main point of static typing -- the compiler *always* knows the
676 scope of every variable at compile time.
677
678 Python uses dynamic types. It is impossible to know in advance which attribute
679 will be referenced at runtime. Member attributes may be added or removed from
680 objects on the fly. This makes it impossible to know, from a simple reading,
681 what attribute is being referenced: a local one, a global one, or a member
682 attribute?
683
684 For instance, take the following incomplete snippet::
685
686 def foo(a):
687 with a:
688 print(x)
689
690 The snippet assumes that ``a`` must have a member attribute called ``x``. However,
691 there is nothing in Python that tells the interpreter this. What should happen
692 if ``a`` is, let us say, an integer? If there is a global variable named ``x``,
693 will it be used inside the :keyword:`with` block? As you see, the dynamic nature of Python
694 makes such choices much harder.
695
696 The primary benefit of :keyword:`with` and similar language features (reduction of code
697 volume) can, however, easily be achieved in Python by assignment. Instead of::
698
699 function(args).mydict[index][index].a = 21
700 function(args).mydict[index][index].b = 42
701 function(args).mydict[index][index].c = 63
702
703 write this::
704
705 ref = function(args).mydict[index][index]
706 ref.a = 21
707 ref.b = 42
708 ref.c = 63
709
710 This also has the side-effect of increasing execution speed because name
711 bindings are resolved at run-time in Python, and the second version only needs
712 to perform the resolution once.
713
714 Similar proposals that would introduce syntax to further reduce code volume,
715 such as using a 'leading dot', have been rejected in favour of explicitness (see
716 https://mail.python.org/pipermail/python-ideas/2016-May/040070.html).
717
718
719 Why don't generators support the with statement?
720 ------------------------------------------------
721
722 For technical reasons, a generator used directly as a context manager
723 would not work correctly. When, as is most common, a generator is used as
724 an iterator run to completion, no closing is needed. When it is, wrap
725 it as :func:`contextlib.closing(generator) <contextlib.closing>`
726 in the :keyword:`with` statement.
727
728
729 Why are colons required for the if/while/def/class statements?
730 --------------------------------------------------------------
731
732 The colon is required primarily to enhance readability (one of the results of
733 the experimental ABC language). Consider this::
734
735 if a == b
736 print(a)
737
738 versus ::
739
740 if a == b:
741 print(a)
742
743 Notice how the second one is slightly easier to read. Notice further how a
744 colon sets off the example in this FAQ answer; it's a standard usage in English.
745
746 Another minor reason is that the colon makes it easier for editors with syntax
747 highlighting; they can look for colons to decide when indentation needs to be
748 increased instead of having to do a more elaborate parsing of the program text.
749
750
751 Why does Python allow commas at the end of lists and tuples?
752 ------------------------------------------------------------
753
754 Python lets you add a trailing comma at the end of lists, tuples, and
755 dictionaries::
756
757 [1, 2, 3,]
758 ('a', 'b', 'c',)
759 d = {
760 "A": [1, 5],
761 "B": [6, 7], # last trailing comma is optional but good style
762 }
763
764
765 There are several reasons to allow this.
766
767 When you have a literal value for a list, tuple, or dictionary spread across
768 multiple lines, it's easier to add more elements because you don't have to
769 remember to add a comma to the previous line. The lines can also be reordered
770 without creating a syntax error.
771
772 Accidentally omitting the comma can lead to errors that are hard to diagnose.
773 For example::
774
775 x = [
776 "fee",
777 "fie"
778 "foo",
779 "fum"
780 ]
781
782 This list looks like it has four elements, but it actually contains three:
783 "fee", "fiefoo" and "fum". Always adding the comma avoids this source of error.
784
785 Allowing the trailing comma may also make programmatic code generation easier.