Debian Codegraph @

Doc/whatsnew/2.6.rst

raw
1 .. _whats-new-in-2.6:
2
3 ****************************
4 What's New in Python 2.6
5 ****************************
6
7 :Author: A.M. Kuchling (amk at amk.ca)
8
9 .. $Id$
10 Rules for maintenance:
11
12 * Anyone can add text to this document. Do not spend very much time
13 on the wording of your changes, because your text will probably
14 get rewritten to some degree.
15
16 * The maintainer will go through Misc/NEWS periodically and add
17 changes; it's therefore more important to add your changes to
18 Misc/NEWS than to this file.
19
20 * This is not a complete list of every single change; completeness
21 is the purpose of Misc/NEWS. Some changes I consider too small
22 or esoteric to include. If such a change is added to the text,
23 I'll just remove it. (This is another reason you shouldn't spend
24 too much time on writing your addition.)
25
26 * If you want to draw your new text to the attention of the
27 maintainer, add 'XXX' to the beginning of the paragraph or
28 section.
29
30 * It's OK to just add a fragmentary note about a change. For
31 example: "XXX Describe the transmogrify() function added to the
32 socket module." The maintainer will research the change and
33 write the necessary text.
34
35 * You can comment out your additions if you like, but it's not
36 necessary (especially when a final release is some months away).
37
38 * Credit the author of a patch or bugfix. Just the name is
39 sufficient; the e-mail address isn't necessary.
40
41 * It's helpful to add the bug/patch number in a parenthetical comment.
42
43 XXX Describe the transmogrify() function added to the socket
44 module.
45 (Contributed by P.Y. Developer; :issue:`12345`.)
46
47 This saves the maintainer some effort going through the SVN logs
48 when researching a change.
49
50 This article explains the new features in Python 2.6, released on October 1,
51 2008. The release schedule is described in :pep:`361`.
52
53 The major theme of Python 2.6 is preparing the migration path to
54 Python 3.0, a major redesign of the language. Whenever possible,
55 Python 2.6 incorporates new features and syntax from 3.0 while
56 remaining compatible with existing code by not removing older features
57 or syntax. When it's not possible to do that, Python 2.6 tries to do
58 what it can, adding compatibility functions in a
59 :mod:`!future_builtins` module and a :option:`!-3` switch to warn about
60 usages that will become unsupported in 3.0.
61
62 Some significant new packages have been added to the standard library,
63 such as the :mod:`multiprocessing` and :mod:`json` modules, but
64 there aren't many new features that aren't related to Python 3.0 in
65 some way.
66
67 Python 2.6 also sees a number of improvements and bugfixes throughout
68 the source. A search through the change logs finds there were 259
69 patches applied and 612 bugs fixed between Python 2.5 and 2.6. Both
70 figures are likely to be underestimates.
71
72 This article doesn't attempt to provide a complete specification of
73 the new features, but instead provides a convenient overview. For
74 full details, you should refer to the documentation for Python 2.6. If
75 you want to understand the rationale for the design and
76 implementation, refer to the PEP for a particular new feature.
77 Whenever possible, "What's New in Python" links to the bug/patch item
78 for each change.
79
80 .. Compare with previous release in 2 - 3 sentences here.
81 add hyperlink when the documentation becomes available online.
82
83 .. ========================================================================
84 .. Large, PEP-level features and changes should be described here.
85 .. ========================================================================
86
87 Python 3.0
88 ================
89
90 The development cycle for Python versions 2.6 and 3.0 was
91 synchronized, with the alpha and beta releases for both versions being
92 made on the same days. The development of 3.0 has influenced many
93 features in 2.6.
94
95 Python 3.0 is a far-ranging redesign of Python that breaks
96 compatibility with the 2.x series. This means that existing Python
97 code will need some conversion in order to run on
98 Python 3.0. However, not all the changes in 3.0 necessarily break
99 compatibility. In cases where new features won't cause existing code
100 to break, they've been backported to 2.6 and are described in this
101 document in the appropriate place. Some of the 3.0-derived features
102 are:
103
104 * A :meth:`__complex__` method for converting objects to a complex number.
105 * Alternate syntax for catching exceptions: ``except TypeError as exc``.
106 * The addition of :func:`functools.reduce` as a synonym for the built-in
107 :func:`reduce` function.
108
109 Python 3.0 adds several new built-in functions and changes the
110 semantics of some existing builtins. Functions that are new in 3.0
111 such as :func:`bin` have simply been added to Python 2.6, but existing
112 builtins haven't been changed; instead, the :mod:`!future_builtins`
113 module has versions with the new 3.0 semantics. Code written to be
114 compatible with 3.0 can do ``from future_builtins import hex, map`` as
115 necessary.
116
117 A new command-line switch, :option:`!-3`, enables warnings
118 about features that will be removed in Python 3.0. You can run code
119 with this switch to see how much work will be necessary to port
120 code to 3.0. The value of this switch is available
121 to Python code as the boolean variable :data:`!sys.py3kwarning`,
122 and to C extension code as :c:data:`!Py_Py3kWarningFlag`.
123
124 .. seealso::
125
126 The 3\ *xxx* series of PEPs, which contains proposals for Python 3.0.
127 :pep:`3000` describes the development process for Python 3.0.
128 Start with :pep:`3100` that describes the general goals for Python
129 3.0, and then explore the higher-numbered PEPs that propose
130 specific features.
131
132
133 Changes to the Development Process
134 ==================================================
135
136 While 2.6 was being developed, the Python development process
137 underwent two significant changes: we switched from SourceForge's
138 issue tracker to a customized Roundup installation, and the
139 documentation was converted from LaTeX to reStructuredText.
140
141
142 New Issue Tracker: Roundup
143 --------------------------------------------------
144
145 For a long time, the Python developers had been growing increasingly
146 annoyed by SourceForge's bug tracker. SourceForge's hosted solution
147 doesn't permit much customization; for example, it wasn't possible to
148 customize the life cycle of issues.
149
150 The infrastructure committee of the Python Software Foundation
151 therefore posted a call for issue trackers, asking volunteers to set
152 up different products and import some of the bugs and patches from
153 SourceForge. Four different trackers were examined: `Jira
154 <https://www.atlassian.com/software/jira/>`__,
155 `Launchpad <https://launchpad.net/>`__,
156 `Roundup <https://roundup.sourceforge.io/>`__, and
157 `Trac <https://trac.edgewall.org/>`__.
158 The committee eventually settled on Jira
159 and Roundup as the two candidates. Jira is a commercial product that
160 offers no-cost hosted instances to free-software projects; Roundup
161 is an open-source project that requires volunteers
162 to administer it and a server to host it.
163
164 After posting a call for volunteers, a new Roundup installation was
165 set up at https://bugs.python.org. One installation of Roundup can
166 host multiple trackers, and this server now also hosts issue trackers
167 for Jython and for the Python web site. It will surely find
168 other uses in the future. Where possible,
169 this edition of "What's New in Python" links to the bug/patch
170 item for each change.
171
172 Hosting of the Python bug tracker is kindly provided by
173 `Upfront Systems <https://upfrontsoftware.co.za>`__
174 of Stellenbosch, South Africa. Martin von Löwis put a
175 lot of effort into importing existing bugs and patches from
176 SourceForge; his scripts for this import operation are at
177 ``https://svn.python.org/view/tracker/importer/`` and may be useful to
178 other projects wishing to move from SourceForge to Roundup.
179
180 .. seealso::
181
182 https://bugs.python.org
183 The Python bug tracker.
184
185 https://bugs.jython.org:
186 The Jython bug tracker.
187
188 https://roundup.sourceforge.io/
189 Roundup downloads and documentation.
190
191 https://svn.python.org/view/tracker/importer/
192 Martin von Löwis's conversion scripts.
193
194 New Documentation Format: reStructuredText Using Sphinx
195 -----------------------------------------------------------
196
197 The Python documentation was written using LaTeX since the project
198 started around 1989. In the 1980s and early 1990s, most documentation
199 was printed out for later study, not viewed online. LaTeX was widely
200 used because it provided attractive printed output while remaining
201 straightforward to write once the basic rules of the markup were
202 learned.
203
204 Today LaTeX is still used for writing publications destined for
205 printing, but the landscape for programming tools has shifted. We no
206 longer print out reams of documentation; instead, we browse through it
207 online and HTML has become the most important format to support.
208 Unfortunately, converting LaTeX to HTML is fairly complicated and Fred
209 L. Drake Jr., the long-time Python documentation editor, spent a lot
210 of time maintaining the conversion process. Occasionally people would
211 suggest converting the documentation into SGML and later XML, but
212 performing a good conversion is a major task and no one ever committed
213 the time required to finish the job.
214
215 During the 2.6 development cycle, Georg Brandl put a lot of effort
216 into building a new toolchain for processing the documentation. The
217 resulting package is called Sphinx, and is available from
218 https://www.sphinx-doc.org/.
219
220 Sphinx concentrates on HTML output, producing attractively styled and
221 modern HTML; printed output is still supported through conversion to
222 LaTeX. The input format is reStructuredText, a markup syntax
223 supporting custom extensions and directives that is commonly used in
224 the Python community.
225
226 Sphinx is a standalone package that can be used for writing, and
227 almost two dozen other projects
228 (`listed on the Sphinx web site <https://www.sphinx-doc.org/en/master/examples.html>`__)
229 have adopted Sphinx as their documentation tool.
230
231 .. seealso::
232
233 `Documenting Python <https://devguide.python.org/documenting/>`__
234 Describes how to write for Python's documentation.
235
236 `Sphinx <https://www.sphinx-doc.org/>`__
237 Documentation and code for the Sphinx toolchain.
238
239 `Docutils <https://docutils.sourceforge.io>`__
240 The underlying reStructuredText parser and toolset.
241
242
243 .. _pep-0343:
244
245 PEP 343: The 'with' statement
246 =============================
247
248 The previous version, Python 2.5, added the ':keyword:`with`'
249 statement as an optional feature, to be enabled by a ``from __future__
250 import with_statement`` directive. In 2.6 the statement no longer needs to
251 be specially enabled; this means that :keyword:`!with` is now always a
252 keyword. The rest of this section is a copy of the corresponding
253 section from the "What's New in Python 2.5" document; if you're
254 familiar with the ':keyword:`!with`' statement
255 from Python 2.5, you can skip this section.
256
257 The ':keyword:`with`' statement clarifies code that previously would use
258 ``try...finally`` blocks to ensure that clean-up code is executed. In this
259 section, I'll discuss the statement as it will commonly be used. In the next
260 section, I'll examine the implementation details and show how to write objects
261 for use with this statement.
262
263 The ':keyword:`with`' statement is a control-flow structure whose basic
264 structure is::
265
266 with expression [as variable]:
267 with-block
268
269 The expression is evaluated, and it should result in an object that supports the
270 context management protocol (that is, has :meth:`~object.__enter__` and :meth:`~object.__exit__`
271 methods).
272
273 The object's :meth:`~object.__enter__` is called before *with-block* is executed and
274 therefore can run set-up code. It also may return a value that is bound to the
275 name *variable*, if given. (Note carefully that *variable* is *not* assigned
276 the result of *expression*.)
277
278 After execution of the *with-block* is finished, the object's :meth:`~object.__exit__`
279 method is called, even if the block raised an exception, and can therefore run
280 clean-up code.
281
282 Some standard Python objects now support the context management protocol and can
283 be used with the ':keyword:`with`' statement. File objects are one example::
284
285 with open('/etc/passwd', 'r') as f:
286 for line in f:
287 print line
288 ... more processing code ...
289
290 After this statement has executed, the file object in *f* will have been
291 automatically closed, even if the :keyword:`for` loop raised an exception
292 part-way through the block.
293
294 .. note::
295
296 In this case, *f* is the same object created by :func:`open`, because
297 :meth:`~object.__enter__` returns *self*.
298
299 The :mod:`threading` module's locks and condition variables also support the
300 ':keyword:`with`' statement::
301
302 lock = threading.Lock()
303 with lock:
304 # Critical section of code
305 ...
306
307 The lock is acquired before the block is executed and always released once the
308 block is complete.
309
310 The :func:`~decimal.localcontext` function in the :mod:`decimal` module makes
311 it easy to save and restore the current decimal context, which encapsulates
312 the desired precision and rounding characteristics for computations::
313
314 from decimal import Decimal, Context, localcontext
315
316 # Displays with default precision of 28 digits
317 v = Decimal('578')
318 print v.sqrt()
319
320 with localcontext(Context(prec=16)):
321 # All code in this block uses a precision of 16 digits.
322 # The original context is restored on exiting the block.
323 print v.sqrt()
324
325
326 .. _new-26-context-managers:
327
328 Writing Context Managers
329 ------------------------
330
331 Under the hood, the ':keyword:`with`' statement is fairly complicated. Most
332 people will only use ':keyword:`!with`' in company with existing objects and
333 don't need to know these details, so you can skip the rest of this section if
334 you like. Authors of new objects will need to understand the details of the
335 underlying implementation and should keep reading.
336
337 A high-level explanation of the context management protocol is:
338
339 * The expression is evaluated and should result in an object called a "context
340 manager". The context manager must have :meth:`~object.__enter__` and
341 :meth:`~object.__exit__` methods.
342
343 * The context manager's :meth:`~object.__enter__` method is called. The value
344 returned is assigned to *VAR*. If no ``as VAR`` clause is present, the
345 value is simply discarded.
346
347 * The code in *BLOCK* is executed.
348
349 * If *BLOCK* raises an exception, the context manager's :meth:`~object.__exit__` method
350 is called with three arguments, the exception details (``type, value, traceback``,
351 the same values returned by :func:`sys.exc_info`, which can also be ``None``
352 if no exception occurred). The method's return value controls whether an exception
353 is re-raised: any false value re-raises the exception, and ``True`` will result
354 in suppressing it. You'll only rarely want to suppress the exception, because
355 if you do the author of the code containing the ':keyword:`with`' statement will
356 never realize anything went wrong.
357
358 * If *BLOCK* didn't raise an exception, the :meth:`~object.__exit__` method is still
359 called, but *type*, *value*, and *traceback* are all ``None``.
360
361 Let's think through an example. I won't present detailed code but will only
362 sketch the methods necessary for a database that supports transactions.
363
364 (For people unfamiliar with database terminology: a set of changes to the
365 database are grouped into a transaction. Transactions can be either committed,
366 meaning that all the changes are written into the database, or rolled back,
367 meaning that the changes are all discarded and the database is unchanged. See
368 any database textbook for more information.)
369
370 Let's assume there's an object representing a database connection. Our goal will
371 be to let the user write code like this::
372
373 db_connection = DatabaseConnection()
374 with db_connection as cursor:
375 cursor.execute('insert into ...')
376 cursor.execute('delete from ...')
377 # ... more operations ...
378
379 The transaction should be committed if the code in the block runs flawlessly or
380 rolled back if there's an exception. Here's the basic interface for
381 :class:`!DatabaseConnection` that I'll assume::
382
383 class DatabaseConnection:
384 # Database interface
385 def cursor(self):
386 "Returns a cursor object and starts a new transaction"
387 def commit(self):
388 "Commits current transaction"
389 def rollback(self):
390 "Rolls back current transaction"
391
392 The :meth:`~object.__enter__` method is pretty easy, having only to start a new
393 transaction. For this application the resulting cursor object would be a useful
394 result, so the method will return it. The user can then add ``as cursor`` to
395 their ':keyword:`with`' statement to bind the cursor to a variable name. ::
396
397 class DatabaseConnection:
398 ...
399 def __enter__(self):
400 # Code to start a new transaction
401 cursor = self.cursor()
402 return cursor
403
404 The :meth:`~object.__exit__` method is the most complicated because it's where most of
405 the work has to be done. The method has to check if an exception occurred. If
406 there was no exception, the transaction is committed. The transaction is rolled
407 back if there was an exception.
408
409 In the code below, execution will just fall off the end of the function,
410 returning the default value of ``None``. ``None`` is false, so the exception
411 will be re-raised automatically. If you wished, you could be more explicit and
412 add a :keyword:`return` statement at the marked location. ::
413
414 class DatabaseConnection:
415 ...
416 def __exit__(self, type, value, tb):
417 if tb is None:
418 # No exception, so commit
419 self.commit()
420 else:
421 # Exception occurred, so rollback.
422 self.rollback()
423 # return False
424
425
426 .. _new-module-contextlib:
427
428 The contextlib module
429 ---------------------
430
431 The :mod:`contextlib` module provides some functions and a decorator that
432 are useful when writing objects for use with the ':keyword:`with`' statement.
433
434 The decorator is called :func:`~contextlib.contextmanager`, and lets you write
435 a single generator function instead of defining a new class. The generator
436 should yield exactly one value. The code up to the :keyword:`yield` will be
437 executed as the :meth:`~object.__enter__` method, and the value yielded will
438 be the method's return value that will get bound to the variable in the
439 ':keyword:`with`' statement's :keyword:`!as` clause, if any. The code after
440 the :keyword:`!yield` will be executed in the :meth:`~object.__exit__` method.
441 Any exception raised in the block will be raised by the :keyword:`!yield`
442 statement.
443
444 Using this decorator, our database example from the previous section
445 could be written as::
446
447 from contextlib import contextmanager
448
449 @contextmanager
450 def db_transaction(connection):
451 cursor = connection.cursor()
452 try:
453 yield cursor
454 except:
455 connection.rollback()
456 raise
457 else:
458 connection.commit()
459
460 db = DatabaseConnection()
461 with db_transaction(db) as cursor:
462 ...
463
464 The :mod:`contextlib` module also has a ``nested(mgr1, mgr2, ...)`` function
465 that combines a number of context managers so you don't need to write nested
466 ':keyword:`with`' statements. In this example, the single ':keyword:`!with`'
467 statement both starts a database transaction and acquires a thread lock::
468
469 lock = threading.Lock()
470 with nested (db_transaction(db), lock) as (cursor, locked):
471 ...
472
473 Finally, the :func:`~contextlib.closing` function returns its argument so that it can be
474 bound to a variable, and calls the argument's ``.close()`` method at the end
475 of the block. ::
476
477 import urllib, sys
478 from contextlib import closing
479
480 with closing(urllib.urlopen('http://www.yahoo.com')) as f:
481 for line in f:
482 sys.stdout.write(line)
483
484
485 .. seealso::
486
487 :pep:`343` - The "with" statement
488 PEP written by Guido van Rossum and Nick Coghlan; implemented by Mike Bland,
489 Guido van Rossum, and Neal Norwitz. The PEP shows the code generated for a
490 ':keyword:`with`' statement, which can be helpful in learning how the statement
491 works.
492
493 The documentation for the :mod:`contextlib` module.
494
495 .. ======================================================================
496
497 .. _pep-0366:
498
499 PEP 366: Explicit Relative Imports From a Main Module
500 ============================================================
501
502 Python's :option:`-m` switch allows running a module as a script.
503 When you ran a module that was located inside a package, relative
504 imports didn't work correctly.
505
506 The fix for Python 2.6 adds a :attr:`module.__package__` attribute.
507 When this attribute is present, relative imports will be
508 relative to the value of this attribute instead of the
509 :attr:`~module.__name__` attribute.
510
511 PEP 302-style importers can then set :attr:`~module.__package__` as necessary.
512 The :mod:`runpy` module that implements the :option:`-m` switch now
513 does this, so relative imports will now work correctly in scripts
514 running from inside a package.
515
516 .. ======================================================================
517
518 .. _pep-0370:
519
520 PEP 370: Per-user ``site-packages`` Directory
521 =====================================================
522
523 When you run Python, the module search path ``sys.path`` usually
524 includes a directory whose path ends in ``"site-packages"``. This
525 directory is intended to hold locally installed packages available to
526 all users using a machine or a particular site installation.
527
528 Python 2.6 introduces a convention for user-specific site directories.
529 The directory varies depending on the platform:
530
531 * Unix and Mac OS X: :file:`~/.local/`
532 * Windows: :file:`%APPDATA%/Python`
533
534 Within this directory, there will be version-specific subdirectories,
535 such as :file:`lib/python2.6/site-packages` on Unix/Mac OS and
536 :file:`Python26/site-packages` on Windows.
537
538 If you don't like the default directory, it can be overridden by an
539 environment variable. :envvar:`PYTHONUSERBASE` sets the root
540 directory used for all Python versions supporting this feature. On
541 Windows, the directory for application-specific data can be changed by
542 setting the :envvar:`!APPDATA` environment variable. You can also
543 modify the :file:`site.py` file for your Python installation.
544
545 The feature can be disabled entirely by running Python with the
546 :option:`-s` option or setting the :envvar:`PYTHONNOUSERSITE`
547 environment variable.
548
549 .. seealso::
550
551 :pep:`370` - Per-user ``site-packages`` Directory
552 PEP written and implemented by Christian Heimes.
553
554
555 .. ======================================================================
556
557 .. _pep-0371:
558
559 PEP 371: The ``multiprocessing`` Package
560 =====================================================
561
562 The new :mod:`multiprocessing` package lets Python programs create new
563 processes that will perform a computation and return a result to the
564 parent. The parent and child processes can communicate using queues
565 and pipes, synchronize their operations using locks and semaphores,
566 and can share simple arrays of data.
567
568 The :mod:`multiprocessing` module started out as an exact emulation of
569 the :mod:`threading` module using processes instead of threads. That
570 goal was discarded along the path to Python 2.6, but the general
571 approach of the module is still similar. The fundamental class
572 is the :class:`~multiprocessing.Process`, which is passed a callable object and
573 a collection of arguments. The :meth:`~multiprocessing.Process.start` method
574 sets the callable running in a subprocess, after which you can call
575 the :meth:`~multiprocessing.Process.is_alive` method to check whether the
576 subprocess is still running and the :meth:`~multiprocessing.Process.join`
577 method to wait for the process to exit.
578
579 Here's a simple example where the subprocess will calculate a
580 factorial. The function doing the calculation is written strangely so
581 that it takes significantly longer when the input argument is a
582 multiple of 4.
583
584 ::
585
586 import time
587 from multiprocessing import Process, Queue
588
589
590 def factorial(queue, N):
591 "Compute a factorial."
592 # If N is a multiple of 4, this function will take much longer.
593 if (N % 4) == 0:
594 time.sleep(.05 * N/4)
595
596 # Calculate the result
597 fact = 1L
598 for i in range(1, N+1):
599 fact = fact * i
600
601 # Put the result on the queue
602 queue.put(fact)
603
604 if __name__ == '__main__':
605 queue = Queue()
606
607 N = 5
608
609 p = Process(target=factorial, args=(queue, N))
610 p.start()
611 p.join()
612
613 result = queue.get()
614 print 'Factorial', N, '=', result
615
616 A :class:`~queue.Queue` is used to communicate the result of the factorial.
617 The :class:`~queue.Queue` object is stored in a global variable.
618 The child process will use the value of the variable when the child
619 was created; because it's a :class:`~queue.Queue`, parent and child can use
620 the object to communicate. (If the parent were to change the value of
621 the global variable, the child's value would be unaffected, and vice
622 versa.)
623
624 Two other classes, :class:`~multiprocessing.pool.Pool` and
625 :class:`~multiprocessing.Manager`, provide higher-level interfaces.
626 :class:`~multiprocessing.pool.Pool` will create a fixed number of worker
627 processes, and requests can then be distributed to the workers by calling
628 :meth:`~multiprocessing.pool.Pool.apply` or
629 :meth:`~multiprocessing.pool.Pool.apply_async` to add a single request, and
630 :meth:`~multiprocessing.pool.Pool.map` or
631 :meth:`~multiprocessing.pool.Pool.map_async` to add a number of
632 requests. The following code uses a :class:`~multiprocessing.pool.Pool` to
633 spread requests across 5 worker processes and retrieve a list of results::
634
635 from multiprocessing import Pool
636
637 def factorial(N, dictionary):
638 "Compute a factorial."
639 ...
640 p = Pool(5)
641 result = p.map(factorial, range(1, 1000, 10))
642 for v in result:
643 print v
644
645 This produces the following output::
646
647 1
648 39916800
649 51090942171709440000
650 8222838654177922817725562880000000
651 33452526613163807108170062053440751665152000000000
652 ...
653
654 The other high-level interface, the :class:`~multiprocessing.Manager` class,
655 creates a separate server process that can hold master copies of Python data
656 structures. Other processes can then access and modify these data
657 structures using proxy objects. The following example creates a
658 shared dictionary by calling the :meth:`dict` method; the worker
659 processes then insert values into the dictionary. (Locking is not
660 done for you automatically, which doesn't matter in this example.
661 :class:`~multiprocessing.Manager`'s methods also include
662 :meth:`~multiprocessing.managers.SyncManager.Lock`,
663 :meth:`~multiprocessing.managers.SyncManager.RLock`,
664 and :meth:`~multiprocessing.managers.SyncManager.Semaphore` to create
665 shared locks.)
666
667 ::
668
669 import time
670 from multiprocessing import Pool, Manager
671
672 def factorial(N, dictionary):
673 "Compute a factorial."
674 # Calculate the result
675 fact = 1L
676 for i in range(1, N+1):
677 fact = fact * i
678
679 # Store result in dictionary
680 dictionary[N] = fact
681
682 if __name__ == '__main__':
683 p = Pool(5)
684 mgr = Manager()
685 d = mgr.dict() # Create shared dictionary
686
687 # Run tasks using the pool
688 for N in range(1, 1000, 10):
689 p.apply_async(factorial, (N, d))
690
691 # Mark pool as closed -- no more tasks can be added.
692 p.close()
693
694 # Wait for tasks to exit
695 p.join()
696
697 # Output results
698 for k, v in sorted(d.items()):
699 print k, v
700
701 This will produce the output::
702
703 1 1
704 11 39916800
705 21 51090942171709440000
706 31 8222838654177922817725562880000000
707 41 33452526613163807108170062053440751665152000000000
708 51 15511187532873822802242430164693032110632597200169861120000...
709
710 .. seealso::
711
712 The documentation for the :mod:`multiprocessing` module.
713
714 :pep:`371` - Addition of the multiprocessing package
715 PEP written by Jesse Noller and Richard Oudkerk;
716 implemented by Richard Oudkerk and Jesse Noller.
717
718
719 .. ======================================================================
720
721 .. _pep-3101:
722
723 PEP 3101: Advanced String Formatting
724 =====================================================
725
726 In Python 3.0, the ``%`` operator is supplemented by a more powerful string
727 formatting method, :meth:`format`. Support for the :meth:`str.format` method
728 has been backported to Python 2.6.
729
730 In 2.6, both 8-bit and Unicode strings have a ``.format()`` method that
731 treats the string as a template and takes the arguments to be formatted.
732 The formatting template uses curly brackets (``{``, ``}``) as special characters::
733
734 >>> # Substitute positional argument 0 into the string.
735 >>> "User ID: {0}".format("root")
736 'User ID: root'
737 >>> # Use the named keyword arguments
738 >>> "User ID: {uid} Last seen: {last_login}".format(
739 ... uid="root",
740 ... last_login = "5 Mar 2008 07:20")
741 'User ID: root Last seen: 5 Mar 2008 07:20'
742
743 Curly brackets can be escaped by doubling them::
744
745 >>> "Empty dict: {{}}".format()
746 "Empty dict: {}"
747
748 Field names can be integers indicating positional arguments, such as
749 ``{0}``, ``{1}``, etc. or names of keyword arguments. You can also
750 supply compound field names that read attributes or access dictionary keys::
751
752 >>> import sys
753 >>> print 'Platform: {0.platform}\nPython version: {0.version}'.format(sys)
754 Platform: darwin
755 Python version: 2.6a1+ (trunk:61261M, Mar 5 2008, 20:29:41)
756 [GCC 4.0.1 (Apple Computer, Inc. build 5367)]'
757
758 >>> import mimetypes
759 >>> 'Content-type: {0[.mp4]}'.format(mimetypes.types_map)
760 'Content-type: video/mp4'
761
762 Note that when using dictionary-style notation such as ``[.mp4]``, you
763 don't need to put any quotation marks around the string; it will look
764 up the value using ``.mp4`` as the key. Strings beginning with a
765 number will be converted to an integer. You can't write more
766 complicated expressions inside a format string.
767
768 So far we've shown how to specify which field to substitute into the
769 resulting string. The precise formatting used is also controllable by
770 adding a colon followed by a format specifier. For example::
771
772 >>> # Field 0: left justify, pad to 15 characters
773 >>> # Field 1: right justify, pad to 6 characters
774 >>> fmt = '{0:15} ${1:>6}'
775 >>> fmt.format('Registration', 35)
776 'Registration $ 35'
777 >>> fmt.format('Tutorial', 50)
778 'Tutorial $ 50'
779 >>> fmt.format('Banquet', 125)
780 'Banquet $ 125'
781
782 Format specifiers can reference other fields through nesting::
783
784 >>> fmt = '{0:{1}}'
785 >>> width = 15
786 >>> fmt.format('Invoice #1234', width)
787 'Invoice #1234 '
788 >>> width = 35
789 >>> fmt.format('Invoice #1234', width)
790 'Invoice #1234 '
791
792 The alignment of a field within the desired width can be specified:
793
794 ================ ============================================
795 Character Effect
796 ================ ============================================
797 < (default) Left-align
798 > Right-align
799 ^ Center
800 = (For numeric types only) Pad after the sign.
801 ================ ============================================
802
803 Format specifiers can also include a presentation type, which
804 controls how the value is formatted. For example, floating-point numbers
805 can be formatted as a general number or in exponential notation::
806
807 >>> '{0:g}'.format(3.75)
808 '3.75'
809 >>> '{0:e}'.format(3.75)
810 '3.750000e+00'
811
812 A variety of presentation types are available. Consult the 2.6
813 documentation for a :ref:`complete list <formatstrings>`; here's a sample:
814
815 ===== ========================================================================
816 ``b`` Binary. Outputs the number in base 2.
817 ``c`` Character. Converts the integer to the corresponding Unicode character
818 before printing.
819 ``d`` Decimal Integer. Outputs the number in base 10.
820 ``o`` Octal format. Outputs the number in base 8.
821 ``x`` Hex format. Outputs the number in base 16, using lower-case letters for
822 the digits above 9.
823 ``e`` Exponent notation. Prints the number in scientific notation using the
824 letter 'e' to indicate the exponent.
825 ``g`` General format. This prints the number as a fixed-point number, unless
826 the number is too large, in which case it switches to 'e' exponent
827 notation.
828 ``n`` Number. This is the same as 'g' (for floats) or 'd' (for integers),
829 except that it uses the current locale setting to insert the appropriate
830 number separator characters.
831 ``%`` Percentage. Multiplies the number by 100 and displays in fixed ('f')
832 format, followed by a percent sign.
833 ===== ========================================================================
834
835 Classes and types can define a :meth:`~object.__format__` method to control how they're
836 formatted. It receives a single argument, the format specifier::
837
838 def __format__(self, format_spec):
839 if isinstance(format_spec, unicode):
840 return unicode(str(self))
841 else:
842 return str(self)
843
844 There's also a :func:`format` builtin that will format a single
845 value. It calls the type's :meth:`~object.__format__` method with the
846 provided specifier::
847
848 >>> format(75.6564, '.2f')
849 '75.66'
850
851
852 .. seealso::
853
854 :ref:`formatstrings`
855 The reference documentation for format fields.
856
857 :pep:`3101` - Advanced String Formatting
858 PEP written by Talin. Implemented by Eric Smith.
859
860 .. ======================================================================
861
862 .. _pep-3105:
863
864 PEP 3105: ``print`` As a Function
865 =====================================================
866
867 The ``print`` statement becomes the :func:`print` function in Python 3.0.
868 Making :func:`print` a function makes it possible to replace the function
869 by doing ``def print(...)`` or importing a new function from somewhere else.
870
871 Python 2.6 has a ``__future__`` import that removes ``print`` as language
872 syntax, letting you use the functional form instead. For example::
873
874 >>> from __future__ import print_function
875 >>> print('# of entries', len(dictionary), file=sys.stderr)
876
877 The signature of the new function is::
878
879 def print(*args, sep=' ', end='\n', file=None)
880
881
882 The parameters are:
883
884 * *args*: positional arguments whose values will be printed out.
885 * *sep*: the separator, which will be printed between arguments.
886 * *end*: the ending text, which will be printed after all of the
887 arguments have been output.
888 * *file*: the file object to which the output will be sent.
889
890 .. seealso::
891
892 :pep:`3105` - Make print a function
893 PEP written by Georg Brandl.
894
895 .. ======================================================================
896
897 .. _pep-3110:
898
899 PEP 3110: Exception-Handling Changes
900 =====================================================
901
902 One error that Python programmers occasionally make
903 is writing the following code::
904
905 try:
906 ...
907 except TypeError, ValueError: # Wrong!
908 ...
909
910 The author is probably trying to catch both :exc:`TypeError` and
911 :exc:`ValueError` exceptions, but this code actually does something
912 different: it will catch :exc:`TypeError` and bind the resulting
913 exception object to the local name ``"ValueError"``. The
914 :exc:`ValueError` exception will not be caught at all. The correct
915 code specifies a tuple of exceptions::
916
917 try:
918 ...
919 except (TypeError, ValueError):
920 ...
921
922 This error happens because the use of the comma here is ambiguous:
923 does it indicate two different nodes in the parse tree, or a single
924 node that's a tuple?
925
926 Python 3.0 makes this unambiguous by replacing the comma with the word
927 "as". To catch an exception and store the exception object in the
928 variable ``exc``, you must write::
929
930 try:
931 ...
932 except TypeError as exc:
933 ...
934
935 Python 3.0 will only support the use of "as", and therefore interprets
936 the first example as catching two different exceptions. Python 2.6
937 supports both the comma and "as", so existing code will continue to
938 work. We therefore suggest using "as" when writing new Python code
939 that will only be executed with 2.6.
940
941 .. seealso::
942
943 :pep:`3110` - Catching Exceptions in Python 3000
944 PEP written and implemented by Collin Winter.
945
946 .. ======================================================================
947
948 .. _pep-3112:
949
950 PEP 3112: Byte Literals
951 =====================================================
952
953 Python 3.0 adopts Unicode as the language's fundamental string type and
954 denotes 8-bit literals differently, either as ``b'string'``
955 or using a :class:`bytes` constructor. For future compatibility,
956 Python 2.6 adds :class:`bytes` as a synonym for the :class:`str` type,
957 and it also supports the ``b''`` notation.
958
959
960 The 2.6 :class:`str` differs from 3.0's :class:`bytes` type in various
961 ways; most notably, the constructor is completely different. In 3.0,
962 ``bytes([65, 66, 67])`` is 3 elements long, containing the bytes
963 representing ``ABC``; in 2.6, ``bytes([65, 66, 67])`` returns the
964 12-byte string representing the :func:`str` of the list.
965
966 The primary use of :class:`bytes` in 2.6 will be to write tests of
967 object type such as ``isinstance(x, bytes)``. This will help the 2to3
968 converter, which can't tell whether 2.x code intends strings to
969 contain either characters or 8-bit bytes; you can now
970 use either :class:`bytes` or :class:`str` to represent your intention
971 exactly, and the resulting code will also be correct in Python 3.0.
972
973 There's also a ``__future__`` import that causes all string literals
974 to become Unicode strings. This means that ``\u`` escape sequences
975 can be used to include Unicode characters::
976
977
978 from __future__ import unicode_literals
979
980 s = ('\u751f\u3080\u304e\u3000\u751f\u3054'
981 '\u3081\u3000\u751f\u305f\u307e\u3054')
982
983 print len(s) # 12 Unicode characters
984
985 At the C level, Python 3.0 will rename the existing 8-bit
986 string type, called :c:type:`!PyStringObject` in Python 2.x,
987 to :c:type:`PyBytesObject`. Python 2.6 uses ``#define``
988 to support using the names :c:func:`PyBytesObject`,
989 :c:func:`PyBytes_Check`, :c:func:`PyBytes_FromStringAndSize`,
990 and all the other functions and macros used with strings.
991
992 Instances of the :class:`bytes` type are immutable just
993 as strings are. A new :class:`bytearray` type stores a mutable
994 sequence of bytes::
995
996 >>> bytearray([65, 66, 67])
997 bytearray(b'ABC')
998 >>> b = bytearray(u'\u21ef\u3244', 'utf-8')
999 >>> b
1000 bytearray(b'\xe2\x87\xaf\xe3\x89\x84')
1001 >>> b[0] = '\xe3'
1002 >>> b
1003 bytearray(b'\xe3\x87\xaf\xe3\x89\x84')
1004 >>> unicode(str(b), 'utf-8')
1005 u'\u31ef \u3244'
1006
1007 Byte arrays support most of the methods of string types, such as
1008 :meth:`~bytearray.startswith`/:meth:`~bytearray.endswith`,
1009 :meth:`~bytearray.find`/:meth:`~bytearray.rfind`,
1010 and some of the methods of lists, such as :meth:`~bytearray.append`,
1011 :meth:`~bytearray.pop`, and :meth:`~bytearray.reverse`.
1012
1013 ::
1014
1015 >>> b = bytearray('ABC')
1016 >>> b.append('d')
1017 >>> b.append(ord('e'))
1018 >>> b
1019 bytearray(b'ABCde')
1020
1021 There's also a corresponding C API, with
1022 :c:func:`PyByteArray_FromObject`,
1023 :c:func:`PyByteArray_FromStringAndSize`,
1024 and various other functions.
1025
1026 .. seealso::
1027
1028 :pep:`3112` - Bytes literals in Python 3000
1029 PEP written by Jason Orendorff; backported to 2.6 by Christian Heimes.
1030
1031 .. ======================================================================
1032
1033 .. _pep-3116:
1034
1035 PEP 3116: New I/O Library
1036 =====================================================
1037
1038 Python's built-in file objects support a number of methods, but
1039 file-like objects don't necessarily support all of them. Objects that
1040 imitate files usually support :meth:`!read` and
1041 :meth:`!write`, but they may not support :meth:`!readline`,
1042 for example. Python 3.0 introduces a layered I/O library in the :mod:`io`
1043 module that separates buffering and text-handling features from the
1044 fundamental read and write operations.
1045
1046 There are three levels of abstract base classes provided by
1047 the :mod:`io` module:
1048
1049 * :class:`~io.RawIOBase` defines raw I/O operations: :meth:`~io.RawIOBase.read`,
1050 :meth:`~io.RawIOBase.readinto`, :meth:`~io.RawIOBase.write`,
1051 :meth:`~io.IOBase.seek`, :meth:`~io.IOBase.tell`, :meth:`~io.IOBase.truncate`,
1052 and :meth:`~io.IOBase.close`.
1053 Most of the methods of this class will often map to a single system call.
1054 There are also :meth:`~io.IOBase.readable`, :meth:`~io.IOBase.writable`,
1055 and :meth:`~io.IOBase.seekable` methods for determining what operations a
1056 given object will allow.
1057
1058 Python 3.0 has concrete implementations of this class for files and
1059 sockets, but Python 2.6 hasn't restructured its file and socket objects
1060 in this way.
1061
1062 * :class:`~io.BufferedIOBase` is an abstract base class that
1063 buffers data in memory to reduce the number of
1064 system calls used, making I/O processing more efficient.
1065 It supports all of the methods of :class:`~io.RawIOBase`,
1066 and adds a :attr:`~io.BufferedIOBase.raw` attribute holding the underlying
1067 raw object.
1068
1069 There are five concrete classes implementing this ABC.
1070 :class:`~io.BufferedWriter` and :class:`~io.BufferedReader` are for objects
1071 that support write-only or read-only usage that have a :meth:`~io.IOBase.seek`
1072 method for random access. :class:`~io.BufferedRandom` objects support
1073 read and write access upon the same underlying stream, and
1074 :class:`~io.BufferedRWPair` is for objects such as TTYs that have both
1075 read and write operations acting upon unconnected streams of data.
1076 The :class:`~io.BytesIO` class supports reading, writing, and seeking
1077 over an in-memory buffer.
1078
1079 .. index::
1080 single: universal newlines; What's new
1081
1082 * :class:`~io.TextIOBase`: Provides functions for reading and writing
1083 strings (remember, strings will be Unicode in Python 3.0),
1084 and supporting :term:`universal newlines`. :class:`~io.TextIOBase` defines
1085 the :meth:`readline` method and supports iteration upon
1086 objects.
1087
1088 There are two concrete implementations. :class:`~io.TextIOWrapper`
1089 wraps a buffered I/O object, supporting all of the methods for
1090 text I/O and adding a :attr:`~io.TextIOBase.buffer` attribute for access
1091 to the underlying object. :class:`~io.StringIO` simply buffers
1092 everything in memory without ever writing anything to disk.
1093
1094 (In Python 2.6, :class:`io.StringIO` is implemented in
1095 pure Python, so it's pretty slow. You should therefore stick with the
1096 existing :mod:`!StringIO` module or :mod:`!cStringIO` for now. At some
1097 point Python 3.0's :mod:`io` module will be rewritten into C for speed,
1098 and perhaps the C implementation will be backported to the 2.x releases.)
1099
1100 In Python 2.6, the underlying implementations haven't been
1101 restructured to build on top of the :mod:`io` module's classes. The
1102 module is being provided to make it easier to write code that's
1103 forward-compatible with 3.0, and to save developers the effort of writing
1104 their own implementations of buffering and text I/O.
1105
1106 .. seealso::
1107
1108 :pep:`3116` - New I/O
1109 PEP written by Daniel Stutzbach, Mike Verdone, and Guido van Rossum.
1110 Code by Guido van Rossum, Georg Brandl, Walter Doerwald,
1111 Jeremy Hylton, Martin von Löwis, Tony Lownds, and others.
1112
1113 .. ======================================================================
1114
1115 .. _pep-3118:
1116
1117 PEP 3118: Revised Buffer Protocol
1118 =====================================================
1119
1120 The buffer protocol is a C-level API that lets Python types
1121 exchange pointers into their internal representations. A
1122 memory-mapped file can be viewed as a buffer of characters, for
1123 example, and this lets another module such as :mod:`re`
1124 treat memory-mapped files as a string of characters to be searched.
1125
1126 The primary users of the buffer protocol are numeric-processing
1127 packages such as NumPy, which expose the internal representation
1128 of arrays so that callers can write data directly into an array instead
1129 of going through a slower API. This PEP updates the buffer protocol in light of experience
1130 from NumPy development, adding a number of new features
1131 such as indicating the shape of an array or locking a memory region.
1132
1133 The most important new C API function is
1134 ``PyObject_GetBuffer(PyObject *obj, Py_buffer *view, int flags)``, which
1135 takes an object and a set of flags, and fills in the
1136 ``Py_buffer`` structure with information
1137 about the object's memory representation. Objects
1138 can use this operation to lock memory in place
1139 while an external caller could be modifying the contents,
1140 so there's a corresponding ``PyBuffer_Release(Py_buffer *view)`` to
1141 indicate that the external caller is done.
1142
1143 The *flags* argument to :c:func:`PyObject_GetBuffer` specifies
1144 constraints upon the memory returned. Some examples are:
1145
1146 * :c:macro:`PyBUF_WRITABLE` indicates that the memory must be writable.
1147
1148 * :c:macro:`PyBUF_LOCK` requests a read-only or exclusive lock on the memory.
1149
1150 * :c:macro:`PyBUF_C_CONTIGUOUS` and :c:macro:`PyBUF_F_CONTIGUOUS`
1151 requests a C-contiguous (last dimension varies the fastest) or
1152 Fortran-contiguous (first dimension varies the fastest) array layout.
1153
1154 Two new argument codes for :c:func:`PyArg_ParseTuple`,
1155 ``s*`` and ``z*``, return locked buffer objects for a parameter.
1156
1157 .. seealso::
1158
1159 :pep:`3118` - Revising the buffer protocol
1160 PEP written by Travis Oliphant and Carl Banks; implemented by
1161 Travis Oliphant.
1162
1163
1164 .. ======================================================================
1165
1166 .. _pep-3119:
1167
1168 PEP 3119: Abstract Base Classes
1169 =====================================================
1170
1171 Some object-oriented languages such as Java support interfaces,
1172 declaring that a class has a given set of methods or supports a given
1173 access protocol. Abstract Base Classes (or ABCs) are an equivalent
1174 feature for Python. The ABC support consists of an :mod:`abc` module
1175 containing a metaclass called :class:`~abc.ABCMeta`, special handling of
1176 this metaclass by the :func:`isinstance` and :func:`issubclass`
1177 builtins, and a collection of basic ABCs that the Python developers
1178 think will be widely useful. Future versions of Python will probably
1179 add more ABCs.
1180
1181 Let's say you have a particular class and wish to know whether it supports
1182 dictionary-style access. The phrase "dictionary-style" is vague, however.
1183 It probably means that accessing items with ``obj[1]`` works.
1184 Does it imply that setting items with ``obj[2] = value`` works?
1185 Or that the object will have :meth:`!keys`, :meth:`!values`, and :meth:`!items`
1186 methods? What about the iterative variants such as :meth:`!iterkeys`?
1187 :meth:`!copy`and :meth:`!update`? Iterating over the object with :func:`!iter`?
1188
1189 The Python 2.6 :mod:`collections` module includes a number of
1190 different ABCs that represent these distinctions. :class:`Iterable`
1191 indicates that a class defines :meth:`~object.__iter__`, and
1192 :class:`Container` means the class defines a :meth:`~object.__contains__`
1193 method and therefore supports ``x in y`` expressions. The basic
1194 dictionary interface of getting items, setting items, and
1195 :meth:`!keys`, :meth:`!values`, and :meth:`!items`, is defined by the
1196 :class:`MutableMapping` ABC.
1197
1198 You can derive your own classes from a particular ABC
1199 to indicate they support that ABC's interface::
1200
1201 import collections
1202
1203 class Storage(collections.MutableMapping):
1204 ...
1205
1206
1207 Alternatively, you could write the class without deriving from
1208 the desired ABC and instead register the class by
1209 calling the ABC's :meth:`~abc.ABCMeta.register` method::
1210
1211 import collections
1212
1213 class Storage:
1214 ...
1215
1216 collections.MutableMapping.register(Storage)
1217
1218 For classes that you write, deriving from the ABC is probably clearer.
1219 The :meth:`~abc.ABCMeta.register` method is useful when you've written a new
1220 ABC that can describe an existing type or class, or if you want
1221 to declare that some third-party class implements an ABC.
1222 For example, if you defined a :class:`!PrintableType` ABC,
1223 it's legal to do::
1224
1225 # Register Python's types
1226 PrintableType.register(int)
1227 PrintableType.register(float)
1228 PrintableType.register(str)
1229
1230 Classes should obey the semantics specified by an ABC, but
1231 Python can't check this; it's up to the class author to
1232 understand the ABC's requirements and to implement the code accordingly.
1233
1234 To check whether an object supports a particular interface, you can
1235 now write::
1236
1237 def func(d):
1238 if not isinstance(d, collections.MutableMapping):
1239 raise ValueError("Mapping object expected, not %r" % d)
1240
1241 Don't feel that you must now begin writing lots of checks as in the
1242 above example. Python has a strong tradition of duck-typing, where
1243 explicit type-checking is never done and code simply calls methods on
1244 an object, trusting that those methods will be there and raising an
1245 exception if they aren't. Be judicious in checking for ABCs and only
1246 do it where it's absolutely necessary.
1247
1248 You can write your own ABCs by using ``abc.ABCMeta`` as the
1249 metaclass in a class definition::
1250
1251 from abc import ABCMeta, abstractmethod
1252
1253 class Drawable():
1254 __metaclass__ = ABCMeta
1255
1256 @abstractmethod
1257 def draw(self, x, y, scale=1.0):
1258 pass
1259
1260 def draw_doubled(self, x, y):
1261 self.draw(x, y, scale=2.0)
1262
1263
1264 class Square(Drawable):
1265 def draw(self, x, y, scale):
1266 ...
1267
1268
1269 In the :class:`!Drawable` ABC above, the :meth:`!draw_doubled` method
1270 renders the object at twice its size and can be implemented in terms
1271 of other methods described in :class:`!Drawable`. Classes implementing
1272 this ABC therefore don't need to provide their own implementation
1273 of :meth:`!draw_doubled`, though they can do so. An implementation
1274 of :meth:`!draw` is necessary, though; the ABC can't provide
1275 a useful generic implementation.
1276
1277 You can apply the :deco:`~abc.abstractmethod` decorator to methods such as
1278 :meth:`!draw` that must be implemented; Python will then raise an
1279 exception for classes that don't define the method.
1280 Note that the exception is only raised when you actually
1281 try to create an instance of a subclass lacking the method::
1282
1283 >>> class Circle(Drawable):
1284 ... pass
1285 ...
1286 >>> c = Circle()
1287 Traceback (most recent call last):
1288 File "<stdin>", line 1, in <module>
1289 TypeError: Can't instantiate abstract class Circle with abstract methods draw
1290 >>>
1291
1292 Abstract data attributes can be declared using the
1293 ``@abstractproperty`` decorator::
1294
1295 from abc import abstractproperty
1296 ...
1297
1298 @abstractproperty
1299 def readonly(self):
1300 return self._x
1301
1302 Subclasses must then define a ``readonly`` property.
1303
1304 .. seealso::
1305
1306 :pep:`3119` - Introducing Abstract Base Classes
1307 PEP written by Guido van Rossum and Talin.
1308 Implemented by Guido van Rossum.
1309 Backported to 2.6 by Benjamin Aranguren, with Alex Martelli.
1310
1311 .. ======================================================================
1312
1313 .. _pep-3127:
1314
1315 PEP 3127: Integer Literal Support and Syntax
1316 =====================================================
1317
1318 Python 3.0 changes the syntax for octal (base-8) integer literals,
1319 prefixing them with "0o" or "0O" instead of a leading zero, and adds
1320 support for binary (base-2) integer literals, signalled by a "0b" or
1321 "0B" prefix.
1322
1323 Python 2.6 doesn't drop support for a leading 0 signalling
1324 an octal number, but it does add support for "0o" and "0b"::
1325
1326 >>> 0o21, 2*8 + 1
1327 (17, 17)
1328 >>> 0b101111
1329 47
1330
1331 The :func:`oct` builtin still returns numbers
1332 prefixed with a leading zero, and a new :func:`bin`
1333 builtin returns the binary representation for a number::
1334
1335 >>> oct(42)
1336 '052'
1337 >>> future_builtins.oct(42)
1338 '0o52'
1339 >>> bin(173)
1340 '0b10101101'
1341
1342 The :func:`int` and :func:`long` builtins will now accept the "0o"
1343 and "0b" prefixes when base-8 or base-2 are requested, or when the
1344 *base* argument is zero (signalling that the base used should be
1345 determined from the string)::
1346
1347 >>> int ('0o52', 0)
1348 42
1349 >>> int('1101', 2)
1350 13
1351 >>> int('0b1101', 2)
1352 13
1353 >>> int('0b1101', 0)
1354 13
1355
1356
1357 .. seealso::
1358
1359 :pep:`3127` - Integer Literal Support and Syntax
1360 PEP written by Patrick Maupin; backported to 2.6 by
1361 Eric Smith.
1362
1363 .. ======================================================================
1364
1365 .. _pep-3129:
1366
1367 PEP 3129: Class Decorators
1368 =====================================================
1369
1370 Decorators have been extended from functions to classes. It's now legal to
1371 write::
1372
1373 @foo
1374 @bar
1375 class A:
1376 pass
1377
1378 This is equivalent to::
1379
1380 class A:
1381 pass
1382
1383 A = foo(bar(A))
1384
1385 .. seealso::
1386
1387 :pep:`3129` - Class Decorators
1388 PEP written by Collin Winter.
1389
1390 .. ======================================================================
1391
1392 .. _pep-3141:
1393
1394 PEP 3141: A Type Hierarchy for Numbers
1395 =====================================================
1396
1397 Python 3.0 adds several abstract base classes for numeric types
1398 inspired by Scheme's numeric tower. These classes were backported to
1399 2.6 as the :mod:`numbers` module.
1400
1401 The most general ABC is :class:`Number`. It defines no operations at
1402 all, and only exists to allow checking if an object is a number by
1403 doing ``isinstance(obj, Number)``.
1404
1405 :class:`Complex` is a subclass of :class:`Number`. Complex numbers
1406 can undergo the basic operations of addition, subtraction,
1407 multiplication, division, and exponentiation, and you can retrieve the
1408 real and imaginary parts and obtain a number's conjugate. Python's built-in
1409 complex type is an implementation of :class:`Complex`.
1410
1411 :class:`Real` further derives from :class:`Complex`, and adds
1412 operations that only work on real numbers: :func:`floor`, :func:`trunc`,
1413 rounding, taking the remainder mod N, floor division,
1414 and comparisons.
1415
1416 :class:`Rational` numbers derive from :class:`Real`, have
1417 :attr:`numerator` and :attr:`denominator` properties, and can be
1418 converted to floats. Python 2.6 adds a simple rational-number class,
1419 :class:`Fraction`, in the :mod:`fractions` module. (It's called
1420 :class:`Fraction` instead of :class:`Rational` to avoid
1421 a name clash with :class:`numbers.Rational`.)
1422
1423 :class:`Integral` numbers derive from :class:`Rational`, and
1424 can be shifted left and right with ``<<`` and ``>>``,
1425 combined using bitwise operations such as ``&`` and ``|``,
1426 and can be used as array indexes and slice boundaries.
1427
1428 In Python 3.0, the PEP slightly redefines the existing builtins
1429 :func:`round`, :func:`math.floor`, :func:`math.ceil`, and adds a new
1430 one, :func:`math.trunc`, that's been backported to Python 2.6.
1431 :func:`math.trunc` rounds toward zero, returning the closest
1432 :class:`Integral` that's between the function's argument and zero.
1433
1434 .. seealso::
1435
1436 :pep:`3141` - A Type Hierarchy for Numbers
1437 PEP written by Jeffrey Yasskin.
1438
1439 `Scheme's numerical tower <https://www.gnu.org/software/guile/manual/html_node/Numerical-Tower.html#Numerical-Tower>`__, from the Guile manual.
1440
1441 `Scheme's number datatypes <https://conservatory.scheme.org/schemers/Documents/Standards/R5RS/HTML/r5rs-Z-H-9.html#%_sec_6.2>`__ from the R5RS Scheme specification.
1442
1443
1444 The :mod:`fractions` Module
1445 --------------------------------------------------
1446
1447 To fill out the hierarchy of numeric types, the :mod:`fractions`
1448 module provides a rational-number class. Rational numbers store their
1449 values as a numerator and denominator forming a fraction, and can
1450 exactly represent numbers such as ``2/3`` that floating-point numbers
1451 can only approximate.
1452
1453 The :class:`Fraction` constructor takes two :class:`Integral` values
1454 that will be the numerator and denominator of the resulting fraction. ::
1455
1456 >>> from fractions import Fraction
1457 >>> a = Fraction(2, 3)
1458 >>> b = Fraction(2, 5)
1459 >>> float(a), float(b)
1460 (0.66666666666666663, 0.40000000000000002)
1461 >>> a+b
1462 Fraction(16, 15)
1463 >>> a/b
1464 Fraction(5, 3)
1465
1466 For converting floating-point numbers to rationals,
1467 the float type now has an :meth:`as_integer_ratio` method that returns
1468 the numerator and denominator for a fraction that evaluates to the same
1469 floating-point value::
1470
1471 >>> (2.5) .as_integer_ratio()
1472 (5, 2)
1473 >>> (3.1415) .as_integer_ratio()
1474 (7074029114692207L, 2251799813685248L)
1475 >>> (1./3) .as_integer_ratio()
1476 (6004799503160661L, 18014398509481984L)
1477
1478 Note that values that can only be approximated by floating-point
1479 numbers, such as 1./3, are not simplified to the number being
1480 approximated; the fraction attempts to match the floating-point value
1481 **exactly**.
1482
1483 The :mod:`fractions` module is based upon an implementation by Sjoerd
1484 Mullender that was in Python's :file:`Demo/classes/` directory for a
1485 long time. This implementation was significantly updated by Jeffrey
1486 Yasskin.
1487
1488
1489 Other Language Changes
1490 ======================
1491
1492 Some smaller changes made to the core Python language are:
1493
1494 * Directories and zip archives containing a :file:`__main__.py` file
1495 can now be executed directly by passing their name to the
1496 interpreter. The directory or zip archive is automatically inserted
1497 as the first entry in sys.path. (Suggestion and initial patch by
1498 Andy Chu, subsequently revised by Phillip J. Eby and Nick Coghlan;
1499 :issue:`1739468`.)
1500
1501 * The :func:`hasattr` function was catching and ignoring all errors,
1502 under the assumption that they meant a :meth:`__getattr__` method
1503 was failing somehow and the return value of :func:`hasattr` would
1504 therefore be ``False``. This logic shouldn't be applied to
1505 :exc:`KeyboardInterrupt` and :exc:`SystemExit`, however; Python 2.6
1506 will no longer discard such exceptions when :func:`hasattr`
1507 encounters them. (Fixed by Benjamin Peterson; :issue:`2196`.)
1508
1509 * When calling a function using the ``**`` syntax to provide keyword
1510 arguments, you are no longer required to use a Python dictionary;
1511 any mapping will now work::
1512
1513 >>> def f(**kw):
1514 ... print sorted(kw)
1515 ...
1516 >>> ud=UserDict.UserDict()
1517 >>> ud['a'] = 1
1518 >>> ud['b'] = 'string'
1519 >>> f(**ud)
1520 ['a', 'b']
1521
1522 (Contributed by Alexander Belopolsky; :issue:`1686487`.)
1523
1524 It's also become legal to provide keyword arguments after a ``*args`` argument
1525 to a function call. ::
1526
1527 >>> def f(*args, **kw):
1528 ... print args, kw
1529 ...
1530 >>> f(1,2,3, *(4,5,6), keyword=13)
1531 (1, 2, 3, 4, 5, 6) {'keyword': 13}
1532
1533 Previously this would have been a syntax error.
1534 (Contributed by Amaury Forgeot d'Arc; :issue:`3473`.)
1535
1536 * A new builtin, ``next(iterator, [default])`` returns the next item
1537 from the specified iterator. If the *default* argument is supplied,
1538 it will be returned if *iterator* has been exhausted; otherwise,
1539 the :exc:`StopIteration` exception will be raised. (Backported
1540 in :issue:`2719`.)
1541
1542 * Tuples now have :meth:`~tuple.index` and :meth:`~tuple.count` methods
1543 matching the list type's :meth:`~list.index` and :meth:`~list.count` methods::
1544
1545 >>> t = (0,1,2,3,4,0,1,2)
1546 >>> t.index(3)
1547 3
1548 >>> t.count(0)
1549 2
1550
1551 (Contributed by Raymond Hettinger)
1552
1553 * The built-in types now have improved support for extended slicing syntax,
1554 accepting various combinations of ``(start, stop, step)``.
1555 Previously, the support was partial and certain corner cases wouldn't work.
1556 (Implemented by Thomas Wouters.)
1557
1558 .. Revision 57619
1559
1560 * Properties now have three attributes, :attr:`getter`, :attr:`setter`
1561 and :attr:`deleter`, that are decorators providing useful shortcuts
1562 for adding a getter, setter or deleter function to an existing
1563 property. You would use them like this::
1564
1565 class C(object):
1566 @property
1567 def x(self):
1568 return self._x
1569
1570 @x.setter
1571 def x(self, value):
1572 self._x = value
1573
1574 @x.deleter
1575 def x(self):
1576 del self._x
1577
1578 class D(C):
1579 @C.x.getter
1580 def x(self):
1581 return self._x * 2
1582
1583 @x.setter
1584 def x(self, value):
1585 self._x = value / 2
1586
1587 * Several methods of the built-in set types now accept multiple iterables:
1588 :meth:`intersection`,
1589 :meth:`intersection_update`,
1590 :meth:`union`, :meth:`update`,
1591 :meth:`difference` and :meth:`difference_update`.
1592
1593 ::
1594
1595 >>> s=set('1234567890')
1596 >>> s.intersection('abc123', 'cdf246') # Intersection between all inputs
1597 set(['2'])
1598 >>> s.difference('246', '789')
1599 set(['1', '0', '3', '5'])
1600
1601 (Contributed by Raymond Hettinger.)
1602
1603 * Many floating-point features were added. The :func:`float` function
1604 will now turn the string ``nan`` into an
1605 IEEE 754 Not A Number value, and ``+inf`` and ``-inf`` into
1606 positive or negative infinity. This works on any platform with
1607 IEEE 754 semantics. (Contributed by Christian Heimes; :issue:`1635`.)
1608
1609 Other functions in the :mod:`math` module, :func:`isinf` and
1610 :func:`isnan`, return true if their floating-point argument is
1611 infinite or Not A Number. (:issue:`1640`)
1612
1613 Conversion functions were added to convert floating-point numbers
1614 into hexadecimal strings (:issue:`3008`). These functions
1615 convert floats to and from a string representation without
1616 introducing rounding errors from the conversion between decimal and
1617 binary. Floats have a :meth:`hex` method that returns a string
1618 representation, and the ``float.fromhex()`` method converts a string
1619 back into a number::
1620
1621 >>> a = 3.75
1622 >>> a.hex()
1623 '0x1.e000000000000p+1'
1624 >>> float.fromhex('0x1.e000000000000p+1')
1625 3.75
1626 >>> b=1./3
1627 >>> b.hex()
1628 '0x1.5555555555555p-2'
1629
1630 * A numerical nicety: when creating a complex number from two floats
1631 on systems that support signed zeros (-0 and +0), the
1632 :func:`complex` constructor will now preserve the sign
1633 of the zero. (Fixed by Mark T. Dickinson; :issue:`1507`.)
1634
1635 * Classes that inherit a :meth:`__hash__` method from a parent class
1636 can set ``__hash__ = None`` to indicate that the class isn't
1637 hashable. This will make ``hash(obj)`` raise a :exc:`TypeError`
1638 and the class will not be indicated as implementing the
1639 :class:`Hashable` ABC.
1640
1641 You should do this when you've defined a :meth:`__cmp__` or
1642 :meth:`__eq__` method that compares objects by their value rather
1643 than by identity. All objects have a default hash method that uses
1644 ``id(obj)`` as the hash value. There's no tidy way to remove the
1645 :meth:`__hash__` method inherited from a parent class, so
1646 assigning ``None`` was implemented as an override. At the
1647 C level, extensions can set ``tp_hash`` to
1648 :c:func:`PyObject_HashNotImplemented`.
1649 (Fixed by Nick Coghlan and Amaury Forgeot d'Arc; :issue:`2235`.)
1650
1651 * The :exc:`GeneratorExit` exception now subclasses
1652 :exc:`BaseException` instead of :exc:`Exception`. This means
1653 that an exception handler that does ``except Exception:``
1654 will not inadvertently catch :exc:`GeneratorExit`.
1655 (Contributed by Chad Austin; :issue:`1537`.)
1656
1657 * Generator objects now have a :attr:`gi_code` attribute that refers to
1658 the original code object backing the generator.
1659 (Contributed by Collin Winter; :issue:`1473257`.)
1660
1661 * The :func:`compile` built-in function now accepts keyword arguments
1662 as well as positional parameters. (Contributed by Thomas Wouters;
1663 :issue:`1444529`.)
1664
1665 * The :func:`complex` constructor now accepts strings containing
1666 parenthesized complex numbers, meaning that ``complex(repr(cplx))``
1667 will now round-trip values. For example, ``complex('(3+4j)')``
1668 now returns the value (3+4j). (:issue:`1491866`)
1669
1670 * The string :meth:`translate` method now accepts ``None`` as the
1671 translation table parameter, which is treated as the identity
1672 transformation. This makes it easier to carry out operations
1673 that only delete characters. (Contributed by Bengt Richter and
1674 implemented by Raymond Hettinger; :issue:`1193128`.)
1675
1676 * The built-in :func:`dir` function now checks for a :meth:`__dir__`
1677 method on the objects it receives. This method must return a list
1678 of strings containing the names of valid attributes for the object,
1679 and lets the object control the value that :func:`dir` produces.
1680 Objects that have :meth:`__getattr__` or :meth:`__getattribute__`
1681 methods can use this to advertise pseudo-attributes they will honor.
1682 (:issue:`1591665`)
1683
1684 * Instance method objects have new attributes for the object and function
1685 comprising the method; the new synonym for :attr:`!im_self` is
1686 :attr:`~method.__self__`, and :attr:`!im_func` is also available as
1687 :attr:`~method.__func__`.
1688 The old names are still supported in Python 2.6, but are gone in 3.0.
1689
1690 * An obscure change: when you use the :func:`locals` function inside a
1691 :keyword:`class` statement, the resulting dictionary no longer returns free
1692 variables. (Free variables, in this case, are variables referenced in the
1693 :keyword:`!class` statement that aren't attributes of the class.)
1694
1695 .. ======================================================================
1696
1697
1698 Optimizations
1699 -------------
1700
1701 * The :mod:`warnings` module has been rewritten in C. This makes
1702 it possible to invoke warnings from the parser, and may also
1703 make the interpreter's startup faster.
1704 (Contributed by Neal Norwitz and Brett Cannon; :issue:`1631171`.)
1705
1706 * Type objects now have a cache of methods that can reduce
1707 the work required to find the correct method implementation
1708 for a particular class; once cached, the interpreter doesn't need to
1709 traverse base classes to figure out the right method to call.
1710 The cache is cleared if a base class or the class itself is modified,
1711 so the cache should remain correct even in the face of Python's dynamic
1712 nature.
1713 (Original optimization implemented by Armin Rigo, updated for
1714 Python 2.6 by Kevin Jacobs; :issue:`1700288`.)
1715
1716 By default, this change is only applied to types that are included with
1717 the Python core. Extension modules may not necessarily be compatible with
1718 this cache,
1719 so they must explicitly add :c:macro:`Py_TPFLAGS_HAVE_VERSION_TAG`
1720 to the module's ``tp_flags`` field to enable the method cache.
1721 (To be compatible with the method cache, the extension module's code
1722 must not directly access and modify the ``tp_dict`` member of
1723 any of the types it implements. Most modules don't do this,
1724 but it's impossible for the Python interpreter to determine that.
1725 See :issue:`1878` for some discussion.)
1726
1727 * Function calls that use keyword arguments are significantly faster
1728 by doing a quick pointer comparison, usually saving the time of a
1729 full string comparison. (Contributed by Raymond Hettinger, after an
1730 initial implementation by Antoine Pitrou; :issue:`1819`.)
1731
1732 * All of the functions in the :mod:`struct` module have been rewritten in
1733 C, thanks to work at the Need For Speed sprint.
1734 (Contributed by Raymond Hettinger.)
1735
1736 * Some of the standard built-in types now set a bit in their type
1737 objects. This speeds up checking whether an object is a subclass of
1738 one of these types. (Contributed by Neal Norwitz.)
1739
1740 * Unicode strings now use faster code for detecting
1741 whitespace and line breaks; this speeds up the :meth:`split` method
1742 by about 25% and :meth:`splitlines` by 35%.
1743 (Contributed by Antoine Pitrou.) Memory usage is reduced
1744 by using pymalloc for the Unicode string's data.
1745
1746 * The ``with`` statement now stores the :meth:`~object.__exit__` method on the stack,
1747 producing a small speedup. (Implemented by Jeffrey Yasskin.)
1748
1749 * To reduce memory usage, the garbage collector will now clear internal
1750 free lists when garbage-collecting the highest generation of objects.
1751 This may return memory to the operating system sooner.
1752
1753 .. ======================================================================
1754
1755 .. _new-26-interpreter:
1756
1757 Interpreter Changes
1758 -------------------------------
1759
1760 Two command-line options have been reserved for use by other Python
1761 implementations. The :option:`!-J` switch has been reserved for use by
1762 Jython for Jython-specific options, such as switches that are passed to
1763 the underlying JVM. :option:`-X` has been reserved for options
1764 specific to a particular implementation of Python such as CPython,
1765 Jython, or IronPython. If either option is used with Python 2.6, the
1766 interpreter will report that the option isn't currently used.
1767
1768 Python can now be prevented from writing :file:`.pyc` or :file:`.pyo`
1769 files by supplying the :option:`-B` switch to the Python interpreter,
1770 or by setting the :envvar:`PYTHONDONTWRITEBYTECODE` environment
1771 variable before running the interpreter. This setting is available to
1772 Python programs as the ``sys.dont_write_bytecode`` variable, and
1773 Python code can change the value to modify the interpreter's
1774 behaviour. (Contributed by Neal Norwitz and Georg Brandl.)
1775
1776 The encoding used for standard input, output, and standard error can
1777 be specified by setting the :envvar:`PYTHONIOENCODING` environment
1778 variable before running the interpreter. The value should be a string
1779 in the form ``<encoding>`` or ``<encoding>:<errorhandler>``.
1780 The *encoding* part specifies the encoding's name, e.g. ``utf-8`` or
1781 ``latin-1``; the optional *errorhandler* part specifies
1782 what to do with characters that can't be handled by the encoding,
1783 and should be one of "error", "ignore", or "replace". (Contributed
1784 by Martin von Löwis.)
1785
1786 .. ======================================================================
1787
1788 New and Improved Modules
1789 ========================
1790
1791 As in every release, Python's standard library received a number of
1792 enhancements and bug fixes. Here's a partial list of the most notable
1793 changes, sorted alphabetically by module name. Consult the
1794 :file:`Misc/NEWS` file in the source tree for a more complete list of
1795 changes, or look through the Subversion logs for all the details.
1796
1797 * The :mod:`!asyncore` and :mod:`!asynchat` modules are
1798 being actively maintained again, and a number of patches and bugfixes
1799 were applied. (Maintained by Josiah Carlson; see :issue:`1736190` for
1800 one patch.)
1801
1802 * The :mod:`bsddb` module also has a new maintainer, Jesús Cea Avión, and the package
1803 is now available as a standalone package. The web page for the package is
1804 `www.jcea.es/programacion/pybsddb.htm
1805 <https://www.jcea.es/programacion/pybsddb.htm>`__.
1806 The plan is to remove the package from the standard library
1807 in Python 3.0, because its pace of releases is much more frequent than
1808 Python's.
1809
1810 The :mod:`bsddb.dbshelve` module now uses the highest pickling protocol
1811 available, instead of restricting itself to protocol 1.
1812 (Contributed by W. Barnes.)
1813
1814 * The :mod:`!cgi` module will now read variables from the query string
1815 of an HTTP POST request. This makes it possible to use form actions
1816 with URLs that include query strings such as
1817 "/cgi-bin/add.py?category=1". (Contributed by Alexandre Fiori and
1818 Nubis; :issue:`1817`.)
1819
1820 The :func:`parse_qs` and :func:`parse_qsl` functions have been
1821 relocated from the :mod:`!cgi` module to the :mod:`urlparse <urllib.parse>` module.
1822 The versions still available in the :mod:`!cgi` module will
1823 trigger :exc:`PendingDeprecationWarning` messages in 2.6
1824 (:issue:`600362`).
1825
1826 * The :mod:`cmath` module underwent extensive revision,
1827 contributed by Mark Dickinson and Christian Heimes.
1828 Five new functions were added:
1829
1830 * :func:`polar` converts a complex number to polar form, returning
1831 the modulus and argument of the complex number.
1832
1833 * :func:`rect` does the opposite, turning a modulus, argument pair
1834 back into the corresponding complex number.
1835
1836 * :func:`phase` returns the argument (also called the angle) of a complex
1837 number.
1838
1839 * :func:`isnan` returns True if either
1840 the real or imaginary part of its argument is a NaN.
1841
1842 * :func:`isinf` returns True if either the real or imaginary part of
1843 its argument is infinite.
1844
1845 The revisions also improved the numerical soundness of the
1846 :mod:`cmath` module. For all functions, the real and imaginary
1847 parts of the results are accurate to within a few units of least
1848 precision (ulps) whenever possible. See :issue:`1381` for the
1849 details. The branch cuts for :func:`asinh`, :func:`atanh`: and
1850 :func:`atan` have also been corrected.
1851
1852 The tests for the module have been greatly expanded; nearly 2000 new
1853 test cases exercise the algebraic functions.
1854
1855 On IEEE 754 platforms, the :mod:`cmath` module now handles IEEE 754
1856 special values and floating-point exceptions in a manner consistent
1857 with Annex 'G' of the C99 standard.
1858
1859 * A new data type in the :mod:`collections` module: ``namedtuple(typename, fieldnames)``
1860 is a factory function that creates subclasses of the standard tuple
1861 whose fields are accessible by name as well as index. For example::
1862
1863 >>> var_type = collections.namedtuple('variable',
1864 ... 'id name type size')
1865 >>> # Names are separated by spaces or commas.
1866 >>> # 'id, name, type, size' would also work.
1867 >>> var_type._fields
1868 ('id', 'name', 'type', 'size')
1869
1870 >>> var = var_type(1, 'frequency', 'int', 4)
1871 >>> print var[0], var.id # Equivalent
1872 1 1
1873 >>> print var[2], var.type # Equivalent
1874 int int
1875 >>> var._asdict()
1876 {'size': 4, 'type': 'int', 'id': 1, 'name': 'frequency'}
1877 >>> v2 = var._replace(name='amplitude')
1878 >>> v2
1879 variable(id=1, name='amplitude', type='int', size=4)
1880
1881 Several places in the standard library that returned tuples have
1882 been modified to return :func:`namedtuple` instances. For example,
1883 the :meth:`Decimal.as_tuple` method now returns a named tuple with
1884 :attr:`sign`, :attr:`digits`, and :attr:`exponent` fields.
1885
1886 (Contributed by Raymond Hettinger.)
1887
1888 * Another change to the :mod:`collections` module is that the
1889 :class:`deque` type now supports an optional *maxlen* parameter;
1890 if supplied, the deque's size will be restricted to no more
1891 than *maxlen* items. Adding more items to a full deque causes
1892 old items to be discarded.
1893
1894 ::
1895
1896 >>> from collections import deque
1897 >>> dq=deque(maxlen=3)
1898 >>> dq
1899 deque([], maxlen=3)
1900 >>> dq.append(1); dq.append(2); dq.append(3)
1901 >>> dq
1902 deque([1, 2, 3], maxlen=3)
1903 >>> dq.append(4)
1904 >>> dq
1905 deque([2, 3, 4], maxlen=3)
1906
1907 (Contributed by Raymond Hettinger.)
1908
1909 * The :mod:`Cookie <http.cookies>` module's :class:`~http.cookies.Morsel` objects now support an
1910 :attr:`~http.cookies.Morsel.httponly` attribute. In some browsers. cookies with this attribute
1911 set cannot be accessed or manipulated by JavaScript code.
1912 (Contributed by Arvin Schnell; :issue:`1638033`.)
1913
1914 * A new window method in the :mod:`curses` module,
1915 :meth:`chgat`, changes the display attributes for a certain number of
1916 characters on a single line. (Contributed by Fabian Kreutz.)
1917
1918 ::
1919
1920 # Boldface text starting at y=0,x=21
1921 # and affecting the rest of the line.
1922 stdscr.chgat(0, 21, curses.A_BOLD)
1923
1924 The :class:`Textbox` class in the :mod:`curses.textpad` module
1925 now supports editing in insert mode as well as overwrite mode.
1926 Insert mode is enabled by supplying a true value for the *insert_mode*
1927 parameter when creating the :class:`Textbox` instance.
1928
1929 * The :mod:`datetime` module's :meth:`strftime` methods now support a
1930 ``%f`` format code that expands to the number of microseconds in the
1931 object, zero-padded on
1932 the left to six places. (Contributed by Skip Montanaro; :issue:`1158`.)
1933
1934 * The :mod:`decimal` module was updated to version 1.66 of
1935 `the General Decimal Specification <https://speleotrove.com/decimal/decarith.html>`__. New features
1936 include some methods for some basic mathematical functions such as
1937 :meth:`exp` and :meth:`log10`::
1938
1939 >>> Decimal(1).exp()
1940 Decimal("2.718281828459045235360287471")
1941 >>> Decimal("2.7182818").ln()
1942 Decimal("0.9999999895305022877376682436")
1943 >>> Decimal(1000).log10()
1944 Decimal("3")
1945
1946 The :meth:`as_tuple` method of :class:`Decimal` objects now returns a
1947 named tuple with :attr:`sign`, :attr:`digits`, and :attr:`exponent` fields.
1948
1949 (Implemented by Facundo Batista and Mark Dickinson. Named tuple
1950 support added by Raymond Hettinger.)
1951
1952 * The :mod:`difflib` module's :class:`SequenceMatcher` class
1953 now returns named tuples representing matches,
1954 with :attr:`a`, :attr:`b`, and :attr:`size` attributes.
1955 (Contributed by Raymond Hettinger.)
1956
1957 * An optional ``timeout`` parameter, specifying a timeout measured in
1958 seconds, was added to the :class:`ftplib.FTP` class constructor as
1959 well as the :meth:`connect` method. (Added by Facundo Batista.)
1960 Also, the :class:`FTP` class's :meth:`storbinary` and
1961 :meth:`storlines` now take an optional *callback* parameter that
1962 will be called with each block of data after the data has been sent.
1963 (Contributed by Phil Schwartz; :issue:`1221598`.)
1964
1965 * The :func:`reduce` built-in function is also available in the
1966 :mod:`functools` module. In Python 3.0, the builtin has been
1967 dropped and :func:`reduce` is only available from :mod:`functools`;
1968 currently there are no plans to drop the builtin in the 2.x series.
1969 (Patched by Christian Heimes; :issue:`1739906`.)
1970
1971 * When possible, the :mod:`getpass` module will now use
1972 :file:`/dev/tty` to print a prompt message and read the password,
1973 falling back to standard error and standard input. If the
1974 password may be echoed to the terminal, a warning is printed before
1975 the prompt is displayed. (Contributed by Gregory P. Smith.)
1976
1977 * The :func:`glob.glob` function can now return Unicode filenames if
1978 a Unicode path was used and Unicode filenames are matched within the
1979 directory. (:issue:`1001604`)
1980
1981 * A new function in the :mod:`heapq` module, ``merge(iter1, iter2, ...)``,
1982 takes any number of iterables returning data in sorted
1983 order, and returns a new generator that returns the contents of all
1984 the iterators, also in sorted order. For example::
1985
1986 >>> list(heapq.merge([1, 3, 5, 9], [2, 8, 16]))
1987 [1, 2, 3, 5, 8, 9, 16]
1988
1989 Another new function, ``heappushpop(heap, item)``,
1990 pushes *item* onto *heap*, then pops off and returns the smallest item.
1991 This is more efficient than making a call to :func:`heappush` and then
1992 :func:`heappop`.
1993
1994 :mod:`heapq` is now implemented to only use less-than comparison,
1995 instead of the less-than-or-equal comparison it previously used.
1996 This makes :mod:`heapq`'s usage of a type match the
1997 :meth:`list.sort` method.
1998 (Contributed by Raymond Hettinger.)
1999
2000 * An optional ``timeout`` parameter, specifying a timeout measured in
2001 seconds, was added to the :class:`httplib.HTTPConnection <http.client.HTTPConnection>` and
2002 :class:`HTTPSConnection <http.client.HTTPSConnection>` class constructors. (Added by Facundo
2003 Batista.)
2004
2005 * Most of the :mod:`inspect` module's functions, such as
2006 :func:`getmoduleinfo` and :func:`getargs`, now return named tuples.
2007 In addition to behaving like tuples, the elements of the return value
2008 can also be accessed as attributes.
2009 (Contributed by Raymond Hettinger.)
2010
2011 Some new functions in the module include
2012 :func:`isgenerator`, :func:`isgeneratorfunction`,
2013 and :func:`isabstract`.
2014
2015 * The :mod:`itertools` module gained several new functions.
2016
2017 ``izip_longest(iter1, iter2, ...[, fillvalue])`` makes tuples from
2018 each of the elements; if some of the iterables are shorter than
2019 others, the missing values are set to *fillvalue*. For example::
2020
2021 >>> tuple(itertools.izip_longest([1,2,3], [1,2,3,4,5]))
2022 ((1, 1), (2, 2), (3, 3), (None, 4), (None, 5))
2023
2024 ``product(iter1, iter2, ..., [repeat=N])`` returns the Cartesian product
2025 of the supplied iterables, a set of tuples containing
2026 every possible combination of the elements returned from each iterable. ::
2027
2028 >>> list(itertools.product([1,2,3], [4,5,6]))
2029 [(1, 4), (1, 5), (1, 6),
2030 (2, 4), (2, 5), (2, 6),
2031 (3, 4), (3, 5), (3, 6)]
2032
2033 The optional *repeat* keyword argument is used for taking the
2034 product of an iterable or a set of iterables with themselves,
2035 repeated *N* times. With a single iterable argument, *N*-tuples
2036 are returned::
2037
2038 >>> list(itertools.product([1,2], repeat=3))
2039 [(1, 1, 1), (1, 1, 2), (1, 2, 1), (1, 2, 2),
2040 (2, 1, 1), (2, 1, 2), (2, 2, 1), (2, 2, 2)]
2041
2042 With two iterables, *2N*-tuples are returned. ::
2043
2044 >>> list(itertools.product([1,2], [3,4], repeat=2))
2045 [(1, 3, 1, 3), (1, 3, 1, 4), (1, 3, 2, 3), (1, 3, 2, 4),
2046 (1, 4, 1, 3), (1, 4, 1, 4), (1, 4, 2, 3), (1, 4, 2, 4),
2047 (2, 3, 1, 3), (2, 3, 1, 4), (2, 3, 2, 3), (2, 3, 2, 4),
2048 (2, 4, 1, 3), (2, 4, 1, 4), (2, 4, 2, 3), (2, 4, 2, 4)]
2049
2050 ``combinations(iterable, r)`` returns sub-sequences of length *r* from
2051 the elements of *iterable*. ::
2052
2053 >>> list(itertools.combinations('123', 2))
2054 [('1', '2'), ('1', '3'), ('2', '3')]
2055 >>> list(itertools.combinations('123', 3))
2056 [('1', '2', '3')]
2057 >>> list(itertools.combinations('1234', 3))
2058 [('1', '2', '3'), ('1', '2', '4'),
2059 ('1', '3', '4'), ('2', '3', '4')]
2060
2061 ``permutations(iter[, r])`` returns all the permutations of length *r* of
2062 the iterable's elements. If *r* is not specified, it will default to the
2063 number of elements produced by the iterable. ::
2064
2065 >>> list(itertools.permutations([1,2,3,4], 2))
2066 [(1, 2), (1, 3), (1, 4),
2067 (2, 1), (2, 3), (2, 4),
2068 (3, 1), (3, 2), (3, 4),
2069 (4, 1), (4, 2), (4, 3)]
2070
2071 ``itertools.chain(*iterables)`` is an existing function in
2072 :mod:`itertools` that gained a new constructor in Python 2.6.
2073 ``itertools.chain.from_iterable(iterable)`` takes a single
2074 iterable that should return other iterables. :func:`chain` will
2075 then return all the elements of the first iterable, then
2076 all the elements of the second, and so on. ::
2077
2078 >>> list(itertools.chain.from_iterable([[1,2,3], [4,5,6]]))
2079 [1, 2, 3, 4, 5, 6]
2080
2081 (All contributed by Raymond Hettinger.)
2082
2083 * The :mod:`logging` module's :class:`FileHandler` class
2084 and its subclasses :class:`WatchedFileHandler`, :class:`RotatingFileHandler`,
2085 and :class:`TimedRotatingFileHandler` now
2086 have an optional *delay* parameter to their constructors. If *delay*
2087 is true, opening of the log file is deferred until the first
2088 :meth:`emit` call is made. (Contributed by Vinay Sajip.)
2089
2090 :class:`TimedRotatingFileHandler` also has a *utc* constructor
2091 parameter. If the argument is true, UTC time will be used
2092 in determining when midnight occurs and in generating filenames;
2093 otherwise local time will be used.
2094
2095 * Several new functions were added to the :mod:`math` module:
2096
2097 * :func:`~math.isinf` and :func:`~math.isnan` determine whether a given float
2098 is a (positive or negative) infinity or a NaN (Not a Number), respectively.
2099
2100 * :func:`~math.copysign` copies the sign bit of an IEEE 754 number,
2101 returning the absolute value of *x* combined with the sign bit of
2102 *y*. For example, ``math.copysign(1, -0.0)`` returns -1.0.
2103 (Contributed by Christian Heimes.)
2104
2105 * :func:`~math.factorial` computes the factorial of a number.
2106 (Contributed by Raymond Hettinger; :issue:`2138`.)
2107
2108 * :func:`~math.fsum` adds up the stream of numbers from an iterable,
2109 and is careful to avoid loss of precision through using partial sums.
2110 (Contributed by Jean Brouwers, Raymond Hettinger, and Mark Dickinson;
2111 :issue:`2819`.)
2112
2113 * :func:`~math.acosh`, :func:`~math.asinh`
2114 and :func:`~math.atanh` compute the inverse hyperbolic functions.
2115
2116 * :func:`~math.log1p` returns the natural logarithm of *1+x*
2117 (base *e*).
2118
2119 * :func:`trunc` rounds a number toward zero, returning the closest
2120 :class:`Integral` that's between the function's argument and zero.
2121 Added as part of the backport of
2122 `PEP 3141's type hierarchy for numbers <#pep-3141>`__.
2123
2124 * The :mod:`math` module has been improved to give more consistent
2125 behaviour across platforms, especially with respect to handling of
2126 floating-point exceptions and IEEE 754 special values.
2127
2128 Whenever possible, the module follows the recommendations of the C99
2129 standard about 754's special values. For example, ``sqrt(-1.)``
2130 should now give a :exc:`ValueError` across almost all platforms,
2131 while ``sqrt(float('NaN'))`` should return a NaN on all IEEE 754
2132 platforms. Where Annex 'F' of the C99 standard recommends signaling
2133 'divide-by-zero' or 'invalid', Python will raise :exc:`ValueError`.
2134 Where Annex 'F' of the C99 standard recommends signaling 'overflow',
2135 Python will raise :exc:`OverflowError`. (See :issue:`711019` and
2136 :issue:`1640`.)
2137
2138 (Contributed by Christian Heimes and Mark Dickinson.)
2139
2140 * :class:`~mmap.mmap` objects now have a :meth:`rfind` method that searches for a
2141 substring beginning at the end of the string and searching
2142 backwards. The :meth:`find` method also gained an *end* parameter
2143 giving an index at which to stop searching.
2144 (Contributed by John Lenton.)
2145
2146 * The :mod:`operator` module gained a
2147 :func:`methodcaller` function that takes a name and an optional
2148 set of arguments, returning a callable that will call
2149 the named function on any arguments passed to it. For example::
2150
2151 >>> # Equivalent to lambda s: s.replace('old', 'new')
2152 >>> replacer = operator.methodcaller('replace', 'old', 'new')
2153 >>> replacer('old wine in old bottles')
2154 'new wine in new bottles'
2155
2156 (Contributed by Georg Brandl, after a suggestion by Gregory Petrosyan.)
2157
2158 The :func:`attrgetter` function now accepts dotted names and performs
2159 the corresponding attribute lookups::
2160
2161 >>> inst_name = operator.attrgetter(
2162 ... '__class__.__name__')
2163 >>> inst_name('')
2164 'str'
2165 >>> inst_name(help)
2166 '_Helper'
2167
2168 (Contributed by Georg Brandl, after a suggestion by Barry Warsaw.)
2169
2170 * The :mod:`os` module now wraps several new system calls.
2171 ``fchmod(fd, mode)`` and ``fchown(fd, uid, gid)`` change the mode
2172 and ownership of an opened file, and ``lchmod(path, mode)`` changes
2173 the mode of a symlink. (Contributed by Georg Brandl and Christian
2174 Heimes.)
2175
2176 :func:`chflags` and :func:`lchflags` are wrappers for the
2177 corresponding system calls (where they're available), changing the
2178 flags set on a file. Constants for the flag values are defined in
2179 the :mod:`stat` module; some possible values include
2180 :const:`UF_IMMUTABLE` to signal the file may not be changed and
2181 :const:`UF_APPEND` to indicate that data can only be appended to the
2182 file. (Contributed by M. Levinson.)
2183
2184 ``os.closerange(low, high)`` efficiently closes all file descriptors
2185 from *low* to *high*, ignoring any errors and not including *high* itself.
2186 This function is now used by the :mod:`subprocess` module to make starting
2187 processes faster. (Contributed by Georg Brandl; :issue:`1663329`.)
2188
2189 * The ``os.environ`` object's :meth:`clear` method will now unset the
2190 environment variables using :func:`os.unsetenv` in addition to clearing
2191 the object's keys. (Contributed by Martin Horcicka; :issue:`1181`.)
2192
2193 * The :func:`os.walk` function now has a ``followlinks`` parameter. If
2194 set to True, it will follow symlinks pointing to directories and
2195 visit the directory's contents. For backward compatibility, the
2196 parameter's default value is false. Note that the function can fall
2197 into an infinite recursion if there's a symlink that points to a
2198 parent directory. (:issue:`1273829`)
2199
2200 * In the :mod:`os.path` module, the :func:`splitext` function
2201 has been changed to not split on leading period characters.
2202 This produces better results when operating on Unix's dot-files.
2203 For example, ``os.path.splitext('.ipython')``
2204 now returns ``('.ipython', '')`` instead of ``('', '.ipython')``.
2205 (:issue:`1115886`)
2206
2207 A new function, ``os.path.relpath(path, start='.')``, returns a relative path
2208 from the ``start`` path, if it's supplied, or from the current
2209 working directory to the destination ``path``. (Contributed by
2210 Richard Barran; :issue:`1339796`.)
2211
2212 On Windows, :func:`os.path.expandvars` will now expand environment variables
2213 given in the form "%var%", and "~user" will be expanded into the
2214 user's home directory path. (Contributed by Josiah Carlson;
2215 :issue:`957650`.)
2216
2217 * The Python debugger provided by the :mod:`pdb` module
2218 gained a new command: "run" restarts the Python program being debugged
2219 and can optionally take new command-line arguments for the program.
2220 (Contributed by Rocky Bernstein; :issue:`1393667`.)
2221
2222 * The :func:`pdb.post_mortem` function, used to begin debugging a
2223 traceback, will now use the traceback returned by :func:`sys.exc_info`
2224 if no traceback is supplied. (Contributed by Facundo Batista;
2225 :issue:`1106316`.)
2226
2227 * The :mod:`pickletools` module now has an :func:`optimize` function
2228 that takes a string containing a pickle and removes some unused
2229 opcodes, returning a shorter pickle that contains the same data structure.
2230 (Contributed by Raymond Hettinger.)
2231
2232 * A :func:`get_data` function was added to the :mod:`pkgutil`
2233 module that returns the contents of resource files included
2234 with an installed Python package. For example::
2235
2236 >>> import pkgutil
2237 >>> print pkgutil.get_data('test', 'exception_hierarchy.txt')
2238 BaseException
2239 +-- SystemExit
2240 +-- KeyboardInterrupt
2241 +-- GeneratorExit
2242 +-- Exception
2243 +-- StopIteration
2244 +-- StandardError
2245 ...
2246
2247 (Contributed by Paul Moore; :issue:`2439`.)
2248
2249 * The :mod:`pyexpat` module's :class:`Parser` objects now allow setting
2250 their :attr:`buffer_size` attribute to change the size of the buffer
2251 used to hold character data.
2252 (Contributed by Achim Gaedke; :issue:`1137`.)
2253
2254 * The :mod:`Queue` module now provides queue variants that retrieve entries
2255 in different orders. The :class:`PriorityQueue` class stores
2256 queued items in a heap and retrieves them in priority order,
2257 and :class:`LifoQueue` retrieves the most recently added entries first,
2258 meaning that it behaves like a stack.
2259 (Contributed by Raymond Hettinger.)
2260
2261 * The :mod:`random` module's :class:`Random` objects can
2262 now be pickled on a 32-bit system and unpickled on a 64-bit
2263 system, and vice versa. Unfortunately, this change also means
2264 that Python 2.6's :class:`Random` objects can't be unpickled correctly
2265 on earlier versions of Python.
2266 (Contributed by Shawn Ligocki; :issue:`1727780`.)
2267
2268 The new ``triangular(low, high, mode)`` function returns random
2269 numbers following a triangular distribution. The returned values
2270 are between *low* and *high*, not including *high* itself, and
2271 with *mode* as the most frequently occurring value
2272 in the distribution. (Contributed by Wladmir van der Laan and
2273 Raymond Hettinger; :issue:`1681432`.)
2274
2275 * Long regular expression searches carried out by the :mod:`re`
2276 module will check for signals being delivered, so
2277 time-consuming searches can now be interrupted.
2278 (Contributed by Josh Hoyt and Ralf Schmitt; :issue:`846388`.)
2279
2280 The regular expression module is implemented by compiling bytecodes
2281 for a tiny regex-specific virtual machine. Untrusted code
2282 could create malicious strings of bytecode directly and cause crashes,
2283 so Python 2.6 includes a verifier for the regex bytecode.
2284 (Contributed by Guido van Rossum from work for Google App Engine;
2285 :issue:`3487`.)
2286
2287 * The :mod:`rlcompleter` module's :meth:`Completer.complete` method
2288 will now ignore exceptions triggered while evaluating a name.
2289 (Fixed by Lorenz Quack; :issue:`2250`.)
2290
2291 * The :mod:`sched` module's :class:`scheduler` instances now
2292 have a read-only :attr:`queue` attribute that returns the
2293 contents of the scheduler's queue, represented as a list of
2294 named tuples with the fields ``(time, priority, action, argument)``.
2295 (Contributed by Raymond Hettinger; :issue:`1861`.)
2296
2297 * The :mod:`select` module now has wrapper functions
2298 for the Linux :c:func:`!epoll` and BSD :c:func:`!kqueue` system calls.
2299 :meth:`modify` method was added to the existing :class:`poll`
2300 objects; ``pollobj.modify(fd, eventmask)`` takes a file descriptor
2301 or file object and an event mask, modifying the recorded event mask
2302 for that file.
2303 (Contributed by Christian Heimes; :issue:`1657`.)
2304
2305 * The :func:`shutil.copytree` function now has an optional *ignore* argument
2306 that takes a callable object. This callable will receive each directory path
2307 and a list of the directory's contents, and returns a list of names that
2308 will be ignored, not copied.
2309
2310 The :mod:`shutil` module also provides an :func:`ignore_patterns`
2311 function for use with this new parameter. :func:`ignore_patterns`
2312 takes an arbitrary number of glob-style patterns and returns a
2313 callable that will ignore any files and directories that match any
2314 of these patterns. The following example copies a directory tree,
2315 but skips both :file:`.svn` directories and Emacs backup files,
2316 which have names ending with '~'::
2317
2318 shutil.copytree('Doc/library', '/tmp/library',
2319 ignore=shutil.ignore_patterns('*~', '.svn'))
2320
2321 (Contributed by Tarek Ziadé; :issue:`2663`.)
2322
2323 * Integrating signal handling with GUI handling event loops
2324 like those used by Tkinter or GTk+ has long been a problem; most
2325 software ends up polling, waking up every fraction of a second to check
2326 if any GUI events have occurred.
2327 The :mod:`signal` module can now make this more efficient.
2328 Calling ``signal.set_wakeup_fd(fd)`` sets a file descriptor
2329 to be used; when a signal is received, a byte is written to that
2330 file descriptor. There's also a C-level function,
2331 :c:func:`PySignal_SetWakeupFd`, for setting the descriptor.
2332
2333 Event loops will use this by opening a pipe to create two descriptors,
2334 one for reading and one for writing. The writable descriptor
2335 will be passed to :func:`set_wakeup_fd`, and the readable descriptor
2336 will be added to the list of descriptors monitored by the event loop via
2337 :c:func:`!select` or :c:func:`!poll`.
2338 On receiving a signal, a byte will be written and the main event loop
2339 will be woken up, avoiding the need to poll.
2340
2341 (Contributed by Adam Olsen; :issue:`1583`.)
2342
2343 The :func:`siginterrupt` function is now available from Python code,
2344 and allows changing whether signals can interrupt system calls or not.
2345 (Contributed by Ralf Schmitt.)
2346
2347 The :func:`setitimer` and :func:`getitimer` functions have also been
2348 added (where they're available). :func:`setitimer`
2349 allows setting interval timers that will cause a signal to be
2350 delivered to the process after a specified time, measured in
2351 wall-clock time, consumed process time, or combined process+system
2352 time. (Contributed by Guilherme Polo; :issue:`2240`.)
2353
2354 * The :mod:`smtplib` module now supports SMTP over SSL thanks to the
2355 addition of the :class:`SMTP_SSL` class. This class supports an
2356 interface identical to the existing :class:`SMTP` class.
2357 (Contributed by Monty Taylor.) Both class constructors also have an
2358 optional ``timeout`` parameter that specifies a timeout for the
2359 initial connection attempt, measured in seconds. (Contributed by
2360 Facundo Batista.)
2361
2362 An implementation of the LMTP protocol (:rfc:`2033`) was also added
2363 to the module. LMTP is used in place of SMTP when transferring
2364 e-mail between agents that don't manage a mail queue. (LMTP
2365 implemented by Leif Hedstrom; :issue:`957003`.)
2366
2367 :meth:`SMTP.starttls` now complies with :rfc:`3207` and forgets any
2368 knowledge obtained from the server not obtained from the TLS
2369 negotiation itself. (Patch contributed by Bill Fenner;
2370 :issue:`829951`.)
2371
2372 * The :mod:`socket` module now supports TIPC (https://tipc.sourceforge.net/),
2373 a high-performance non-IP-based protocol designed for use in clustered
2374 environments. TIPC addresses are 4- or 5-tuples.
2375 (Contributed by Alberto Bertogli; :issue:`1646`.)
2376
2377 A new function, :func:`create_connection`, takes an address and
2378 connects to it using an optional timeout value, returning the
2379 connected socket object. This function also looks up the address's
2380 type and connects to it using IPv4 or IPv6 as appropriate. Changing
2381 your code to use :func:`create_connection` instead of
2382 ``socket(socket.AF_INET, ...)`` may be all that's required to make
2383 your code work with IPv6.
2384
2385 * The base classes in the :mod:`SocketServer <socketserver>` module now support
2386 calling a :meth:`~socketserver.BaseServer.handle_timeout` method after a span of inactivity
2387 specified by the server's :attr:`~socketserver.BaseServer.timeout` attribute. (Contributed
2388 by Michael Pomraning.) The :meth:`~socketserver.BaseServer.serve_forever` method
2389 now takes an optional poll interval measured in seconds,
2390 controlling how often the server will check for a shutdown request.
2391 (Contributed by Pedro Werneck and Jeffrey Yasskin;
2392 :issue:`742598`, :issue:`1193577`.)
2393
2394 * The :mod:`sqlite3` module, maintained by Gerhard Häring,
2395 has been updated from version 2.3.2 in Python 2.5 to
2396 version 2.4.1.
2397
2398 * The :mod:`struct` module now supports the C99 :c:expr:`_Bool` type,
2399 using the format character ``'?'``.
2400 (Contributed by David Remahl.)
2401
2402 * The :class:`~subprocess.Popen` objects provided by the :mod:`subprocess` module
2403 now have :meth:`~subprocess.Popen.terminate`, :meth:`~subprocess.Popen.kill`, and :meth:`~subprocess.Popen.send_signal` methods.
2404 On Windows, :meth:`!send_signal` only supports the :py:const:`~signal.SIGTERM`
2405 signal, and all these methods are aliases for the Win32 API function
2406 :c:func:`!TerminateProcess`.
2407 (Contributed by Christian Heimes.)
2408
2409 * A new variable in the :mod:`sys` module, :attr:`float_info`, is an
2410 object containing information derived from the :file:`float.h` file
2411 about the platform's floating-point support. Attributes of this
2412 object include :attr:`mant_dig` (number of digits in the mantissa),
2413 :attr:`epsilon` (smallest difference between 1.0 and the next
2414 largest value representable), and several others. (Contributed by
2415 Christian Heimes; :issue:`1534`.)
2416
2417 Another new variable, :attr:`dont_write_bytecode`, controls whether Python
2418 writes any :file:`.pyc` or :file:`.pyo` files on importing a module.
2419 If this variable is true, the compiled files are not written. The
2420 variable is initially set on start-up by supplying the :option:`-B`
2421 switch to the Python interpreter, or by setting the
2422 :envvar:`PYTHONDONTWRITEBYTECODE` environment variable before
2423 running the interpreter. Python code can subsequently
2424 change the value of this variable to control whether bytecode files
2425 are written or not.
2426 (Contributed by Neal Norwitz and Georg Brandl.)
2427
2428 Information about the command-line arguments supplied to the Python
2429 interpreter is available by reading attributes of a named
2430 tuple available as ``sys.flags``. For example, the :attr:`verbose`
2431 attribute is true if Python
2432 was executed in verbose mode, :attr:`debug` is true in debugging mode, etc.
2433 These attributes are all read-only.
2434 (Contributed by Christian Heimes.)
2435
2436 A new function, :func:`getsizeof`, takes a Python object and returns
2437 the amount of memory used by the object, measured in bytes. Built-in
2438 objects return correct results; third-party extensions may not,
2439 but can define a :meth:`__sizeof__` method to return the
2440 object's size.
2441 (Contributed by Robert Schuppenies; :issue:`2898`.)
2442
2443 It's now possible to determine the current profiler and tracer functions
2444 by calling :func:`sys.getprofile` and :func:`sys.gettrace`.
2445 (Contributed by Georg Brandl; :issue:`1648`.)
2446
2447 * The :mod:`tarfile` module now supports POSIX.1-2001 (pax) tarfiles in
2448 addition to the POSIX.1-1988 (ustar) and GNU tar formats that were
2449 already supported. The default format is GNU tar; specify the
2450 ``format`` parameter to open a file using a different format::
2451
2452 tar = tarfile.open("output.tar", "w",
2453 format=tarfile.PAX_FORMAT)
2454
2455 The new ``encoding`` and ``errors`` parameters specify an encoding and
2456 an error handling scheme for character conversions. ``'strict'``,
2457 ``'ignore'``, and ``'replace'`` are the three standard ways Python can
2458 handle errors,;
2459 ``'utf-8'`` is a special value that replaces bad characters with
2460 their UTF-8 representation. (Character conversions occur because the
2461 PAX format supports Unicode filenames, defaulting to UTF-8 encoding.)
2462
2463 The :meth:`TarFile.add` method now accepts an ``exclude`` argument that's
2464 a function that can be used to exclude certain filenames from
2465 an archive.
2466 The function must take a filename and return true if the file
2467 should be excluded or false if it should be archived.
2468 The function is applied to both the name initially passed to :meth:`add`
2469 and to the names of files in recursively added directories.
2470
2471 (All changes contributed by Lars Gustäbel).
2472
2473 * An optional ``timeout`` parameter was added to the
2474 :class:`!telnetlib.Telnet` class constructor, specifying a timeout
2475 measured in seconds. (Added by Facundo Batista.)
2476
2477 * The :class:`tempfile.NamedTemporaryFile` class usually deletes
2478 the temporary file it created when the file is closed. This
2479 behaviour can now be changed by passing ``delete=False`` to the
2480 constructor. (Contributed by Damien Miller; :issue:`1537850`.)
2481
2482 A new class, :class:`SpooledTemporaryFile`, behaves like
2483 a temporary file but stores its data in memory until a maximum size is
2484 exceeded. On reaching that limit, the contents will be written to
2485 an on-disk temporary file. (Contributed by Dustin J. Mitchell.)
2486
2487 The :class:`NamedTemporaryFile` and :class:`SpooledTemporaryFile` classes
2488 both work as context managers, so you can write
2489 ``with tempfile.NamedTemporaryFile() as tmp: ...``.
2490 (Contributed by Alexander Belopolsky; :issue:`2021`.)
2491
2492 * The :mod:`test.test_support <test.support>` module gained a number
2493 of context managers useful for writing tests.
2494 :func:`~test.support.os_helper.EnvironmentVarGuard` is a
2495 context manager that temporarily changes environment variables and
2496 automatically restores them to their old values.
2497
2498 Another context manager, :class:`TransientResource`, can surround calls
2499 to resources that may or may not be available; it will catch and
2500 ignore a specified list of exceptions. For example,
2501 a network test may ignore certain failures when connecting to an
2502 external web site::
2503
2504 with test_support.TransientResource(IOError,
2505 errno=errno.ETIMEDOUT):
2506 f = urllib.urlopen('https://sf.net')
2507 ...
2508
2509 Finally, :func:`check_warnings` resets the :mod:`warning` module's
2510 warning filters and returns an object that will record all warning
2511 messages triggered (:issue:`3781`)::
2512
2513 with test_support.check_warnings() as wrec:
2514 warnings.simplefilter("always")
2515 # ... code that triggers a warning ...
2516 assert str(wrec.message) == "function is outdated"
2517 assert len(wrec.warnings) == 1, "Multiple warnings raised"
2518
2519 (Contributed by Brett Cannon.)
2520
2521 * The :mod:`textwrap` module can now preserve existing whitespace
2522 at the beginnings and ends of the newly created lines
2523 by specifying ``drop_whitespace=False``
2524 as an argument::
2525
2526 >>> S = """This sentence has a bunch of
2527 ... extra whitespace."""
2528 >>> print textwrap.fill(S, width=15)
2529 This sentence
2530 has a bunch
2531 of extra
2532 whitespace.
2533 >>> print textwrap.fill(S, drop_whitespace=False, width=15)
2534 This sentence
2535 has a bunch
2536 of extra
2537 whitespace.
2538 >>>
2539
2540 (Contributed by Dwayne Bailey; :issue:`1581073`.)
2541
2542 * The :mod:`threading` module API is being changed to use properties
2543 such as :attr:`daemon` instead of :meth:`setDaemon` and
2544 :meth:`isDaemon` methods, and some methods have been renamed to use
2545 underscores instead of camel-case; for example, the
2546 :meth:`activeCount` method is renamed to :meth:`active_count`. Both
2547 the 2.6 and 3.0 versions of the module support the same properties
2548 and renamed methods, but don't remove the old methods. No date has been set
2549 for the deprecation of the old APIs in Python 3.x; the old APIs won't
2550 be removed in any 2.x version.
2551 (Carried out by several people, most notably Benjamin Peterson.)
2552
2553 The :mod:`threading` module's :class:`Thread` objects
2554 gained an :attr:`ident` property that returns the thread's
2555 identifier, a nonzero integer. (Contributed by Gregory P. Smith;
2556 :issue:`2871`.)
2557
2558 * The :mod:`timeit` module now accepts callables as well as strings
2559 for the statement being timed and for the setup code.
2560 Two convenience functions were added for creating
2561 :class:`Timer` instances:
2562 ``repeat(stmt, setup, time, repeat, number)`` and
2563 ``timeit(stmt, setup, time, number)`` create an instance and call
2564 the corresponding method. (Contributed by Erik Demaine;
2565 :issue:`1533909`.)
2566
2567 * The :mod:`Tkinter` module now accepts lists and tuples for options,
2568 separating the elements by spaces before passing the resulting value to
2569 Tcl/Tk.
2570 (Contributed by Guilherme Polo; :issue:`2906`.)
2571
2572 * The :mod:`turtle` module for turtle graphics was greatly enhanced by
2573 Gregor Lingl. New features in the module include:
2574
2575 * Better animation of turtle movement and rotation.
2576 * Control over turtle movement using the new :meth:`delay`,
2577 :meth:`tracer`, and :meth:`speed` methods.
2578 * The ability to set new shapes for the turtle, and to
2579 define a new coordinate system.
2580 * Turtles now have an :meth:`undo` method that can roll back actions.
2581 * Simple support for reacting to input events such as mouse and keyboard
2582 activity, making it possible to write simple games.
2583 * A :file:`turtle.cfg` file can be used to customize the starting appearance
2584 of the turtle's screen.
2585 * The module's docstrings can be replaced by new docstrings that have been
2586 translated into another language.
2587
2588 (:issue:`1513695`)
2589
2590 * An optional ``timeout`` parameter was added to the
2591 :func:`urllib.urlopen <urllib.request.urlopen>` function and the
2592 :class:`urllib.ftpwrapper` class constructor, as well as the
2593 :func:`urllib2.urlopen <urllib.request.urlopen>` function. The parameter specifies a timeout
2594 measured in seconds. For example::
2595
2596 >>> u = urllib2.urlopen("http://slow.example.com",
2597 timeout=3)
2598 Traceback (most recent call last):
2599 ...
2600 urllib2.URLError: <urlopen error timed out>
2601 >>>
2602
2603 (Added by Facundo Batista.)
2604
2605 * The Unicode database provided by the :mod:`unicodedata` module
2606 has been updated to version 5.1.0. (Updated by
2607 Martin von Löwis; :issue:`3811`.)
2608
2609 * The :mod:`warnings` module's :func:`formatwarning` and :func:`showwarning`
2610 gained an optional *line* argument that can be used to supply the
2611 line of source code. (Added as part of :issue:`1631171`, which re-implemented
2612 part of the :mod:`warnings` module in C code.)
2613
2614 A new function, :func:`catch_warnings`, is a context manager
2615 intended for testing purposes that lets you temporarily modify the
2616 warning filters and then restore their original values (:issue:`3781`).
2617
2618 * The XML-RPC :class:`SimpleXMLRPCServer <xmlrpc.server>` and :class:`DocXMLRPCServer <xmlrpc.server>`
2619 classes can now be prevented from immediately opening and binding to
2620 their socket by passing ``False`` as the *bind_and_activate*
2621 constructor parameter. This can be used to modify the instance's
2622 :attr:`allow_reuse_address` attribute before calling the
2623 :meth:`server_bind` and :meth:`server_activate` methods to
2624 open the socket and begin listening for connections.
2625 (Contributed by Peter Parente; :issue:`1599845`.)
2626
2627 :class:`SimpleXMLRPCServer` also has a :attr:`_send_traceback_header`
2628 attribute; if true, the exception and formatted traceback are returned
2629 as HTTP headers "X-Exception" and "X-Traceback". This feature is
2630 for debugging purposes only and should not be used on production servers
2631 because the tracebacks might reveal passwords or other sensitive
2632 information. (Contributed by Alan McIntyre as part of his
2633 project for Google's Summer of Code 2007.)
2634
2635 * The :mod:`xmlrpclib <xmlrpc.client>` module no longer automatically converts
2636 :class:`datetime.date` and :class:`datetime.time` to the
2637 :class:`xmlrpclib.DateTime <xmlrpc.client.DateTime>` type; the conversion semantics were
2638 not necessarily correct for all applications. Code using
2639 :mod:`!xmlrpclib` should convert :class:`date` and :class:`~datetime.time`
2640 instances. (:issue:`1330538`) The code can also handle
2641 dates before 1900 (contributed by Ralf Schmitt; :issue:`2014`)
2642 and 64-bit integers represented by using ``<i8>`` in XML-RPC responses
2643 (contributed by Riku Lindblad; :issue:`2985`).
2644
2645 * The :mod:`zipfile` module's :class:`ZipFile` class now has
2646 :meth:`extract` and :meth:`extractall` methods that will unpack
2647 a single file or all the files in the archive to the current directory, or
2648 to a specified directory::
2649
2650 z = zipfile.ZipFile('python-251.zip')
2651
2652 # Unpack a single file, writing it relative
2653 # to the /tmp directory.
2654 z.extract('Python/sysmodule.c', '/tmp')
2655
2656 # Unpack all the files in the archive.
2657 z.extractall()
2658
2659 (Contributed by Alan McIntyre; :issue:`467924`.)
2660
2661 The :meth:`open`, :meth:`read` and :meth:`extract` methods can now
2662 take either a filename or a :class:`ZipInfo` object. This is useful when an
2663 archive accidentally contains a duplicated filename.
2664 (Contributed by Graham Horler; :issue:`1775025`.)
2665
2666 Finally, :mod:`zipfile` now supports using Unicode filenames
2667 for archived files. (Contributed by Alexey Borzenkov; :issue:`1734346`.)
2668
2669 .. ======================================================================
2670 .. whole new modules get described in subsections here
2671
2672 The :mod:`ast` module
2673 ----------------------
2674
2675 The :mod:`ast` module provides an Abstract Syntax Tree
2676 representation of Python code, and Armin Ronacher
2677 contributed a set of helper functions that perform a variety of
2678 common tasks. These will be useful for HTML templating
2679 packages, code analyzers, and similar tools that process
2680 Python code.
2681
2682 The :func:`parse` function takes an expression and returns an AST.
2683 The :func:`dump` function outputs a representation of a tree, suitable
2684 for debugging::
2685
2686 import ast
2687
2688 t = ast.parse("""
2689 d = {}
2690 for i in 'abcdefghijklm':
2691 d[i + i] = ord(i) - ord('a') + 1
2692 print d
2693 """)
2694 print ast.dump(t)
2695
2696 This outputs a deeply nested tree::
2697
2698 Module(body=[
2699 Assign(targets=[
2700 Name(id='d', ctx=Store())
2701 ], value=Dict(keys=[], values=[]))
2702 For(target=Name(id='i', ctx=Store()),
2703 iter=Str(s='abcdefghijklm'), body=[
2704 Assign(targets=[
2705 Subscript(value=
2706 Name(id='d', ctx=Load()),
2707 slice=
2708 Index(value=
2709 BinOp(left=Name(id='i', ctx=Load()), op=Add(),
2710 right=Name(id='i', ctx=Load()))), ctx=Store())
2711 ], value=
2712 BinOp(left=
2713 BinOp(left=
2714 Call(func=
2715 Name(id='ord', ctx=Load()), args=[
2716 Name(id='i', ctx=Load())
2717 ], keywords=[], starargs=None, kwargs=None),
2718 op=Sub(), right=Call(func=
2719 Name(id='ord', ctx=Load()), args=[
2720 Str(s='a')
2721 ], keywords=[], starargs=None, kwargs=None)),
2722 op=Add(), right=Num(n=1)))
2723 ], orelse=[])
2724 Print(dest=None, values=[
2725 Name(id='d', ctx=Load())
2726 ], nl=True)
2727 ])
2728
2729 The :func:`literal_eval` method takes a string or an AST
2730 representing a literal expression, parses and evaluates it, and
2731 returns the resulting value. A literal expression is a Python
2732 expression containing only strings, numbers, dictionaries,
2733 etc. but no statements or function calls. If you need to
2734 evaluate an expression but cannot accept the security risk of using an
2735 :func:`eval` call, :func:`literal_eval` will handle it safely::
2736
2737 >>> literal = '("a", "b", {2:4, 3:8, 1:2})'
2738 >>> print ast.literal_eval(literal)
2739 ('a', 'b', {1: 2, 2: 4, 3: 8})
2740 >>> print ast.literal_eval('"a" + "b"')
2741 Traceback (most recent call last):
2742 ...
2743 ValueError: malformed string
2744
2745 The module also includes :class:`NodeVisitor` and
2746 :class:`NodeTransformer` classes for traversing and modifying an AST,
2747 and functions for common transformations such as changing line
2748 numbers.
2749
2750 .. ======================================================================
2751
2752 The :mod:`!future_builtins` module
2753 --------------------------------------
2754
2755 Python 3.0 makes many changes to the repertoire of built-in
2756 functions, and most of the changes can't be introduced in the Python
2757 2.x series because they would break compatibility.
2758 The :mod:`!future_builtins` module provides versions
2759 of these built-in functions that can be imported when writing
2760 3.0-compatible code.
2761
2762 The functions in this module currently include:
2763
2764 * ``ascii(obj)``: equivalent to :func:`repr`. In Python 3.0,
2765 :func:`repr` will return a Unicode string, while :func:`ascii` will
2766 return a pure ASCII bytestring.
2767
2768 * ``filter(predicate, iterable)``,
2769 ``map(func, iterable1, ...)``: the 3.0 versions
2770 return iterators, unlike the 2.x builtins which return lists.
2771
2772 * ``hex(value)``, ``oct(value)``: instead of calling the
2773 :meth:`__hex__` or :meth:`__oct__` methods, these versions will
2774 call the :meth:`__index__` method and convert the result to hexadecimal
2775 or octal. :func:`oct` will use the new ``0o`` notation for its
2776 result.
2777
2778 .. ======================================================================
2779
2780 The :mod:`json` module: JavaScript Object Notation
2781 --------------------------------------------------------------------
2782
2783 The new :mod:`json` module supports the encoding and decoding of Python types in
2784 JSON (Javascript Object Notation). JSON is a lightweight interchange format
2785 often used in web applications. For more information about JSON, see
2786 http://www.json.org.
2787
2788 :mod:`json` comes with support for decoding and encoding most built-in Python
2789 types. The following example encodes and decodes a dictionary::
2790
2791 >>> import json
2792 >>> data = {"spam": "foo", "parrot": 42}
2793 >>> in_json = json.dumps(data) # Encode the data
2794 >>> in_json
2795 '{"parrot": 42, "spam": "foo"}'
2796 >>> json.loads(in_json) # Decode into a Python object
2797 {"spam": "foo", "parrot": 42}
2798
2799 It's also possible to write your own decoders and encoders to support
2800 more types. Pretty-printing of the JSON strings is also supported.
2801
2802 :mod:`json` (originally called simplejson) was written by Bob
2803 Ippolito.
2804
2805
2806 .. ======================================================================
2807
2808 The :mod:`plistlib` module: A Property-List Parser
2809 --------------------------------------------------
2810
2811 The ``.plist`` format is commonly used on Mac OS X to
2812 store basic data types (numbers, strings, lists,
2813 and dictionaries) by serializing them into an XML-based format.
2814 It resembles the XML-RPC serialization of data types.
2815
2816 Despite being primarily used on Mac OS X, the format
2817 has nothing Mac-specific about it and the Python implementation works
2818 on any platform that Python supports, so the :mod:`plistlib` module
2819 has been promoted to the standard library.
2820
2821 Using the module is simple::
2822
2823 import sys
2824 import plistlib
2825 import datetime as dt
2826
2827 # Create data structure
2828 data_struct = dict(lastAccessed=dt.datetime.now(),
2829 version=1,
2830 categories=('Personal','Shared','Private'))
2831
2832 # Create string containing XML.
2833 plist_str = plistlib.writePlistToString(data_struct)
2834 new_struct = plistlib.readPlistFromString(plist_str)
2835 print data_struct
2836 print new_struct
2837
2838 # Write data structure to a file and read it back.
2839 plistlib.writePlist(data_struct, '/tmp/customizations.plist')
2840 new_struct = plistlib.readPlist('/tmp/customizations.plist')
2841
2842 # read/writePlist accepts file-like objects as well as paths.
2843 plistlib.writePlist(data_struct, sys.stdout)
2844
2845 .. ======================================================================
2846
2847 ctypes Enhancements
2848 --------------------------------------------------
2849
2850 Thomas Heller continued to maintain and enhance the
2851 :mod:`ctypes` module.
2852
2853 :mod:`ctypes` now supports a :class:`c_bool` datatype
2854 that represents the C99 ``bool`` type. (Contributed by David Remahl;
2855 :issue:`1649190`.)
2856
2857 The :mod:`ctypes` string, buffer and array types have improved
2858 support for extended slicing syntax,
2859 where various combinations of ``(start, stop, step)`` are supplied.
2860 (Implemented by Thomas Wouters.)
2861
2862 .. Revision 57769
2863
2864 All :mod:`ctypes` data types now support
2865 :meth:`from_buffer` and :meth:`from_buffer_copy`
2866 methods that create a ctypes instance based on a
2867 provided buffer object. :meth:`from_buffer_copy` copies
2868 the contents of the object,
2869 while :meth:`from_buffer` will share the same memory area.
2870
2871 A new calling convention tells :mod:`ctypes` to clear the ``errno`` or
2872 Win32 LastError variables at the outset of each wrapped call.
2873 (Implemented by Thomas Heller; :issue:`1798`.)
2874
2875 You can now retrieve the Unix ``errno`` variable after a function
2876 call. When creating a wrapped function, you can supply
2877 ``use_errno=True`` as a keyword parameter to the :func:`DLL` function
2878 and then call the module-level methods :meth:`set_errno` and
2879 :meth:`get_errno` to set and retrieve the error value.
2880
2881 The Win32 LastError variable is similarly supported by
2882 the :func:`DLL`, :func:`OleDLL`, and :func:`WinDLL` functions.
2883 You supply ``use_last_error=True`` as a keyword parameter
2884 and then call the module-level methods :meth:`set_last_error`
2885 and :meth:`get_last_error`.
2886
2887 The :func:`byref` function, used to retrieve a pointer to a ctypes
2888 instance, now has an optional *offset* parameter that is a byte
2889 count that will be added to the returned pointer.
2890
2891 .. ======================================================================
2892
2893 Improved SSL Support
2894 --------------------------------------------------
2895
2896 Bill Janssen made extensive improvements to Python 2.6's support for
2897 the Secure Sockets Layer by adding a new module, :mod:`ssl`, that's
2898 built atop the `OpenSSL <https://www.openssl.org/>`__ library.
2899 This new module provides more control over the protocol negotiated,
2900 the X.509 certificates used, and has better support for writing SSL
2901 servers (as opposed to clients) in Python. The existing SSL support
2902 in the :mod:`socket` module hasn't been removed and continues to work,
2903 though it will be removed in Python 3.0.
2904
2905 To use the new module, you must first create a TCP connection in the
2906 usual way and then pass it to the :func:`ssl.wrap_socket` function.
2907 It's possible to specify whether a certificate is required, and to
2908 obtain certificate info by calling the :meth:`getpeercert` method.
2909
2910 .. seealso::
2911
2912 The documentation for the :mod:`ssl` module.
2913
2914 .. ======================================================================
2915
2916 Deprecations and Removals
2917 =========================
2918
2919 * String exceptions have been removed. Attempting to use them raises a
2920 :exc:`TypeError`.
2921
2922 * Changes to the :class:`Exception` interface
2923 as dictated by :pep:`352` continue to be made. For 2.6,
2924 the :attr:`!message` attribute is being deprecated in favor of the
2925 :attr:`~BaseException.args` attribute.
2926
2927 * (3.0-warning mode) Python 3.0 will feature a reorganized standard
2928 library that will drop many outdated modules and rename others.
2929 Python 2.6 running in 3.0-warning mode will warn about these modules
2930 when they are imported.
2931
2932 The list of deprecated modules is:
2933 :mod:`!audiodev`,
2934 :mod:`!bgenlocations`,
2935 :mod:`!buildtools`,
2936 :mod:`!bundlebuilder`,
2937 :mod:`!Canvas`,
2938 :mod:`!compiler`,
2939 :mod:`!dircache`,
2940 :mod:`!dl`,
2941 :mod:`!fpformat`,
2942 :mod:`!gensuitemodule`,
2943 :mod:`!ihooks`,
2944 :mod:`!imageop`,
2945 :mod:`!imgfile`,
2946 :mod:`!linuxaudiodev`,
2947 :mod:`!mhlib`,
2948 :mod:`!mimetools`,
2949 :mod:`!multifile`,
2950 :mod:`!new`,
2951 :mod:`!pure`,
2952 :mod:`!statvfs`,
2953 :mod:`!sunaudiodev`,
2954 :mod:`!test.testall`, and
2955 :mod:`!toaiff`.
2956
2957 * The :mod:`!gopherlib` module has been removed.
2958
2959 * The :mod:`!MimeWriter` module and :mod:`!mimify` module
2960 have been deprecated; use the :mod:`email`
2961 package instead.
2962
2963 * The :mod:`!md5` module has been deprecated; use the :mod:`hashlib` module
2964 instead.
2965
2966 * The :mod:`!posixfile` module has been deprecated; :func:`fcntl.lockf`
2967 provides better locking.
2968
2969 * The :mod:`!popen2` module has been deprecated; use the :mod:`subprocess`
2970 module.
2971
2972 * The :mod:`!rgbimg` module has been removed.
2973
2974 * The :mod:`!sets` module has been deprecated; it's better to
2975 use the built-in :class:`set` and :class:`frozenset` types.
2976
2977 * The :mod:`!sha` module has been deprecated; use the :mod:`hashlib` module
2978 instead.
2979
2980
2981 .. ======================================================================
2982
2983
2984 Build and C API Changes
2985 =======================
2986
2987 Changes to Python's build process and to the C API include:
2988
2989 * Python now must be compiled with C89 compilers (after 19
2990 years!). This means that the Python source tree has dropped its
2991 own implementations of :c:func:`!memmove` and :c:func:`!strerror`, which
2992 are in the C89 standard library.
2993
2994 * Python 2.6 can be built with Microsoft Visual Studio 2008 (version
2995 9.0), and this is the new default compiler. See the
2996 :file:`PCbuild` directory for the build files. (Implemented by
2997 Christian Heimes.)
2998
2999 * On Mac OS X, Python 2.6 can be compiled as a 4-way universal build.
3000 The :program:`configure` script
3001 can take a :option:`!--with-universal-archs=[32-bit|64-bit|all]`
3002 switch, controlling whether the binaries are built for 32-bit
3003 architectures (x86, PowerPC), 64-bit (x86-64 and PPC-64), or both.
3004 (Contributed by Ronald Oussoren.)
3005
3006 * A new function added in Python 2.6.6, :c:func:`!PySys_SetArgvEx`, sets
3007 the value of ``sys.argv`` and can optionally update ``sys.path`` to
3008 include the directory containing the script named by ``sys.argv[0]``
3009 depending on the value of an *updatepath* parameter.
3010
3011 This function was added to close a security hole for applications
3012 that embed Python. The old function, :c:func:`!PySys_SetArgv`, would
3013 always update ``sys.path``, and sometimes it would add the current
3014 directory. This meant that, if you ran an application embedding
3015 Python in a directory controlled by someone else, attackers could
3016 put a Trojan-horse module in the directory (say, a file named
3017 :file:`os.py`) that your application would then import and run.
3018
3019 If you maintain a C/C++ application that embeds Python, check
3020 whether you're calling :c:func:`!PySys_SetArgv` and carefully consider
3021 whether the application should be using :c:func:`!PySys_SetArgvEx`
3022 with *updatepath* set to false. Note that using this function will
3023 break compatibility with Python versions 2.6.5 and earlier; if you
3024 have to continue working with earlier versions, you can leave
3025 the call to :c:func:`!PySys_SetArgv` alone and call
3026 ``PyRun_SimpleString("sys.path.pop(0)\n")`` afterwards to discard
3027 the first ``sys.path`` component.
3028
3029 Security issue reported as :cve:`2008-5983`;
3030 discussed in :gh:`50003`, and fixed by Antoine Pitrou.
3031
3032 * The BerkeleyDB module now has a C API object, available as
3033 ``bsddb.db.api``. This object can be used by other C extensions
3034 that wish to use the :mod:`bsddb` module for their own purposes.
3035 (Contributed by Duncan Grisby.)
3036
3037 * The new buffer interface, previously described in
3038 `the PEP 3118 section <#pep-3118-revised-buffer-protocol>`__,
3039 adds :c:func:`PyObject_GetBuffer` and :c:func:`PyBuffer_Release`,
3040 as well as a few other functions.
3041
3042 * Python's use of the C stdio library is now thread-safe, or at least
3043 as thread-safe as the underlying library is. A long-standing potential
3044 bug occurred if one thread closed a file object while another thread
3045 was reading from or writing to the object. In 2.6 file objects
3046 have a reference count, manipulated by the
3047 :c:func:`!PyFile_IncUseCount` and :c:func:`!PyFile_DecUseCount`
3048 functions. File objects can't be closed unless the reference count
3049 is zero. :c:func:`!PyFile_IncUseCount` should be called while the GIL
3050 is still held, before carrying out an I/O operation using the
3051 ``FILE *`` pointer, and :c:func:`!PyFile_DecUseCount` should be called
3052 immediately after the GIL is re-acquired.
3053 (Contributed by Antoine Pitrou and Gregory P. Smith.)
3054
3055 * Importing modules simultaneously in two different threads no longer
3056 deadlocks; it will now raise an :exc:`ImportError`. A new API
3057 function, :c:func:`PyImport_ImportModuleNoBlock`, will look for a
3058 module in ``sys.modules`` first, then try to import it after
3059 acquiring an import lock. If the import lock is held by another
3060 thread, an :exc:`ImportError` is raised.
3061 (Contributed by Christian Heimes.)
3062
3063 * Several functions return information about the platform's
3064 floating-point support. :c:func:`PyFloat_GetMax` returns
3065 the maximum representable floating-point value,
3066 and :c:func:`PyFloat_GetMin` returns the minimum
3067 positive value. :c:func:`PyFloat_GetInfo` returns an object
3068 containing more information from the :file:`float.h` file, such as
3069 ``"mant_dig"`` (number of digits in the mantissa), ``"epsilon"``
3070 (smallest difference between 1.0 and the next largest value
3071 representable), and several others.
3072 (Contributed by Christian Heimes; :issue:`1534`.)
3073
3074 * C functions and methods that use
3075 :c:func:`PyComplex_AsCComplex` will now accept arguments that
3076 have a :meth:`__complex__` method. In particular, the functions in the
3077 :mod:`cmath` module will now accept objects with this method.
3078 This is a backport of a Python 3.0 change.
3079 (Contributed by Mark Dickinson; :issue:`1675423`.)
3080
3081 * Python's C API now includes two functions for case-insensitive string
3082 comparisons, ``PyOS_stricmp(char*, char*)``
3083 and ``PyOS_strnicmp(char*, char*, Py_ssize_t)``.
3084 (Contributed by Christian Heimes; :issue:`1635`.)
3085
3086 * Many C extensions define their own little macro for adding
3087 integers and strings to the module's dictionary in the
3088 ``init*`` function. Python 2.6 finally defines standard macros
3089 for adding values to a module, :c:macro:`PyModule_AddStringMacro`
3090 and :c:macro:`PyModule_AddIntMacro()`. (Contributed by
3091 Christian Heimes.)
3092
3093 * Some macros were renamed in both 3.0 and 2.6 to make it clearer that
3094 they are macros,
3095 not functions. :c:macro:`!Py_Size()` became :c:macro:`Py_SIZE()`,
3096 :c:macro:`!Py_Type()` became :c:macro:`Py_TYPE()`, and
3097 :c:macro:`!Py_Refcnt()` became :c:macro:`Py_REFCNT()`.
3098 The mixed-case macros are still available
3099 in Python 2.6 for backward compatibility.
3100 (:issue:`1629`)
3101
3102 * Distutils now places C extensions it builds in a
3103 different directory when running on a debug version of Python.
3104 (Contributed by Collin Winter; :issue:`1530959`.)
3105
3106 * Several basic data types, such as integers and strings, maintain
3107 internal free lists of objects that can be re-used. The data
3108 structures for these free lists now follow a naming convention: the
3109 variable is always named ``free_list``, the counter is always named
3110 ``numfree``, and a macro ``Py<typename>_MAXFREELIST`` is
3111 always defined.
3112
3113 * A new Makefile target, "make patchcheck", prepares the Python source tree
3114 for making a patch: it fixes trailing whitespace in all modified
3115 ``.py`` files, checks whether the documentation has been changed,
3116 and reports whether the :file:`Misc/ACKS` and :file:`Misc/NEWS` files
3117 have been updated.
3118 (Contributed by Brett Cannon.)
3119
3120 Another new target, "make profile-opt", compiles a Python binary
3121 using GCC's profile-guided optimization. It compiles Python with
3122 profiling enabled, runs the test suite to obtain a set of profiling
3123 results, and then compiles using these results for optimization.
3124 (Contributed by Gregory P. Smith.)
3125
3126 .. ======================================================================
3127
3128 Port-Specific Changes: Windows
3129 -----------------------------------
3130
3131 * The support for Windows 95, 98, ME and NT4 has been dropped.
3132 Python 2.6 requires at least Windows 2000 SP4.
3133
3134 * The new default compiler on Windows is Visual Studio 2008 (version
3135 9.0). The build directories for Visual Studio 2003 (version 7.1) and
3136 2005 (version 8.0) were moved into the PC/ directory. The new
3137 :file:`PCbuild` directory supports cross compilation for X64, debug
3138 builds and Profile Guided Optimization (PGO). PGO builds are roughly
3139 10% faster than normal builds. (Contributed by Christian Heimes
3140 with help from Amaury Forgeot d'Arc and Martin von Löwis.)
3141
3142 * The :mod:`msvcrt` module now supports
3143 both the normal and wide char variants of the console I/O
3144 API. The :func:`~msvcrt.getwch` function reads a keypress and returns a Unicode
3145 value, as does the :func:`~msvcrt.getwche` function. The :func:`~msvcrt.putwch` function
3146 takes a Unicode character and writes it to the console.
3147 (Contributed by Christian Heimes.)
3148
3149 * :func:`os.path.expandvars` will now expand environment variables in
3150 the form "%var%", and "~user" will be expanded into the user's home
3151 directory path. (Contributed by Josiah Carlson; :issue:`957650`.)
3152
3153 * The :mod:`socket` module's socket objects now have an
3154 :meth:`~socket.socket.ioctl` method that provides a limited interface to the
3155 :c:func:`WSAIoctl` system interface.
3156
3157 * The :mod:`_winreg <winreg>` module now has a function,
3158 :func:`~winreg.ExpandEnvironmentStrings`,
3159 that expands environment variable references such as ``%NAME%``
3160 in an input string. The handle objects provided by this
3161 module now support the context protocol, so they can be used
3162 in :keyword:`with` statements. (Contributed by Christian Heimes.)
3163
3164 :mod:`_winreg <winreg>` also has better support for x64 systems,
3165 exposing the :func:`~winreg.DisableReflectionKey`, :func:`~winreg.EnableReflectionKey`,
3166 and :func:`~winreg.QueryReflectionKey` functions, which enable and disable
3167 registry reflection for 32-bit processes running on 64-bit systems.
3168 (:issue:`1753245`)
3169
3170 * The :mod:`!msilib` module's :class:`!Record` object
3171 gained :meth:`!GetInteger` and :meth:`!GetString` methods that
3172 return field values as an integer or a string.
3173 (Contributed by Floris Bruynooghe; :issue:`2125`.)
3174
3175 .. ======================================================================
3176
3177 Port-Specific Changes: Mac OS X
3178 -----------------------------------
3179
3180 * When compiling a framework build of Python, you can now specify the
3181 framework name to be used by providing the
3182 :option:`!--with-framework-name=` option to the
3183 :program:`configure` script.
3184
3185 * The :mod:`!macfs` module has been removed. This in turn required the
3186 :func:`!macostools.touched` function to be removed because it depended on the
3187 :mod:`!macfs` module. (:issue:`1490190`)
3188
3189 * Many other Mac OS modules have been deprecated and will be removed in
3190 Python 3.0:
3191 :mod:`!_builtinSuites`,
3192 :mod:`!aepack`,
3193 :mod:`!aetools`,
3194 :mod:`!aetypes`,
3195 :mod:`!applesingle`,
3196 :mod:`!appletrawmain`,
3197 :mod:`!appletrunner`,
3198 :mod:`!argvemulator`,
3199 :mod:`!Audio_mac`,
3200 :mod:`!autoGIL`,
3201 :mod:`!Carbon`,
3202 :mod:`!cfmfile`,
3203 :mod:`!CodeWarrior`,
3204 :mod:`!ColorPicker`,
3205 :mod:`!EasyDialogs`,
3206 :mod:`!Explorer`,
3207 :mod:`!Finder`,
3208 :mod:`!FrameWork`,
3209 :mod:`!findertools`,
3210 :mod:`!ic`,
3211 :mod:`!icglue`,
3212 :mod:`!icopen`,
3213 :mod:`!macerrors`,
3214 :mod:`!MacOS`,
3215 :mod:`!macfs`,
3216 :mod:`!macostools`,
3217 :mod:`!macresource`,
3218 :mod:`!MiniAEFrame`,
3219 :mod:`!Nav`,
3220 :mod:`!Netscape`,
3221 :mod:`!OSATerminology`,
3222 :mod:`!pimp`,
3223 :mod:`!PixMapWrapper`,
3224 :mod:`!StdSuites`,
3225 :mod:`!SystemEvents`,
3226 :mod:`!Terminal`, and
3227 :mod:`!terminalcommand`.
3228
3229 .. ======================================================================
3230
3231 Port-Specific Changes: IRIX
3232 -----------------------------------
3233
3234 A number of old IRIX-specific modules were deprecated and will
3235 be removed in Python 3.0:
3236 :mod:`!al` and :mod:`!AL`,
3237 :mod:`!cd`,
3238 :mod:`!cddb`,
3239 :mod:`!cdplayer`,
3240 :mod:`!CL` and :mod:`!cl`,
3241 :mod:`!DEVICE`,
3242 :mod:`!ERRNO`,
3243 :mod:`!FILE`,
3244 :mod:`!FL` and :mod:`!fl`,
3245 :mod:`!flp`,
3246 :mod:`!fm`,
3247 :mod:`!GET`,
3248 :mod:`!GLWS`,
3249 :mod:`!GL` and :mod:`!gl`,
3250 :mod:`!IN`,
3251 :mod:`!IOCTL`,
3252 :mod:`!jpeg`,
3253 :mod:`!panelparser`,
3254 :mod:`!readcd`,
3255 :mod:`!SV` and :mod:`!sv`,
3256 :mod:`!torgb`,
3257 :mod:`!videoreader`, and
3258 :mod:`!WAIT`.
3259
3260 .. ======================================================================
3261
3262
3263 Porting to Python 2.6
3264 =====================
3265
3266 This section lists previously described changes and other bugfixes
3267 that may require changes to your code:
3268
3269 * Classes that aren't supposed to be hashable should
3270 set ``__hash__ = None`` in their definitions to indicate
3271 the fact.
3272
3273 * String exceptions have been removed. Attempting to use them raises a
3274 :exc:`TypeError`.
3275
3276 * The :meth:`__init__` method of :class:`collections.deque`
3277 now clears any existing contents of the deque
3278 before adding elements from the iterable. This change makes the
3279 behavior match ``list.__init__()``.
3280
3281 * :meth:`object.__init__` previously accepted arbitrary arguments and
3282 keyword arguments, ignoring them. In Python 2.6, this is no longer
3283 allowed and will result in a :exc:`TypeError`. This will affect
3284 :meth:`__init__` methods that end up calling the corresponding
3285 method on :class:`object` (perhaps through using :func:`super`).
3286 See :issue:`1683368` for discussion.
3287
3288 * The :class:`Decimal` constructor now accepts leading and trailing
3289 whitespace when passed a string. Previously it would raise an
3290 :exc:`InvalidOperation` exception. On the other hand, the
3291 :meth:`create_decimal` method of :class:`Context` objects now
3292 explicitly disallows extra whitespace, raising a
3293 :exc:`ConversionSyntax` exception.
3294
3295 * Due to an implementation accident, if you passed a file path to
3296 the built-in :func:`__import__` function, it would actually import
3297 the specified file. This was never intended to work, however, and
3298 the implementation now explicitly checks for this case and raises
3299 an :exc:`ImportError`.
3300
3301 * C API: the :c:func:`PyImport_Import` and :c:func:`PyImport_ImportModule`
3302 functions now default to absolute imports, not relative imports.
3303 This will affect C extensions that import other modules.
3304
3305 * C API: extension data types that shouldn't be hashable
3306 should define their ``tp_hash`` slot to
3307 :c:func:`PyObject_HashNotImplemented`.
3308
3309 * The :mod:`socket` module exception :exc:`socket.error` now inherits
3310 from :exc:`IOError`. Previously it wasn't a subclass of
3311 :exc:`StandardError` but now it is, through :exc:`IOError`.
3312 (Implemented by Gregory P. Smith; :issue:`1706815`.)
3313
3314 * The :mod:`xmlrpclib <xmlrpc.client>` module no longer automatically converts
3315 :class:`datetime.date` and :class:`datetime.time` to the
3316 :class:`xmlrpclib.DateTime <xmlrpc.client.DateTime>` type; the conversion semantics were
3317 not necessarily correct for all applications. Code using
3318 :mod:`!xmlrpclib` should convert :class:`date` and :class:`~datetime.time`
3319 instances. (:issue:`1330538`)
3320
3321 * (3.0-warning mode) The :class:`Exception` class now warns
3322 when accessed using slicing or index access; having
3323 :class:`Exception` behave like a tuple is being phased out.
3324
3325 * (3.0-warning mode) inequality comparisons between two dictionaries
3326 or two objects that don't implement comparison methods are reported
3327 as warnings. ``dict1 == dict2`` still works, but ``dict1 < dict2``
3328 is being phased out.
3329
3330 Comparisons between cells, which are an implementation detail of Python's
3331 scoping rules, also cause warnings because such comparisons are forbidden
3332 entirely in 3.0.
3333
3334 For applications that embed Python:
3335
3336 * The :c:func:`!PySys_SetArgvEx` function was added in Python 2.6.6,
3337 letting applications close a security hole when the existing
3338 :c:func:`!PySys_SetArgv` function was used. Check whether you're
3339 calling :c:func:`!PySys_SetArgv` and carefully consider whether the
3340 application should be using :c:func:`!PySys_SetArgvEx` with
3341 *updatepath* set to false.
3342
3343 .. ======================================================================
3344
3345
3346 .. _26acks:
3347
3348 Acknowledgements
3349 ================
3350
3351 The author would like to thank the following people for offering
3352 suggestions, corrections and assistance with various drafts of this
3353 article: Georg Brandl, Steve Brown, Nick Coghlan, Ralph Corderoy,
3354 Jim Jewett, Kent Johnson, Chris Lambacher, Martin Michlmayr,
3355 Antoine Pitrou, Brian Warner.
3356