Debian Codegraph @

Doc/whatsnew/2.7.rst

raw
1 ****************************
2 What's New in Python 2.7
3 ****************************
4
5 :Author: A.M. Kuchling (amk at amk.ca)
6
7 .. hyperlink all the methods & functions.
8
9 .. T_STRING_INPLACE not described in main docs
10
11 .. $Id$
12 Rules for maintenance:
13
14 * Anyone can add text to this document. Do not spend very much time
15 on the wording of your changes, because your text will probably
16 get rewritten to some degree.
17
18 * The maintainer will go through Misc/NEWS periodically and add
19 changes; it's therefore more important to add your changes to
20 Misc/NEWS than to this file.
21
22 * This is not a complete list of every single change; completeness
23 is the purpose of Misc/NEWS. Some changes I consider too small
24 or esoteric to include. If such a change is added to the text,
25 I'll just remove it. (This is another reason you shouldn't spend
26 too much time on writing your addition.)
27
28 * If you want to draw your new text to the attention of the
29 maintainer, add 'XXX' to the beginning of the paragraph or
30 section.
31
32 * It's OK to just add a fragmentary note about a change. For
33 example: "XXX Describe the transmogrify() function added to the
34 socket module." The maintainer will research the change and
35 write the necessary text.
36
37 * You can comment out your additions if you like, but it's not
38 necessary (especially when a final release is some months away).
39
40 * Credit the author of a patch or bugfix. Just the name is
41 sufficient; the e-mail address isn't necessary.
42
43 * It's helpful to add the bug/patch number in a parenthetical comment.
44
45 XXX Describe the transmogrify() function added to the socket
46 module.
47 (Contributed by P.Y. Developer; :issue:`12345`.)
48
49 This saves the maintainer some effort going through the SVN logs
50 when researching a change.
51
52 This article explains the new features in Python 2.7. Python 2.7 was released
53 on July 3, 2010.
54
55 Numeric handling has been improved in many ways, for both
56 floating-point numbers and for the :class:`~decimal.Decimal` class.
57 There are some useful additions to the standard library, such as a
58 greatly enhanced :mod:`unittest` module, the :mod:`argparse` module
59 for parsing command-line options, convenient :class:`~collections.OrderedDict`
60 and :class:`~collections.Counter` classes in the :mod:`collections` module,
61 and many other improvements.
62
63 Python 2.7 is planned to be the last of the 2.x releases, so we worked
64 on making it a good release for the long term. To help with porting
65 to Python 3, several new features from the Python 3.x series have been
66 included in 2.7.
67
68 This article doesn't attempt to provide a complete specification of
69 the new features, but instead provides a convenient overview. For
70 full details, you should refer to the documentation for Python 2.7 at
71 https://docs.python.org. If you want to understand the rationale for
72 the design and implementation, refer to the PEP for a particular new
73 feature or the issue on https://bugs.python.org in which a change was
74 discussed. Whenever possible, "What's New in Python" links to the
75 bug/patch item for each change.
76
77 .. _whatsnew27-python31:
78
79 The Future for Python 2.x
80 =========================
81
82 Python 2.7 is the last major release in the 2.x series, as the Python
83 maintainers have shifted the focus of their new feature development efforts
84 to the Python 3.x series. This means that while Python 2 continues to
85 receive bug fixes, and to be updated to build correctly on new hardware and
86 versions of supported operated systems, there will be no new full feature
87 releases for the language or standard library.
88
89 However, while there is a large common subset between Python 2.7 and Python
90 3, and many of the changes involved in migrating to that common subset, or
91 directly to Python 3, can be safely automated, some other changes (notably
92 those associated with Unicode handling) may require careful consideration,
93 and preferably robust automated regression test suites, to migrate
94 effectively.
95
96 This means that Python 2.7 will remain in place for a long time, providing a
97 stable and supported base platform for production systems that have not yet
98 been ported to Python 3. The full expected lifecycle of the Python 2.7
99 series is detailed in :pep:`373`.
100
101 Some key consequences of the long-term significance of 2.7 are:
102
103 * As noted above, the 2.7 release has a much longer period of maintenance
104 when compared to earlier 2.x versions. Python 2.7 is currently expected to
105 remain supported by the core development team (receiving security updates
106 and other bug fixes) until at least 2020 (10 years after its initial
107 release, compared to the more typical support period of 18--24 months).
108
109 * As the Python 2.7 standard library ages, making effective use of the
110 Python Package Index (either directly or via a redistributor) becomes
111 more important for Python 2 users. In addition to a wide variety of third
112 party packages for various tasks, the available packages include backports
113 of new modules and features from the Python 3 standard library that are
114 compatible with Python 2, as well as various tools and libraries that can
115 make it easier to migrate to Python 3. The `Python Packaging User Guide
116 <https://packaging.python.org>`__ provides guidance on downloading and
117 installing software from the Python Package Index.
118
119 * While the preferred approach to enhancing Python 2 is now the publication
120 of new packages on the Python Package Index, this approach doesn't
121 necessarily work in all cases, especially those related to network
122 security. In exceptional cases that cannot be handled adequately by
123 publishing new or updated packages on PyPI, the Python Enhancement
124 Proposal process may be used to make the case for adding new features
125 directly to the Python 2 standard library. Any such additions, and the
126 maintenance releases where they were added, will be noted in the
127 :ref:`py27-maintenance-enhancements` section below.
128
129 For projects wishing to migrate from Python 2 to Python 3, or for library
130 and framework developers wishing to support users on both Python 2 and
131 Python 3, there are a variety of tools and guides available to help decide
132 on a suitable approach and manage some of the technical details involved.
133 The recommended starting point is the :ref:`pyporting-howto` HOWTO guide.
134
135
136 Changes to the Handling of Deprecation Warnings
137 ===============================================
138
139 For Python 2.7, a policy decision was made to silence warnings only of
140 interest to developers by default. :exc:`DeprecationWarning` and its
141 descendants are now ignored unless otherwise requested, preventing
142 users from seeing warnings triggered by an application. This change
143 was also made in the branch that became Python 3.2. (Discussed
144 on stdlib-sig and carried out in :issue:`7319`.)
145
146 In previous releases, :exc:`DeprecationWarning` messages were
147 enabled by default, providing Python developers with a clear
148 indication of where their code may break in a future major version
149 of Python.
150
151 However, there are increasingly many users of Python-based
152 applications who are not directly involved in the development of
153 those applications. :exc:`DeprecationWarning` messages are
154 irrelevant to such users, making them worry about an application
155 that's actually working correctly and burdening application developers
156 with responding to these concerns.
157
158 You can re-enable display of :exc:`DeprecationWarning` messages by
159 running Python with the :option:`-Wdefault <-W>` (short form:
160 :option:`-Wd <-W>`) switch, or by setting the :envvar:`PYTHONWARNINGS`
161 environment variable to ``"default"`` (or ``"d"``) before running
162 Python. Python code can also re-enable them
163 by calling ``warnings.simplefilter('default')``.
164
165 The ``unittest`` module also automatically reenables deprecation warnings
166 when running tests.
167
168
169 Python 3.1 Features
170 =======================
171
172 Much as Python 2.6 incorporated features from Python 3.0,
173 version 2.7 incorporates some of the new features
174 in Python 3.1. The 2.x series continues to provide tools
175 for migrating to the 3.x series.
176
177 A partial list of 3.1 features that were backported to 2.7:
178
179 * The syntax for set literals (``{1,2,3}`` is a mutable set).
180 * Dictionary and set comprehensions (``{i: i*2 for i in range(3)}``).
181 * Multiple context managers in a single :keyword:`with` statement.
182 * A new version of the :mod:`io` library, rewritten in C for performance.
183 * The ordered-dictionary type described in :ref:`pep-0372`.
184 * The new ``","`` format specifier described in :ref:`pep-0378`.
185 * The :class:`memoryview` object.
186 * A small subset of the :mod:`importlib` module,
187 `described below <#importlib-section>`__.
188 * The :func:`repr` of a float ``x`` is shorter in many cases: it's now
189 based on the shortest decimal string that's guaranteed to round back
190 to ``x``. As in previous versions of Python, it's guaranteed that
191 ``float(repr(x))`` recovers ``x``.
192 * Float-to-string and string-to-float conversions are correctly rounded.
193 The :func:`round` function is also now correctly rounded.
194 * The :c:type:`PyCapsule` type, used to provide a C API for extension modules.
195 * The :c:func:`PyLong_AsLongAndOverflow` C API function.
196
197 Other new Python3-mode warnings include:
198
199 * :func:`!operator.isCallable` and :func:`!operator.sequenceIncludes`,
200 which are not supported in 3.x, now trigger warnings.
201 * The :option:`!-3` switch now automatically
202 enables the :option:`!-Qwarn` switch that causes warnings
203 about using classic division with integers and long integers.
204
205
206
207 .. ========================================================================
208 .. Large, PEP-level features and changes should be described here.
209 .. ========================================================================
210
211 .. _pep-0372:
212
213 PEP 372: Adding an Ordered Dictionary to collections
214 ====================================================
215
216 Regular Python dictionaries iterate over key/value pairs in arbitrary order.
217 Over the years, a number of authors have written alternative implementations
218 that remember the order that the keys were originally inserted. Based on
219 the experiences from those implementations, 2.7 introduces a new
220 :class:`~collections.OrderedDict` class in the :mod:`collections` module.
221
222 The :class:`~collections.OrderedDict` API provides the same interface as regular
223 dictionaries but iterates over keys and values in a guaranteed order
224 depending on when a key was first inserted::
225
226 >>> from collections import OrderedDict
227 >>> d = OrderedDict([('first', 1),
228 ... ('second', 2),
229 ... ('third', 3)])
230 >>> d.items()
231 [('first', 1), ('second', 2), ('third', 3)]
232
233 If a new entry overwrites an existing entry, the original insertion
234 position is left unchanged::
235
236 >>> d['second'] = 4
237 >>> d.items()
238 [('first', 1), ('second', 4), ('third', 3)]
239
240 Deleting an entry and reinserting it will move it to the end::
241
242 >>> del d['second']
243 >>> d['second'] = 5
244 >>> d.items()
245 [('first', 1), ('third', 3), ('second', 5)]
246
247 The :meth:`~collections.OrderedDict.popitem` method has an optional *last*
248 argument that defaults to ``True``. If *last* is true, the most recently
249 added key is returned and removed; if it's false, the
250 oldest key is selected::
251
252 >>> od = OrderedDict([(x,0) for x in range(20)])
253 >>> od.popitem()
254 (19, 0)
255 >>> od.popitem()
256 (18, 0)
257 >>> od.popitem(last=False)
258 (0, 0)
259 >>> od.popitem(last=False)
260 (1, 0)
261
262 Comparing two ordered dictionaries checks both the keys and values,
263 and requires that the insertion order was the same::
264
265 >>> od1 = OrderedDict([('first', 1),
266 ... ('second', 2),
267 ... ('third', 3)])
268 >>> od2 = OrderedDict([('third', 3),
269 ... ('first', 1),
270 ... ('second', 2)])
271 >>> od1 == od2
272 False
273 >>> # Move 'third' key to the end
274 >>> del od2['third']; od2['third'] = 3
275 >>> od1 == od2
276 True
277
278 Comparing an :class:`~collections.OrderedDict` with a regular dictionary
279 ignores the insertion order and just compares the keys and values.
280
281 How does the :class:`~collections.OrderedDict` work? It maintains a
282 doubly linked list of keys, appending new keys to the list as they're inserted.
283 A secondary dictionary maps keys to their corresponding list node, so
284 deletion doesn't have to traverse the entire linked list and therefore
285 remains *O*\ (1).
286
287 The standard library now supports use of ordered dictionaries in several
288 modules.
289
290 * The :mod:`ConfigParser <configparser>` module uses them by default, meaning that
291 configuration files can now be read, modified, and then written back
292 in their original order.
293
294 * The :meth:`~collections.somenamedtuple._asdict` method for
295 :func:`collections.namedtuple` now returns an ordered dictionary with the
296 values appearing in the same order as the underlying tuple indices.
297
298 * The :mod:`json` module's :class:`~json.JSONDecoder` class
299 constructor was extended with an *object_pairs_hook* parameter to
300 allow :class:`OrderedDict` instances to be built by the decoder.
301 Support was also added for third-party tools like
302 `PyYAML <https://pyyaml.org/>`_.
303
304 .. seealso::
305
306 :pep:`372` - Adding an ordered dictionary to collections
307 PEP written by Armin Ronacher and Raymond Hettinger;
308 implemented by Raymond Hettinger.
309
310 .. _pep-0378:
311
312 PEP 378: Format Specifier for Thousands Separator
313 =================================================
314
315 To make program output more readable, it can be useful to add
316 separators to large numbers, rendering them as
317 18,446,744,073,709,551,616 instead of 18446744073709551616.
318
319 The fully general solution for doing this is the :mod:`locale` module,
320 which can use different separators ("," in North America, "." in
321 Europe) and different grouping sizes, but :mod:`locale` is complicated
322 to use and unsuitable for multi-threaded applications where different
323 threads are producing output for different locales.
324
325 Therefore, a simple comma-grouping mechanism has been added to the
326 mini-language used by the :meth:`str.format` method. When
327 formatting a floating-point number, simply include a comma between the
328 width and the precision::
329
330 >>> '{:20,.2f}'.format(18446744073709551616.0)
331 '18,446,744,073,709,551,616.00'
332
333 When formatting an integer, include the comma after the width:
334
335 >>> '{:20,d}'.format(18446744073709551616)
336 '18,446,744,073,709,551,616'
337
338 This mechanism is not adaptable at all; commas are always used as the
339 separator and the grouping is always into three-digit groups. The
340 comma-formatting mechanism isn't as general as the :mod:`locale`
341 module, but it's easier to use.
342
343 .. seealso::
344
345 :pep:`378` - Format Specifier for Thousands Separator
346 PEP written by Raymond Hettinger; implemented by Eric Smith.
347
348 PEP 389: The argparse Module for Parsing Command Lines
349 ======================================================
350
351 The :mod:`argparse` module for parsing command-line arguments was
352 added as a more powerful replacement for the
353 :mod:`optparse` module.
354
355 This means Python now supports three different modules for parsing
356 command-line arguments: :mod:`getopt`, :mod:`optparse`, and
357 :mod:`argparse`. The :mod:`getopt` module closely resembles the C
358 library's :c:func:`!getopt` function, so it remains useful if you're writing a
359 Python prototype that will eventually be rewritten in C.
360 :mod:`optparse` becomes redundant, but there are no plans to remove it
361 because there are many scripts still using it, and there's no
362 automated way to update these scripts. (Making the :mod:`argparse`
363 API consistent with :mod:`optparse`'s interface was discussed but
364 rejected as too messy and difficult.)
365
366 In short, if you're writing a new script and don't need to worry
367 about compatibility with earlier versions of Python, use
368 :mod:`argparse` instead of :mod:`optparse`.
369
370 Here's an example::
371
372 import argparse
373
374 parser = argparse.ArgumentParser(description='Command-line example.')
375
376 # Add optional switches
377 parser.add_argument('-v', action='store_true', dest='is_verbose',
378 help='produce verbose output')
379 parser.add_argument('-o', action='store', dest='output',
380 metavar='FILE',
381 help='direct output to FILE instead of stdout')
382 parser.add_argument('-C', action='store', type=int, dest='context',
383 metavar='NUM', default=0,
384 help='display NUM lines of added context')
385
386 # Allow any number of additional arguments.
387 parser.add_argument(nargs='*', action='store', dest='inputs',
388 help='input filenames (default is stdin)')
389
390 args = parser.parse_args()
391 print args.__dict__
392
393 Unless you override it, :option:`!-h` and :option:`!--help` switches
394 are automatically added, and produce neatly formatted output::
395
396 -> ./python.exe argparse-example.py --help
397 usage: argparse-example.py [-h] [-v] [-o FILE] [-C NUM] [inputs [inputs ...]]
398
399 Command-line example.
400
401 positional arguments:
402 inputs input filenames (default is stdin)
403
404 optional arguments:
405 -h, --help show this help message and exit
406 -v produce verbose output
407 -o FILE direct output to FILE instead of stdout
408 -C NUM display NUM lines of added context
409
410 As with :mod:`optparse`, the command-line switches and arguments
411 are returned as an object with attributes named by the *dest* parameters::
412
413 -> ./python.exe argparse-example.py -v
414 {'output': None,
415 'is_verbose': True,
416 'context': 0,
417 'inputs': []}
418
419 -> ./python.exe argparse-example.py -v -o /tmp/output -C 4 file1 file2
420 {'output': '/tmp/output',
421 'is_verbose': True,
422 'context': 4,
423 'inputs': ['file1', 'file2']}
424
425 :mod:`argparse` has much fancier validation than :mod:`optparse`; you
426 can specify an exact number of arguments as an integer, 0 or more
427 arguments by passing ``'*'``, 1 or more by passing ``'+'``, or an
428 optional argument with ``'?'``. A top-level parser can contain
429 sub-parsers to define subcommands that have different sets of
430 switches, as in ``svn commit``, ``svn checkout``, etc. You can
431 specify an argument's type as :class:`~argparse.FileType`, which will
432 automatically open files for you and understands that ``'-'`` means
433 standard input or output.
434
435 .. seealso::
436
437 :mod:`argparse` documentation
438 The documentation page of the argparse module.
439
440 :ref:`upgrading-optparse-code`
441 Part of the Python documentation, describing how to convert
442 code that uses :mod:`optparse`.
443
444 :pep:`389` - argparse - New Command Line Parsing Module
445 PEP written and implemented by Steven Bethard.
446
447 PEP 391: Dictionary-Based Configuration For Logging
448 ====================================================
449
450 The :mod:`logging` module is very flexible; applications can define
451 a tree of logging subsystems, and each logger in this tree can filter
452 out certain messages, format them differently, and direct messages to
453 a varying number of handlers.
454
455 All this flexibility can require a lot of configuration. You can
456 write Python statements to create objects and set their properties,
457 but a complex set-up requires verbose but boring code.
458 :mod:`logging` also supports a :func:`~logging.config.fileConfig`
459 function that parses a file, but the file format doesn't support
460 configuring filters, and it's messier to generate programmatically.
461
462 Python 2.7 adds a :func:`~logging.config.dictConfig` function that
463 uses a dictionary to configure logging. There are many ways to
464 produce a dictionary from different sources: construct one with code;
465 parse a file containing JSON; or use a YAML parsing library if one is
466 installed. For more information see :ref:`logging-config-api`.
467
468 The following example configures two loggers, the root logger and a
469 logger named "network". Messages sent to the root logger will be
470 sent to the system log using the syslog protocol, and messages
471 to the "network" logger will be written to a :file:`network.log` file
472 that will be rotated once the log reaches 1MB.
473
474 ::
475
476 import logging
477 import logging.config
478
479 configdict = {
480 'version': 1, # Configuration schema in use; must be 1 for now
481 'formatters': {
482 'standard': {
483 'format': ('%(asctime)s %(name)-15s '
484 '%(levelname)-8s %(message)s')}},
485
486 'handlers': {'netlog': {'backupCount': 10,
487 'class': 'logging.handlers.RotatingFileHandler',
488 'filename': '/logs/network.log',
489 'formatter': 'standard',
490 'level': 'INFO',
491 'maxBytes': 1000000},
492 'syslog': {'class': 'logging.handlers.SysLogHandler',
493 'formatter': 'standard',
494 'level': 'ERROR'}},
495
496 # Specify all the subordinate loggers
497 'loggers': {
498 'network': {
499 'handlers': ['netlog']
500 }
501 },
502 # Specify properties of the root logger
503 'root': {
504 'handlers': ['syslog']
505 },
506 }
507
508 # Set up configuration
509 logging.config.dictConfig(configdict)
510
511 # As an example, log two error messages
512 logger = logging.getLogger('/')
513 logger.error('Database not found')
514
515 netlogger = logging.getLogger('network')
516 netlogger.error('Connection failed')
517
518 Three smaller enhancements to the :mod:`logging` module, all
519 implemented by Vinay Sajip, are:
520
521 .. rev79293
522
523 * The :class:`~logging.handlers.SysLogHandler` class now supports
524 syslogging over TCP. The constructor has a *socktype* parameter
525 giving the type of socket to use, either :const:`socket.SOCK_DGRAM`
526 for UDP or :const:`socket.SOCK_STREAM` for TCP. The default
527 protocol remains UDP.
528
529 * :class:`~logging.Logger` instances gained a :meth:`~logging.Logger.getChild`
530 method that retrieves a descendant logger using a relative path.
531 For example, once you retrieve a logger by doing ``log = getLogger('app')``,
532 calling ``log.getChild('network.listen')`` is equivalent to
533 ``getLogger('app.network.listen')``.
534
535 * The :class:`~logging.LoggerAdapter` class gained an
536 :meth:`~logging.Logger.isEnabledFor` method that takes a
537 *level* and returns whether the underlying logger would
538 process a message of that level of importance.
539
540 .. XXX: Logger objects don't have a class declaration so the link don't work
541
542 .. seealso::
543
544 :pep:`391` - Dictionary-Based Configuration For Logging
545 PEP written and implemented by Vinay Sajip.
546
547 PEP 3106: Dictionary Views
548 ====================================================
549
550 The dictionary methods :meth:`~dict.keys`, :meth:`~dict.values`, and
551 :meth:`~dict.items` are different in Python 3.x. They return an object
552 called a :dfn:`view` instead of a fully materialized list.
553
554 It's not possible to change the return values of :meth:`~dict.keys`,
555 :meth:`~dict.values`, and :meth:`~dict.items` in Python 2.7 because
556 too much code would break. Instead the 3.x versions were added
557 under the new names :meth:`!viewkeys`, :meth:`!viewvalues`,
558 and :meth:`!viewitems`.
559
560 ::
561
562 >>> d = dict((i*10, chr(65+i)) for i in range(26))
563 >>> d
564 {0: 'A', 130: 'N', 10: 'B', 140: 'O', 20: ..., 250: 'Z'}
565 >>> d.viewkeys()
566 dict_keys([0, 130, 10, 140, 20, 150, 30, ..., 250])
567
568 Views can be iterated over, but the key and item views also behave
569 like sets. The ``&`` operator performs intersection, and ``|``
570 performs a union::
571
572 >>> d1 = dict((i*10, chr(65+i)) for i in range(26))
573 >>> d2 = dict((i**.5, i) for i in range(1000))
574 >>> d1.viewkeys() & d2.viewkeys()
575 set([0.0, 10.0, 20.0, 30.0])
576 >>> d1.viewkeys() | range(0, 30)
577 set([0, 1, 130, 3, 4, 5, 6, ..., 120, 250])
578
579 The view keeps track of the dictionary and its contents change as the
580 dictionary is modified::
581
582 >>> vk = d.viewkeys()
583 >>> vk
584 dict_keys([0, 130, 10, ..., 250])
585 >>> d[260] = '&'
586 >>> vk
587 dict_keys([0, 130, 260, 10, ..., 250])
588
589 However, note that you can't add or remove keys while you're iterating
590 over the view::
591
592 >>> for k in vk:
593 ... d[k*2] = k
594 ...
595 Traceback (most recent call last):
596 File "<stdin>", line 1, in <module>
597 RuntimeError: dictionary changed size during iteration
598
599 You can use the view methods in Python 2.x code, and the 2to3
600 converter will change them to the standard :meth:`~dict.keys`,
601 :meth:`~dict.values`, and :meth:`~dict.items` methods.
602
603 .. seealso::
604
605 :pep:`3106` - Revamping dict.keys(), .values() and .items()
606 PEP written by Guido van Rossum.
607 Backported to 2.7 by Alexandre Vassalotti; :issue:`1967`.
608
609
610 PEP 3137: The memoryview Object
611 ====================================================
612
613 The :class:`memoryview` object provides a view of another object's
614 memory content that matches the :class:`bytes` type's interface.
615
616 .. doctest::
617 :options: +SKIP
618
619 >>> import string
620 >>> m = memoryview(string.letters)
621 >>> m
622 <memory at 0x37f850>
623 >>> len(m) # Returns length of underlying object
624 52
625 >>> m[0], m[25], m[26] # Indexing returns one byte
626 ('a', 'z', 'A')
627 >>> m2 = m[0:26] # Slicing returns another memoryview
628 >>> m2
629 <memory at 0x37f080>
630
631 The content of the view can be converted to a string of bytes or
632 a list of integers:
633
634 .. doctest::
635 :options: +SKIP
636
637 >>> m2.tobytes()
638 'abcdefghijklmnopqrstuvwxyz'
639 >>> m2.tolist()
640 [97, 98, 99, 100, 101, 102, 103, ... 121, 122]
641 >>>
642
643 :class:`memoryview` objects allow modifying the underlying object if
644 it's a mutable object.
645
646 .. doctest::
647 :options: +SKIP
648
649 >>> m2[0] = 75
650 Traceback (most recent call last):
651 File "<stdin>", line 1, in <module>
652 TypeError: cannot modify read-only memory
653 >>> b = bytearray(string.letters) # Creating a mutable object
654 >>> b
655 bytearray(b'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ')
656 >>> mb = memoryview(b)
657 >>> mb[0] = '*' # Assign to view, changing the bytearray.
658 >>> b[0:5] # The bytearray has been changed.
659 bytearray(b'*bcde')
660 >>>
661
662 .. seealso::
663
664 :pep:`3137` - Immutable Bytes and Mutable Buffer
665 PEP written by Guido van Rossum.
666 Implemented by Travis Oliphant, Antoine Pitrou and others.
667 Backported to 2.7 by Antoine Pitrou; :issue:`2396`.
668
669
670
671 Other Language Changes
672 ======================
673
674 Some smaller changes made to the core Python language are:
675
676 * The syntax for set literals has been backported from Python 3.x.
677 Curly brackets are used to surround the contents of the resulting
678 mutable set; set literals are
679 distinguished from dictionaries by not containing colons and values.
680 ``{}`` continues to represent an empty dictionary; use
681 ``set()`` for an empty set.
682
683 .. doctest::
684 :options: +SKIP
685
686 >>> {1, 2, 3, 4, 5}
687 set([1, 2, 3, 4, 5])
688 >>> set() # empty set
689 set([])
690 >>> {} # empty dict
691 {}
692
693 Backported by Alexandre Vassalotti; :issue:`2335`.
694
695 * Dictionary and set comprehensions are another feature backported from
696 3.x, generalizing list/generator comprehensions to use
697 the literal syntax for sets and dictionaries.
698
699 .. doctest::
700 :options: +SKIP
701
702 >>> {x: x*x for x in range(6)}
703 {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25}
704 >>> {('a'*x) for x in range(6)}
705 set(['', 'a', 'aa', 'aaa', 'aaaa', 'aaaaa'])
706
707 Backported by Alexandre Vassalotti; :issue:`2333`.
708
709 * The :keyword:`with` statement can now use multiple context managers
710 in one statement. Context managers are processed from left to right
711 and each one is treated as beginning a new :keyword:`!with` statement.
712 This means that::
713
714 with A() as a, B() as b:
715 ... suite of statements ...
716
717 is equivalent to::
718
719 with A() as a:
720 with B() as b:
721 ... suite of statements ...
722
723 The :func:`!contextlib.nested` function provides a very similar
724 function, so it's no longer necessary and has been deprecated.
725
726 (Proposed in https://codereview.appspot.com/53094; implemented by
727 Georg Brandl.)
728
729 * Conversions between floating-point numbers and strings are
730 now correctly rounded on most platforms. These conversions occur
731 in many different places: :func:`str` on
732 floats and complex numbers; the :class:`float` and :class:`complex`
733 constructors;
734 numeric formatting; serializing and
735 deserializing floats and complex numbers using the
736 :mod:`marshal`, :mod:`pickle`
737 and :mod:`json` modules;
738 parsing of float and imaginary literals in Python code;
739 and :class:`~decimal.Decimal`-to-float conversion.
740
741 Related to this, the :func:`repr` of a floating-point number *x*
742 now returns a result based on the shortest decimal string that's
743 guaranteed to round back to *x* under correct rounding (with
744 round-half-to-even rounding mode). Previously it gave a string
745 based on rounding x to 17 decimal digits.
746
747 .. maybe add an example?
748
749 The rounding library responsible for this improvement works on
750 Windows and on Unix platforms using the gcc, icc, or suncc
751 compilers. There may be a small number of platforms where correct
752 operation of this code cannot be guaranteed, so the code is not
753 used on such systems. You can find out which code is being used
754 by checking :data:`sys.float_repr_style`, which will be ``short``
755 if the new code is in use and ``legacy`` if it isn't.
756
757 Implemented by Eric Smith and Mark Dickinson, using David Gay's
758 :file:`dtoa.c` library; :issue:`7117`.
759
760 * Conversions from long integers and regular integers to floating
761 point now round differently, returning the floating-point number
762 closest to the number. This doesn't matter for small integers that
763 can be converted exactly, but for large numbers that will
764 unavoidably lose precision, Python 2.7 now approximates more
765 closely. For example, Python 2.6 computed the following::
766
767 >>> n = 295147905179352891391
768 >>> float(n)
769 2.9514790517935283e+20
770 >>> n - long(float(n))
771 65535L
772
773 Python 2.7's floating-point result is larger, but much closer to the
774 true value::
775
776 >>> n = 295147905179352891391
777 >>> float(n)
778 2.9514790517935289e+20
779 >>> n - long(float(n))
780 -1L
781
782 (Implemented by Mark Dickinson; :issue:`3166`.)
783
784 Integer division is also more accurate in its rounding behaviours. (Also
785 implemented by Mark Dickinson; :issue:`1811`.)
786
787 * Implicit coercion for complex numbers has been removed; the interpreter
788 will no longer ever attempt to call a :meth:`!__coerce__` method on complex
789 objects. (Removed by Meador Inge and Mark Dickinson; :issue:`5211`.)
790
791 * The :meth:`str.format` method now supports automatic numbering of the replacement
792 fields. This makes using :meth:`str.format` more closely resemble using
793 ``%s`` formatting::
794
795 >>> '{}:{}:{}'.format(2009, 04, 'Sunday')
796 '2009:4:Sunday'
797 >>> '{}:{}:{day}'.format(2009, 4, day='Sunday')
798 '2009:4:Sunday'
799
800 The auto-numbering takes the fields from left to right, so the first ``{...}``
801 specifier will use the first argument to :meth:`str.format`, the next
802 specifier will use the next argument, and so on. You can't mix auto-numbering
803 and explicit numbering -- either number all of your specifier fields or none
804 of them -- but you can mix auto-numbering and named fields, as in the second
805 example above. (Contributed by Eric Smith; :issue:`5237`.)
806
807 Complex numbers now correctly support usage with :func:`format`,
808 and default to being right-aligned.
809 Specifying a precision or comma-separation applies to both the real
810 and imaginary parts of the number, but a specified field width and
811 alignment is applied to the whole of the resulting ``1.5+3j``
812 output. (Contributed by Eric Smith; :issue:`1588` and :issue:`7988`.)
813
814 The 'F' format code now always formats its output using uppercase characters,
815 so it will now produce 'INF' and 'NAN'.
816 (Contributed by Eric Smith; :issue:`3382`.)
817
818 A low-level change: the :meth:`object.__format__` method now triggers
819 a :exc:`PendingDeprecationWarning` if it's passed a format string,
820 because the :meth:`!__format__` method for :class:`object` converts
821 the object to a string representation and formats that. Previously
822 the method silently applied the format string to the string
823 representation, but that could hide mistakes in Python code. If
824 you're supplying formatting information such as an alignment or
825 precision, presumably you're expecting the formatting to be applied
826 in some object-specific way. (Fixed by Eric Smith; :issue:`7994`.)
827
828 * The :func:`int` and :func:`!long` types gained a ``bit_length``
829 method that returns the number of bits necessary to represent
830 its argument in binary::
831
832 >>> n = 37
833 >>> bin(n)
834 '0b100101'
835 >>> n.bit_length()
836 6
837 >>> n = 2**123-1
838 >>> n.bit_length()
839 123
840 >>> (n+1).bit_length()
841 124
842
843 (Contributed by Fredrik Johansson and Victor Stinner; :issue:`3439`.)
844
845 * The :keyword:`import` statement will no longer try an absolute import
846 if a relative import (e.g. ``from .os import sep``) fails. This
847 fixes a bug, but could possibly break certain :keyword:`!import`
848 statements that were only working by accident. (Fixed by Meador Inge;
849 :issue:`7902`.)
850
851 * It's now possible for a subclass of the built-in :class:`!unicode` type
852 to override the :meth:`!__unicode__` method. (Implemented by
853 Victor Stinner; :issue:`1583863`.)
854
855 * The :class:`bytearray` type's :meth:`~bytearray.translate` method now accepts
856 ``None`` as its first argument. (Fixed by Georg Brandl;
857 :issue:`4759`.)
858
859 .. XXX bytearray doesn't seem to be documented
860
861 * When using :deco:`classmethod` and
862 :deco:`staticmethod` to wrap
863 methods as class or static methods, the wrapper object now
864 exposes the wrapped function as their :attr:`~method.__func__`
865 attribute.
866 (Contributed by Amaury Forgeot d'Arc, after a suggestion by
867 George Sakkis; :issue:`5982`.)
868
869 * When a restricted set of attributes were set using ``__slots__``,
870 deleting an unset attribute would not raise :exc:`AttributeError`
871 as you would expect. Fixed by Benjamin Peterson; :issue:`7604`.)
872
873 * Two new encodings are now supported: "cp720", used primarily for
874 Arabic text; and "cp858", a variant of CP 850 that adds the euro
875 symbol. (CP720 contributed by Alexander Belchenko and Amaury
876 Forgeot d'Arc in :issue:`1616979`; CP858 contributed by Tim Hatch in
877 :issue:`8016`.)
878
879 * The :class:`!file` object will now set the :attr:`!filename` attribute
880 on the :exc:`IOError` exception when trying to open a directory
881 on POSIX platforms (noted by Jan Kaliszewski; :issue:`4764`), and
882 now explicitly checks for and forbids writing to read-only file objects
883 instead of trusting the C library to catch and report the error
884 (fixed by Stefan Krah; :issue:`5677`).
885
886 * The Python tokenizer now translates line endings itself, so the
887 :func:`compile` built-in function now accepts code using any
888 line-ending convention. Additionally, it no longer requires that the
889 code end in a newline.
890
891 * Extra parentheses in function definitions are illegal in Python 3.x,
892 meaning that you get a syntax error from ``def f((x)): pass``. In
893 Python3-warning mode, Python 2.7 will now warn about this odd usage.
894 (Noted by James Lingard; :issue:`7362`.)
895
896 * It's now possible to create weak references to old-style class
897 objects. New-style classes were always weak-referenceable. (Fixed
898 by Antoine Pitrou; :issue:`8268`.)
899
900 * When a module object is garbage-collected, the module's dictionary is
901 now only cleared if no one else is holding a reference to the
902 dictionary (:issue:`7140`).
903
904 .. ======================================================================
905
906 .. _new-27-interpreter:
907
908 Interpreter Changes
909 -------------------------------
910
911 A new environment variable, :envvar:`PYTHONWARNINGS`,
912 allows controlling warnings. It should be set to a string
913 containing warning settings, equivalent to those
914 used with the :option:`-W` switch, separated by commas.
915 (Contributed by Brian Curtin; :issue:`7301`.)
916
917 For example, the following setting will print warnings every time
918 they occur, but turn warnings from the :mod:`Cookie <http.cookies>` module into an
919 error. (The exact syntax for setting an environment variable varies
920 across operating systems and shells.)
921
922 ::
923
924 export PYTHONWARNINGS=all,error:::Cookie:0
925
926 .. ======================================================================
927
928
929 Optimizations
930 -------------
931
932 Several performance enhancements have been added:
933
934 * A new opcode was added to perform the initial setup for
935 :keyword:`with` statements, looking up the :meth:`~object.__enter__` and
936 :meth:`~object.__exit__` methods. (Contributed by Benjamin Peterson.)
937
938 * The garbage collector now performs better for one common usage
939 pattern: when many objects are being allocated without deallocating
940 any of them. This would previously take quadratic
941 time for garbage collection, but now the number of full garbage collections
942 is reduced as the number of objects on the heap grows.
943 The new logic only performs a full garbage collection pass when
944 the middle generation has been collected 10 times and when the
945 number of survivor objects from the middle generation exceeds 10% of
946 the number of objects in the oldest generation. (Suggested by Martin
947 von Löwis and implemented by Antoine Pitrou; :issue:`4074`.)
948
949 * The garbage collector tries to avoid tracking simple containers
950 which can't be part of a cycle. In Python 2.7, this is now true for
951 tuples and dicts containing atomic types (such as ints, strings,
952 etc.). Transitively, a dict containing tuples of atomic types won't
953 be tracked either. This helps reduce the cost of each
954 garbage collection by decreasing the number of objects to be
955 considered and traversed by the collector.
956 (Contributed by Antoine Pitrou; :issue:`4688`.)
957
958 * Long integers are now stored internally either in base ``2**15`` or in base
959 ``2**30``, the base being determined at build time. Previously, they
960 were always stored in base ``2**15``. Using base ``2**30`` gives
961 significant performance improvements on 64-bit machines, but
962 benchmark results on 32-bit machines have been mixed. Therefore,
963 the default is to use base ``2**30`` on 64-bit machines and base ``2**15``
964 on 32-bit machines; on Unix, there's a new configure option
965 :option:`!--enable-big-digits` that can be used to override this default.
966
967 Apart from the performance improvements this change should be
968 invisible to end users, with one exception: for testing and
969 debugging purposes there's a new structseq :data:`!sys.long_info` that
970 provides information about the internal format, giving the number of
971 bits per digit and the size in bytes of the C type used to store
972 each digit::
973
974 >>> import sys
975 >>> sys.long_info
976 sys.long_info(bits_per_digit=30, sizeof_digit=4)
977
978 (Contributed by Mark Dickinson; :issue:`4258`.)
979
980 Another set of changes made long objects a few bytes smaller: 2 bytes
981 smaller on 32-bit systems and 6 bytes on 64-bit.
982 (Contributed by Mark Dickinson; :issue:`5260`.)
983
984 * The division algorithm for long integers has been made faster
985 by tightening the inner loop, doing shifts instead of multiplications,
986 and fixing an unnecessary extra iteration.
987 Various benchmarks show speedups of between 50% and 150% for long
988 integer divisions and modulo operations.
989 (Contributed by Mark Dickinson; :issue:`5512`.)
990 Bitwise operations are also significantly faster (initial patch by
991 Gregory Smith; :issue:`1087418`).
992
993 * The implementation of ``%`` checks for the left-side operand being
994 a Python string and special-cases it; this results in a 1--3%
995 performance increase for applications that frequently use ``%``
996 with strings, such as templating libraries.
997 (Implemented by Collin Winter; :issue:`5176`.)
998
999 * List comprehensions with an ``if`` condition are compiled into
1000 faster bytecode. (Patch by Antoine Pitrou, back-ported to 2.7
1001 by Jeffrey Yasskin; :issue:`4715`.)
1002
1003 * Converting an integer or long integer to a decimal string was made
1004 faster by special-casing base 10 instead of using a generalized
1005 conversion function that supports arbitrary bases.
1006 (Patch by Gawain Bolton; :issue:`6713`.)
1007
1008 * The :meth:`!split`, :meth:`!replace`, :meth:`!rindex`,
1009 :meth:`!rpartition`, and :meth:`!rsplit` methods of string-like types
1010 (strings, Unicode strings, and :class:`bytearray` objects) now use a
1011 fast reverse-search algorithm instead of a character-by-character
1012 scan. This is sometimes faster by a factor of 10. (Added by
1013 Florent Xicluna; :issue:`7462` and :issue:`7622`.)
1014
1015 * The :mod:`pickle` and :mod:`!cPickle` modules now automatically
1016 intern the strings used for attribute names, reducing memory usage
1017 of the objects resulting from unpickling. (Contributed by Jake
1018 McGuire; :issue:`5084`.)
1019
1020 * The :mod:`!cPickle` module now special-cases dictionaries,
1021 nearly halving the time required to pickle them.
1022 (Contributed by Collin Winter; :issue:`5670`.)
1023
1024 .. ======================================================================
1025
1026 New and Improved Modules
1027 ========================
1028
1029 As in every release, Python's standard library received a number of
1030 enhancements and bug fixes. Here's a partial list of the most notable
1031 changes, sorted alphabetically by module name. Consult the
1032 :file:`Misc/NEWS` file in the source tree for a more complete list of
1033 changes, or look through the Subversion logs for all the details.
1034
1035 * The :mod:`bdb` module's base debugging class :class:`~bdb.Bdb`
1036 gained a feature for skipping modules. The constructor
1037 now takes an iterable containing glob-style patterns such as
1038 ``django.*``; the debugger will not step into stack frames
1039 from a module that matches one of these patterns.
1040 (Contributed by Maru Newby after a suggestion by
1041 Senthil Kumaran; :issue:`5142`.)
1042
1043 * The :mod:`binascii` module now supports the buffer API, so it can be
1044 used with :class:`memoryview` instances and other similar buffer objects.
1045 (Backported from 3.x by Florent Xicluna; :issue:`7703`.)
1046
1047 * Updated module: the :mod:`!bsddb` module has been updated from 4.7.2devel9
1048 to version 4.8.4 of
1049 `the pybsddb package <https://www.jcea.es/programacion/pybsddb.htm>`__.
1050 The new version features better Python 3.x compatibility, various bug fixes,
1051 and adds several new BerkeleyDB flags and methods.
1052 (Updated by Jesús Cea Avión; :issue:`8156`. The pybsddb
1053 changelog can be read at https://hg.jcea.es/pybsddb/file/tip/ChangeLog.)
1054
1055 * The :mod:`bz2` module's :class:`~bz2.BZ2File` now supports the context
1056 management protocol, so you can write ``with bz2.BZ2File(...) as f:``.
1057 (Contributed by Hagen Fürstenau; :issue:`3860`.)
1058
1059 * New class: the :class:`~collections.Counter` class in the :mod:`collections`
1060 module is useful for tallying data. :class:`~collections.Counter` instances
1061 behave mostly like dictionaries but return zero for missing keys instead of
1062 raising a :exc:`KeyError`:
1063
1064 .. doctest::
1065 :options: +NORMALIZE_WHITESPACE
1066
1067 >>> from collections import Counter
1068 >>> c = Counter()
1069 >>> for letter in 'here is a sample of english text':
1070 ... c[letter] += 1
1071 ...
1072 >>> c # doctest: +SKIP
1073 Counter({' ': 6, 'e': 5, 's': 3, 'a': 2, 'i': 2, 'h': 2,
1074 'l': 2, 't': 2, 'g': 1, 'f': 1, 'm': 1, 'o': 1, 'n': 1,
1075 'p': 1, 'r': 1, 'x': 1})
1076 >>> c['e']
1077 5
1078 >>> c['z']
1079 0
1080
1081 There are three additional :class:`~collections.Counter` methods.
1082 :meth:`~collections.Counter.most_common` returns the N most common
1083 elements and their counts. :meth:`~collections.Counter.elements`
1084 returns an iterator over the contained elements, repeating each
1085 element as many times as its count.
1086 :meth:`~collections.Counter.subtract` takes an iterable and
1087 subtracts one for each element instead of adding; if the argument is
1088 a dictionary or another :class:`Counter`, the counts are
1089 subtracted. ::
1090
1091 >>> c.most_common(5)
1092 [(' ', 6), ('e', 5), ('s', 3), ('a', 2), ('i', 2)]
1093 >>> c.elements() ->
1094 'a', 'a', ' ', ' ', ' ', ' ', ' ', ' ',
1095 'e', 'e', 'e', 'e', 'e', 'g', 'f', 'i', 'i',
1096 'h', 'h', 'm', 'l', 'l', 'o', 'n', 'p', 's',
1097 's', 's', 'r', 't', 't', 'x'
1098 >>> c['e']
1099 5
1100 >>> c.subtract('very heavy on the letter e')
1101 >>> c['e'] # Count is now lower
1102 -1
1103
1104 Contributed by Raymond Hettinger; :issue:`1696199`.
1105
1106 .. revision 79660
1107
1108 New class: :class:`~collections.OrderedDict` is described in the earlier
1109 section :ref:`pep-0372`.
1110
1111 New method: The :class:`~collections.deque` data type now has a
1112 :meth:`~collections.deque.count` method that returns the number of
1113 contained elements equal to the supplied argument *x*, and a
1114 :meth:`~collections.deque.reverse` method that reverses the elements
1115 of the deque in-place. :class:`~collections.deque` also exposes its maximum
1116 length as the read-only :attr:`~collections.deque.maxlen` attribute.
1117 (Both features added by Raymond Hettinger.)
1118
1119 The :class:`~collections.namedtuple` class now has an optional *rename* parameter.
1120 If *rename* is true, field names that are invalid because they've
1121 been repeated or aren't legal Python identifiers will be
1122 renamed to legal names that are derived from the field's
1123 position within the list of fields:
1124
1125 >>> from collections import namedtuple
1126 >>> T = namedtuple('T', ['field1', '$illegal', 'for', 'field2'], rename=True)
1127 >>> T._fields
1128 ('field1', '_1', '_2', 'field2')
1129
1130 (Added by Raymond Hettinger; :issue:`1818`.)
1131
1132 Finally, the :class:`~collections.abc.Mapping` abstract base class now
1133 returns :data:`NotImplemented` if a mapping is compared to
1134 another type that isn't a :class:`Mapping`.
1135 (Fixed by Daniel Stutzbach; :issue:`8729`.)
1136
1137 * Constructors for the parsing classes in the :mod:`ConfigParser <configparser>` module now
1138 take an *allow_no_value* parameter, defaulting to false; if true,
1139 options without values will be allowed. For example::
1140
1141 >>> import ConfigParser, StringIO
1142 >>> sample_config = """
1143 ... [mysqld]
1144 ... user = mysql
1145 ... pid-file = /var/run/mysqld/mysqld.pid
1146 ... skip-bdb
1147 ... """
1148 >>> config = ConfigParser.RawConfigParser(allow_no_value=True)
1149 >>> config.readfp(StringIO.StringIO(sample_config))
1150 >>> config.get('mysqld', 'user')
1151 'mysql'
1152 >>> print config.get('mysqld', 'skip-bdb')
1153 None
1154 >>> print config.get('mysqld', 'unknown')
1155 Traceback (most recent call last):
1156 ...
1157 NoOptionError: No option 'unknown' in section: 'mysqld'
1158
1159 (Contributed by Mats Kindahl; :issue:`7005`.)
1160
1161 * Deprecated function: :func:`!contextlib.nested`, which allows
1162 handling more than one context manager with a single :keyword:`with`
1163 statement, has been deprecated, because the :keyword:`!with` statement
1164 now supports multiple context managers.
1165
1166 * The :mod:`cookielib <http.cookiejar>` module now ignores cookies that have an invalid
1167 version field, one that doesn't contain an integer value. (Fixed by
1168 John J. Lee; :issue:`3924`.)
1169
1170 * The :mod:`copy` module's :func:`~copy.deepcopy` function will now
1171 correctly copy bound instance methods. (Implemented by
1172 Robert Collins; :issue:`1515`.)
1173
1174 * The :mod:`ctypes` module now always converts ``None`` to a C ``NULL``
1175 pointer for arguments declared as pointers. (Changed by Thomas
1176 Heller; :issue:`4606`.) The underlying `libffi library
1177 <https://sourceware.org/libffi/>`__ has been updated to version
1178 3.0.9, containing various fixes for different platforms. (Updated
1179 by Matthias Klose; :issue:`8142`.)
1180
1181 * New method: the :mod:`datetime` module's :class:`~datetime.timedelta` class
1182 gained a :meth:`~datetime.timedelta.total_seconds` method that returns the
1183 number of seconds in the duration. (Contributed by Brian Quinlan; :issue:`5788`.)
1184
1185 * New method: the :class:`~decimal.Decimal` class gained a
1186 :meth:`~decimal.Decimal.from_float` class method that performs an exact
1187 conversion of a floating-point number to a :class:`!Decimal`.
1188 This exact conversion strives for the
1189 closest decimal approximation to the floating-point representation's value;
1190 the resulting decimal value will therefore still include the inaccuracy,
1191 if any.
1192 For example, ``Decimal.from_float(0.1)`` returns
1193 ``Decimal('0.1000000000000000055511151231257827021181583404541015625')``.
1194 (Implemented by Raymond Hettinger; :issue:`4796`.)
1195
1196 Comparing instances of :class:`~decimal.Decimal` with floating-point
1197 numbers now produces sensible results based on the numeric values
1198 of the operands. Previously such comparisons would fall back to
1199 Python's default rules for comparing objects, which produced arbitrary
1200 results based on their type. Note that you still cannot combine
1201 :class:`!Decimal` and floating point in other operations such as addition,
1202 since you should be explicitly choosing how to convert between float and
1203 :class:`!Decimal`. (Fixed by Mark Dickinson; :issue:`2531`.)
1204
1205 The constructor for :class:`~decimal.Decimal` now accepts
1206 floating-point numbers (added by Raymond Hettinger; :issue:`8257`)
1207 and non-European Unicode characters such as Arabic-Indic digits
1208 (contributed by Mark Dickinson; :issue:`6595`).
1209
1210 Most of the methods of the :class:`~decimal.Context` class now accept integers
1211 as well as :class:`~decimal.Decimal` instances; the only exceptions are the
1212 :meth:`~decimal.Context.canonical` and :meth:`~decimal.Context.is_canonical`
1213 methods. (Patch by Juan José Conti; :issue:`7633`.)
1214
1215 When using :class:`~decimal.Decimal` instances with a string's
1216 :meth:`~str.format` method, the default alignment was previously
1217 left-alignment. This has been changed to right-alignment, which is
1218 more sensible for numeric types. (Changed by Mark Dickinson; :issue:`6857`.)
1219
1220 Comparisons involving a signaling NaN value (or ``sNAN``) now signal
1221 :const:`~decimal.InvalidOperation` instead of silently returning a true or
1222 false value depending on the comparison operator. Quiet NaN values
1223 (or ``NaN``) are now hashable. (Fixed by Mark Dickinson;
1224 :issue:`7279`.)
1225
1226 * The :mod:`difflib` module now produces output that is more
1227 compatible with modern :command:`diff`/:command:`patch` tools
1228 through one small change, using a tab character instead of spaces as
1229 a separator in the header giving the filename. (Fixed by Anatoly
1230 Techtonik; :issue:`7585`.)
1231
1232 * The Distutils ``sdist`` command now always regenerates the
1233 :file:`MANIFEST` file, since even if the :file:`MANIFEST.in` or
1234 :file:`setup.py` files haven't been modified, the user might have
1235 created some new files that should be included.
1236 (Fixed by Tarek Ziadé; :issue:`8688`.)
1237
1238 * The :mod:`doctest` module's :const:`~doctest.IGNORE_EXCEPTION_DETAIL` flag
1239 will now ignore the name of the module containing the exception
1240 being tested. (Patch by Lennart Regebro; :issue:`7490`.)
1241
1242 * The :mod:`email` module's :class:`~email.message.Message` class will
1243 now accept a Unicode-valued payload, automatically converting the
1244 payload to the encoding specified by :attr:`!output_charset`.
1245 (Added by R. David Murray; :issue:`1368247`.)
1246
1247 * The :class:`~fractions.Fraction` class now accepts a single float or
1248 :class:`~decimal.Decimal` instance, or two rational numbers, as
1249 arguments to its constructor. (Implemented by Mark Dickinson;
1250 rationals added in :issue:`5812`, and float/decimal in
1251 :issue:`8294`.)
1252
1253 Ordering comparisons (``<``, ``<=``, ``>``, ``>=``) between
1254 fractions and complex numbers now raise a :exc:`TypeError`.
1255 This fixes an oversight, making the :class:`~fractions.Fraction`
1256 match the other numeric types.
1257
1258 .. revision 79455
1259
1260 * New class: :class:`~ftplib.FTP_TLS` in
1261 the :mod:`ftplib` module provides secure FTP
1262 connections using TLS encapsulation of authentication as well as
1263 subsequent control and data transfers.
1264 (Contributed by Giampaolo Rodola; :issue:`2054`.)
1265
1266 The :meth:`~ftplib.FTP.storbinary` method for binary uploads can now restart
1267 uploads thanks to an added *rest* parameter (patch by Pablo Mouzo;
1268 :issue:`6845`.)
1269
1270 * New class decorator: :func:`~functools.total_ordering` in the :mod:`functools`
1271 module takes a class that defines an :meth:`~object.__eq__` method and one of
1272 :meth:`~object.__lt__`, :meth:`~object.__le__`, :meth:`~object.__gt__`, or :meth:`~object.__ge__`,
1273 and generates the missing comparison methods. Since the
1274 :meth:`!__cmp__` method is being deprecated in Python 3.x,
1275 this decorator makes it easier to define ordered classes.
1276 (Added by Raymond Hettinger; :issue:`5479`.)
1277
1278 New function: :func:`~functools.cmp_to_key` will take an old-style comparison
1279 function that expects two arguments and return a new callable that
1280 can be used as the *key* parameter to functions such as
1281 :func:`sorted`, :func:`min` and :func:`max`, etc. The primary
1282 intended use is to help with making code compatible with Python 3.x.
1283 (Added by Raymond Hettinger.)
1284
1285 * New function: the :mod:`gc` module's :func:`~gc.is_tracked` returns
1286 true if a given instance is tracked by the garbage collector, false
1287 otherwise. (Contributed by Antoine Pitrou; :issue:`4688`.)
1288
1289 * The :mod:`gzip` module's :class:`~gzip.GzipFile` now supports the context
1290 management protocol, so you can write ``with gzip.GzipFile(...) as f:``
1291 (contributed by Hagen Fürstenau; :issue:`3860`), and it now implements
1292 the :class:`io.BufferedIOBase` ABC, so you can wrap it with
1293 :class:`io.BufferedReader` for faster processing
1294 (contributed by Nir Aides; :issue:`7471`).
1295 It's also now possible to override the modification time
1296 recorded in a gzipped file by providing an optional timestamp to
1297 the constructor. (Contributed by Jacques Frechet; :issue:`4272`.)
1298
1299 Files in gzip format can be padded with trailing zero bytes; the
1300 :mod:`gzip` module will now consume these trailing bytes. (Fixed by
1301 Tadek Pietraszek and Brian Curtin; :issue:`2846`.)
1302
1303 * New attribute: the :mod:`hashlib` module now has an :attr:`!algorithms`
1304 attribute containing a tuple naming the supported algorithms.
1305 In Python 2.7, ``hashlib.algorithms`` contains
1306 ``('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')``.
1307 (Contributed by Carl Chenet; :issue:`7418`.)
1308
1309 * The default :class:`~http.client.HTTPResponse` class used by the :mod:`httplib <http>` module now
1310 supports buffering, resulting in much faster reading of HTTP responses.
1311 (Contributed by Kristján Valur Jónsson; :issue:`4879`.)
1312
1313 The :class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection` classes
1314 now support a *source_address* parameter, a ``(host, port)`` 2-tuple
1315 giving the source address that will be used for the connection.
1316 (Contributed by Eldon Ziegler; :issue:`3972`.)
1317
1318 * The :mod:`!ihooks` module now supports relative imports. Note that
1319 :mod:`!ihooks` is an older module for customizing imports,
1320 superseded by the :mod:`!imputil` module added in Python 2.0.
1321 (Relative import support added by Neil Schemenauer.)
1322
1323 .. revision 75423
1324
1325 * The :mod:`imaplib` module now supports IPv6 addresses.
1326 (Contributed by Derek Morr; :issue:`1655`.)
1327
1328 * New function: the :mod:`inspect` module's :func:`~inspect.getcallargs`
1329 takes a callable and its positional and keyword arguments,
1330 and figures out which of the callable's parameters will receive each argument,
1331 returning a dictionary mapping argument names to their values. For example::
1332
1333 >>> from inspect import getcallargs
1334 >>> def f(a, b=1, *pos, **named):
1335 ... pass
1336 ...
1337 >>> getcallargs(f, 1, 2, 3)
1338 {'a': 1, 'b': 2, 'pos': (3,), 'named': {}}
1339 >>> getcallargs(f, a=2, x=4)
1340 {'a': 2, 'b': 1, 'pos': (), 'named': {'x': 4}}
1341 >>> getcallargs(f)
1342 Traceback (most recent call last):
1343 ...
1344 TypeError: f() takes at least 1 argument (0 given)
1345
1346 Contributed by George Sakkis; :issue:`3135`.
1347
1348 * Updated module: The :mod:`io` library has been upgraded to the version shipped with
1349 Python 3.1. For 3.1, the I/O library was entirely rewritten in C
1350 and is 2 to 20 times faster depending on the task being performed. The
1351 original Python version was renamed to the :mod:`!_pyio` module.
1352
1353 One minor resulting change: the :class:`io.TextIOBase` class now
1354 has an :attr:`~io.TextIOBase.errors` attribute giving the error setting
1355 used for encoding and decoding errors (one of ``'strict'``, ``'replace'``,
1356 ``'ignore'``).
1357
1358 The :class:`io.FileIO` class now raises an :exc:`OSError` when passed
1359 an invalid file descriptor. (Implemented by Benjamin Peterson;
1360 :issue:`4991`.) The :meth:`~io.IOBase.truncate` method now preserves the
1361 file position; previously it would change the file position to the
1362 end of the new file. (Fixed by Pascal Chambon; :issue:`6939`.)
1363
1364 * New function: ``itertools.compress(data, selectors)`` takes two
1365 iterators. Elements of *data* are returned if the corresponding
1366 value in *selectors* is true::
1367
1368 itertools.compress('ABCDEF', [1,0,1,0,1,1]) =>
1369 A, C, E, F
1370
1371 .. maybe here is better to use >>> list(itertools.compress(...)) instead
1372
1373 New function: ``itertools.combinations_with_replacement(iter, r)``
1374 returns all the possible *r*-length combinations of elements from the
1375 iterable *iter*. Unlike :func:`~itertools.combinations`, individual elements
1376 can be repeated in the generated combinations::
1377
1378 itertools.combinations_with_replacement('abc', 2) =>
1379 ('a', 'a'), ('a', 'b'), ('a', 'c'),
1380 ('b', 'b'), ('b', 'c'), ('c', 'c')
1381
1382 Note that elements are treated as unique depending on their position
1383 in the input, not their actual values.
1384
1385 The :func:`itertools.count` function now has a *step* argument that
1386 allows incrementing by values other than 1. :func:`~itertools.count` also
1387 now allows keyword arguments, and using non-integer values such as
1388 floats or :class:`~decimal.Decimal` instances. (Implemented by Raymond
1389 Hettinger; :issue:`5032`.)
1390
1391 :func:`itertools.combinations` and :func:`itertools.product`
1392 previously raised :exc:`ValueError` for values of *r* larger than
1393 the input iterable. This was deemed a specification error, so they
1394 now return an empty iterator. (Fixed by Raymond Hettinger; :issue:`4816`.)
1395
1396 * Updated module: The :mod:`json` module was upgraded to version 2.0.9 of the
1397 simplejson package, which includes a C extension that makes
1398 encoding and decoding faster.
1399 (Contributed by Bob Ippolito; :issue:`4136`.)
1400
1401 To support the new :class:`collections.OrderedDict` type, :func:`json.load`
1402 now has an optional *object_pairs_hook* parameter that will be called
1403 with any object literal that decodes to a list of pairs.
1404 (Contributed by Raymond Hettinger; :issue:`5381`.)
1405
1406 * The :mod:`mailbox` module's :class:`~mailbox.Maildir` class now records the
1407 timestamp on the directories it reads, and only re-reads them if the
1408 modification time has subsequently changed. This improves
1409 performance by avoiding unneeded directory scans. (Fixed by
1410 A.M. Kuchling and Antoine Pitrou; :issue:`1607951`, :issue:`6896`.)
1411
1412 * New functions: the :mod:`math` module gained
1413 :func:`~math.erf` and :func:`~math.erfc` for the error function and the complementary error function,
1414 :func:`~math.expm1` which computes ``e**x - 1`` with more precision than
1415 using :func:`~math.exp` and subtracting 1,
1416 :func:`~math.gamma` for the Gamma function, and
1417 :func:`~math.lgamma` for the natural log of the Gamma function.
1418 (Contributed by Mark Dickinson and nirinA raseliarison; :issue:`3366`.)
1419
1420 * The :mod:`multiprocessing` module's :class:`!Manager*` classes
1421 can now be passed a callable that will be called whenever
1422 a subprocess is started, along with a set of arguments that will be
1423 passed to the callable.
1424 (Contributed by lekma; :issue:`5585`.)
1425
1426 The :class:`~multiprocessing.pool.Pool` class, which controls a pool of worker processes,
1427 now has an optional *maxtasksperchild* parameter. Worker processes
1428 will perform the specified number of tasks and then exit, causing the
1429 :class:`!Pool` to start a new worker. This is useful if tasks may leak
1430 memory or other resources, or if some tasks will cause the worker to
1431 become very large.
1432 (Contributed by Charles Cazabon; :issue:`6963`.)
1433
1434 * The :mod:`!nntplib` module now supports IPv6 addresses.
1435 (Contributed by Derek Morr; :issue:`1664`.)
1436
1437 * New functions: the :mod:`os` module wraps the following POSIX system
1438 calls: :func:`~os.getresgid` and :func:`~os.getresuid`, which return the
1439 real, effective, and saved GIDs and UIDs;
1440 :func:`~os.setresgid` and :func:`~os.setresuid`, which set
1441 real, effective, and saved GIDs and UIDs to new values;
1442 :func:`~os.initgroups`, which initialize the group access list
1443 for the current process. (GID/UID functions
1444 contributed by Travis H.; :issue:`6508`. Support for initgroups added
1445 by Jean-Paul Calderone; :issue:`7333`.)
1446
1447 The :func:`os.fork` function now re-initializes the import lock in
1448 the child process; this fixes problems on Solaris when :func:`~os.fork`
1449 is called from a thread. (Fixed by Zsolt Cserna; :issue:`7242`.)
1450
1451 * In the :mod:`os.path` module, the :func:`~os.path.normpath` and
1452 :func:`~os.path.abspath` functions now preserve Unicode; if their input path
1453 is a Unicode string, the return value is also a Unicode string.
1454 (:meth:`~os.path.normpath` fixed by Matt Giuca in :issue:`5827`;
1455 :meth:`~os.path.abspath` fixed by Ezio Melotti in :issue:`3426`.)
1456
1457 * The :mod:`pydoc` module now has help for the various symbols that Python
1458 uses. You can now do ``help('<<')`` or ``help('@')``, for example.
1459 (Contributed by David Laban; :issue:`4739`.)
1460
1461 * The :mod:`re` module's :func:`~re.split`, :func:`~re.sub`, and :func:`~re.subn`
1462 now accept an optional *flags* argument, for consistency with the
1463 other functions in the module. (Added by Gregory P. Smith.)
1464
1465 * New function: :func:`~runpy.run_path` in the :mod:`runpy` module
1466 will execute the code at a provided *path* argument. *path* can be
1467 the path of a Python source file (:file:`example.py`), a compiled
1468 bytecode file (:file:`example.pyc`), a directory
1469 (:file:`./package/`), or a zip archive (:file:`example.zip`). If a
1470 directory or zip path is provided, it will be added to the front of
1471 ``sys.path`` and the module :mod:`__main__` will be imported. It's
1472 expected that the directory or zip contains a :file:`__main__.py`;
1473 if it doesn't, some other :file:`__main__.py` might be imported from
1474 a location later in ``sys.path``. This makes more of the machinery
1475 of :mod:`runpy` available to scripts that want to mimic the way
1476 Python's command line processes an explicit path name.
1477 (Added by Nick Coghlan; :issue:`6816`.)
1478
1479 * New function: in the :mod:`shutil` module, :func:`~shutil.make_archive`
1480 takes a filename, archive type (zip or tar-format), and a directory
1481 path, and creates an archive containing the directory's contents.
1482 (Added by Tarek Ziadé.)
1483
1484 :mod:`shutil`'s :func:`~shutil.copyfile` and :func:`~shutil.copytree`
1485 functions now raise a :exc:`~shutil.SpecialFileError` exception when
1486 asked to copy a named pipe. Previously the code would treat
1487 named pipes like a regular file by opening them for reading, and
1488 this would block indefinitely. (Fixed by Antoine Pitrou; :issue:`3002`.)
1489
1490 * The :mod:`signal` module no longer re-installs the signal handler
1491 unless this is truly necessary, which fixes a bug that could make it
1492 impossible to catch the EINTR signal robustly. (Fixed by
1493 Charles-Francois Natali; :issue:`8354`.)
1494
1495 * New functions: in the :mod:`site` module, three new functions
1496 return various site- and user-specific paths.
1497 :func:`~site.getsitepackages` returns a list containing all
1498 global site-packages directories,
1499 :func:`~site.getusersitepackages` returns the path of the user's
1500 site-packages directory, and
1501 :func:`~site.getuserbase` returns the value of the :data:`~site.USER_BASE`
1502 environment variable, giving the path to a directory that can be used
1503 to store data.
1504 (Contributed by Tarek Ziadé; :issue:`6693`.)
1505
1506 The :mod:`site` module now reports exceptions occurring
1507 when the :mod:`sitecustomize` module is imported, and will no longer
1508 catch and swallow the :exc:`KeyboardInterrupt` exception. (Fixed by
1509 Victor Stinner; :issue:`3137`.)
1510
1511 * The :func:`~socket.create_connection` function
1512 gained a *source_address* parameter, a ``(host, port)`` 2-tuple
1513 giving the source address that will be used for the connection.
1514 (Contributed by Eldon Ziegler; :issue:`3972`.)
1515
1516 The :meth:`~socket.socket.recv_into` and :meth:`~socket.socket.recvfrom_into`
1517 methods will now write into objects that support the buffer API, most usefully
1518 the :class:`bytearray` and :class:`memoryview` objects. (Implemented by
1519 Antoine Pitrou; :issue:`8104`.)
1520
1521 * The :mod:`SocketServer <socketserver>` module's :class:`~socketserver.TCPServer` class now
1522 supports socket timeouts and disabling the Nagle algorithm.
1523 The :attr:`!disable_nagle_algorithm` class attribute
1524 defaults to ``False``; if overridden to be true,
1525 new request connections will have the TCP_NODELAY option set to
1526 prevent buffering many small sends into a single TCP packet.
1527 The :attr:`~socketserver.BaseServer.timeout` class attribute can hold
1528 a timeout in seconds that will be applied to the request socket; if
1529 no request is received within that time, :meth:`~socketserver.BaseServer.handle_timeout`
1530 will be called and :meth:`~socketserver.BaseServer.handle_request` will return.
1531 (Contributed by Kristján Valur Jónsson; :issue:`6192` and :issue:`6267`.)
1532
1533 * Updated module: the :mod:`sqlite3` module has been updated to
1534 version 2.6.0 of the `pysqlite package <https://github.com/ghaering/pysqlite>`__. Version 2.6.0 includes a number of bugfixes, and adds
1535 the ability to load SQLite extensions from shared libraries.
1536 Call the ``enable_load_extension(True)`` method to enable extensions,
1537 and then call :meth:`~sqlite3.Connection.load_extension` to load a particular shared library.
1538 (Updated by Gerhard Häring.)
1539
1540 * The :mod:`ssl` module's :class:`~ssl.SSLSocket` objects now support the
1541 buffer API, which fixed a test suite failure (fix by Antoine Pitrou;
1542 :issue:`7133`) and automatically set
1543 OpenSSL's :c:macro:`!SSL_MODE_AUTO_RETRY`, which will prevent an error
1544 code being returned from :meth:`!recv` operations that trigger an SSL
1545 renegotiation (fix by Antoine Pitrou; :issue:`8222`).
1546
1547 The :func:`~ssl.SSLContext.wrap_socket` constructor function now takes a
1548 *ciphers* argument that's a string listing the encryption algorithms
1549 to be allowed; the format of the string is described
1550 `in the OpenSSL documentation
1551 <https://docs.openssl.org/1.0.2/man1/ciphers/>`__.
1552 (Added by Antoine Pitrou; :issue:`8322`.)
1553
1554 Another change makes the extension load all of OpenSSL's ciphers and
1555 digest algorithms so that they're all available. Some SSL
1556 certificates couldn't be verified, reporting an "unknown algorithm"
1557 error. (Reported by Beda Kosata, and fixed by Antoine Pitrou;
1558 :issue:`8484`.)
1559
1560 The version of OpenSSL being used is now available as the module
1561 attributes :const:`ssl.OPENSSL_VERSION` (a string),
1562 :const:`ssl.OPENSSL_VERSION_INFO` (a 5-tuple), and
1563 :const:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by Antoine
1564 Pitrou; :issue:`8321`.)
1565
1566 * The :mod:`struct` module will no longer silently ignore overflow
1567 errors when a value is too large for a particular integer format
1568 code (one of ``bBhHiIlLqQ``); it now always raises a
1569 :exc:`struct.error` exception. (Changed by Mark Dickinson;
1570 :issue:`1523`.) The :func:`~struct.pack` function will also
1571 attempt to use :meth:`~object.__index__` to convert and pack non-integers
1572 before trying the :meth:`~object.__int__` method or reporting an error.
1573 (Changed by Mark Dickinson; :issue:`8300`.)
1574
1575 * New function: the :mod:`subprocess` module's
1576 :func:`~subprocess.check_output` runs a command with a specified set of arguments
1577 and returns the command's output as a string when the command runs without
1578 error, or raises a :exc:`~subprocess.CalledProcessError` exception otherwise.
1579
1580 ::
1581
1582 >>> subprocess.check_output(['df', '-h', '.'])
1583 'Filesystem Size Used Avail Capacity Mounted on\n
1584 /dev/disk0s2 52G 49G 3.0G 94% /\n'
1585
1586 >>> subprocess.check_output(['df', '-h', '/bogus'])
1587 ...
1588 subprocess.CalledProcessError: Command '['df', '-h', '/bogus']' returned non-zero exit status 1
1589
1590 (Contributed by Gregory P. Smith.)
1591
1592 The :mod:`subprocess` module will now retry its internal system calls
1593 on receiving an :const:`~errno.EINTR` signal. (Reported by several people; final
1594 patch by Gregory P. Smith in :issue:`1068268`.)
1595
1596 * New function: :func:`~symtable.Symbol.is_declared_global` in the :mod:`symtable` module
1597 returns true for variables that are explicitly declared to be global,
1598 false for ones that are implicitly global.
1599 (Contributed by Jeremy Hylton.)
1600
1601 * The :mod:`syslog` module will now use the value of ``sys.argv[0]`` as the
1602 identifier instead of the previous default value of ``'python'``.
1603 (Changed by Sean Reifschneider; :issue:`8451`.)
1604
1605 * The :data:`sys.version_info` value is now a named tuple, with attributes
1606 named :attr:`!major`, :attr:`!minor`, :attr:`!micro`,
1607 :attr:`!releaselevel`, and :attr:`!serial`. (Contributed by Ross
1608 Light; :issue:`4285`.)
1609
1610 :func:`sys.getwindowsversion` also returns a named tuple,
1611 with attributes named :attr:`!major`, :attr:`!minor`, :attr:`!build`,
1612 :attr:`!platform`, :attr:`!service_pack`, :attr:`!service_pack_major`,
1613 :attr:`!service_pack_minor`, :attr:`!suite_mask`, and
1614 :attr:`!product_type`. (Contributed by Brian Curtin; :issue:`7766`.)
1615
1616 * The :mod:`tarfile` module's default error handling has changed, to
1617 no longer suppress fatal errors. The default error level was previously 0,
1618 which meant that errors would only result in a message being written to the
1619 debug log, but because the debug log is not activated by default,
1620 these errors go unnoticed. The default error level is now 1,
1621 which raises an exception if there's an error.
1622 (Changed by Lars Gustäbel; :issue:`7357`.)
1623
1624 :mod:`tarfile` now supports filtering the :class:`~tarfile.TarInfo`
1625 objects being added to a tar file. When you call :meth:`~tarfile.TarFile.add`,
1626 you may supply an optional *filter* argument
1627 that's a callable. The *filter* callable will be passed the
1628 :class:`~tarfile.TarInfo` for every file being added, and can modify and return it.
1629 If the callable returns ``None``, the file will be excluded from the
1630 resulting archive. This is more powerful than the existing
1631 *exclude* argument, which has therefore been deprecated.
1632 (Added by Lars Gustäbel; :issue:`6856`.)
1633 The :class:`~tarfile.TarFile` class also now supports the context management protocol.
1634 (Added by Lars Gustäbel; :issue:`7232`.)
1635
1636 * The :meth:`~threading.Event.wait` method of the :class:`threading.Event` class
1637 now returns the internal flag on exit. This means the method will usually
1638 return true because :meth:`~threading.Event.wait` is supposed to block until the
1639 internal flag becomes true. The return value will only be false if
1640 a timeout was provided and the operation timed out.
1641 (Contributed by Tim Lesher; :issue:`1674032`.)
1642
1643 * The Unicode database provided by the :mod:`unicodedata` module is
1644 now used internally to determine which characters are numeric,
1645 whitespace, or represent line breaks. The database also
1646 includes information from the :file:`Unihan.txt` data file (patch
1647 by Anders Chrigström and Amaury Forgeot d'Arc; :issue:`1571184`)
1648 and has been updated to version 5.2.0 (updated by
1649 Florent Xicluna; :issue:`8024`).
1650
1651 * The :mod:`urlparse <urllib.parse>` module's :func:`~urllib.parse.urlsplit` now handles
1652 unknown URL schemes in a fashion compliant with :rfc:`3986`: if the
1653 URL is of the form ``"<something>://..."``, the text before the
1654 ``://`` is treated as the scheme, even if it's a made-up scheme that
1655 the module doesn't know about. This change may break code that
1656 worked around the old behaviour. For example, Python 2.6.4 or 2.5
1657 will return the following:
1658
1659 .. doctest::
1660 :options: +SKIP
1661
1662 >>> import urlparse
1663 >>> urlparse.urlsplit('invented://host/filename?query')
1664 ('invented', '', '//host/filename?query', '', '')
1665
1666 Python 2.7 (and Python 2.6.5) will return:
1667
1668 .. doctest::
1669 :options: +SKIP
1670
1671 >>> import urlparse
1672 >>> urlparse.urlsplit('invented://host/filename?query')
1673 ('invented', 'host', '/filename?query', '', '')
1674
1675 (Python 2.7 actually produces slightly different output, since it
1676 returns a named tuple instead of a standard tuple.)
1677
1678 The :mod:`urlparse <urllib.parse>` module also supports IPv6 literal addresses as defined by
1679 :rfc:`2732` (contributed by Senthil Kumaran; :issue:`2987`).
1680
1681 .. doctest::
1682 :options: +SKIP
1683
1684 >>> urlparse.urlparse('http://[1080::8:800:200C:417A]/foo')
1685 ParseResult(scheme='http', netloc='[1080::8:800:200C:417A]',
1686 path='/foo', params='', query='', fragment='')
1687
1688 * New class: the :class:`~weakref.WeakSet` class in the :mod:`weakref`
1689 module is a set that only holds weak references to its elements; elements
1690 will be removed once there are no references pointing to them.
1691 (Originally implemented in Python 3.x by Raymond Hettinger, and backported
1692 to 2.7 by Michael Foord.)
1693
1694 * The :mod:`xml.etree.ElementTree` library, no longer escapes
1695 ampersands and angle brackets when outputting an XML processing
1696 instruction (which looks like ``<?xml-stylesheet href="#style1"?>``)
1697 or comment (which looks like ``<!-- comment -->``).
1698 (Patch by Neil Muller; :issue:`2746`.)
1699
1700 * The XML-RPC client and server, provided by the :mod:`xmlrpclib <xmlrpc.client>` and
1701 :mod:`SimpleXMLRPCServer <xmlrpc.server>` modules, have improved performance by
1702 supporting HTTP/1.1 keep-alive and by optionally using gzip encoding
1703 to compress the XML being exchanged. The gzip compression is
1704 controlled by the :attr:`!encode_threshold` attribute of
1705 :class:`~xmlrpc.server.SimpleXMLRPCRequestHandler`, which contains a size in bytes;
1706 responses larger than this will be compressed.
1707 (Contributed by Kristján Valur Jónsson; :issue:`6267`.)
1708
1709 * The :mod:`zipfile` module's :class:`~zipfile.ZipFile` now supports the context
1710 management protocol, so you can write ``with zipfile.ZipFile(...) as f:``.
1711 (Contributed by Brian Curtin; :issue:`5511`.)
1712
1713 :mod:`zipfile` now also supports archiving empty directories and
1714 extracts them correctly. (Fixed by Kuba Wieczorek; :issue:`4710`.)
1715 Reading files out of an archive is faster, and interleaving
1716 :meth:`read() <io.BufferedIOBase.read>` and
1717 :meth:`readline() <io.IOBase.readline>` now works correctly.
1718 (Contributed by Nir Aides; :issue:`7610`.)
1719
1720 The :func:`~zipfile.is_zipfile` function now
1721 accepts a file object, in addition to the path names accepted in earlier
1722 versions. (Contributed by Gabriel Genellina; :issue:`4756`.)
1723
1724 The :meth:`~zipfile.ZipFile.writestr` method now has an optional *compress_type* parameter
1725 that lets you override the default compression method specified in the
1726 :class:`~zipfile.ZipFile` constructor. (Contributed by Ronald Oussoren;
1727 :issue:`6003`.)
1728
1729
1730 .. ======================================================================
1731 .. whole new modules get described in subsections here
1732
1733
1734 .. _importlib-section:
1735
1736 New module: importlib
1737 ------------------------------
1738
1739 Python 3.1 includes the :mod:`importlib` package, a re-implementation
1740 of the logic underlying Python's :keyword:`import` statement.
1741 :mod:`importlib` is useful for implementers of Python interpreters and
1742 to users who wish to write new importers that can participate in the
1743 import process. Python 2.7 doesn't contain the complete
1744 :mod:`importlib` package, but instead has a tiny subset that contains
1745 a single function, :func:`~importlib.import_module`.
1746
1747 ``import_module(name, package=None)`` imports a module. *name* is
1748 a string containing the module or package's name. It's possible to do
1749 relative imports by providing a string that begins with a ``.``
1750 character, such as ``..utils.errors``. For relative imports, the
1751 *package* argument must be provided and is the name of the package that
1752 will be used as the anchor for
1753 the relative import. :func:`~importlib.import_module` both inserts the imported
1754 module into ``sys.modules`` and returns the module object.
1755
1756 Here are some examples::
1757
1758 >>> from importlib import import_module
1759 >>> anydbm = import_module('anydbm') # Standard absolute import
1760 >>> anydbm
1761 <module 'anydbm' from '/p/python/Lib/anydbm.py'>
1762 >>> # Relative import
1763 >>> file_util = import_module('..file_util', 'distutils.command')
1764 >>> file_util
1765 <module 'distutils.file_util' from '/python/Lib/distutils/file_util.pyc'>
1766
1767 :mod:`importlib` was implemented by Brett Cannon and introduced in
1768 Python 3.1.
1769
1770
1771 New module: sysconfig
1772 ---------------------------------
1773
1774 The :mod:`sysconfig` module has been pulled out of the Distutils
1775 package, becoming a new top-level module in its own right.
1776 :mod:`sysconfig` provides functions for getting information about
1777 Python's build process: compiler switches, installation paths, the
1778 platform name, and whether Python is running from its source
1779 directory.
1780
1781 Some of the functions in the module are:
1782
1783 * :func:`~sysconfig.get_config_var` returns variables from Python's
1784 Makefile and the :file:`pyconfig.h` file.
1785 * :func:`~sysconfig.get_config_vars` returns a dictionary containing
1786 all of the configuration variables.
1787 * :func:`~sysconfig.get_path` returns the configured path for
1788 a particular type of module: the standard library,
1789 site-specific modules, platform-specific modules, etc.
1790 * :func:`~sysconfig.is_python_build` returns true if you're running a
1791 binary from a Python source tree, and false otherwise.
1792
1793 Consult the :mod:`sysconfig` documentation for more details and for
1794 a complete list of functions.
1795
1796 The Distutils package and :mod:`sysconfig` are now maintained by Tarek
1797 Ziadé, who has also started a Distutils2 package (source repository at
1798 https://hg.python.org/distutils2/) for developing a next-generation
1799 version of Distutils.
1800
1801
1802 ttk: Themed Widgets for Tk
1803 --------------------------
1804
1805 Tcl/Tk 8.5 includes a set of themed widgets that re-implement basic Tk
1806 widgets but have a more customizable appearance and can therefore more
1807 closely resemble the native platform's widgets. This widget
1808 set was originally called Tile, but was renamed to Ttk (for "themed Tk")
1809 on being added to Tcl/Tck release 8.5.
1810
1811 To learn more, read the :mod:`~tkinter.ttk` module documentation. You may also
1812 wish to read the Tcl/Tk manual page describing the
1813 Ttk theme engine, available at
1814 https://www.tcl.tk/man/tcl8.5/TkCmd/ttk_intro.html. Some
1815 screenshots of the Python/Ttk code in use are at
1816 https://code.google.com/archive/p/python-ttk/wikis/Screenshots.wiki.
1817
1818 The :mod:`tkinter.ttk` module was written by Guilherme Polo and added in
1819 :issue:`2983`. An alternate version called ``Tile.py``, written by
1820 Martin Franklin and maintained by Kevin Walzer, was proposed for
1821 inclusion in :issue:`2618`, but the authors argued that Guilherme
1822 Polo's work was more comprehensive.
1823
1824
1825 .. _unittest-section:
1826
1827 Updated module: unittest
1828 ---------------------------------
1829
1830 The :mod:`unittest` module was greatly enhanced; many
1831 new features were added. Most of these features were implemented
1832 by Michael Foord, unless otherwise noted. The enhanced version of
1833 the module is downloadable separately for use with Python versions 2.4 to 2.6,
1834 packaged as the :mod:`!unittest2` package, from :pypi:`unittest2`.
1835
1836 When used from the command line, the module can automatically discover
1837 tests. It's not as fancy as `py.test <https://pytest.org>`__ or
1838 `nose <https://nose.readthedocs.io/>`__, but provides a
1839 simple way to run tests kept within a set of package directories. For example,
1840 the following command will search the :file:`test/` subdirectory for
1841 any importable test files named ``test*.py``::
1842
1843 python -m unittest discover -s test
1844
1845 Consult the :mod:`unittest` module documentation for more details.
1846 (Developed in :issue:`6001`.)
1847
1848 The :func:`~unittest.main` function supports some other new options:
1849
1850 * :option:`-b <unittest -b>` or :option:`!--buffer` will buffer the standard output
1851 and standard error streams during each test. If the test passes,
1852 any resulting output will be discarded; on failure, the buffered
1853 output will be displayed.
1854
1855 * :option:`-c <unittest -c>` or :option:`!--catch` will cause the control-C interrupt
1856 to be handled more gracefully. Instead of interrupting the test
1857 process immediately, the currently running test will be completed
1858 and then the partial results up to the interruption will be reported.
1859 If you're impatient, a second press of control-C will cause an immediate
1860 interruption.
1861
1862 This control-C handler tries to avoid causing problems when the code
1863 being tested or the tests being run have defined a signal handler of
1864 their own, by noticing that a signal handler was already set and
1865 calling it. If this doesn't work for you, there's a
1866 :func:`~unittest.removeHandler` decorator that can be used to mark tests that
1867 should have the control-C handling disabled.
1868
1869 * :option:`-f <unittest -f>` or :option:`!--failfast` makes
1870 test execution stop immediately when a test fails instead of
1871 continuing to execute further tests. (Suggested by Cliff Dyer and
1872 implemented by Michael Foord; :issue:`8074`.)
1873
1874 The progress messages now show 'x' for expected failures
1875 and 'u' for unexpected successes when run in verbose mode.
1876 (Contributed by Benjamin Peterson.)
1877
1878 Test cases can raise the :exc:`~unittest.SkipTest` exception to skip a
1879 test (:issue:`1034053`).
1880
1881 The error messages for :meth:`~unittest.TestCase.assertEqual`,
1882 :meth:`~unittest.TestCase.assertTrue`, and :meth:`~unittest.TestCase.assertFalse`
1883 failures now provide more information. If you set the
1884 :attr:`~unittest.TestCase.longMessage` attribute of your :class:`~unittest.TestCase` classes to
1885 true, both the standard error message and any additional message you
1886 provide will be printed for failures. (Added by Michael Foord; :issue:`5663`.)
1887
1888 The :meth:`~unittest.TestCase.assertRaises` method now
1889 returns a context handler when called without providing a callable
1890 object to run. For example, you can write this::
1891
1892 with self.assertRaises(KeyError):
1893 {}['foo']
1894
1895 (Implemented by Antoine Pitrou; :issue:`4444`.)
1896
1897 .. rev 78774
1898
1899 Module- and class-level setup and teardown fixtures are now supported.
1900 Modules can contain :func:`~unittest.setUpModule` and :func:`~unittest.tearDownModule`
1901 functions. Classes can have :meth:`~unittest.TestCase.setUpClass` and
1902 :meth:`~unittest.TestCase.tearDownClass` methods that must be defined as class methods
1903 (using ``@classmethod`` or equivalent). These functions and
1904 methods are invoked when the test runner switches to a test case in a
1905 different module or class.
1906
1907 The methods :meth:`~unittest.TestCase.addCleanup` and
1908 :meth:`~unittest.TestCase.doCleanups` were added.
1909 :meth:`~unittest.TestCase.addCleanup` lets you add cleanup functions that
1910 will be called unconditionally (after :meth:`~unittest.TestCase.setUp` if
1911 :meth:`~unittest.TestCase.setUp` fails, otherwise after :meth:`~unittest.TestCase.tearDown`). This allows
1912 for much simpler resource allocation and deallocation during tests
1913 (:issue:`5679`).
1914
1915 A number of new methods were added that provide more specialized
1916 tests. Many of these methods were written by Google engineers
1917 for use in their test suites; Gregory P. Smith, Michael Foord, and
1918 GvR worked on merging them into Python's version of :mod:`unittest`.
1919
1920 * :meth:`~unittest.TestCase.assertIsNone` and :meth:`~unittest.TestCase.assertIsNotNone` take one
1921 expression and verify that the result is or is not ``None``.
1922
1923 * :meth:`~unittest.TestCase.assertIs` and :meth:`~unittest.TestCase.assertIsNot`
1924 take two values and check whether the two values evaluate to the same object or not.
1925 (Added by Michael Foord; :issue:`2578`.)
1926
1927 * :meth:`~unittest.TestCase.assertIsInstance` and
1928 :meth:`~unittest.TestCase.assertNotIsInstance` check whether
1929 the resulting object is an instance of a particular class, or of
1930 one of a tuple of classes. (Added by Georg Brandl; :issue:`7031`.)
1931
1932 * :meth:`~unittest.TestCase.assertGreater`, :meth:`~unittest.TestCase.assertGreaterEqual`,
1933 :meth:`~unittest.TestCase.assertLess`, and :meth:`~unittest.TestCase.assertLessEqual` compare
1934 two quantities.
1935
1936 * :meth:`~unittest.TestCase.assertMultiLineEqual` compares two strings, and if they're
1937 not equal, displays a helpful comparison that highlights the
1938 differences in the two strings. This comparison is now used by
1939 default when Unicode strings are compared with :meth:`~unittest.TestCase.assertEqual`.
1940
1941 * :meth:`assertRegexpMatches() <unittest.TestCase.assertRegex>` and
1942 :meth:`assertNotRegexpMatches() <unittest.TestCase.assertNotRegex>` checks whether the
1943 first argument is a string matching or not matching the regular
1944 expression provided as the second argument (:issue:`8038`).
1945
1946 * :meth:`assertRaisesRegexp() <unittest.TestCase.assertRaisesRegex>` checks
1947 whether a particular exception
1948 is raised, and then also checks that the string representation of
1949 the exception matches the provided regular expression.
1950
1951 * :meth:`~unittest.TestCase.assertIn` and :meth:`~unittest.TestCase.assertNotIn`
1952 tests whether *first* is or is not in *second*.
1953
1954 * :meth:`assertItemsEqual() <unittest.TestCase.assertCountEqual>` tests whether two provided sequences
1955 contain the same elements.
1956
1957 * :meth:`~unittest.TestCase.assertSetEqual` compares whether two sets are equal, and
1958 only reports the differences between the sets in case of error.
1959
1960 * Similarly, :meth:`~unittest.TestCase.assertListEqual` and :meth:`~unittest.TestCase.assertTupleEqual`
1961 compare the specified types and explain any differences without necessarily
1962 printing their full values; these methods are now used by default
1963 when comparing lists and tuples using :meth:`~unittest.TestCase.assertEqual`.
1964 More generally, :meth:`~unittest.TestCase.assertSequenceEqual` compares two sequences
1965 and can optionally check whether both sequences are of a
1966 particular type.
1967
1968 * :meth:`~unittest.TestCase.assertDictEqual` compares two dictionaries and reports the
1969 differences; it's now used by default when you compare two dictionaries
1970 using :meth:`~unittest.TestCase.assertEqual`. :meth:`!assertDictContainsSubset` checks whether
1971 all of the key/value pairs in *first* are found in *second*.
1972
1973 * :meth:`~unittest.TestCase.assertAlmostEqual` and :meth:`~unittest.TestCase.assertNotAlmostEqual` test
1974 whether *first* and *second* are approximately equal. This method
1975 can either round their difference to an optionally specified number
1976 of *places* (the default is 7) and compare it to zero, or require
1977 the difference to be smaller than a supplied *delta* value.
1978
1979 * :meth:`~unittest.TestLoader.loadTestsFromName` properly honors the
1980 :attr:`~unittest.TestLoader.suiteClass` attribute of
1981 the :class:`~unittest.TestLoader`. (Fixed by Mark Roddy; :issue:`6866`.)
1982
1983 * A new hook lets you extend the :meth:`~unittest.TestCase.assertEqual` method to handle
1984 new data types. The :meth:`~unittest.TestCase.addTypeEqualityFunc` method takes a type
1985 object and a function. The function will be used when both of the
1986 objects being compared are of the specified type. This function
1987 should compare the two objects and raise an exception if they don't
1988 match; it's a good idea for the function to provide additional
1989 information about why the two objects aren't matching, much as the new
1990 sequence comparison methods do.
1991
1992 :func:`unittest.main` now takes an optional ``exit`` argument. If
1993 false, :func:`~unittest.main` doesn't call :func:`sys.exit`, allowing
1994 :func:`~unittest.main` to be used from the interactive interpreter.
1995 (Contributed by J. Pablo Fernández; :issue:`3379`.)
1996
1997 :class:`~unittest.TestResult` has new :meth:`~unittest.TestResult.startTestRun` and
1998 :meth:`~unittest.TestResult.stopTestRun` methods that are called immediately before
1999 and after a test run. (Contributed by Robert Collins; :issue:`5728`.)
2000
2001 With all these changes, the :file:`unittest.py` was becoming awkwardly
2002 large, so the module was turned into a package and the code split into
2003 several files (by Benjamin Peterson). This doesn't affect how the
2004 module is imported or used.
2005
2006 .. seealso::
2007
2008 https://web.archive.org/web/20210619163128/http://www.voidspace.org.uk/python/articles/unittest2.shtml
2009 Describes the new features, how to use them, and the
2010 rationale for various design decisions. (By Michael Foord.)
2011
2012 .. _elementtree-section:
2013
2014 Updated module: ElementTree 1.3
2015 ---------------------------------
2016
2017 The version of the ElementTree library included with Python was updated to
2018 version 1.3. Some of the new features are:
2019
2020 * The various parsing functions now take a *parser* keyword argument
2021 giving an :class:`~xml.etree.ElementTree.XMLParser` instance that will
2022 be used. This makes it possible to override the file's internal encoding::
2023
2024 p = ET.XMLParser(encoding='utf-8')
2025 t = ET.XML("""<root/>""", parser=p)
2026
2027 Errors in parsing XML now raise a :exc:`~xml.etree.ElementTree.ParseError` exception, whose
2028 instances have a :attr:`!position` attribute
2029 containing a (*line*, *column*) tuple giving the location of the problem.
2030
2031 * ElementTree's code for converting trees to a string has been
2032 significantly reworked, making it roughly twice as fast in many
2033 cases. The :meth:`ElementTree.write() <xml.etree.ElementTree.ElementTree.write>`
2034 and :meth:`!Element.write` methods now have a *method* parameter that can be
2035 "xml" (the default), "html", or "text". HTML mode will output empty
2036 elements as ``<empty></empty>`` instead of ``<empty/>``, and text
2037 mode will skip over elements and only output the text chunks. If
2038 you set the :attr:`~xml.etree.ElementTree.Element.tag` attribute of an
2039 element to ``None`` but
2040 leave its children in place, the element will be omitted when the
2041 tree is written out, so you don't need to do more extensive rearrangement
2042 to remove a single element.
2043
2044 Namespace handling has also been improved. All ``xmlns:<whatever>``
2045 declarations are now output on the root element, not scattered throughout
2046 the resulting XML. You can set the default namespace for a tree
2047 by setting the :attr:`!default_namespace` attribute and can
2048 register new prefixes with :meth:`~xml.etree.ElementTree.register_namespace`. In XML mode,
2049 you can use the true/false *xml_declaration* parameter to suppress the
2050 XML declaration.
2051
2052 * New :class:`~xml.etree.ElementTree.Element` method:
2053 :meth:`~xml.etree.ElementTree.Element.extend` appends the items from a
2054 sequence to the element's children. Elements themselves behave like
2055 sequences, so it's easy to move children from one element to
2056 another::
2057
2058 from xml.etree import ElementTree as ET
2059
2060 t = ET.XML("""<list>
2061 <item>1</item> <item>2</item> <item>3</item>
2062 </list>""")
2063 new = ET.XML('<root/>')
2064 new.extend(t)
2065
2066 # Outputs <root><item>1</item>...</root>
2067 print ET.tostring(new)
2068
2069 * New :class:`~xml.etree.ElementTree.Element` method:
2070 :meth:`~xml.etree.ElementTree.Element.iter` yields the children of the
2071 element as a generator. It's also possible to write ``for child in
2072 elem:`` to loop over an element's children. The existing method
2073 :meth:`!getiterator` is now deprecated, as is :meth:`!getchildren`
2074 which constructs and returns a list of children.
2075
2076 * New :class:`~xml.etree.ElementTree.Element` method:
2077 :meth:`~xml.etree.ElementTree.Element.itertext` yields all chunks of
2078 text that are descendants of the element. For example::
2079
2080 t = ET.XML("""<list>
2081 <item>1</item> <item>2</item> <item>3</item>
2082 </list>""")
2083
2084 # Outputs ['\n ', '1', ' ', '2', ' ', '3', '\n']
2085 print list(t.itertext())
2086
2087 * Deprecated: using an element as a Boolean (i.e., ``if elem:``) would
2088 return true if the element had any children, or false if there were
2089 no children. This behaviour is confusing -- ``None`` is false, but
2090 so is a childless element? -- so it will now trigger a
2091 :exc:`FutureWarning`. In your code, you should be explicit: write
2092 ``len(elem) != 0`` if you're interested in the number of children,
2093 or ``elem is not None``.
2094
2095 Fredrik Lundh develops ElementTree and produced the 1.3 version;
2096 you can read his article describing 1.3 at
2097 https://web.archive.org/web/20200703234532/http://effbot.org/zone/elementtree-13-intro.htm.
2098 Florent Xicluna updated the version included with
2099 Python, after discussions on python-dev and in :issue:`6472`.)
2100
2101 .. ======================================================================
2102
2103
2104 Build and C API Changes
2105 =======================
2106
2107 Changes to Python's build process and to the C API include:
2108
2109 * The latest release of the GNU Debugger, GDB 7, can be `scripted
2110 using Python
2111 <https://web.archive.org/web/20110715084810/http://sourceware.org/gdb/current/onlinedocs/gdb/Python.html>`__.
2112 When you begin debugging an executable program P, GDB will look for
2113 a file named ``P-gdb.py`` and automatically read it. Dave Malcolm
2114 contributed a :file:`python-gdb.py` that adds a number of
2115 commands useful when debugging Python itself. For example,
2116 ``py-up`` and ``py-down`` go up or down one Python stack frame,
2117 which usually corresponds to several C stack frames. ``py-print``
2118 prints the value of a Python variable, and ``py-bt`` prints the
2119 Python stack trace. (Added as a result of :issue:`8032`.)
2120
2121 * If you use the :file:`.gdbinit` file provided with Python,
2122 the "pyo" macro in the 2.7 version now works correctly when the thread being
2123 debugged doesn't hold the GIL; the macro now acquires it before printing.
2124 (Contributed by Victor Stinner; :issue:`3632`.)
2125
2126 * :c:func:`Py_AddPendingCall` is now thread-safe, letting any
2127 worker thread submit notifications to the main Python thread. This
2128 is particularly useful for asynchronous IO operations.
2129 (Contributed by Kristján Valur Jónsson; :issue:`4293`.)
2130
2131 * New function: :c:func:`PyCode_NewEmpty` creates an empty code object;
2132 only the filename, function name, and first line number are required.
2133 This is useful for extension modules that are attempting to
2134 construct a more useful traceback stack. Previously such
2135 extensions needed to call :c:func:`!PyCode_New`, which had many
2136 more arguments. (Added by Jeffrey Yasskin.)
2137
2138 * New function: :c:func:`PyErr_NewExceptionWithDoc` creates a new
2139 exception class, just as the existing :c:func:`PyErr_NewException` does,
2140 but takes an extra ``char *`` argument containing the docstring for the
2141 new exception class. (Added by 'lekma' on the Python bug tracker;
2142 :issue:`7033`.)
2143
2144 * New function: :c:func:`PyFrame_GetLineNumber` takes a frame object
2145 and returns the line number that the frame is currently executing.
2146 Previously code would need to get the index of the bytecode
2147 instruction currently executing, and then look up the line number
2148 corresponding to that address. (Added by Jeffrey Yasskin.)
2149
2150 * New functions: :c:func:`PyLong_AsLongAndOverflow` and
2151 :c:func:`PyLong_AsLongLongAndOverflow` approximates a Python long
2152 integer as a C :c:expr:`long` or :c:expr:`long long`.
2153 If the number is too large to fit into
2154 the output type, an *overflow* flag is set and returned to the caller.
2155 (Contributed by Case Van Horsen; :issue:`7528` and :issue:`7767`.)
2156
2157 * New function: stemming from the rewrite of string-to-float conversion,
2158 a new :c:func:`PyOS_string_to_double` function was added. The old
2159 :c:func:`!PyOS_ascii_strtod` and :c:func:`!PyOS_ascii_atof` functions
2160 are now deprecated.
2161
2162 * New function: :c:func:`!PySys_SetArgvEx` sets the value of
2163 ``sys.argv`` and can optionally update ``sys.path`` to include the
2164 directory containing the script named by ``sys.argv[0]`` depending
2165 on the value of an *updatepath* parameter.
2166
2167 This function was added to close a security hole for applications
2168 that embed Python. The old function, :c:func:`!PySys_SetArgv`, would
2169 always update ``sys.path``, and sometimes it would add the current
2170 directory. This meant that, if you ran an application embedding
2171 Python in a directory controlled by someone else, attackers could
2172 put a Trojan-horse module in the directory (say, a file named
2173 :file:`os.py`) that your application would then import and run.
2174
2175 If you maintain a C/C++ application that embeds Python, check
2176 whether you're calling :c:func:`!PySys_SetArgv` and carefully consider
2177 whether the application should be using :c:func:`!PySys_SetArgvEx`
2178 with *updatepath* set to false.
2179
2180 Security issue reported as :cve:`2008-5983`;
2181 discussed in :issue:`5753`, and fixed by Antoine Pitrou.
2182
2183 * New macros: the Python header files now define the following macros:
2184 :c:macro:`Py_ISALNUM`,
2185 :c:macro:`Py_ISALPHA`,
2186 :c:macro:`Py_ISDIGIT`,
2187 :c:macro:`Py_ISLOWER`,
2188 :c:macro:`Py_ISSPACE`,
2189 :c:macro:`Py_ISUPPER`,
2190 :c:macro:`Py_ISXDIGIT`,
2191 :c:macro:`Py_TOLOWER`, and :c:macro:`Py_TOUPPER`.
2192 All of these functions are analogous to the C
2193 standard macros for classifying characters, but ignore the current
2194 locale setting, because in
2195 several places Python needs to analyze characters in a
2196 locale-independent way. (Added by Eric Smith;
2197 :issue:`5793`.)
2198
2199 * Removed function: :c:func:`!PyEval_CallObject` is now only available
2200 as a macro. A function version was being kept around to preserve
2201 ABI linking compatibility, but that was in 1997; it can certainly be
2202 deleted by now. (Removed by Antoine Pitrou; :issue:`8276`.)
2203
2204 * New format codes: the :c:func:`!PyString_FromFormat`,
2205 :c:func:`!PyString_FromFormatV`, and :c:func:`PyErr_Format` functions now
2206 accept ``%lld`` and ``%llu`` format codes for displaying
2207 C's :c:expr:`long long` types.
2208 (Contributed by Mark Dickinson; :issue:`7228`.)
2209
2210 * The complicated interaction between threads and process forking has
2211 been changed. Previously, the child process created by
2212 :func:`os.fork` might fail because the child is created with only a
2213 single thread running, the thread performing the :func:`os.fork`.
2214 If other threads were holding a lock, such as Python's import lock,
2215 when the fork was performed, the lock would still be marked as
2216 "held" in the new process. But in the child process nothing would
2217 ever release the lock, since the other threads weren't replicated,
2218 and the child process would no longer be able to perform imports.
2219
2220 Python 2.7 acquires the import lock before performing an
2221 :func:`os.fork`, and will also clean up any locks created using the
2222 :mod:`threading` module. C extension modules that have internal
2223 locks, or that call :c:func:`fork()` themselves, will not benefit
2224 from this clean-up.
2225
2226 (Fixed by Thomas Wouters; :issue:`1590864`.)
2227
2228 * The :c:func:`Py_Finalize` function now calls the internal
2229 :func:`!threading._shutdown` function; this prevents some exceptions from
2230 being raised when an interpreter shuts down.
2231 (Patch by Adam Olsen; :issue:`1722344`.)
2232
2233 * When using the :c:type:`PyMemberDef` structure to define attributes
2234 of a type, Python will no longer let you try to delete or set a
2235 :c:macro:`!T_STRING_INPLACE` attribute.
2236
2237 .. rev 79644
2238
2239 * Global symbols defined by the :mod:`ctypes` module are now prefixed
2240 with ``Py``, or with ``_ctypes``. (Implemented by Thomas
2241 Heller; :issue:`3102`.)
2242
2243 * New configure option: the :option:`!--with-system-expat` switch allows
2244 building the :mod:`pyexpat <xml.parsers.expat>` module to use the system Expat library.
2245 (Contributed by Arfrever Frehtes Taifersar Arahesis; :issue:`7609`.)
2246
2247 * New configure option: the
2248 :option:`!--with-valgrind` option will now disable the pymalloc
2249 allocator, which is difficult for the Valgrind memory-error detector
2250 to analyze correctly.
2251 Valgrind will therefore be better at detecting memory leaks and
2252 overruns. (Contributed by James Henstridge; :issue:`2422`.)
2253
2254 * New configure option: you can now supply an empty string to
2255 :option:`!--with-dbmliborder=` in order to disable all of the various
2256 DBM modules. (Added by Arfrever Frehtes Taifersar Arahesis;
2257 :issue:`6491`.)
2258
2259 * The :program:`configure` script now checks for floating-point rounding bugs
2260 on certain 32-bit Intel chips and defines a :c:macro:`!X87_DOUBLE_ROUNDING`
2261 preprocessor definition. No code currently uses this definition,
2262 but it's available if anyone wishes to use it.
2263 (Added by Mark Dickinson; :issue:`2937`.)
2264
2265 :program:`configure` also now sets a :envvar:`!LDCXXSHARED` Makefile
2266 variable for supporting C++ linking. (Contributed by Arfrever
2267 Frehtes Taifersar Arahesis; :issue:`1222585`.)
2268
2269 * The build process now creates the necessary files for pkg-config
2270 support. (Contributed by Clinton Roy; :issue:`3585`.)
2271
2272 * The build process now supports Subversion 1.7. (Contributed by
2273 Arfrever Frehtes Taifersar Arahesis; :issue:`6094`.)
2274
2275
2276 .. _whatsnew27-capsules:
2277
2278 Capsules
2279 -------------------
2280
2281 Python 3.1 adds a new C datatype, :c:type:`PyCapsule`, for providing a
2282 C API to an extension module. A capsule is essentially the holder of
2283 a C ``void *`` pointer, and is made available as a module attribute; for
2284 example, the :mod:`socket` module's API is exposed as ``socket.CAPI``,
2285 and :mod:`unicodedata` exposes ``ucnhash_CAPI``. Other extensions
2286 can import the module, access its dictionary to get the capsule
2287 object, and then get the ``void *`` pointer, which will usually point
2288 to an array of pointers to the module's various API functions.
2289
2290 There is an existing data type already used for this,
2291 :c:type:`!PyCObject`, but it doesn't provide type safety. Evil code
2292 written in pure Python could cause a segmentation fault by taking a
2293 :c:type:`!PyCObject` from module A and somehow substituting it for the
2294 :c:type:`!PyCObject` in module B. Capsules know their own name,
2295 and getting the pointer requires providing the name:
2296
2297 .. code-block:: c
2298
2299 void *vtable;
2300
2301 if (!PyCapsule_IsValid(capsule, "mymodule.CAPI") {
2302 PyErr_SetString(PyExc_ValueError, "argument type invalid");
2303 return NULL;
2304 }
2305
2306 vtable = PyCapsule_GetPointer(capsule, "mymodule.CAPI");
2307
2308 You are assured that ``vtable`` points to whatever you're expecting.
2309 If a different capsule was passed in, :c:func:`PyCapsule_IsValid` would
2310 detect the mismatched name and return false. Refer to
2311 :ref:`using-capsules` for more information on using these objects.
2312
2313 Python 2.7 now uses capsules internally to provide various
2314 extension-module APIs, but the :c:func:`!PyCObject_AsVoidPtr` was
2315 modified to handle capsules, preserving compile-time compatibility
2316 with the :c:type:`!PyCObject` interface. Use of
2317 :c:func:`!PyCObject_AsVoidPtr` will signal a
2318 :exc:`PendingDeprecationWarning`, which is silent by default.
2319
2320 Implemented in Python 3.1 and backported to 2.7 by Larry Hastings;
2321 discussed in :issue:`5630`.
2322
2323
2324 .. ======================================================================
2325
2326 Port-Specific Changes: Windows
2327 -----------------------------------
2328
2329 * The :mod:`msvcrt` module now contains some constants from
2330 the :file:`crtassem.h` header file:
2331 :data:`~msvcrt.CRT_ASSEMBLY_VERSION`,
2332 :data:`~msvcrt.VC_ASSEMBLY_PUBLICKEYTOKEN`,
2333 and :data:`~msvcrt.LIBRARIES_ASSEMBLY_NAME_PREFIX`.
2334 (Contributed by David Cournapeau; :issue:`4365`.)
2335
2336 * The :mod:`_winreg <winreg>` module for accessing the registry now implements
2337 the :func:`~winreg.CreateKeyEx` and :func:`~winreg.DeleteKeyEx`
2338 functions, extended versions of previously supported functions that
2339 take several extra arguments. The :func:`~winreg.DisableReflectionKey`,
2340 :func:`~winreg.EnableReflectionKey`, and :func:`~winreg.QueryReflectionKey`
2341 were also tested and documented.
2342 (Implemented by Brian Curtin: :issue:`7347`.)
2343
2344 * The new :c:func:`!_beginthreadex` API is used to start threads, and
2345 the native thread-local storage functions are now used.
2346 (Contributed by Kristján Valur Jónsson; :issue:`3582`.)
2347
2348 * The :func:`os.kill` function now works on Windows. The signal value
2349 can be the constants :const:`~signal.CTRL_C_EVENT`,
2350 :const:`~signal.CTRL_BREAK_EVENT`, or any integer. The first two constants
2351 will send :kbd:`Control-C` and :kbd:`Control-Break` keystroke events to
2352 subprocesses; any other value will use the :c:func:`!TerminateProcess`
2353 API. (Contributed by Miki Tebeka; :issue:`1220212`.)
2354
2355 * The :func:`os.listdir` function now correctly fails
2356 for an empty path. (Fixed by Hirokazu Yamamoto; :issue:`5913`.)
2357
2358 * The :mod:`mimetypes` module will now read the MIME database from
2359 the Windows registry when initializing.
2360 (Patch by Gabriel Genellina; :issue:`4969`.)
2361
2362 .. ======================================================================
2363
2364 Port-Specific Changes: Mac OS X
2365 -----------------------------------
2366
2367 * The path ``/Library/Python/2.7/site-packages`` is now appended to
2368 ``sys.path``, in order to share added packages between the system
2369 installation and a user-installed copy of the same version.
2370 (Changed by Ronald Oussoren; :issue:`4865`.)
2371
2372 .. versionchanged:: 2.7.13
2373
2374 As of 2.7.13, this change was removed.
2375 ``/Library/Python/2.7/site-packages``, the site-packages directory
2376 used by the Apple-supplied system Python 2.7 is no longer appended to
2377 ``sys.path`` for user-installed Pythons such as from the python.org
2378 installers. As of macOS 10.12, Apple changed how the system
2379 site-packages directory is configured, which could cause installation
2380 of pip components, like setuptools, to fail. Packages installed for
2381 the system Python will no longer be shared with user-installed
2382 Pythons. (:issue:`28440`)
2383
2384 Port-Specific Changes: FreeBSD
2385 -----------------------------------
2386
2387 * FreeBSD 7.1's :const:`!SO_SETFIB` constant, used with the :func:`~socket.socket` methods
2388 :func:`~socket.socket.getsockopt`/:func:`~socket.socket.setsockopt` to select an
2389 alternate routing table, is now available in the :mod:`socket`
2390 module. (Added by Kyle VanderBeek; :issue:`8235`.)
2391
2392 Other Changes and Fixes
2393 =======================
2394
2395 * Two benchmark scripts, :file:`iobench` and :file:`ccbench`, were
2396 added to the :file:`Tools` directory. :file:`iobench` measures the
2397 speed of the built-in file I/O objects returned by :func:`open`
2398 while performing various operations, and :file:`ccbench` is a
2399 concurrency benchmark that tries to measure computing throughput,
2400 thread switching latency, and IO processing bandwidth when
2401 performing several tasks using a varying number of threads.
2402
2403 * The :file:`Tools/i18n/msgfmt.py` script now understands plural
2404 forms in :file:`.po` files. (Fixed by Martin von Löwis;
2405 :issue:`5464`.)
2406
2407 * When importing a module from a :file:`.pyc` or :file:`.pyo` file
2408 with an existing :file:`.py` counterpart, the :attr:`~codeobject.co_filename`
2409 attributes of the resulting code objects are overwritten when the
2410 original filename is obsolete. This can happen if the file has been
2411 renamed, moved, or is accessed through different paths. (Patch by
2412 Ziga Seilnacht and Jean-Paul Calderone; :issue:`1180193`.)
2413
2414 * The :file:`regrtest.py` script now takes a :option:`!--randseed=`
2415 switch that takes an integer that will be used as the random seed
2416 for the :option:`!-r` option that executes tests in random order.
2417 The :option:`!-r` option also reports the seed that was used
2418 (Added by Collin Winter.)
2419
2420 * Another :file:`regrtest.py` switch is :option:`!-j`, which
2421 takes an integer specifying how many tests run in parallel. This
2422 allows reducing the total runtime on multi-core machines.
2423 This option is compatible with several other options, including the
2424 :option:`!-R` switch which is known to produce long runtimes.
2425 (Added by Antoine Pitrou, :issue:`6152`.) This can also be used
2426 with a new :option:`!-F` switch that runs selected tests in a loop
2427 until they fail. (Added by Antoine Pitrou; :issue:`7312`.)
2428
2429 * When executed as a script, the :file:`py_compile.py` module now
2430 accepts ``'-'`` as an argument, which will read standard input for
2431 the list of filenames to be compiled. (Contributed by Piotr
2432 Ożarowski; :issue:`8233`.)
2433
2434 .. ======================================================================
2435
2436 Porting to Python 2.7
2437 =====================
2438
2439 This section lists previously described changes and other bugfixes
2440 that may require changes to your code:
2441
2442 * The :func:`range` function processes its arguments more
2443 consistently; it will now call :meth:`~object.__int__` on non-float,
2444 non-integer arguments that are supplied to it. (Fixed by Alexander
2445 Belopolsky; :issue:`1533`.)
2446
2447 * The string :meth:`format` method changed the default precision used
2448 for floating-point and complex numbers from 6 decimal
2449 places to 12, which matches the precision used by :func:`str`.
2450 (Changed by Eric Smith; :issue:`5920`.)
2451
2452 * Because of an optimization for the :keyword:`with` statement, the special
2453 methods :meth:`~object.__enter__` and :meth:`~object.__exit__` must belong to the object's
2454 type, and cannot be directly attached to the object's instance. This
2455 affects new-style classes (derived from :class:`object`) and C extension
2456 types. (:issue:`6101`.)
2457
2458 * Due to a bug in Python 2.6, the *exc_value* parameter to
2459 :meth:`~object.__exit__` methods was often the string representation of the
2460 exception, not an instance. This was fixed in 2.7, so *exc_value*
2461 will be an instance as expected. (Fixed by Florent Xicluna;
2462 :issue:`7853`.)
2463
2464 * When a restricted set of attributes were set using ``__slots__``,
2465 deleting an unset attribute would not raise :exc:`AttributeError`
2466 as you would expect. Fixed by Benjamin Peterson; :issue:`7604`.)
2467
2468 In the standard library:
2469
2470 * Operations with :class:`~datetime.datetime` instances that resulted in a year
2471 falling outside the supported range didn't always raise
2472 :exc:`OverflowError`. Such errors are now checked more carefully
2473 and will now raise the exception. (Reported by Mark Leander, patch
2474 by Anand B. Pillai and Alexander Belopolsky; :issue:`7150`.)
2475
2476 * When using :class:`~decimal.Decimal` instances with a string's
2477 :meth:`format` method, the default alignment was previously
2478 left-alignment. This has been changed to right-alignment, which might
2479 change the output of your programs.
2480 (Changed by Mark Dickinson; :issue:`6857`.)
2481
2482 Comparisons involving a signaling NaN value (or ``sNAN``) now signal
2483 :const:`~decimal.InvalidOperation` instead of silently returning a true or
2484 false value depending on the comparison operator. Quiet NaN values
2485 (or ``NaN``) are now hashable. (Fixed by Mark Dickinson;
2486 :issue:`7279`.)
2487
2488 * The :mod:`xml.etree.ElementTree` library no longer escapes
2489 ampersands and angle brackets when outputting an XML processing
2490 instruction (which looks like ``<?xml-stylesheet href="#style1"?>``)
2491 or comment (which looks like ``<!-- comment -->``).
2492 (Patch by Neil Muller; :issue:`2746`.)
2493
2494 * The :meth:`!readline` method of :class:`~io.StringIO` objects now does
2495 nothing when a negative length is requested, as other file-like
2496 objects do. (:issue:`7348`).
2497
2498 * The :mod:`syslog` module will now use the value of ``sys.argv[0]`` as the
2499 identifier instead of the previous default value of ``'python'``.
2500 (Changed by Sean Reifschneider; :issue:`8451`.)
2501
2502 * The :mod:`tarfile` module's default error handling has changed, to
2503 no longer suppress fatal errors. The default error level was previously 0,
2504 which meant that errors would only result in a message being written to the
2505 debug log, but because the debug log is not activated by default,
2506 these errors go unnoticed. The default error level is now 1,
2507 which raises an exception if there's an error.
2508 (Changed by Lars Gustäbel; :issue:`7357`.)
2509
2510 * The :mod:`urlparse <urllib.parse>` module's :func:`~urllib.parse.urlsplit` now handles
2511 unknown URL schemes in a fashion compliant with :rfc:`3986`: if the
2512 URL is of the form ``"<something>://..."``, the text before the
2513 ``://`` is treated as the scheme, even if it's a made-up scheme that
2514 the module doesn't know about. This change may break code that
2515 worked around the old behaviour. For example, Python 2.6.4 or 2.5
2516 will return the following:
2517
2518 .. doctest::
2519 :options: +SKIP
2520
2521 >>> import urlparse
2522 >>> urlparse.urlsplit('invented://host/filename?query')
2523 ('invented', '', '//host/filename?query', '', '')
2524
2525 Python 2.7 (and Python 2.6.5) will return:
2526
2527 .. doctest::
2528 :options: +SKIP
2529
2530 >>> import urlparse
2531 >>> urlparse.urlsplit('invented://host/filename?query')
2532 ('invented', 'host', '/filename?query', '', '')
2533
2534 (Python 2.7 actually produces slightly different output, since it
2535 returns a named tuple instead of a standard tuple.)
2536
2537 For C extensions:
2538
2539 * C extensions that use integer format codes with the ``PyArg_Parse*``
2540 family of functions will now raise a :exc:`TypeError` exception
2541 instead of triggering a :exc:`DeprecationWarning` (:issue:`5080`).
2542
2543 * Use the new :c:func:`PyOS_string_to_double` function instead of the old
2544 :c:func:`!PyOS_ascii_strtod` and :c:func:`!PyOS_ascii_atof` functions,
2545 which are now deprecated.
2546
2547 For applications that embed Python:
2548
2549 * The :c:func:`!PySys_SetArgvEx` function was added, letting
2550 applications close a security hole when the existing
2551 :c:func:`!PySys_SetArgv` function was used. Check whether you're
2552 calling :c:func:`!PySys_SetArgv` and carefully consider whether the
2553 application should be using :c:func:`!PySys_SetArgvEx` with
2554 *updatepath* set to false.
2555
2556 .. ======================================================================
2557
2558
2559 .. _py27-maintenance-enhancements:
2560
2561 New Features Added to Python 2.7 Maintenance Releases
2562 =====================================================
2563
2564 New features may be added to Python 2.7 maintenance releases when the
2565 situation genuinely calls for it. Any such additions must go through
2566 the Python Enhancement Proposal process, and make a compelling case for why
2567 they can't be adequately addressed by either adding the new feature solely to
2568 Python 3, or else by publishing it on the Python Package Index.
2569
2570 In addition to the specific proposals listed below, there is a general
2571 exemption allowing new ``-3`` warnings to be added in any Python 2.7
2572 maintenance release.
2573
2574
2575 Two new environment variables for debug mode
2576 --------------------------------------------
2577
2578 In debug mode, the ``[xxx refs]`` statistic is not written by default, the
2579 :envvar:`!PYTHONSHOWREFCOUNT` environment variable now must also be set.
2580 (Contributed by Victor Stinner; :issue:`31733`.)
2581
2582 When Python is compiled with ``COUNT_ALLOC`` defined, allocation counts are no
2583 longer dumped by default anymore: the :envvar:`!PYTHONSHOWALLOCCOUNT` environment
2584 variable must now also be set. Moreover, allocation counts are now dumped into
2585 stderr, rather than stdout. (Contributed by Victor Stinner; :issue:`31692`.)
2586
2587 .. versionadded:: 2.7.15
2588
2589
2590 PEP 434: IDLE Enhancement Exception for All Branches
2591 ----------------------------------------------------
2592
2593 :pep:`434` describes a general exemption for changes made to the IDLE
2594 development environment shipped along with Python. This exemption makes it
2595 possible for the IDLE developers to provide a more consistent user
2596 experience across all supported versions of Python 2 and 3.
2597
2598 For details of any IDLE changes, refer to the NEWS file for the specific
2599 release.
2600
2601
2602 PEP 466: Network Security Enhancements for Python 2.7
2603 -----------------------------------------------------
2604
2605 :pep:`466` describes a number of network security enhancement proposals
2606 that have been approved for inclusion in Python 2.7 maintenance releases,
2607 with the first of those changes appearing in the Python 2.7.7 release.
2608
2609 :pep:`466` related features added in Python 2.7.7:
2610
2611 * :func:`hmac.compare_digest` was backported from Python 3 to make a timing
2612 attack resistant comparison operation available to Python 2 applications.
2613 (Contributed by Alex Gaynor; :issue:`21306`.)
2614
2615 * OpenSSL 1.0.1g was upgraded in the official Windows installers published on
2616 python.org. (Contributed by Zachary Ware; :issue:`21462`.)
2617
2618 :pep:`466` related features added in Python 2.7.8:
2619
2620 * :func:`hashlib.pbkdf2_hmac` was backported from Python 3 to make a hashing
2621 algorithm suitable for secure password storage broadly available to Python
2622 2 applications. (Contributed by Alex Gaynor; :issue:`21304`.)
2623
2624 * OpenSSL 1.0.1h was upgraded for the official Windows installers published on
2625 python.org. (Contributed by Zachary Ware in :issue:`21671` for :cve:`2014-0224`.)
2626
2627 :pep:`466` related features added in Python 2.7.9:
2628
2629 * Most of Python 3.4's :mod:`ssl` module was backported. This means :mod:`ssl`
2630 now supports Server Name Indication, TLS1.x settings, access to the platform
2631 certificate store, the :class:`~ssl.SSLContext` class, and other
2632 features. (Contributed by Alex Gaynor and David Reid; :issue:`21308`.)
2633
2634 Refer to the "Version added: 2.7.9" notes in the module documentation for
2635 specific details.
2636
2637 * :func:`os.urandom` was changed to cache a file descriptor to ``/dev/urandom``
2638 instead of reopening ``/dev/urandom`` on every call. (Contributed by Alex
2639 Gaynor; :issue:`21305`.)
2640
2641 * :data:`hashlib.algorithms_guaranteed` and
2642 :data:`hashlib.algorithms_available` were backported from Python 3 to make
2643 it easier for Python 2 applications to select the strongest available hash
2644 algorithm. (Contributed by Alex Gaynor in :issue:`21307`)
2645
2646
2647 PEP 477: Backport ensurepip (PEP 453) to Python 2.7
2648 ---------------------------------------------------
2649
2650 :pep:`477` approves the inclusion of the :pep:`453` ensurepip module and the
2651 improved documentation that was enabled by it in the Python 2.7 maintenance
2652 releases, appearing first in the Python 2.7.9 release.
2653
2654
2655 Bootstrapping pip By Default
2656 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2657
2658 The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard
2659 cross-platform mechanism to bootstrap the pip installer into Python
2660 installations. The version of ``pip`` included with Python 2.7.9 is ``pip``
2661 1.5.6, and future 2.7.x maintenance releases will update the bundled version to
2662 the latest version of ``pip`` that is available at the time of creating the
2663 release candidate.
2664
2665 By default, the commands ``pip``, ``pipX`` and ``pipX.Y`` will be installed on
2666 all platforms (where X.Y stands for the version of the Python installation),
2667 along with the ``pip`` Python package and its dependencies.
2668
2669 For CPython :ref:`source builds on POSIX systems <building-python-on-unix>`,
2670 the ``make install`` and ``make altinstall`` commands do not bootstrap ``pip``
2671 by default. This behaviour can be controlled through configure options, and
2672 overridden through Makefile options.
2673
2674 On Windows and Mac OS X, the CPython installers now default to installing
2675 ``pip`` along with CPython itself (users may opt out of installing it
2676 during the installation process). Window users will need to opt in to the
2677 automatic ``PATH`` modifications to have ``pip`` available from the command
2678 line by default, otherwise it can still be accessed through the Python
2679 launcher for Windows as ``py -m pip``.
2680
2681 As :pep:`discussed in the PEP <0477#disabling-ensurepip-by-downstream-distributors>`,
2682 platform packagers may choose not to install
2683 these commands by default, as long as, when invoked, they provide clear and
2684 simple directions on how to install them on that platform (usually using
2685 the system package manager).
2686
2687 Documentation Changes
2688 ~~~~~~~~~~~~~~~~~~~~~
2689
2690 As part of this change, the :ref:`installing-index` and
2691 :ref:`distributing-index` sections of the documentation have been
2692 completely redesigned as short getting started and FAQ documents. Most
2693 packaging documentation has now been moved out to the Python Packaging
2694 Authority maintained `Python Packaging User Guide
2695 <https://packaging.python.org>`__ and the documentation of the individual
2696 projects.
2697
2698 However, as this migration is currently still incomplete, the legacy
2699 versions of those guides remaining available as :ref:`install-index`
2700 and :ref:`setuptools-index`.
2701
2702 .. seealso::
2703
2704 :pep:`453` -- Explicit bootstrapping of pip in Python installations
2705 PEP written by Donald Stufft and Nick Coghlan, implemented by
2706 Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily.
2707
2708 PEP 476: Enabling certificate verification by default for stdlib http clients
2709 -----------------------------------------------------------------------------
2710
2711 :pep:`476` updated :mod:`httplib <http>` and modules which use it, such as
2712 :mod:`urllib2 <urllib.request>` and :mod:`xmlrpclib <xmlrpc.client>`, to now
2713 verify that the server
2714 presents a certificate which is signed by a Certificate Authority in the
2715 platform trust store and whose hostname matches the hostname being requested
2716 by default, significantly improving security for many applications. This
2717 change was made in the Python 2.7.9 release.
2718
2719 For applications which require the old previous behavior, they can pass an
2720 alternate context::
2721
2722 import urllib2
2723 import ssl
2724
2725 # This disables all verification
2726 context = ssl._create_unverified_context()
2727
2728 # This allows using a specific certificate for the host, which doesn't need
2729 # to be in the trust store
2730 context = ssl.create_default_context(cafile="/path/to/file.crt")
2731
2732 urllib2.urlopen("https://invalid-cert", context=context)
2733
2734
2735 PEP 493: HTTPS verification migration tools for Python 2.7
2736 ----------------------------------------------------------
2737
2738 :pep:`493` provides additional migration tools to support a more incremental
2739 infrastructure upgrade process for environments containing applications and
2740 services relying on the historically permissive processing of server
2741 certificates when establishing client HTTPS connections. These additions were
2742 made in the Python 2.7.12 release.
2743
2744 These tools are intended for use in cases where affected applications and
2745 services can't be modified to explicitly pass a more permissive SSL context
2746 when establishing the connection.
2747
2748 For applications and services which can't be modified at all, the new
2749 ``PYTHONHTTPSVERIFY`` environment variable may be set to ``0`` to revert an
2750 entire Python process back to the default permissive behaviour of Python 2.7.8
2751 and earlier.
2752
2753 For cases where the connection establishment code can't be modified, but the
2754 overall application can be, the new :func:`!ssl._https_verify_certificates`
2755 function can be used to adjust the default behaviour at runtime.
2756
2757
2758 New ``make regen-all`` build target
2759 -----------------------------------
2760
2761 To simplify cross-compilation, and to ensure that CPython can reliably be
2762 compiled without requiring an existing version of Python to already be
2763 available, the autotools-based build system no longer attempts to implicitly
2764 recompile generated files based on file modification times.
2765
2766 Instead, a new ``make regen-all`` command has been added to force regeneration
2767 of these files when desired (e.g. after an initial version of Python has
2768 already been built based on the pregenerated versions).
2769
2770 More selective regeneration targets are also defined - see
2771 :source:`Makefile.pre.in` for details.
2772
2773 (Contributed by Victor Stinner in :issue:`23404`.)
2774
2775 .. versionadded:: 2.7.14
2776
2777
2778 Removal of ``make touch`` build target
2779 --------------------------------------
2780
2781 The ``make touch`` build target previously used to request implicit regeneration
2782 of generated files by updating their modification times has been removed.
2783
2784 It has been replaced by the new ``make regen-all`` target.
2785
2786 (Contributed by Victor Stinner in :issue:`23404`.)
2787
2788 .. versionchanged:: 2.7.14
2789
2790 .. ======================================================================
2791
2792 .. _acks27:
2793
2794 Acknowledgements
2795 ================
2796
2797 The author would like to thank the following people for offering
2798 suggestions, corrections and assistance with various drafts of this
2799 article: Nick Coghlan, Philip Jenvey, Ryan Lovett, R. David Murray,
2800 Hugh Secker-Walker.