| 1 |
**************************** |
| 2 |
What's New In Python 3.3 |
| 3 |
**************************** |
| 4 |
|
| 5 |
.. Rules for maintenance: |
| 6 |
|
| 7 |
* Anyone can add text to this document. Do not spend very much time |
| 8 |
on the wording of your changes, because your text will probably |
| 9 |
get rewritten to some degree. |
| 10 |
|
| 11 |
* The maintainer will go through Misc/NEWS periodically and add |
| 12 |
changes; it's therefore more important to add your changes to |
| 13 |
Misc/NEWS than to this file. |
| 14 |
|
| 15 |
* This is not a complete list of every single change; completeness |
| 16 |
is the purpose of Misc/NEWS. Some changes I consider too small |
| 17 |
or esoteric to include. If such a change is added to the text, |
| 18 |
I'll just remove it. (This is another reason you shouldn't spend |
| 19 |
too much time on writing your addition.) |
| 20 |
|
| 21 |
* If you want to draw your new text to the attention of the |
| 22 |
maintainer, add 'XXX' to the beginning of the paragraph or |
| 23 |
section. |
| 24 |
|
| 25 |
* It's OK to just add a fragmentary note about a change. For |
| 26 |
example: "XXX Describe the transmogrify() function added to the |
| 27 |
socket module." The maintainer will research the change and |
| 28 |
write the necessary text. |
| 29 |
|
| 30 |
* You can comment out your additions if you like, but it's not |
| 31 |
necessary (especially when a final release is some months away). |
| 32 |
|
| 33 |
* Credit the author of a patch or bugfix. Just the name is |
| 34 |
sufficient; the e-mail address isn't necessary. |
| 35 |
|
| 36 |
* It's helpful to add the bug/patch number as a comment: |
| 37 |
|
| 38 |
XXX Describe the transmogrify() function added to the socket |
| 39 |
module. |
| 40 |
(Contributed by P.Y. Developer in :issue:`12345`.) |
| 41 |
|
| 42 |
This saves the maintainer the effort of going through the Mercurial log |
| 43 |
when researching a change. |
| 44 |
|
| 45 |
This article explains the new features in Python 3.3, compared to 3.2. |
| 46 |
Python 3.3 was released on September 29, 2012. For full details, |
| 47 |
see the `changelog <https://docs.python.org/3.3/whatsnew/changelog.html>`_. |
| 48 |
|
| 49 |
.. seealso:: |
| 50 |
|
| 51 |
:pep:`398` - Python 3.3 Release Schedule |
| 52 |
|
| 53 |
|
| 54 |
Summary -- Release highlights |
| 55 |
============================= |
| 56 |
|
| 57 |
.. This section singles out the most important changes in Python 3.3. |
| 58 |
Brevity is key. |
| 59 |
|
| 60 |
New syntax features: |
| 61 |
|
| 62 |
* New ``yield from`` expression for :ref:`generator delegation <pep-380>`. |
| 63 |
* The ``u'unicode'`` syntax is accepted again for :class:`str` objects. |
| 64 |
|
| 65 |
New library modules: |
| 66 |
|
| 67 |
* :mod:`faulthandler` (helps debugging low-level crashes) |
| 68 |
* :mod:`ipaddress` (high-level objects representing IP addresses and masks) |
| 69 |
* :mod:`lzma` (compress data using the XZ / LZMA algorithm) |
| 70 |
* :mod:`unittest.mock` (replace parts of your system under test with mock objects) |
| 71 |
* :mod:`venv` (Python :ref:`virtual environments <pep-405>`, as in the |
| 72 |
popular ``virtualenv`` package) |
| 73 |
|
| 74 |
New built-in features: |
| 75 |
|
| 76 |
* Reworked :ref:`I/O exception hierarchy <pep-3151>`. |
| 77 |
|
| 78 |
Implementation improvements: |
| 79 |
|
| 80 |
* Rewritten :ref:`import machinery <importlib>` based on :mod:`importlib`. |
| 81 |
* More compact :ref:`unicode strings <pep-393>`. |
| 82 |
* More compact :ref:`attribute dictionaries <pep-412>`. |
| 83 |
|
| 84 |
Significantly Improved Library Modules: |
| 85 |
|
| 86 |
* C Accelerator for the :ref:`decimal <new-decimal>` module. |
| 87 |
* Better unicode handling in the :ref:`email <new-email>` module |
| 88 |
(:term:`provisional <provisional package>`). |
| 89 |
|
| 90 |
Security improvements: |
| 91 |
|
| 92 |
* Hash randomization is switched on by default. |
| 93 |
|
| 94 |
Please read on for a comprehensive list of user-facing changes. |
| 95 |
|
| 96 |
|
| 97 |
.. _pep-405: |
| 98 |
|
| 99 |
PEP 405: Virtual Environments |
| 100 |
============================= |
| 101 |
|
| 102 |
Virtual environments help create separate Python setups while sharing a |
| 103 |
system-wide base install, for ease of maintenance. Virtual environments |
| 104 |
have their own set of private site packages (i.e. locally installed |
| 105 |
libraries), and are optionally segregated from the system-wide site |
| 106 |
packages. Their concept and implementation are inspired by the popular |
| 107 |
``virtualenv`` third-party package, but benefit from tighter integration |
| 108 |
with the interpreter core. |
| 109 |
|
| 110 |
This PEP adds the :mod:`venv` module for programmatic access, and the |
| 111 |
``pyvenv`` script for command-line access and |
| 112 |
administration. The Python interpreter checks for a ``pyvenv.cfg``, |
| 113 |
file whose existence signals the base of a virtual environment's directory |
| 114 |
tree. |
| 115 |
|
| 116 |
.. seealso:: |
| 117 |
|
| 118 |
:pep:`405` - Python Virtual Environments |
| 119 |
PEP written by Carl Meyer; implementation by Carl Meyer and Vinay Sajip |
| 120 |
|
| 121 |
|
| 122 |
PEP 420: Implicit Namespace Packages |
| 123 |
==================================== |
| 124 |
|
| 125 |
Native support for package directories that don't require ``__init__.py`` |
| 126 |
marker files and can automatically span multiple path segments (inspired by |
| 127 |
various third party approaches to namespace packages, as described in |
| 128 |
:pep:`420`) |
| 129 |
|
| 130 |
.. seealso:: |
| 131 |
|
| 132 |
:pep:`420` - Implicit Namespace Packages |
| 133 |
PEP written by Eric V. Smith; implementation by Eric V. Smith |
| 134 |
and Barry Warsaw |
| 135 |
|
| 136 |
|
| 137 |
.. _pep-3118-update: |
| 138 |
|
| 139 |
PEP 3118: New memoryview implementation and buffer protocol documentation |
| 140 |
========================================================================= |
| 141 |
|
| 142 |
The implementation of :pep:`3118` has been significantly improved. |
| 143 |
|
| 144 |
The new memoryview implementation comprehensively fixes all ownership and |
| 145 |
lifetime issues of dynamically allocated fields in the Py_buffer struct |
| 146 |
that led to multiple crash reports. Additionally, several functions that |
| 147 |
crashed or returned incorrect results for non-contiguous or multi-dimensional |
| 148 |
input have been fixed. |
| 149 |
|
| 150 |
The memoryview object now has a PEP-3118 compliant getbufferproc() |
| 151 |
that checks the consumer's request type. Many new features have been |
| 152 |
added, most of them work in full generality for non-contiguous arrays |
| 153 |
and arrays with suboffsets. |
| 154 |
|
| 155 |
The documentation has been updated, clearly spelling out responsibilities |
| 156 |
for both exporters and consumers. Buffer request flags are grouped into |
| 157 |
basic and compound flags. The memory layout of non-contiguous and |
| 158 |
multi-dimensional NumPy-style arrays is explained. |
| 159 |
|
| 160 |
Features |
| 161 |
-------- |
| 162 |
|
| 163 |
* All native single character format specifiers in struct module syntax |
| 164 |
(optionally prefixed with '@') are now supported. |
| 165 |
|
| 166 |
* With some restrictions, the cast() method allows changing of format and |
| 167 |
shape of C-contiguous arrays. |
| 168 |
|
| 169 |
* Multi-dimensional list representations are supported for any array type. |
| 170 |
|
| 171 |
* Multi-dimensional comparisons are supported for any array type. |
| 172 |
|
| 173 |
* One-dimensional memoryviews of hashable (read-only) types with formats B, |
| 174 |
b or c are now hashable. (Contributed by Antoine Pitrou in :issue:`13411`.) |
| 175 |
|
| 176 |
* Arbitrary slicing of any 1-D arrays type is supported. For example, it |
| 177 |
is now possible to reverse a memoryview in *O*\ (1) by using a negative step. |
| 178 |
|
| 179 |
API changes |
| 180 |
----------- |
| 181 |
|
| 182 |
* The maximum number of dimensions is officially limited to 64. |
| 183 |
|
| 184 |
* The representation of empty shape, strides and suboffsets is now |
| 185 |
an empty tuple instead of ``None``. |
| 186 |
|
| 187 |
* Accessing a memoryview element with format 'B' (unsigned bytes) |
| 188 |
now returns an integer (in accordance with the struct module syntax). |
| 189 |
For returning a bytes object the view must be cast to 'c' first. |
| 190 |
|
| 191 |
* memoryview comparisons now use the logical structure of the operands |
| 192 |
and compare all array elements by value. All format strings in struct |
| 193 |
module syntax are supported. Views with unrecognised format strings |
| 194 |
are still permitted, but will always compare as unequal, regardless |
| 195 |
of view contents. |
| 196 |
|
| 197 |
* For further changes see `Build and C API Changes`_ and `Porting C code`_. |
| 198 |
|
| 199 |
(Contributed by Stefan Krah in :issue:`10181`.) |
| 200 |
|
| 201 |
.. seealso:: |
| 202 |
|
| 203 |
:pep:`3118` - Revising the Buffer Protocol |
| 204 |
|
| 205 |
|
| 206 |
.. _pep-393: |
| 207 |
|
| 208 |
PEP 393: Flexible String Representation |
| 209 |
======================================= |
| 210 |
|
| 211 |
The Unicode string type is changed to support multiple internal |
| 212 |
representations, depending on the character with the largest Unicode ordinal |
| 213 |
(1, 2, or 4 bytes) in the represented string. This allows a space-efficient |
| 214 |
representation in common cases, but gives access to full UCS-4 on all |
| 215 |
systems. For compatibility with existing APIs, several representations may |
| 216 |
exist in parallel; over time, this compatibility should be phased out. |
| 217 |
|
| 218 |
On the Python side, there should be no downside to this change. |
| 219 |
|
| 220 |
On the C API side, :pep:`393` is fully backward compatible. The legacy API |
| 221 |
should remain available at least five years. Applications using the legacy |
| 222 |
API will not fully benefit of the memory reduction, or - worse - may use |
| 223 |
a bit more memory, because Python may have to maintain two versions of each |
| 224 |
string (in the legacy format and in the new efficient storage). |
| 225 |
|
| 226 |
Functionality |
| 227 |
------------- |
| 228 |
|
| 229 |
Changes introduced by :pep:`393` are the following: |
| 230 |
|
| 231 |
* Python now always supports the full range of Unicode code points, including |
| 232 |
non-BMP ones (i.e. from ``U+0000`` to ``U+10FFFF``). The distinction between |
| 233 |
narrow and wide builds no longer exists and Python now behaves like a wide |
| 234 |
build, even under Windows. |
| 235 |
|
| 236 |
* With the death of narrow builds, the problems specific to narrow builds have |
| 237 |
also been fixed, for example: |
| 238 |
|
| 239 |
* :func:`len` now always returns 1 for non-BMP characters, |
| 240 |
so ``len('\U0010FFFF') == 1``; |
| 241 |
|
| 242 |
* surrogate pairs are not recombined in string literals, |
| 243 |
so ``'\uDBFF\uDFFF' != '\U0010FFFF'``; |
| 244 |
|
| 245 |
* indexing or slicing non-BMP characters returns the expected value, |
| 246 |
so ``'\U0010FFFF'[0]`` now returns ``'\U0010FFFF'`` and not ``'\uDBFF'``; |
| 247 |
|
| 248 |
* all other functions in the standard library now correctly handle |
| 249 |
non-BMP code points. |
| 250 |
|
| 251 |
* The value of :data:`sys.maxunicode` is now always ``1114111`` (``0x10FFFF`` |
| 252 |
in hexadecimal). The :c:func:`!PyUnicode_GetMax` function still returns |
| 253 |
either ``0xFFFF`` or ``0x10FFFF`` for backward compatibility, and it should |
| 254 |
not be used with the new Unicode API (see :issue:`13054`). |
| 255 |
|
| 256 |
* The :file:`./configure` flag ``--with-wide-unicode`` has been removed. |
| 257 |
|
| 258 |
Performance and resource usage |
| 259 |
------------------------------ |
| 260 |
|
| 261 |
The storage of Unicode strings now depends on the highest code point in the string: |
| 262 |
|
| 263 |
* pure ASCII and Latin1 strings (``U+0000-U+00FF``) use 1 byte per code point; |
| 264 |
|
| 265 |
* BMP strings (``U+0000-U+FFFF``) use 2 bytes per code point; |
| 266 |
|
| 267 |
* non-BMP strings (``U+10000-U+10FFFF``) use 4 bytes per code point. |
| 268 |
|
| 269 |
The net effect is that for most applications, memory usage of string |
| 270 |
storage should decrease significantly - especially compared to former |
| 271 |
wide unicode builds - as, in many cases, strings will be pure ASCII |
| 272 |
even in international contexts (because many strings store non-human |
| 273 |
language data, such as XML fragments, HTTP headers, JSON-encoded data, |
| 274 |
etc.). We also hope that it will, for the same reasons, increase CPU |
| 275 |
cache efficiency on non-trivial applications. The memory usage of |
| 276 |
Python 3.3 is two to three times smaller than Python 3.2, and a little |
| 277 |
bit better than Python 2.7, on a Django benchmark (see the PEP for |
| 278 |
details). |
| 279 |
|
| 280 |
.. seealso:: |
| 281 |
|
| 282 |
:pep:`393` - Flexible String Representation |
| 283 |
PEP written by Martin von Löwis; implementation by Torsten Becker |
| 284 |
and Martin von Löwis. |
| 285 |
|
| 286 |
|
| 287 |
.. _pep-397: |
| 288 |
|
| 289 |
PEP 397: Python Launcher for Windows |
| 290 |
==================================== |
| 291 |
|
| 292 |
The Python 3.3 Windows installer now includes a ``py`` launcher application |
| 293 |
that can be used to launch Python applications in a version independent |
| 294 |
fashion. |
| 295 |
|
| 296 |
This launcher is invoked implicitly when double-clicking ``*.py`` files. |
| 297 |
If only a single Python version is installed on the system, that version |
| 298 |
will be used to run the file. If multiple versions are installed, the most |
| 299 |
recent version is used by default, but this can be overridden by including |
| 300 |
a Unix-style "shebang line" in the Python script. |
| 301 |
|
| 302 |
The launcher can also be used explicitly from the command line as the ``py`` |
| 303 |
application. Running ``py`` follows the same version selection rules as |
| 304 |
implicitly launching scripts, but a more specific version can be selected |
| 305 |
by passing appropriate arguments (such as ``-3`` to request Python 3 when |
| 306 |
Python 2 is also installed, or ``-2.6`` to specifically request an earlier |
| 307 |
Python version when a more recent version is installed). |
| 308 |
|
| 309 |
In addition to the launcher, the Windows installer now includes an |
| 310 |
option to add the newly installed Python to the system PATH. (Contributed |
| 311 |
by Brian Curtin in :issue:`3561`.) |
| 312 |
|
| 313 |
.. seealso:: |
| 314 |
|
| 315 |
:pep:`397` - Python Launcher for Windows |
| 316 |
PEP written by Mark Hammond and Martin v. Löwis; implementation by |
| 317 |
Vinay Sajip. |
| 318 |
|
| 319 |
Launcher documentation: :ref:`launcher` |
| 320 |
|
| 321 |
Installer PATH modification: :ref:`windows-path-mod` |
| 322 |
|
| 323 |
|
| 324 |
.. _pep-3151: |
| 325 |
|
| 326 |
PEP 3151: Reworking the OS and IO exception hierarchy |
| 327 |
===================================================== |
| 328 |
|
| 329 |
The hierarchy of exceptions raised by operating system errors is now both |
| 330 |
simplified and finer-grained. |
| 331 |
|
| 332 |
You don't have to worry anymore about choosing the appropriate exception |
| 333 |
type between :exc:`OSError`, :exc:`IOError`, :exc:`EnvironmentError`, |
| 334 |
:exc:`WindowsError`, :exc:`!mmap.error`, :exc:`socket.error` or |
| 335 |
:exc:`select.error`. All these exception types are now only one: |
| 336 |
:exc:`OSError`. The other names are kept as aliases for compatibility |
| 337 |
reasons. |
| 338 |
|
| 339 |
Also, it is now easier to catch a specific error condition. Instead of |
| 340 |
inspecting the ``errno`` attribute (or ``args[0]``) for a particular |
| 341 |
constant from the :mod:`errno` module, you can catch the adequate |
| 342 |
:exc:`OSError` subclass. The available subclasses are the following: |
| 343 |
|
| 344 |
* :exc:`BlockingIOError` |
| 345 |
* :exc:`ChildProcessError` |
| 346 |
* :exc:`ConnectionError` |
| 347 |
* :exc:`FileExistsError` |
| 348 |
* :exc:`FileNotFoundError` |
| 349 |
* :exc:`InterruptedError` |
| 350 |
* :exc:`IsADirectoryError` |
| 351 |
* :exc:`NotADirectoryError` |
| 352 |
* :exc:`PermissionError` |
| 353 |
* :exc:`ProcessLookupError` |
| 354 |
* :exc:`TimeoutError` |
| 355 |
|
| 356 |
And the :exc:`ConnectionError` itself has finer-grained subclasses: |
| 357 |
|
| 358 |
* :exc:`BrokenPipeError` |
| 359 |
* :exc:`ConnectionAbortedError` |
| 360 |
* :exc:`ConnectionRefusedError` |
| 361 |
* :exc:`ConnectionResetError` |
| 362 |
|
| 363 |
Thanks to the new exceptions, common usages of the :mod:`errno` can now be |
| 364 |
avoided. For example, the following code written for Python 3.2:: |
| 365 |
|
| 366 |
from errno import ENOENT, EACCES, EPERM |
| 367 |
|
| 368 |
try: |
| 369 |
with open("document.txt") as f: |
| 370 |
content = f.read() |
| 371 |
except IOError as err: |
| 372 |
if err.errno == ENOENT: |
| 373 |
print("document.txt file is missing") |
| 374 |
elif err.errno in (EACCES, EPERM): |
| 375 |
print("You are not allowed to read document.txt") |
| 376 |
else: |
| 377 |
raise |
| 378 |
|
| 379 |
can now be written without the :mod:`errno` import and without manual |
| 380 |
inspection of exception attributes:: |
| 381 |
|
| 382 |
try: |
| 383 |
with open("document.txt") as f: |
| 384 |
content = f.read() |
| 385 |
except FileNotFoundError: |
| 386 |
print("document.txt file is missing") |
| 387 |
except PermissionError: |
| 388 |
print("You are not allowed to read document.txt") |
| 389 |
|
| 390 |
.. seealso:: |
| 391 |
|
| 392 |
:pep:`3151` - Reworking the OS and IO Exception Hierarchy |
| 393 |
PEP written and implemented by Antoine Pitrou |
| 394 |
|
| 395 |
|
| 396 |
.. index:: |
| 397 |
single: yield; yield from (in What's New) |
| 398 |
|
| 399 |
.. _pep-380: |
| 400 |
|
| 401 |
PEP 380: Syntax for Delegating to a Subgenerator |
| 402 |
================================================ |
| 403 |
|
| 404 |
PEP 380 adds the ``yield from`` expression, allowing a :term:`generator` to |
| 405 |
delegate |
| 406 |
part of its operations to another generator. This allows a section of code |
| 407 |
containing :keyword:`yield` to be factored out and placed in another generator. |
| 408 |
Additionally, the subgenerator is allowed to return with a value, and the |
| 409 |
value is made available to the delegating generator. |
| 410 |
|
| 411 |
While designed primarily for use in delegating to a subgenerator, the ``yield |
| 412 |
from`` expression actually allows delegation to arbitrary subiterators. |
| 413 |
|
| 414 |
For simple iterators, ``yield from iterable`` is essentially just a shortened |
| 415 |
form of ``for item in iterable: yield item``:: |
| 416 |
|
| 417 |
>>> def g(x): |
| 418 |
... yield from range(x, 0, -1) |
| 419 |
... yield from range(x) |
| 420 |
... |
| 421 |
>>> list(g(5)) |
| 422 |
[5, 4, 3, 2, 1, 0, 1, 2, 3, 4] |
| 423 |
|
| 424 |
However, unlike an ordinary loop, ``yield from`` allows subgenerators to |
| 425 |
receive sent and thrown values directly from the calling scope, and |
| 426 |
return a final value to the outer generator:: |
| 427 |
|
| 428 |
>>> def accumulate(): |
| 429 |
... tally = 0 |
| 430 |
... while 1: |
| 431 |
... next = yield |
| 432 |
... if next is None: |
| 433 |
... return tally |
| 434 |
... tally += next |
| 435 |
... |
| 436 |
>>> def gather_tallies(tallies): |
| 437 |
... while 1: |
| 438 |
... tally = yield from accumulate() |
| 439 |
... tallies.append(tally) |
| 440 |
... |
| 441 |
>>> tallies = [] |
| 442 |
>>> acc = gather_tallies(tallies) |
| 443 |
>>> next(acc) # Ensure the accumulator is ready to accept values |
| 444 |
>>> for i in range(4): |
| 445 |
... acc.send(i) |
| 446 |
... |
| 447 |
>>> acc.send(None) # Finish the first tally |
| 448 |
>>> for i in range(5): |
| 449 |
... acc.send(i) |
| 450 |
... |
| 451 |
>>> acc.send(None) # Finish the second tally |
| 452 |
>>> tallies |
| 453 |
[6, 10] |
| 454 |
|
| 455 |
The main principle driving this change is to allow even generators that are |
| 456 |
designed to be used with the ``send`` and ``throw`` methods to be split into |
| 457 |
multiple subgenerators as easily as a single large function can be split into |
| 458 |
multiple subfunctions. |
| 459 |
|
| 460 |
.. seealso:: |
| 461 |
|
| 462 |
:pep:`380` - Syntax for Delegating to a Subgenerator |
| 463 |
PEP written by Greg Ewing; implementation by Greg Ewing, integrated into |
| 464 |
3.3 by Renaud Blanch, Ryan Kelly and Nick Coghlan; documentation by |
| 465 |
Zbigniew Jędrzejewski-Szmek and Nick Coghlan |
| 466 |
|
| 467 |
|
| 468 |
PEP 409: Suppressing exception context |
| 469 |
====================================== |
| 470 |
|
| 471 |
PEP 409 introduces new syntax that allows the display of the chained |
| 472 |
exception context to be disabled. This allows cleaner error messages in |
| 473 |
applications that convert between exception types:: |
| 474 |
|
| 475 |
>>> class D: |
| 476 |
... def __init__(self, extra): |
| 477 |
... self._extra_attributes = extra |
| 478 |
... def __getattr__(self, attr): |
| 479 |
... try: |
| 480 |
... return self._extra_attributes[attr] |
| 481 |
... except KeyError: |
| 482 |
... raise AttributeError(attr) from None |
| 483 |
... |
| 484 |
>>> D({}).x |
| 485 |
Traceback (most recent call last): |
| 486 |
File "<stdin>", line 1, in <module> |
| 487 |
File "<stdin>", line 8, in __getattr__ |
| 488 |
AttributeError: x |
| 489 |
|
| 490 |
Without the ``from None`` suffix to suppress the cause, the original |
| 491 |
exception would be displayed by default:: |
| 492 |
|
| 493 |
>>> class C: |
| 494 |
... def __init__(self, extra): |
| 495 |
... self._extra_attributes = extra |
| 496 |
... def __getattr__(self, attr): |
| 497 |
... try: |
| 498 |
... return self._extra_attributes[attr] |
| 499 |
... except KeyError: |
| 500 |
... raise AttributeError(attr) |
| 501 |
... |
| 502 |
>>> C({}).x |
| 503 |
Traceback (most recent call last): |
| 504 |
File "<stdin>", line 6, in __getattr__ |
| 505 |
KeyError: 'x' |
| 506 |
|
| 507 |
During handling of the above exception, another exception occurred: |
| 508 |
|
| 509 |
Traceback (most recent call last): |
| 510 |
File "<stdin>", line 1, in <module> |
| 511 |
File "<stdin>", line 8, in __getattr__ |
| 512 |
AttributeError: x |
| 513 |
|
| 514 |
No debugging capability is lost, as the original exception context remains |
| 515 |
available if needed (for example, if an intervening library has incorrectly |
| 516 |
suppressed valuable underlying details):: |
| 517 |
|
| 518 |
>>> try: |
| 519 |
... D({}).x |
| 520 |
... except AttributeError as exc: |
| 521 |
... print(repr(exc.__context__)) |
| 522 |
... |
| 523 |
KeyError('x',) |
| 524 |
|
| 525 |
.. seealso:: |
| 526 |
|
| 527 |
:pep:`409` - Suppressing exception context |
| 528 |
PEP written by Ethan Furman; implemented by Ethan Furman and Nick |
| 529 |
Coghlan. |
| 530 |
|
| 531 |
|
| 532 |
PEP 414: Explicit Unicode literals |
| 533 |
====================================== |
| 534 |
|
| 535 |
To ease the transition from Python 2 for Unicode aware Python applications |
| 536 |
that make heavy use of Unicode literals, Python 3.3 once again supports the |
| 537 |
"``u``" prefix for string literals. This prefix has no semantic significance |
| 538 |
in Python 3, it is provided solely to reduce the number of purely mechanical |
| 539 |
changes in migrating to Python 3, making it easier for developers to focus on |
| 540 |
the more significant semantic changes (such as the stricter default |
| 541 |
separation of binary and text data). |
| 542 |
|
| 543 |
.. seealso:: |
| 544 |
|
| 545 |
:pep:`414` - Explicit Unicode literals |
| 546 |
PEP written by Armin Ronacher. |
| 547 |
|
| 548 |
|
| 549 |
PEP 3155: Qualified name for classes and functions |
| 550 |
================================================== |
| 551 |
|
| 552 |
Functions and class objects have a new :attr:`~definition.__qualname__` |
| 553 |
attribute representing |
| 554 |
the "path" from the module top-level to their definition. For global functions |
| 555 |
and classes, this is the same as :attr:`~definition.__name__`. |
| 556 |
For other functions and classes, |
| 557 |
it provides better information about where they were actually defined, and |
| 558 |
how they might be accessible from the global scope. |
| 559 |
|
| 560 |
Example with (non-bound) methods:: |
| 561 |
|
| 562 |
>>> class C: |
| 563 |
... def meth(self): |
| 564 |
... pass |
| 565 |
... |
| 566 |
>>> C.meth.__name__ |
| 567 |
'meth' |
| 568 |
>>> C.meth.__qualname__ |
| 569 |
'C.meth' |
| 570 |
|
| 571 |
Example with nested classes:: |
| 572 |
|
| 573 |
>>> class C: |
| 574 |
... class D: |
| 575 |
... def meth(self): |
| 576 |
... pass |
| 577 |
... |
| 578 |
>>> C.D.__name__ |
| 579 |
'D' |
| 580 |
>>> C.D.__qualname__ |
| 581 |
'C.D' |
| 582 |
>>> C.D.meth.__name__ |
| 583 |
'meth' |
| 584 |
>>> C.D.meth.__qualname__ |
| 585 |
'C.D.meth' |
| 586 |
|
| 587 |
Example with nested functions:: |
| 588 |
|
| 589 |
>>> def outer(): |
| 590 |
... def inner(): |
| 591 |
... pass |
| 592 |
... return inner |
| 593 |
... |
| 594 |
>>> outer().__name__ |
| 595 |
'inner' |
| 596 |
>>> outer().__qualname__ |
| 597 |
'outer.<locals>.inner' |
| 598 |
|
| 599 |
The string representation of those objects is also changed to include the |
| 600 |
new, more precise information:: |
| 601 |
|
| 602 |
>>> str(C.D) |
| 603 |
"<class '__main__.C.D'>" |
| 604 |
>>> str(C.D.meth) |
| 605 |
'<function C.D.meth at 0x7f46b9fe31e0>' |
| 606 |
|
| 607 |
.. seealso:: |
| 608 |
|
| 609 |
:pep:`3155` - Qualified name for classes and functions |
| 610 |
PEP written and implemented by Antoine Pitrou. |
| 611 |
|
| 612 |
|
| 613 |
.. _pep-412: |
| 614 |
|
| 615 |
PEP 412: Key-Sharing Dictionary |
| 616 |
=============================== |
| 617 |
|
| 618 |
Dictionaries used for the storage of objects' attributes are now able to |
| 619 |
share part of their internal storage between each other (namely, the part |
| 620 |
which stores the keys and their respective hashes). This reduces the memory |
| 621 |
consumption of programs creating many instances of non-builtin types. |
| 622 |
|
| 623 |
.. seealso:: |
| 624 |
|
| 625 |
:pep:`412` - Key-Sharing Dictionary |
| 626 |
PEP written and implemented by Mark Shannon. |
| 627 |
|
| 628 |
|
| 629 |
PEP 362: Function Signature Object |
| 630 |
================================== |
| 631 |
|
| 632 |
A new function :func:`inspect.signature` makes introspection of python |
| 633 |
callables easy and straightforward. A broad range of callables is supported: |
| 634 |
python functions, decorated or not, classes, and :func:`functools.partial` |
| 635 |
objects. New classes :class:`inspect.Signature`, :class:`inspect.Parameter` |
| 636 |
and :class:`inspect.BoundArguments` hold information about the call signatures, |
| 637 |
such as, annotations, default values, parameters kinds, and bound arguments, |
| 638 |
which considerably simplifies writing decorators and any code that validates |
| 639 |
or amends calling signatures or arguments. |
| 640 |
|
| 641 |
.. seealso:: |
| 642 |
|
| 643 |
:pep:`362`: - Function Signature Object |
| 644 |
PEP written by Brett Cannon, Yury Selivanov, Larry Hastings, Jiwon Seo; |
| 645 |
implemented by Yury Selivanov. |
| 646 |
|
| 647 |
|
| 648 |
PEP 421: Adding sys.implementation |
| 649 |
================================== |
| 650 |
|
| 651 |
A new attribute on the :mod:`sys` module exposes details specific to the |
| 652 |
implementation of the currently running interpreter. The initial set of |
| 653 |
attributes on :data:`sys.implementation` are ``name``, ``version``, |
| 654 |
``hexversion``, and ``cache_tag``. |
| 655 |
|
| 656 |
The intention of ``sys.implementation`` is to consolidate into one namespace |
| 657 |
the implementation-specific data used by the standard library. This allows |
| 658 |
different Python implementations to share a single standard library code base |
| 659 |
much more easily. In its initial state, ``sys.implementation`` holds only a |
| 660 |
small portion of the implementation-specific data. Over time that ratio will |
| 661 |
shift in order to make the standard library more portable. |
| 662 |
|
| 663 |
One example of improved standard library portability is ``cache_tag``. As of |
| 664 |
Python 3.3, ``sys.implementation.cache_tag`` is used by :mod:`importlib` to |
| 665 |
support :pep:`3147` compliance. Any Python implementation that uses |
| 666 |
``importlib`` for its built-in import system may use ``cache_tag`` to control |
| 667 |
the caching behavior for modules. |
| 668 |
|
| 669 |
SimpleNamespace |
| 670 |
--------------- |
| 671 |
|
| 672 |
The implementation of ``sys.implementation`` also introduces a new type to |
| 673 |
Python: :class:`types.SimpleNamespace`. In contrast to a mapping-based |
| 674 |
namespace, like :class:`dict`, ``SimpleNamespace`` is attribute-based, like |
| 675 |
:class:`object`. However, unlike ``object``, ``SimpleNamespace`` instances |
| 676 |
are writable. This means that you can add, remove, and modify the namespace |
| 677 |
through normal attribute access. |
| 678 |
|
| 679 |
.. seealso:: |
| 680 |
|
| 681 |
:pep:`421` - Adding sys.implementation |
| 682 |
PEP written and implemented by Eric Snow. |
| 683 |
|
| 684 |
|
| 685 |
.. _importlib: |
| 686 |
|
| 687 |
Using importlib as the Implementation of Import |
| 688 |
=============================================== |
| 689 |
:issue:`2377` - Replace __import__ w/ importlib.__import__ |
| 690 |
:issue:`13959` - Re-implement parts of :mod:`!imp` in pure Python |
| 691 |
:issue:`14605` - Make import machinery explicit |
| 692 |
:issue:`14646` - Require loaders set __loader__ and __package__ |
| 693 |
|
| 694 |
The :func:`__import__` function is now powered by :func:`importlib.__import__`. |
| 695 |
This work leads to the completion of "phase 2" of :pep:`302`. There are |
| 696 |
multiple benefits to this change. First, it has allowed for more of the |
| 697 |
machinery powering import to be exposed instead of being implicit and hidden |
| 698 |
within the C code. It also provides a single implementation for all Python VMs |
| 699 |
supporting Python 3.3 to use, helping to end any VM-specific deviations in |
| 700 |
import semantics. And finally it eases the maintenance of import, allowing for |
| 701 |
future growth to occur. |
| 702 |
|
| 703 |
For the common user, there should be no visible change in semantics. For |
| 704 |
those whose code currently manipulates import or calls import |
| 705 |
programmatically, the code changes that might possibly be required are covered |
| 706 |
in the `Porting Python code`_ section of this document. |
| 707 |
|
| 708 |
New APIs |
| 709 |
-------- |
| 710 |
One of the large benefits of this work is the exposure of what goes into |
| 711 |
making the import statement work. That means the various importers that were |
| 712 |
once implicit are now fully exposed as part of the :mod:`importlib` package. |
| 713 |
|
| 714 |
The abstract base classes defined in :mod:`importlib.abc` have been expanded |
| 715 |
to properly delineate between :term:`meta path finders <meta path finder>` |
| 716 |
and :term:`path entry finders <path entry finder>` by introducing |
| 717 |
:class:`importlib.abc.MetaPathFinder` and |
| 718 |
:class:`importlib.abc.PathEntryFinder`, respectively. The old ABC of |
| 719 |
:class:`!importlib.abc.Finder` is now only provided for backwards-compatibility |
| 720 |
and does not enforce any method requirements. |
| 721 |
|
| 722 |
In terms of finders, :class:`importlib.machinery.FileFinder` exposes the |
| 723 |
mechanism used to search for source and bytecode files of a module. Previously |
| 724 |
this class was an implicit member of :data:`sys.path_hooks`. |
| 725 |
|
| 726 |
For loaders, the new abstract base class :class:`importlib.abc.FileLoader` helps |
| 727 |
write a loader that uses the file system as the storage mechanism for a module's |
| 728 |
code. The loader for source files |
| 729 |
(:class:`importlib.machinery.SourceFileLoader`), sourceless bytecode files |
| 730 |
(:class:`importlib.machinery.SourcelessFileLoader`), and extension modules |
| 731 |
(:class:`importlib.machinery.ExtensionFileLoader`) are now available for |
| 732 |
direct use. |
| 733 |
|
| 734 |
:exc:`ImportError` now has ``name`` and ``path`` attributes which are set when |
| 735 |
there is relevant data to provide. The message for failed imports will also |
| 736 |
provide the full name of the module now instead of just the tail end of the |
| 737 |
module's name. |
| 738 |
|
| 739 |
The :func:`importlib.invalidate_caches` function will now call the method with |
| 740 |
the same name on all finders cached in :data:`sys.path_importer_cache` to help |
| 741 |
clean up any stored state as necessary. |
| 742 |
|
| 743 |
Visible Changes |
| 744 |
--------------- |
| 745 |
|
| 746 |
For potential required changes to code, see the `Porting Python code`_ |
| 747 |
section. |
| 748 |
|
| 749 |
Beyond the expanse of what :mod:`importlib` now exposes, there are other |
| 750 |
visible changes to import. The biggest is that :data:`sys.meta_path` and |
| 751 |
:data:`sys.path_hooks` now store all of the meta path finders and path entry |
| 752 |
hooks used by import. Previously the finders were implicit and hidden within |
| 753 |
the C code of import instead of being directly exposed. This means that one can |
| 754 |
now easily remove or change the order of the various finders to fit one's needs. |
| 755 |
|
| 756 |
Another change is that all modules have a ``__loader__`` attribute, storing the |
| 757 |
loader used to create the module. :pep:`302` has been updated to make this |
| 758 |
attribute mandatory for loaders to implement, so in the future once 3rd-party |
| 759 |
loaders have been updated people will be able to rely on the existence of the |
| 760 |
attribute. Until such time, though, import is setting the module post-load. |
| 761 |
|
| 762 |
Loaders are also now expected to set the ``__package__`` attribute from |
| 763 |
:pep:`366`. Once again, import itself is already setting this on all loaders |
| 764 |
from :mod:`importlib` and import itself is setting the attribute post-load. |
| 765 |
|
| 766 |
``None`` is now inserted into :data:`sys.path_importer_cache` when no finder |
| 767 |
can be found on :data:`sys.path_hooks`. Since :class:`!imp.NullImporter` is not |
| 768 |
directly exposed on :data:`sys.path_hooks` it could no longer be relied upon to |
| 769 |
always be available to use as a value representing no finder found. |
| 770 |
|
| 771 |
All other changes relate to semantic changes which should be taken into |
| 772 |
consideration when updating code for Python 3.3, and thus should be read about |
| 773 |
in the `Porting Python code`_ section of this document. |
| 774 |
|
| 775 |
(Implementation by Brett Cannon) |
| 776 |
|
| 777 |
|
| 778 |
Other Language Changes |
| 779 |
====================== |
| 780 |
|
| 781 |
Some smaller changes made to the core Python language are: |
| 782 |
|
| 783 |
* Added support for Unicode name aliases and named sequences. |
| 784 |
Both :func:`unicodedata.lookup` and ``'\N{...}'`` now resolve name aliases, |
| 785 |
and :func:`unicodedata.lookup` resolves named sequences too. |
| 786 |
|
| 787 |
(Contributed by Ezio Melotti in :issue:`12753`.) |
| 788 |
|
| 789 |
* Unicode database updated to UCD version 6.1.0 |
| 790 |
|
| 791 |
* Equality comparisons on :func:`range` objects now return a result reflecting |
| 792 |
the equality of the underlying sequences generated by those range objects. |
| 793 |
(:issue:`13201`) |
| 794 |
|
| 795 |
* The ``count()``, ``find()``, ``rfind()``, ``index()`` and ``rindex()`` |
| 796 |
methods of :class:`bytes` and :class:`bytearray` objects now accept an |
| 797 |
integer between 0 and 255 as their first argument. |
| 798 |
|
| 799 |
(Contributed by Petri Lehtinen in :issue:`12170`.) |
| 800 |
|
| 801 |
* The ``rjust()``, ``ljust()``, and ``center()`` methods of :class:`bytes` |
| 802 |
and :class:`bytearray` now accept a :class:`bytearray` for the ``fill`` |
| 803 |
argument. (Contributed by Petri Lehtinen in :issue:`12380`.) |
| 804 |
|
| 805 |
* New methods have been added to :class:`list` and :class:`bytearray`: |
| 806 |
``copy()`` and ``clear()`` (:issue:`10516`). Consequently, |
| 807 |
:class:`~collections.abc.MutableSequence` now also defines a |
| 808 |
:meth:`!clear` method (:issue:`11388`). |
| 809 |
|
| 810 |
* Raw bytes literals can now be written ``rb"..."`` as well as ``br"..."``. |
| 811 |
|
| 812 |
(Contributed by Antoine Pitrou in :issue:`13748`.) |
| 813 |
|
| 814 |
* :meth:`dict.setdefault` now does only one lookup for the given key, making |
| 815 |
it atomic when used with built-in types. |
| 816 |
|
| 817 |
(Contributed by Filip Gruszczyński in :issue:`13521`.) |
| 818 |
|
| 819 |
* The error messages produced when a function call does not match the function |
| 820 |
signature have been significantly improved. |
| 821 |
|
| 822 |
(Contributed by Benjamin Peterson.) |
| 823 |
|
| 824 |
|
| 825 |
A Finer-Grained Import Lock |
| 826 |
=========================== |
| 827 |
|
| 828 |
Previous versions of CPython have always relied on a global import lock. |
| 829 |
This led to unexpected annoyances, such as deadlocks when importing a module |
| 830 |
would trigger code execution in a different thread as a side-effect. |
| 831 |
Clumsy workarounds were sometimes employed, such as the |
| 832 |
:c:func:`PyImport_ImportModuleNoBlock` C API function. |
| 833 |
|
| 834 |
In Python 3.3, importing a module takes a per-module lock. This correctly |
| 835 |
serializes importation of a given module from multiple threads (preventing |
| 836 |
the exposure of incompletely initialized modules), while eliminating the |
| 837 |
aforementioned annoyances. |
| 838 |
|
| 839 |
(Contributed by Antoine Pitrou in :issue:`9260`.) |
| 840 |
|
| 841 |
|
| 842 |
Builtin functions and types |
| 843 |
=========================== |
| 844 |
|
| 845 |
* :func:`open` gets a new *opener* parameter: the underlying file descriptor |
| 846 |
for the file object is then obtained by calling *opener* with (*file*, |
| 847 |
*flags*). It can be used to use custom flags like :const:`os.O_CLOEXEC` for |
| 848 |
example. The ``'x'`` mode was added: open for exclusive creation, failing if |
| 849 |
the file already exists. |
| 850 |
* :func:`print`: added the *flush* keyword argument. If the *flush* keyword |
| 851 |
argument is true, the stream is forcibly flushed. |
| 852 |
* :func:`hash`: hash randomization is enabled by default, see |
| 853 |
:meth:`object.__hash__` and :envvar:`PYTHONHASHSEED`. |
| 854 |
* The :class:`str` type gets a new :meth:`~str.casefold` method: return a |
| 855 |
casefolded copy of the string, casefolded strings may be used for caseless |
| 856 |
matching. For example, ``'ß'.casefold()`` returns ``'ss'``. |
| 857 |
* The sequence documentation has been substantially rewritten to better |
| 858 |
explain the binary/text sequence distinction and to provide specific |
| 859 |
documentation sections for the individual builtin sequence types |
| 860 |
(:issue:`4966`). |
| 861 |
|
| 862 |
|
| 863 |
New Modules |
| 864 |
=========== |
| 865 |
|
| 866 |
faulthandler |
| 867 |
------------ |
| 868 |
|
| 869 |
This new debug module :mod:`faulthandler` contains functions to dump Python tracebacks explicitly, |
| 870 |
on a fault (a crash like a segmentation fault), after a timeout, or on a user |
| 871 |
signal. Call :func:`faulthandler.enable` to install fault handlers for the |
| 872 |
:const:`~signal.SIGSEGV`, :const:`~signal.SIGFPE`, :const:`~signal.SIGABRT`, |
| 873 |
:const:`~signal.SIGBUS`, and :const:`~signal.SIGILL` signals. |
| 874 |
You can also enable them at startup by setting the :envvar:`PYTHONFAULTHANDLER` |
| 875 |
environment variable or by using :option:`-X` ``faulthandler`` command line option. |
| 876 |
|
| 877 |
Example of a segmentation fault on Linux: |
| 878 |
|
| 879 |
.. code-block:: shell-session |
| 880 |
|
| 881 |
$ python -q -X faulthandler |
| 882 |
>>> import ctypes |
| 883 |
>>> ctypes.string_at(0) |
| 884 |
Fatal Python error: Segmentation fault |
| 885 |
|
| 886 |
Current thread 0x00007fb899f39700: |
| 887 |
File "/home/python/cpython/Lib/ctypes/__init__.py", line 486 in string_at |
| 888 |
File "<stdin>", line 1 in <module> |
| 889 |
Segmentation fault |
| 890 |
|
| 891 |
|
| 892 |
ipaddress |
| 893 |
--------- |
| 894 |
|
| 895 |
The new :mod:`ipaddress` module provides tools for creating and manipulating |
| 896 |
objects representing IPv4 and IPv6 addresses, networks and interfaces (i.e. |
| 897 |
an IP address associated with a specific IP subnet). |
| 898 |
|
| 899 |
(Contributed by Google and Peter Moody in :pep:`3144`.) |
| 900 |
|
| 901 |
lzma |
| 902 |
---- |
| 903 |
|
| 904 |
The newly added :mod:`lzma` module provides data compression and decompression |
| 905 |
using the LZMA algorithm, including support for the ``.xz`` and ``.lzma`` |
| 906 |
file formats. |
| 907 |
|
| 908 |
(Contributed by Nadeem Vawda and Per Øyvind Karlsen in :issue:`6715`.) |
| 909 |
|
| 910 |
|
| 911 |
Improved Modules |
| 912 |
================ |
| 913 |
|
| 914 |
abc |
| 915 |
--- |
| 916 |
|
| 917 |
Improved support for abstract base classes containing descriptors composed with |
| 918 |
abstract methods. The recommended approach to declaring abstract descriptors is |
| 919 |
now to provide :attr:`!__isabstractmethod__` as a dynamically updated |
| 920 |
property. The built-in descriptors have been updated accordingly. |
| 921 |
|
| 922 |
* :class:`abc.abstractproperty` has been deprecated, use :class:`property` |
| 923 |
with :func:`abc.abstractmethod` instead. |
| 924 |
* :class:`abc.abstractclassmethod` has been deprecated, use |
| 925 |
:class:`classmethod` with :func:`abc.abstractmethod` instead. |
| 926 |
* :class:`abc.abstractstaticmethod` has been deprecated, use |
| 927 |
:class:`staticmethod` with :func:`abc.abstractmethod` instead. |
| 928 |
|
| 929 |
(Contributed by Darren Dale in :issue:`11610`.) |
| 930 |
|
| 931 |
:meth:`abc.ABCMeta.register` now returns the registered subclass, which means |
| 932 |
it can now be used as a class decorator (:issue:`10868`). |
| 933 |
|
| 934 |
|
| 935 |
array |
| 936 |
----- |
| 937 |
|
| 938 |
The :mod:`array` module supports the :c:expr:`long long` type using ``q`` and |
| 939 |
``Q`` type codes. |
| 940 |
|
| 941 |
(Contributed by Oren Tirosh and Hirokazu Yamamoto in :issue:`1172711`.) |
| 942 |
|
| 943 |
|
| 944 |
base64 |
| 945 |
------ |
| 946 |
|
| 947 |
ASCII-only Unicode strings are now accepted by the decoding functions of the |
| 948 |
:mod:`base64` modern interface. For example, ``base64.b64decode('YWJj')`` |
| 949 |
returns ``b'abc'``. (Contributed by Catalin Iacob in :issue:`13641`.) |
| 950 |
|
| 951 |
|
| 952 |
binascii |
| 953 |
-------- |
| 954 |
|
| 955 |
In addition to the binary objects they normally accept, the ``a2b_`` functions |
| 956 |
now all also accept ASCII-only strings as input. (Contributed by Antoine |
| 957 |
Pitrou in :issue:`13637`.) |
| 958 |
|
| 959 |
|
| 960 |
bz2 |
| 961 |
--- |
| 962 |
|
| 963 |
The :mod:`bz2` module has been rewritten from scratch. In the process, several |
| 964 |
new features have been added: |
| 965 |
|
| 966 |
* New :func:`bz2.open` function: open a bzip2-compressed file in binary or |
| 967 |
text mode. |
| 968 |
|
| 969 |
* :class:`bz2.BZ2File` can now read from and write to arbitrary file-like |
| 970 |
objects, by means of its constructor's *fileobj* argument. |
| 971 |
|
| 972 |
(Contributed by Nadeem Vawda in :issue:`5863`.) |
| 973 |
|
| 974 |
* :class:`bz2.BZ2File` and :func:`bz2.decompress` can now decompress |
| 975 |
multi-stream inputs (such as those produced by the :program:`pbzip2` tool). |
| 976 |
:class:`bz2.BZ2File` can now also be used to create this type of file, using |
| 977 |
the ``'a'`` (append) mode. |
| 978 |
|
| 979 |
(Contributed by Nir Aides in :issue:`1625`.) |
| 980 |
|
| 981 |
* :class:`bz2.BZ2File` now implements all of the :class:`io.BufferedIOBase` API, |
| 982 |
except for the :meth:`!detach` and :meth:`!truncate` methods. |
| 983 |
|
| 984 |
|
| 985 |
codecs |
| 986 |
------ |
| 987 |
|
| 988 |
The :mod:`~encodings.mbcs` codec has been rewritten to handle correctly |
| 989 |
``replace`` and ``ignore`` error handlers on all Windows versions. The |
| 990 |
:mod:`~encodings.mbcs` codec now supports all error handlers, instead of only |
| 991 |
``replace`` to encode and ``ignore`` to decode. |
| 992 |
|
| 993 |
A new Windows-only codec has been added: ``cp65001`` (:issue:`13216`). It is the |
| 994 |
Windows code page 65001 (Windows UTF-8, ``CP_UTF8``). For example, it is used |
| 995 |
by ``sys.stdout`` if the console output code page is set to cp65001 (e.g., using |
| 996 |
``chcp 65001`` command). |
| 997 |
|
| 998 |
Multibyte CJK decoders now resynchronize faster. They only ignore the first |
| 999 |
byte of an invalid byte sequence. For example, ``b'\xff\n'.decode('gb2312', |
| 1000 |
'replace')`` now returns a ``\n`` after the replacement character. |
| 1001 |
|
| 1002 |
(:issue:`12016`) |
| 1003 |
|
| 1004 |
Incremental CJK codec encoders are no longer reset at each call to their |
| 1005 |
encode() methods. For example:: |
| 1006 |
|
| 1007 |
>>> import codecs |
| 1008 |
>>> encoder = codecs.getincrementalencoder('hz')('strict') |
| 1009 |
>>> b''.join(encoder.encode(x) for x in '\u52ff\u65bd\u65bc\u4eba\u3002 Bye.') |
| 1010 |
b'~{NpJ)l6HK!#~} Bye.' |
| 1011 |
|
| 1012 |
This example gives ``b'~{Np~}~{J)~}~{l6~}~{HK~}~{!#~} Bye.'`` with older Python |
| 1013 |
versions. |
| 1014 |
|
| 1015 |
(:issue:`12100`) |
| 1016 |
|
| 1017 |
The ``unicode_internal`` codec has been deprecated. |
| 1018 |
|
| 1019 |
|
| 1020 |
collections |
| 1021 |
----------- |
| 1022 |
|
| 1023 |
Addition of a new :class:`~collections.ChainMap` class to allow treating a |
| 1024 |
number of mappings as a single unit. (Written by Raymond Hettinger for |
| 1025 |
:issue:`11089`, made public in :issue:`11297`.) |
| 1026 |
|
| 1027 |
The abstract base classes have been moved in a new :mod:`collections.abc` |
| 1028 |
module, to better differentiate between the abstract and the concrete |
| 1029 |
collections classes. Aliases for ABCs are still present in the |
| 1030 |
:mod:`collections` module to preserve existing imports. (:issue:`11085`) |
| 1031 |
|
| 1032 |
.. XXX addition of __slots__ to ABCs not recorded here: internal detail |
| 1033 |
|
| 1034 |
The :class:`~collections.Counter` class now supports the unary ``+`` and ``-`` |
| 1035 |
operators, as well as the in-place operators ``+=``, ``-=``, ``|=``, and |
| 1036 |
``&=``. (Contributed by Raymond Hettinger in :issue:`13121`.) |
| 1037 |
|
| 1038 |
|
| 1039 |
contextlib |
| 1040 |
---------- |
| 1041 |
|
| 1042 |
:class:`~contextlib.ExitStack` now provides a solid foundation for |
| 1043 |
programmatic manipulation of context managers and similar cleanup |
| 1044 |
functionality. Unlike the previous ``contextlib.nested`` API (which was |
| 1045 |
deprecated and removed), the new API is designed to work correctly |
| 1046 |
regardless of whether context managers acquire their resources in |
| 1047 |
their ``__init__`` method (for example, file objects) or in their |
| 1048 |
``__enter__`` method (for example, synchronisation objects from the |
| 1049 |
:mod:`threading` module). |
| 1050 |
|
| 1051 |
(:issue:`13585`) |
| 1052 |
|
| 1053 |
|
| 1054 |
crypt |
| 1055 |
----- |
| 1056 |
|
| 1057 |
Addition of salt and modular crypt format (hashing method) and the :func:`!mksalt` |
| 1058 |
function to the :mod:`!crypt` module. |
| 1059 |
|
| 1060 |
(:issue:`10924`) |
| 1061 |
|
| 1062 |
curses |
| 1063 |
------ |
| 1064 |
|
| 1065 |
* If the :mod:`curses` module is linked to the ncursesw library, use Unicode |
| 1066 |
functions when Unicode strings or characters are passed (e.g. |
| 1067 |
:c:func:`!waddwstr`), and bytes functions otherwise (e.g. :c:func:`!waddstr`). |
| 1068 |
* Use the locale encoding instead of ``utf-8`` to encode Unicode strings. |
| 1069 |
* :class:`curses.window` has a new :attr:`curses.window.encoding` attribute. |
| 1070 |
* The :class:`curses.window` class has a new :meth:`~curses.window.get_wch` |
| 1071 |
method to get a wide character |
| 1072 |
* The :mod:`curses` module has a new :meth:`~curses.unget_wch` function to |
| 1073 |
push a wide character so the next :meth:`~curses.window.get_wch` will return |
| 1074 |
it |
| 1075 |
|
| 1076 |
(Contributed by Iñigo Serna in :issue:`6755`.) |
| 1077 |
|
| 1078 |
datetime |
| 1079 |
-------- |
| 1080 |
|
| 1081 |
* Equality comparisons between naive and aware :class:`~datetime.datetime` |
| 1082 |
instances now return :const:`False` instead of raising :exc:`TypeError` |
| 1083 |
(:issue:`15006`). |
| 1084 |
* New :meth:`datetime.datetime.timestamp` method: Return POSIX timestamp |
| 1085 |
corresponding to the :class:`~datetime.datetime` instance. |
| 1086 |
* The :meth:`datetime.datetime.strftime` method supports formatting years |
| 1087 |
older than 1000. |
| 1088 |
* The :meth:`datetime.datetime.astimezone` method can now be |
| 1089 |
called without arguments to convert datetime instance to the system |
| 1090 |
timezone. |
| 1091 |
|
| 1092 |
|
| 1093 |
.. _new-decimal: |
| 1094 |
|
| 1095 |
decimal |
| 1096 |
------- |
| 1097 |
|
| 1098 |
:issue:`7652` - integrate fast native decimal arithmetic. |
| 1099 |
C-module and libmpdec written by Stefan Krah. |
| 1100 |
|
| 1101 |
The new C version of the decimal module integrates the high speed libmpdec |
| 1102 |
library for arbitrary precision correctly rounded decimal floating-point |
| 1103 |
arithmetic. libmpdec conforms to IBM's General Decimal Arithmetic Specification. |
| 1104 |
|
| 1105 |
Performance gains range from 10x for database applications to 100x for |
| 1106 |
numerically intensive applications. These numbers are expected gains |
| 1107 |
for standard precisions used in decimal floating-point arithmetic. Since |
| 1108 |
the precision is user configurable, the exact figures may vary. For example, |
| 1109 |
in integer bignum arithmetic the differences can be significantly higher. |
| 1110 |
|
| 1111 |
The following table is meant as an illustration. Benchmarks are available |
| 1112 |
at https://www.bytereef.org/mpdecimal/quickstart.html. |
| 1113 |
|
| 1114 |
+---------+-------------+--------------+-------------+ |
| 1115 |
| | decimal.py | _decimal | speedup | |
| 1116 |
+=========+=============+==============+=============+ |
| 1117 |
| pi | 42.02s | 0.345s | 120x | |
| 1118 |
+---------+-------------+--------------+-------------+ |
| 1119 |
| telco | 172.19s | 5.68s | 30x | |
| 1120 |
+---------+-------------+--------------+-------------+ |
| 1121 |
| psycopg | 3.57s | 0.29s | 12x | |
| 1122 |
+---------+-------------+--------------+-------------+ |
| 1123 |
|
| 1124 |
Features |
| 1125 |
~~~~~~~~ |
| 1126 |
|
| 1127 |
* The :exc:`~decimal.FloatOperation` signal optionally enables stricter |
| 1128 |
semantics for mixing floats and Decimals. |
| 1129 |
|
| 1130 |
* If Python is compiled without threads, the C version automatically |
| 1131 |
disables the expensive thread local context machinery. In this case, |
| 1132 |
the variable :const:`~decimal.HAVE_THREADS` is set to ``False``. |
| 1133 |
|
| 1134 |
API changes |
| 1135 |
~~~~~~~~~~~ |
| 1136 |
|
| 1137 |
* The C module has the following context limits, depending on the machine |
| 1138 |
architecture: |
| 1139 |
|
| 1140 |
+----------------------------+----------------+-------------------------+ |
| 1141 |
| | 32-bit | 64-bit | |
| 1142 |
+============================+================+=========================+ |
| 1143 |
| :const:`~decimal.MAX_PREC` | ``425000000`` | ``999999999999999999`` | |
| 1144 |
+----------------------------+----------------+-------------------------+ |
| 1145 |
| :const:`~decimal.MAX_EMAX` | ``425000000`` | ``999999999999999999`` | |
| 1146 |
+----------------------------+----------------+-------------------------+ |
| 1147 |
| :const:`~decimal.MIN_EMIN` | ``-425000000`` | ``-999999999999999999`` | |
| 1148 |
+----------------------------+----------------+-------------------------+ |
| 1149 |
|
| 1150 |
* In the context templates (:const:`~decimal.DefaultContext`, |
| 1151 |
:const:`~decimal.BasicContext` and :const:`~decimal.ExtendedContext`) |
| 1152 |
the magnitude of :attr:`~decimal.Context.Emax` and |
| 1153 |
:attr:`~decimal.Context.Emin` has changed to ``999999``. |
| 1154 |
|
| 1155 |
* The :class:`~decimal.Decimal` constructor in decimal.py does not observe |
| 1156 |
the context limits and converts values with arbitrary exponents or precision |
| 1157 |
exactly. Since the C version has internal limits, the following scheme is |
| 1158 |
used: If possible, values are converted exactly, otherwise |
| 1159 |
:exc:`~decimal.InvalidOperation` is raised and the result is NaN. In the |
| 1160 |
latter case it is always possible to use :meth:`~decimal.Context.create_decimal` |
| 1161 |
in order to obtain a rounded or inexact value. |
| 1162 |
|
| 1163 |
|
| 1164 |
* The power function in decimal.py is always correctly rounded. In the |
| 1165 |
C version, it is defined in terms of the correctly rounded |
| 1166 |
:meth:`~decimal.Decimal.exp` and :meth:`~decimal.Decimal.ln` functions, |
| 1167 |
but the final result is only "almost always correctly rounded". |
| 1168 |
|
| 1169 |
|
| 1170 |
* In the C version, the context dictionary containing the signals is a |
| 1171 |
:class:`~collections.abc.MutableMapping`. For speed reasons, |
| 1172 |
:attr:`~decimal.Context.flags` and :attr:`~decimal.Context.traps` always |
| 1173 |
refer to the same :class:`~collections.abc.MutableMapping` that the context |
| 1174 |
was initialized with. If a new signal dictionary is assigned, |
| 1175 |
:attr:`~decimal.Context.flags` and :attr:`~decimal.Context.traps` |
| 1176 |
are updated with the new values, but they do not reference the RHS |
| 1177 |
dictionary. |
| 1178 |
|
| 1179 |
|
| 1180 |
* Pickling a :class:`~decimal.Context` produces a different output in order |
| 1181 |
to have a common interchange format for the Python and C versions. |
| 1182 |
|
| 1183 |
|
| 1184 |
* The order of arguments in the :class:`~decimal.Context` constructor has been |
| 1185 |
changed to match the order displayed by :func:`repr`. |
| 1186 |
|
| 1187 |
|
| 1188 |
* The ``watchexp`` parameter in the :meth:`~decimal.Decimal.quantize` method |
| 1189 |
is deprecated. |
| 1190 |
|
| 1191 |
|
| 1192 |
.. _new-email: |
| 1193 |
|
| 1194 |
email |
| 1195 |
----- |
| 1196 |
|
| 1197 |
Policy Framework |
| 1198 |
~~~~~~~~~~~~~~~~ |
| 1199 |
|
| 1200 |
The email package now has a :mod:`~email.policy` framework. A |
| 1201 |
:class:`~email.policy.Policy` is an object with several methods and properties |
| 1202 |
that control how the email package behaves. The primary policy for Python 3.3 |
| 1203 |
is the :class:`~email.policy.Compat32` policy, which provides backward |
| 1204 |
compatibility with the email package in Python 3.2. A ``policy`` can be |
| 1205 |
specified when an email message is parsed by a :mod:`~email.parser`, or when a |
| 1206 |
:class:`~email.message.Message` object is created, or when an email is |
| 1207 |
serialized using a :mod:`~email.generator`. Unless overridden, a policy passed |
| 1208 |
to a ``parser`` is inherited by all the ``Message`` object and sub-objects |
| 1209 |
created by the ``parser``. By default a ``generator`` will use the policy of |
| 1210 |
the ``Message`` object it is serializing. The default policy is |
| 1211 |
:data:`~email.policy.compat32`. |
| 1212 |
|
| 1213 |
The minimum set of controls implemented by all ``policy`` objects are: |
| 1214 |
|
| 1215 |
.. tabularcolumns:: |l|L| |
| 1216 |
|
| 1217 |
=============== ======================================================= |
| 1218 |
max_line_length The maximum length, excluding the linesep character(s), |
| 1219 |
individual lines may have when a ``Message`` is |
| 1220 |
serialized. Defaults to 78. |
| 1221 |
|
| 1222 |
linesep The character used to separate individual lines when a |
| 1223 |
``Message`` is serialized. Defaults to ``\n``. |
| 1224 |
|
| 1225 |
cte_type ``7bit`` or ``8bit``. ``8bit`` applies only to a |
| 1226 |
``Bytes`` ``generator``, and means that non-ASCII may |
| 1227 |
be used where allowed by the protocol (or where it |
| 1228 |
exists in the original input). |
| 1229 |
|
| 1230 |
raise_on_defect Causes a ``parser`` to raise error when defects are |
| 1231 |
encountered instead of adding them to the ``Message`` |
| 1232 |
object's ``defects`` list. |
| 1233 |
=============== ======================================================= |
| 1234 |
|
| 1235 |
A new policy instance, with new settings, is created using the |
| 1236 |
:meth:`~email.policy.Policy.clone` method of policy objects. ``clone`` takes |
| 1237 |
any of the above controls as keyword arguments. Any control not specified in |
| 1238 |
the call retains its default value. Thus you can create a policy that uses |
| 1239 |
``\r\n`` linesep characters like this:: |
| 1240 |
|
| 1241 |
mypolicy = compat32.clone(linesep='\r\n') |
| 1242 |
|
| 1243 |
Policies can be used to make the generation of messages in the format needed by |
| 1244 |
your application simpler. Instead of having to remember to specify |
| 1245 |
``linesep='\r\n'`` in all the places you call a ``generator``, you can specify |
| 1246 |
it once, when you set the policy used by the ``parser`` or the ``Message``, |
| 1247 |
whichever your program uses to create ``Message`` objects. On the other hand, |
| 1248 |
if you need to generate messages in multiple forms, you can still specify the |
| 1249 |
parameters in the appropriate ``generator`` call. Or you can have custom |
| 1250 |
policy instances for your different cases, and pass those in when you create |
| 1251 |
the ``generator``. |
| 1252 |
|
| 1253 |
|
| 1254 |
Provisional Policy with New Header API |
| 1255 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1256 |
|
| 1257 |
While the policy framework is worthwhile all by itself, the main motivation for |
| 1258 |
introducing it is to allow the creation of new policies that implement new |
| 1259 |
features for the email package in a way that maintains backward compatibility |
| 1260 |
for those who do not use the new policies. Because the new policies introduce a |
| 1261 |
new API, we are releasing them in Python 3.3 as a :term:`provisional policy |
| 1262 |
<provisional package>`. Backwards incompatible changes (up to and including |
| 1263 |
removal of the code) may occur if deemed necessary by the core developers. |
| 1264 |
|
| 1265 |
The new policies are instances of :class:`~email.policy.EmailPolicy`, |
| 1266 |
and add the following additional controls: |
| 1267 |
|
| 1268 |
.. tabularcolumns:: |l|L| |
| 1269 |
|
| 1270 |
=============== ======================================================= |
| 1271 |
refold_source Controls whether or not headers parsed by a |
| 1272 |
:mod:`~email.parser` are refolded by the |
| 1273 |
:mod:`~email.generator`. It can be ``none``, ``long``, |
| 1274 |
or ``all``. The default is ``long``, which means that |
| 1275 |
source headers with a line longer than |
| 1276 |
``max_line_length`` get refolded. ``none`` means no |
| 1277 |
line get refolded, and ``all`` means that all lines |
| 1278 |
get refolded. |
| 1279 |
|
| 1280 |
header_factory A callable that take a ``name`` and ``value`` and |
| 1281 |
produces a custom header object. |
| 1282 |
=============== ======================================================= |
| 1283 |
|
| 1284 |
The ``header_factory`` is the key to the new features provided by the new |
| 1285 |
policies. When one of the new policies is used, any header retrieved from |
| 1286 |
a ``Message`` object is an object produced by the ``header_factory``, and any |
| 1287 |
time you set a header on a ``Message`` it becomes an object produced by |
| 1288 |
``header_factory``. All such header objects have a ``name`` attribute equal |
| 1289 |
to the header name. Address and Date headers have additional attributes |
| 1290 |
that give you access to the parsed data of the header. This means you can now |
| 1291 |
do things like this:: |
| 1292 |
|
| 1293 |
>>> m = Message(policy=SMTP) |
| 1294 |
>>> m['To'] = 'Éric <[email protected]>' |
| 1295 |
>>> m['to'] |
| 1296 |
'Éric <[email protected]>' |
| 1297 |
>>> m['to'].addresses |
| 1298 |
(Address(display_name='Éric', username='foo', domain='example.com'),) |
| 1299 |
>>> m['to'].addresses[0].username |
| 1300 |
'foo' |
| 1301 |
>>> m['to'].addresses[0].display_name |
| 1302 |
'Éric' |
| 1303 |
>>> m['Date'] = email.utils.localtime() |
| 1304 |
>>> m['Date'].datetime |
| 1305 |
datetime.datetime(2012, 5, 25, 21, 39, 24, 465484, tzinfo=datetime.timezone(datetime.timedelta(-1, 72000), 'EDT')) |
| 1306 |
>>> m['Date'] |
| 1307 |
'Fri, 25 May 2012 21:44:27 -0400' |
| 1308 |
>>> print(m) |
| 1309 |
To: =?utf-8?q?=C3=89ric?= <[email protected]> |
| 1310 |
Date: Fri, 25 May 2012 21:44:27 -0400 |
| 1311 |
|
| 1312 |
You will note that the unicode display name is automatically encoded as |
| 1313 |
``utf-8`` when the message is serialized, but that when the header is accessed |
| 1314 |
directly, you get the unicode version. This eliminates any need to deal with |
| 1315 |
the :mod:`email.header` :meth:`~email.header.decode_header` or |
| 1316 |
:meth:`~email.header.make_header` functions. |
| 1317 |
|
| 1318 |
You can also create addresses from parts:: |
| 1319 |
|
| 1320 |
>>> m['cc'] = [Group('pals', [Address('Bob', 'bob', 'example.com'), |
| 1321 |
... Address('Sally', 'sally', 'example.com')]), |
| 1322 |
... Address('Bonzo', addr_spec='[email protected]')] |
| 1323 |
>>> print(m) |
| 1324 |
To: =?utf-8?q?=C3=89ric?= <[email protected]> |
| 1325 |
Date: Fri, 25 May 2012 21:44:27 -0400 |
| 1326 |
cc: pals: Bob <[email protected]>, Sally <[email protected]>;, Bonzo <[email protected]> |
| 1327 |
|
| 1328 |
Decoding to unicode is done automatically:: |
| 1329 |
|
| 1330 |
>>> m2 = message_from_string(str(m)) |
| 1331 |
>>> m2['to'] |
| 1332 |
'Éric <[email protected]>' |
| 1333 |
|
| 1334 |
When you parse a message, you can use the ``addresses`` and ``groups`` |
| 1335 |
attributes of the header objects to access the groups and individual |
| 1336 |
addresses:: |
| 1337 |
|
| 1338 |
>>> m2['cc'].addresses |
| 1339 |
(Address(display_name='Bob', username='bob', domain='example.com'), Address(display_name='Sally', username='sally', domain='example.com'), Address(display_name='Bonzo', username='bonz', domain='laugh.com')) |
| 1340 |
>>> m2['cc'].groups |
| 1341 |
(Group(display_name='pals', addresses=(Address(display_name='Bob', username='bob', domain='example.com'), Address(display_name='Sally', username='sally', domain='example.com')), Group(display_name=None, addresses=(Address(display_name='Bonzo', username='bonz', domain='laugh.com'),)) |
| 1342 |
|
| 1343 |
In summary, if you use one of the new policies, header manipulation works the |
| 1344 |
way it ought to: your application works with unicode strings, and the email |
| 1345 |
package transparently encodes and decodes the unicode to and from the RFC |
| 1346 |
standard Content Transfer Encodings. |
| 1347 |
|
| 1348 |
Other API Changes |
| 1349 |
~~~~~~~~~~~~~~~~~ |
| 1350 |
|
| 1351 |
New :class:`~email.parser.BytesHeaderParser`, added to the :mod:`~email.parser` |
| 1352 |
module to complement :class:`~email.parser.HeaderParser` and complete the Bytes |
| 1353 |
API. |
| 1354 |
|
| 1355 |
New utility functions: |
| 1356 |
|
| 1357 |
* :func:`~email.utils.format_datetime`: given a :class:`~datetime.datetime`, |
| 1358 |
produce a string formatted for use in an email header. |
| 1359 |
|
| 1360 |
* :func:`~email.utils.parsedate_to_datetime`: given a date string from |
| 1361 |
an email header, convert it into an aware :class:`~datetime.datetime`, |
| 1362 |
or a naive :class:`~datetime.datetime` if the offset is ``-0000``. |
| 1363 |
|
| 1364 |
* :func:`~email.utils.localtime`: With no argument, returns the |
| 1365 |
current local time as an aware :class:`~datetime.datetime` using the local |
| 1366 |
:class:`~datetime.timezone`. Given an aware :class:`~datetime.datetime`, |
| 1367 |
converts it into an aware :class:`~datetime.datetime` using the |
| 1368 |
local :class:`~datetime.timezone`. |
| 1369 |
|
| 1370 |
|
| 1371 |
ftplib |
| 1372 |
------ |
| 1373 |
|
| 1374 |
* :class:`ftplib.FTP` now accepts a ``source_address`` keyword argument to |
| 1375 |
specify the ``(host, port)`` to use as the source address in the bind call |
| 1376 |
when creating the outgoing socket. (Contributed by Giampaolo Rodolà |
| 1377 |
in :issue:`8594`.) |
| 1378 |
|
| 1379 |
* The :class:`~ftplib.FTP_TLS` class now provides a new |
| 1380 |
:func:`~ftplib.FTP_TLS.ccc` function to revert control channel back to |
| 1381 |
plaintext. This can be useful to take advantage of firewalls that know how |
| 1382 |
to handle NAT with non-secure FTP without opening fixed ports. (Contributed |
| 1383 |
by Giampaolo Rodolà in :issue:`12139`.) |
| 1384 |
|
| 1385 |
* Added :meth:`ftplib.FTP.mlsd` method which provides a parsable directory |
| 1386 |
listing format and deprecates :meth:`ftplib.FTP.nlst` and |
| 1387 |
:meth:`ftplib.FTP.dir`. (Contributed by Giampaolo Rodolà in :issue:`11072`.) |
| 1388 |
|
| 1389 |
|
| 1390 |
functools |
| 1391 |
--------- |
| 1392 |
|
| 1393 |
The :func:`functools.lru_cache` decorator now accepts a ``typed`` keyword |
| 1394 |
argument (that defaults to ``False`` to ensure that it caches values of |
| 1395 |
different types that compare equal in separate cache slots. (Contributed |
| 1396 |
by Raymond Hettinger in :issue:`13227`.) |
| 1397 |
|
| 1398 |
|
| 1399 |
gc |
| 1400 |
-- |
| 1401 |
|
| 1402 |
It is now possible to register callbacks invoked by the garbage collector |
| 1403 |
before and after collection using the new :data:`~gc.callbacks` list. |
| 1404 |
|
| 1405 |
|
| 1406 |
hmac |
| 1407 |
---- |
| 1408 |
|
| 1409 |
A new :func:`~hmac.compare_digest` function has been added to prevent side |
| 1410 |
channel attacks on digests through timing analysis. (Contributed by Nick |
| 1411 |
Coghlan and Christian Heimes in :issue:`15061`.) |
| 1412 |
|
| 1413 |
|
| 1414 |
http |
| 1415 |
---- |
| 1416 |
|
| 1417 |
:class:`http.server.BaseHTTPRequestHandler` now buffers the headers and writes |
| 1418 |
them all at once when :meth:`~http.server.BaseHTTPRequestHandler.end_headers` is |
| 1419 |
called. A new method :meth:`~http.server.BaseHTTPRequestHandler.flush_headers` |
| 1420 |
can be used to directly manage when the accumulated headers are sent. |
| 1421 |
(Contributed by Andrew Schaaf in :issue:`3709`.) |
| 1422 |
|
| 1423 |
:class:`http.server` now produces valid ``HTML 4.01 strict`` output. |
| 1424 |
(Contributed by Ezio Melotti in :issue:`13295`.) |
| 1425 |
|
| 1426 |
:class:`http.client.HTTPResponse` now has a |
| 1427 |
:meth:`~http.client.HTTPResponse.readinto` method, which means it can be used |
| 1428 |
as an :class:`io.RawIOBase` class. (Contributed by John Kuhn in |
| 1429 |
:issue:`13464`.) |
| 1430 |
|
| 1431 |
|
| 1432 |
html |
| 1433 |
---- |
| 1434 |
|
| 1435 |
:class:`html.parser.HTMLParser` is now able to parse broken markup without |
| 1436 |
raising errors, therefore the *strict* argument of the constructor and the |
| 1437 |
:exc:`!HTMLParseError` exception are now deprecated. |
| 1438 |
The ability to parse broken markup is the result of a number of bug fixes that |
| 1439 |
are also available on the latest bug fix releases of Python 2.7/3.2. |
| 1440 |
(Contributed by Ezio Melotti in :issue:`15114`, and :issue:`14538`, |
| 1441 |
:issue:`13993`, :issue:`13960`, :issue:`13358`, :issue:`1745761`, |
| 1442 |
:issue:`755670`, :issue:`13357`, :issue:`12629`, :issue:`1200313`, |
| 1443 |
:issue:`670664`, :issue:`13273`, :issue:`12888`, :issue:`7311`.) |
| 1444 |
|
| 1445 |
A new :data:`~html.entities.html5` dictionary that maps HTML5 named character |
| 1446 |
references to the equivalent Unicode character(s) (e.g. ``html5['gt;'] == |
| 1447 |
'>'``) has been added to the :mod:`html.entities` module. The dictionary is |
| 1448 |
now also used by :class:`~html.parser.HTMLParser`. (Contributed by Ezio |
| 1449 |
Melotti in :issue:`11113` and :issue:`15156`.) |
| 1450 |
|
| 1451 |
|
| 1452 |
imaplib |
| 1453 |
------- |
| 1454 |
|
| 1455 |
The :class:`~imaplib.IMAP4_SSL` constructor now accepts an SSLContext |
| 1456 |
parameter to control parameters of the secure channel. |
| 1457 |
|
| 1458 |
(Contributed by Sijin Joseph in :issue:`8808`.) |
| 1459 |
|
| 1460 |
|
| 1461 |
inspect |
| 1462 |
------- |
| 1463 |
|
| 1464 |
A new :func:`~inspect.getclosurevars` function has been added. This function |
| 1465 |
reports the current binding of all names referenced from the function body and |
| 1466 |
where those names were resolved, making it easier to verify correct internal |
| 1467 |
state when testing code that relies on stateful closures. |
| 1468 |
|
| 1469 |
(Contributed by Meador Inge and Nick Coghlan in :issue:`13062`.) |
| 1470 |
|
| 1471 |
A new :func:`~inspect.getgeneratorlocals` function has been added. This |
| 1472 |
function reports the current binding of local variables in the generator's |
| 1473 |
stack frame, making it easier to verify correct internal state when testing |
| 1474 |
generators. |
| 1475 |
|
| 1476 |
(Contributed by Meador Inge in :issue:`15153`.) |
| 1477 |
|
| 1478 |
io |
| 1479 |
-- |
| 1480 |
|
| 1481 |
The :func:`~io.open` function has a new ``'x'`` mode that can be used to |
| 1482 |
exclusively create a new file, and raise a :exc:`FileExistsError` if the file |
| 1483 |
already exists. It is based on the C11 'x' mode to fopen(). |
| 1484 |
|
| 1485 |
(Contributed by David Townshend in :issue:`12760`.) |
| 1486 |
|
| 1487 |
The constructor of the :class:`~io.TextIOWrapper` class has a new |
| 1488 |
*write_through* optional argument. If *write_through* is ``True``, calls to |
| 1489 |
:meth:`!write` are guaranteed not to be buffered: any data |
| 1490 |
written on the :class:`~io.TextIOWrapper` object is immediately handled to its |
| 1491 |
underlying binary buffer. |
| 1492 |
|
| 1493 |
|
| 1494 |
itertools |
| 1495 |
--------- |
| 1496 |
|
| 1497 |
:func:`~itertools.accumulate` now takes an optional ``func`` argument for |
| 1498 |
providing a user-supplied binary function. |
| 1499 |
|
| 1500 |
|
| 1501 |
logging |
| 1502 |
------- |
| 1503 |
|
| 1504 |
The :func:`~logging.basicConfig` function now supports an optional ``handlers`` |
| 1505 |
argument taking an iterable of handlers to be added to the root logger. |
| 1506 |
|
| 1507 |
A class level attribute :attr:`!append_nul` has |
| 1508 |
been added to :class:`~logging.handlers.SysLogHandler` to allow control of the |
| 1509 |
appending of the ``NUL`` (``\000``) byte to syslog records, since for some |
| 1510 |
daemons it is required while for others it is passed through to the log. |
| 1511 |
|
| 1512 |
|
| 1513 |
|
| 1514 |
math |
| 1515 |
---- |
| 1516 |
|
| 1517 |
The :mod:`math` module has a new function, :func:`~math.log2`, which returns |
| 1518 |
the base-2 logarithm of *x*. |
| 1519 |
|
| 1520 |
(Written by Mark Dickinson in :issue:`11888`.) |
| 1521 |
|
| 1522 |
|
| 1523 |
mmap |
| 1524 |
---- |
| 1525 |
|
| 1526 |
The :meth:`~mmap.mmap.read` method is now more compatible with other file-like |
| 1527 |
objects: if the argument is omitted or specified as ``None``, it returns the |
| 1528 |
bytes from the current file position to the end of the mapping. (Contributed |
| 1529 |
by Petri Lehtinen in :issue:`12021`.) |
| 1530 |
|
| 1531 |
|
| 1532 |
multiprocessing |
| 1533 |
--------------- |
| 1534 |
|
| 1535 |
The new :func:`multiprocessing.connection.wait` function allows polling |
| 1536 |
multiple objects (such as connections, sockets and pipes) with a timeout. |
| 1537 |
(Contributed by Richard Oudkerk in :issue:`12328`.) |
| 1538 |
|
| 1539 |
:class:`multiprocessing.connection.Connection` objects can now be transferred |
| 1540 |
over multiprocessing connections. |
| 1541 |
(Contributed by Richard Oudkerk in :issue:`4892`.) |
| 1542 |
|
| 1543 |
:class:`multiprocessing.Process` now accepts a ``daemon`` keyword argument |
| 1544 |
to override the default behavior of inheriting the ``daemon`` flag from |
| 1545 |
the parent process (:issue:`6064`). |
| 1546 |
|
| 1547 |
New attribute :data:`multiprocessing.Process.sentinel` allows a |
| 1548 |
program to wait on multiple :class:`~multiprocessing.Process` objects at one |
| 1549 |
time using the appropriate OS primitives (for example, :mod:`select` on |
| 1550 |
posix systems). |
| 1551 |
|
| 1552 |
New methods :meth:`multiprocessing.pool.Pool.starmap` and |
| 1553 |
:meth:`~multiprocessing.pool.Pool.starmap_async` provide |
| 1554 |
:func:`itertools.starmap` equivalents to the existing |
| 1555 |
:meth:`multiprocessing.pool.Pool.map` and |
| 1556 |
:meth:`~multiprocessing.pool.Pool.map_async` functions. (Contributed by Hynek |
| 1557 |
Schlawack in :issue:`12708`.) |
| 1558 |
|
| 1559 |
|
| 1560 |
nntplib |
| 1561 |
------- |
| 1562 |
|
| 1563 |
The :class:`!nntplib.NNTP` class now supports the context management protocol to |
| 1564 |
unconditionally consume :exc:`socket.error` exceptions and to close the NNTP |
| 1565 |
connection when done:: |
| 1566 |
|
| 1567 |
>>> from nntplib import NNTP |
| 1568 |
>>> with NNTP('news.gmane.org') as n: |
| 1569 |
... n.group('gmane.comp.python.committers') |
| 1570 |
... |
| 1571 |
('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers') |
| 1572 |
>>> |
| 1573 |
|
| 1574 |
(Contributed by Giampaolo Rodolà in :issue:`9795`.) |
| 1575 |
|
| 1576 |
|
| 1577 |
os |
| 1578 |
-- |
| 1579 |
|
| 1580 |
* The :mod:`os` module has a new :func:`~os.pipe2` function that makes it |
| 1581 |
possible to create a pipe with :const:`~os.O_CLOEXEC` or |
| 1582 |
:const:`~os.O_NONBLOCK` flags set atomically. This is especially useful to |
| 1583 |
avoid race conditions in multi-threaded programs. |
| 1584 |
|
| 1585 |
* The :mod:`os` module has a new :func:`~os.sendfile` function which provides |
| 1586 |
an efficient "zero-copy" way for copying data from one file (or socket) |
| 1587 |
descriptor to another. The phrase "zero-copy" refers to the fact that all of |
| 1588 |
the copying of data between the two descriptors is done entirely by the |
| 1589 |
kernel, with no copying of data into userspace buffers. :func:`~os.sendfile` |
| 1590 |
can be used to efficiently copy data from a file on disk to a network socket, |
| 1591 |
e.g. for downloading a file. |
| 1592 |
|
| 1593 |
(Patch submitted by Ross Lagerwall and Giampaolo Rodolà in :issue:`10882`.) |
| 1594 |
|
| 1595 |
* To avoid race conditions like symlink attacks and issues with temporary |
| 1596 |
files and directories, it is more reliable (and also faster) to manipulate |
| 1597 |
file descriptors instead of file names. Python 3.3 enhances existing functions |
| 1598 |
and introduces new functions to work on file descriptors (:issue:`4761`, |
| 1599 |
:issue:`10755` and :issue:`14626`). |
| 1600 |
|
| 1601 |
- The :mod:`os` module has a new :func:`~os.fwalk` function similar to |
| 1602 |
:func:`~os.walk` except that it also yields file descriptors referring to the |
| 1603 |
directories visited. This is especially useful to avoid symlink races. |
| 1604 |
|
| 1605 |
- The following functions get new optional *dir_fd* (:ref:`paths relative to |
| 1606 |
directory descriptors <dir_fd>`) and/or *follow_symlinks* (:ref:`not |
| 1607 |
following symlinks <follow_symlinks>`): |
| 1608 |
:func:`~os.access`, :func:`~os.chflags`, :func:`~os.chmod`, :func:`~os.chown`, |
| 1609 |
:func:`~os.link`, :func:`~os.lstat`, :func:`~os.mkdir`, :func:`~os.mkfifo`, |
| 1610 |
:func:`~os.mknod`, :func:`~os.open`, :func:`~os.readlink`, :func:`~os.remove`, |
| 1611 |
:func:`~os.rename`, :func:`~os.replace`, :func:`~os.rmdir`, :func:`~os.stat`, |
| 1612 |
:func:`~os.symlink`, :func:`~os.unlink`, :func:`~os.utime`. Platform |
| 1613 |
support for using these parameters can be checked via the sets |
| 1614 |
:data:`os.supports_dir_fd` and :data:`os.supports_follow_symlinks`. |
| 1615 |
|
| 1616 |
- The following functions now support a file descriptor for their path argument: |
| 1617 |
:func:`~os.chdir`, :func:`~os.chmod`, :func:`~os.chown`, |
| 1618 |
:func:`~os.execve`, :func:`~os.listdir`, :func:`~os.pathconf`, :func:`~os.path.exists`, |
| 1619 |
:func:`~os.stat`, :func:`~os.statvfs`, :func:`~os.utime`. Platform support |
| 1620 |
for this can be checked via the :data:`os.supports_fd` set. |
| 1621 |
|
| 1622 |
* :func:`~os.access` accepts an ``effective_ids`` keyword argument to turn on |
| 1623 |
using the effective uid/gid rather than the real uid/gid in the access check. |
| 1624 |
Platform support for this can be checked via the |
| 1625 |
:data:`~os.supports_effective_ids` set. |
| 1626 |
|
| 1627 |
* The :mod:`os` module has two new functions: :func:`~os.getpriority` and |
| 1628 |
:func:`~os.setpriority`. They can be used to get or set process |
| 1629 |
niceness/priority in a fashion similar to :func:`os.nice` but extended to all |
| 1630 |
processes instead of just the current one. |
| 1631 |
|
| 1632 |
(Patch submitted by Giampaolo Rodolà in :issue:`10784`.) |
| 1633 |
|
| 1634 |
* The new :func:`os.replace` function allows cross-platform renaming of a |
| 1635 |
file with overwriting the destination. With :func:`os.rename`, an existing |
| 1636 |
destination file is overwritten under POSIX, but raises an error under |
| 1637 |
Windows. |
| 1638 |
(Contributed by Antoine Pitrou in :issue:`8828`.) |
| 1639 |
|
| 1640 |
* The stat family of functions (:func:`~os.stat`, :func:`~os.fstat`, |
| 1641 |
and :func:`~os.lstat`) now support reading a file's timestamps |
| 1642 |
with nanosecond precision. Symmetrically, :func:`~os.utime` |
| 1643 |
can now write file timestamps with nanosecond precision. (Contributed by |
| 1644 |
Larry Hastings in :issue:`14127`.) |
| 1645 |
|
| 1646 |
* The new :func:`os.get_terminal_size` function queries the size of the |
| 1647 |
terminal attached to a file descriptor. See also |
| 1648 |
:func:`shutil.get_terminal_size`. |
| 1649 |
(Contributed by Zbigniew Jędrzejewski-Szmek in :issue:`13609`.) |
| 1650 |
|
| 1651 |
.. XXX sort out this mess after beta1 |
| 1652 |
|
| 1653 |
* New functions to support Linux extended attributes (:issue:`12720`): |
| 1654 |
:func:`~os.getxattr`, :func:`~os.listxattr`, :func:`~os.removexattr`, |
| 1655 |
:func:`~os.setxattr`. |
| 1656 |
|
| 1657 |
* New interface to the scheduler. These functions |
| 1658 |
control how a process is allocated CPU time by the operating system. New |
| 1659 |
functions: |
| 1660 |
:func:`~os.sched_get_priority_max`, :func:`~os.sched_get_priority_min`, |
| 1661 |
:func:`~os.sched_getaffinity`, :func:`~os.sched_getparam`, |
| 1662 |
:func:`~os.sched_getscheduler`, :func:`~os.sched_rr_get_interval`, |
| 1663 |
:func:`~os.sched_setaffinity`, :func:`~os.sched_setparam`, |
| 1664 |
:func:`~os.sched_setscheduler`, :func:`~os.sched_yield`, |
| 1665 |
|
| 1666 |
* New functions to control the file system: |
| 1667 |
|
| 1668 |
* :func:`~os.posix_fadvise`: Announces an intention to access data in a |
| 1669 |
specific pattern thus allowing the kernel to make optimizations. |
| 1670 |
* :func:`~os.posix_fallocate`: Ensures that enough disk space is allocated |
| 1671 |
for a file. |
| 1672 |
* :func:`~os.sync`: Force write of everything to disk. |
| 1673 |
|
| 1674 |
* Additional new posix functions: |
| 1675 |
|
| 1676 |
* :func:`~os.lockf`: Apply, test or remove a POSIX lock on an open file descriptor. |
| 1677 |
* :func:`~os.pread`: Read from a file descriptor at an offset, the file |
| 1678 |
offset remains unchanged. |
| 1679 |
* :func:`~os.pwrite`: Write to a file descriptor from an offset, leaving |
| 1680 |
the file offset unchanged. |
| 1681 |
* :func:`~os.readv`: Read from a file descriptor into a number of writable buffers. |
| 1682 |
* :func:`~os.truncate`: Truncate the file corresponding to *path*, so that |
| 1683 |
it is at most *length* bytes in size. |
| 1684 |
* :func:`~os.waitid`: Wait for the completion of one or more child processes. |
| 1685 |
* :func:`~os.writev`: Write the contents of *buffers* to a file descriptor, |
| 1686 |
where *buffers* is an arbitrary sequence of buffers. |
| 1687 |
* :func:`~os.getgrouplist` (:issue:`9344`): Return list of group ids that |
| 1688 |
specified user belongs to. |
| 1689 |
|
| 1690 |
* :func:`~os.times` and :func:`~os.uname`: Return type changed from a tuple to |
| 1691 |
a tuple-like object with named attributes. |
| 1692 |
|
| 1693 |
* Some platforms now support additional constants for the :func:`~os.lseek` |
| 1694 |
function, such as ``os.SEEK_HOLE`` and ``os.SEEK_DATA``. |
| 1695 |
|
| 1696 |
* New constants :const:`~os.RTLD_LAZY`, :const:`~os.RTLD_NOW`, |
| 1697 |
:const:`~os.RTLD_GLOBAL`, :const:`~os.RTLD_LOCAL`, :const:`~os.RTLD_NODELETE`, |
| 1698 |
:const:`~os.RTLD_NOLOAD`, and :const:`~os.RTLD_DEEPBIND` are available on |
| 1699 |
platforms that support them. These are for use with the |
| 1700 |
:func:`sys.setdlopenflags` function, and supersede the similar constants |
| 1701 |
defined in :mod:`ctypes` and :mod:`!DLFCN`. (Contributed by Victor Stinner |
| 1702 |
in :issue:`13226`.) |
| 1703 |
|
| 1704 |
* :func:`os.symlink` now accepts (and ignores) the ``target_is_directory`` |
| 1705 |
keyword argument on non-Windows platforms, to ease cross-platform support. |
| 1706 |
|
| 1707 |
|
| 1708 |
pdb |
| 1709 |
--- |
| 1710 |
|
| 1711 |
Tab-completion is now available not only for command names, but also their |
| 1712 |
arguments. For example, for the ``break`` command, function and file names |
| 1713 |
are completed. |
| 1714 |
|
| 1715 |
(Contributed by Georg Brandl in :issue:`14210`) |
| 1716 |
|
| 1717 |
|
| 1718 |
pickle |
| 1719 |
------ |
| 1720 |
|
| 1721 |
:class:`pickle.Pickler` objects now have an optional |
| 1722 |
:attr:`~pickle.Pickler.dispatch_table` attribute allowing per-pickler |
| 1723 |
reduction functions to be set. |
| 1724 |
|
| 1725 |
(Contributed by Richard Oudkerk in :issue:`14166`.) |
| 1726 |
|
| 1727 |
|
| 1728 |
pydoc |
| 1729 |
----- |
| 1730 |
|
| 1731 |
The Tk GUI and the :func:`!serve` function have been removed from the |
| 1732 |
:mod:`pydoc` module: ``pydoc -g`` and :func:`!serve` have been deprecated |
| 1733 |
in Python 3.2. |
| 1734 |
|
| 1735 |
|
| 1736 |
re |
| 1737 |
-- |
| 1738 |
|
| 1739 |
:class:`str` regular expressions now support ``\u`` and ``\U`` escapes. |
| 1740 |
|
| 1741 |
(Contributed by Serhiy Storchaka in :issue:`3665`.) |
| 1742 |
|
| 1743 |
|
| 1744 |
sched |
| 1745 |
----- |
| 1746 |
|
| 1747 |
* :meth:`~sched.scheduler.run` now accepts a *blocking* parameter which when |
| 1748 |
set to false makes the method execute the scheduled events due to expire |
| 1749 |
soonest (if any) and then return immediately. |
| 1750 |
This is useful in case you want to use the :class:`~sched.scheduler` in |
| 1751 |
non-blocking applications. (Contributed by Giampaolo Rodolà in :issue:`13449`.) |
| 1752 |
|
| 1753 |
* :class:`~sched.scheduler` class can now be safely used in multi-threaded |
| 1754 |
environments. (Contributed by Josiah Carlson and Giampaolo Rodolà in |
| 1755 |
:issue:`8684`.) |
| 1756 |
|
| 1757 |
* *timefunc* and *delayfunct* parameters of :class:`~sched.scheduler` class |
| 1758 |
constructor are now optional and defaults to :func:`time.time` and |
| 1759 |
:func:`time.sleep` respectively. (Contributed by Chris Clark in |
| 1760 |
:issue:`13245`.) |
| 1761 |
|
| 1762 |
* :meth:`~sched.scheduler.enter` and :meth:`~sched.scheduler.enterabs` |
| 1763 |
*argument* parameter is now optional. (Contributed by Chris Clark in |
| 1764 |
:issue:`13245`.) |
| 1765 |
|
| 1766 |
* :meth:`~sched.scheduler.enter` and :meth:`~sched.scheduler.enterabs` |
| 1767 |
now accept a *kwargs* parameter. (Contributed by Chris Clark in |
| 1768 |
:issue:`13245`.) |
| 1769 |
|
| 1770 |
|
| 1771 |
select |
| 1772 |
------ |
| 1773 |
|
| 1774 |
Solaris and derivative platforms have a new class :class:`select.devpoll` |
| 1775 |
for high performance asynchronous sockets via :file:`/dev/poll`. |
| 1776 |
(Contributed by Jesús Cea Avión in :issue:`6397`.) |
| 1777 |
|
| 1778 |
|
| 1779 |
shlex |
| 1780 |
----- |
| 1781 |
|
| 1782 |
The previously undocumented helper function ``quote`` from the |
| 1783 |
:mod:`!pipes` modules has been moved to the :mod:`shlex` module and |
| 1784 |
documented. :func:`~shlex.quote` properly escapes all characters in a string |
| 1785 |
that might be otherwise given special meaning by the shell. |
| 1786 |
|
| 1787 |
|
| 1788 |
shutil |
| 1789 |
------ |
| 1790 |
|
| 1791 |
* New functions: |
| 1792 |
|
| 1793 |
* :func:`~shutil.disk_usage`: provides total, used and free disk space |
| 1794 |
statistics. (Contributed by Giampaolo Rodolà in :issue:`12442`.) |
| 1795 |
* :func:`~shutil.chown`: allows one to change user and/or group of the given |
| 1796 |
path also specifying the user/group names and not only their numeric |
| 1797 |
ids. (Contributed by Sandro Tosi in :issue:`12191`.) |
| 1798 |
* :func:`shutil.get_terminal_size`: returns the size of the terminal window |
| 1799 |
to which the interpreter is attached. (Contributed by Zbigniew |
| 1800 |
Jędrzejewski-Szmek in :issue:`13609`.) |
| 1801 |
|
| 1802 |
* :func:`~shutil.copy2` and :func:`~shutil.copystat` now preserve file |
| 1803 |
timestamps with nanosecond precision on platforms that support it. |
| 1804 |
They also preserve file "extended attributes" on Linux. (Contributed |
| 1805 |
by Larry Hastings in :issue:`14127` and :issue:`15238`.) |
| 1806 |
|
| 1807 |
* Several functions now take an optional ``symlinks`` argument: when that |
| 1808 |
parameter is true, symlinks aren't dereferenced and the operation instead |
| 1809 |
acts on the symlink itself (or creates one, if relevant). |
| 1810 |
(Contributed by Hynek Schlawack in :issue:`12715`.) |
| 1811 |
|
| 1812 |
* When copying files to a different file system, :func:`~shutil.move` now |
| 1813 |
handles symlinks the way the posix ``mv`` command does, recreating the |
| 1814 |
symlink rather than copying the target file contents. (Contributed by |
| 1815 |
Jonathan Niehof in :issue:`9993`.) :func:`~shutil.move` now also returns |
| 1816 |
the ``dst`` argument as its result. |
| 1817 |
|
| 1818 |
* :func:`~shutil.rmtree` is now resistant to symlink attacks on platforms |
| 1819 |
which support the new ``dir_fd`` parameter in :func:`os.open` and |
| 1820 |
:func:`os.unlink`. (Contributed by Martin von Löwis and Hynek Schlawack |
| 1821 |
in :issue:`4489`.) |
| 1822 |
|
| 1823 |
|
| 1824 |
signal |
| 1825 |
------ |
| 1826 |
|
| 1827 |
* The :mod:`signal` module has new functions: |
| 1828 |
|
| 1829 |
* :func:`~signal.pthread_sigmask`: fetch and/or change the signal mask of the |
| 1830 |
calling thread (Contributed by Jean-Paul Calderone in :issue:`8407`); |
| 1831 |
* :func:`~signal.pthread_kill`: send a signal to a thread; |
| 1832 |
* :func:`~signal.sigpending`: examine pending functions; |
| 1833 |
* :func:`~signal.sigwait`: wait a signal; |
| 1834 |
* :func:`~signal.sigwaitinfo`: wait for a signal, returning detailed |
| 1835 |
information about it; |
| 1836 |
* :func:`~signal.sigtimedwait`: like :func:`~signal.sigwaitinfo` but with a |
| 1837 |
timeout. |
| 1838 |
|
| 1839 |
* The signal handler writes the signal number as a single byte instead of |
| 1840 |
a nul byte into the wakeup file descriptor. So it is possible to wait more |
| 1841 |
than one signal and know which signals were raised. |
| 1842 |
|
| 1843 |
* :func:`signal.signal` and :func:`signal.siginterrupt` raise an OSError, |
| 1844 |
instead of a RuntimeError: OSError has an errno attribute. |
| 1845 |
|
| 1846 |
|
| 1847 |
smtpd |
| 1848 |
----- |
| 1849 |
|
| 1850 |
The :mod:`!smtpd` module now supports :rfc:`5321` (extended SMTP) and :rfc:`1870` |
| 1851 |
(size extension). Per the standard, these extensions are enabled if and only |
| 1852 |
if the client initiates the session with an ``EHLO`` command. |
| 1853 |
|
| 1854 |
(Initial ``ELHO`` support by Alberto Trevino. Size extension by Juhana |
| 1855 |
Jauhiainen. Substantial additional work on the patch contributed by Michele |
| 1856 |
Orrù and Dan Boswell. :issue:`8739`) |
| 1857 |
|
| 1858 |
|
| 1859 |
smtplib |
| 1860 |
------- |
| 1861 |
|
| 1862 |
The :class:`~smtplib.SMTP`, :class:`~smtplib.SMTP_SSL`, and |
| 1863 |
:class:`~smtplib.LMTP` classes now accept a ``source_address`` keyword argument |
| 1864 |
to specify the ``(host, port)`` to use as the source address in the bind call |
| 1865 |
when creating the outgoing socket. (Contributed by Paulo Scardine in |
| 1866 |
:issue:`11281`.) |
| 1867 |
|
| 1868 |
:class:`~smtplib.SMTP` now supports the context management protocol, allowing an |
| 1869 |
``SMTP`` instance to be used in a ``with`` statement. (Contributed |
| 1870 |
by Giampaolo Rodolà in :issue:`11289`.) |
| 1871 |
|
| 1872 |
The :class:`~smtplib.SMTP_SSL` constructor and the :meth:`~smtplib.SMTP.starttls` |
| 1873 |
method now accept an SSLContext parameter to control parameters of the secure |
| 1874 |
channel. (Contributed by Kasun Herath in :issue:`8809`.) |
| 1875 |
|
| 1876 |
|
| 1877 |
socket |
| 1878 |
------ |
| 1879 |
|
| 1880 |
* The :class:`~socket.socket` class now exposes additional methods to process |
| 1881 |
ancillary data when supported by the underlying platform: |
| 1882 |
|
| 1883 |
* :func:`~socket.socket.sendmsg` |
| 1884 |
* :func:`~socket.socket.recvmsg` |
| 1885 |
* :func:`~socket.socket.recvmsg_into` |
| 1886 |
|
| 1887 |
(Contributed by David Watson in :issue:`6560`, based on an earlier patch by |
| 1888 |
Heiko Wundram) |
| 1889 |
|
| 1890 |
* The :class:`~socket.socket` class now supports the PF_CAN protocol family |
| 1891 |
(https://en.wikipedia.org/wiki/Socketcan), on Linux |
| 1892 |
(https://lwn.net/Articles/253425). |
| 1893 |
|
| 1894 |
(Contributed by Matthias Fuchs, updated by Tiago Gonçalves in :issue:`10141`.) |
| 1895 |
|
| 1896 |
* The :class:`~socket.socket` class now supports the PF_RDS protocol family |
| 1897 |
(https://en.wikipedia.org/wiki/Reliable_Datagram_Sockets and |
| 1898 |
`https://oss.oracle.com/projects/rds <https://web.archive.org/web/20130115155505/https://oss.oracle.com/projects/rds/>`__). |
| 1899 |
|
| 1900 |
* The :class:`~socket.socket` class now supports the ``PF_SYSTEM`` protocol |
| 1901 |
family on OS X. (Contributed by Michael Goderbauer in :issue:`13777`.) |
| 1902 |
|
| 1903 |
* New function :func:`~socket.sethostname` allows the hostname to be set |
| 1904 |
on Unix systems if the calling process has sufficient privileges. |
| 1905 |
(Contributed by Ross Lagerwall in :issue:`10866`.) |
| 1906 |
|
| 1907 |
|
| 1908 |
socketserver |
| 1909 |
------------ |
| 1910 |
|
| 1911 |
:class:`~socketserver.BaseServer` now has an overridable method |
| 1912 |
:meth:`~socketserver.BaseServer.service_actions` that is called by the |
| 1913 |
:meth:`~socketserver.BaseServer.serve_forever` method in the service loop. |
| 1914 |
:class:`~socketserver.ForkingMixIn` now uses this to clean up zombie |
| 1915 |
child processes. (Contributed by Justin Warkentin in :issue:`11109`.) |
| 1916 |
|
| 1917 |
|
| 1918 |
sqlite3 |
| 1919 |
------- |
| 1920 |
|
| 1921 |
New :class:`sqlite3.Connection` method |
| 1922 |
:meth:`~sqlite3.Connection.set_trace_callback` can be used to capture a trace of |
| 1923 |
all sql commands processed by sqlite. (Contributed by Torsten Landschoff |
| 1924 |
in :issue:`11688`.) |
| 1925 |
|
| 1926 |
|
| 1927 |
ssl |
| 1928 |
--- |
| 1929 |
|
| 1930 |
* The :mod:`ssl` module has two new random generation functions: |
| 1931 |
|
| 1932 |
* :func:`~ssl.RAND_bytes`: generate cryptographically strong |
| 1933 |
pseudo-random bytes. |
| 1934 |
* :func:`!RAND_pseudo_bytes`: generate pseudo-random bytes. |
| 1935 |
|
| 1936 |
(Contributed by Victor Stinner in :issue:`12049`.) |
| 1937 |
|
| 1938 |
* The :mod:`ssl` module now exposes a finer-grained exception hierarchy |
| 1939 |
in order to make it easier to inspect the various kinds of errors. |
| 1940 |
(Contributed by Antoine Pitrou in :issue:`11183`.) |
| 1941 |
|
| 1942 |
* :meth:`~ssl.SSLContext.load_cert_chain` now accepts a *password* argument |
| 1943 |
to be used if the private key is encrypted. |
| 1944 |
(Contributed by Adam Simpkins in :issue:`12803`.) |
| 1945 |
|
| 1946 |
* Diffie-Hellman key exchange, both regular and Elliptic Curve-based, is |
| 1947 |
now supported through the :meth:`~ssl.SSLContext.load_dh_params` and |
| 1948 |
:meth:`~ssl.SSLContext.set_ecdh_curve` methods. |
| 1949 |
(Contributed by Antoine Pitrou in :issue:`13626` and :issue:`13627`.) |
| 1950 |
|
| 1951 |
* SSL sockets have a new :meth:`~ssl.SSLSocket.get_channel_binding` method |
| 1952 |
allowing the implementation of certain authentication mechanisms such as |
| 1953 |
SCRAM-SHA-1-PLUS. (Contributed by Jacek Konieczny in :issue:`12551`.) |
| 1954 |
|
| 1955 |
* You can query the SSL compression algorithm used by an SSL socket, thanks |
| 1956 |
to its new :meth:`~ssl.SSLSocket.compression` method. The new attribute |
| 1957 |
:const:`~ssl.OP_NO_COMPRESSION` can be used to disable compression. |
| 1958 |
(Contributed by Antoine Pitrou in :issue:`13634`.) |
| 1959 |
|
| 1960 |
* Support has been added for the Next Protocol Negotiation extension using |
| 1961 |
the :meth:`ssl.SSLContext.set_npn_protocols` method. |
| 1962 |
(Contributed by Colin Marc in :issue:`14204`.) |
| 1963 |
|
| 1964 |
* SSL errors can now be introspected more easily thanks to |
| 1965 |
:attr:`~ssl.SSLError.library` and :attr:`~ssl.SSLError.reason` attributes. |
| 1966 |
(Contributed by Antoine Pitrou in :issue:`14837`.) |
| 1967 |
|
| 1968 |
* The :func:`~ssl.get_server_certificate` function now supports IPv6. |
| 1969 |
(Contributed by Charles-François Natali in :issue:`11811`.) |
| 1970 |
|
| 1971 |
* New attribute :const:`~ssl.OP_CIPHER_SERVER_PREFERENCE` allows setting |
| 1972 |
SSLv3 server sockets to use the server's cipher ordering preference rather |
| 1973 |
than the client's (:issue:`13635`). |
| 1974 |
|
| 1975 |
|
| 1976 |
stat |
| 1977 |
---- |
| 1978 |
|
| 1979 |
The undocumented tarfile.filemode function has been moved to |
| 1980 |
:func:`stat.filemode`. It can be used to convert a file's mode to a string of |
| 1981 |
the form '-rwxrwxrwx'. |
| 1982 |
|
| 1983 |
(Contributed by Giampaolo Rodolà in :issue:`14807`.) |
| 1984 |
|
| 1985 |
|
| 1986 |
struct |
| 1987 |
------ |
| 1988 |
|
| 1989 |
The :mod:`struct` module now supports :c:type:`ssize_t` and :c:type:`size_t` via the |
| 1990 |
new codes ``n`` and ``N``, respectively. (Contributed by Antoine Pitrou |
| 1991 |
in :issue:`3163`.) |
| 1992 |
|
| 1993 |
|
| 1994 |
subprocess |
| 1995 |
---------- |
| 1996 |
|
| 1997 |
Command strings can now be bytes objects on posix platforms. (Contributed by |
| 1998 |
Victor Stinner in :issue:`8513`.) |
| 1999 |
|
| 2000 |
A new constant :const:`~subprocess.DEVNULL` allows suppressing output in a |
| 2001 |
platform-independent fashion. (Contributed by Ross Lagerwall in |
| 2002 |
:issue:`5870`.) |
| 2003 |
|
| 2004 |
|
| 2005 |
sys |
| 2006 |
--- |
| 2007 |
|
| 2008 |
The :mod:`sys` module has a new :data:`~sys.thread_info` :term:`named |
| 2009 |
tuple` holding information about the thread implementation |
| 2010 |
(:issue:`11223`). |
| 2011 |
|
| 2012 |
|
| 2013 |
tarfile |
| 2014 |
------- |
| 2015 |
|
| 2016 |
:mod:`tarfile` now supports ``lzma`` encoding via the :mod:`lzma` module. |
| 2017 |
(Contributed by Lars Gustäbel in :issue:`5689`.) |
| 2018 |
|
| 2019 |
|
| 2020 |
tempfile |
| 2021 |
-------- |
| 2022 |
|
| 2023 |
:class:`tempfile.SpooledTemporaryFile`\'s :meth:`!truncate` method now accepts |
| 2024 |
a ``size`` parameter. (Contributed by Ryan Kelly in :issue:`9957`.) |
| 2025 |
|
| 2026 |
|
| 2027 |
textwrap |
| 2028 |
-------- |
| 2029 |
|
| 2030 |
The :mod:`textwrap` module has a new :func:`~textwrap.indent` that makes |
| 2031 |
it straightforward to add a common prefix to selected lines in a block |
| 2032 |
of text (:issue:`13857`). |
| 2033 |
|
| 2034 |
|
| 2035 |
threading |
| 2036 |
--------- |
| 2037 |
|
| 2038 |
:class:`threading.Condition`, :class:`threading.Semaphore`, |
| 2039 |
:class:`threading.BoundedSemaphore`, :class:`threading.Event`, and |
| 2040 |
:class:`threading.Timer`, all of which used to be factory functions returning a |
| 2041 |
class instance, are now classes and may be subclassed. (Contributed by Éric |
| 2042 |
Araujo in :issue:`10968`.) |
| 2043 |
|
| 2044 |
The :class:`threading.Thread` constructor now accepts a ``daemon`` keyword |
| 2045 |
argument to override the default behavior of inheriting the ``daemon`` flag |
| 2046 |
value from the parent thread (:issue:`6064`). |
| 2047 |
|
| 2048 |
The formerly private function ``_thread.get_ident`` is now available as the |
| 2049 |
public function :func:`threading.get_ident`. This eliminates several cases of |
| 2050 |
direct access to the ``_thread`` module in the stdlib. Third party code that |
| 2051 |
used ``_thread.get_ident`` should likewise be changed to use the new public |
| 2052 |
interface. |
| 2053 |
|
| 2054 |
|
| 2055 |
time |
| 2056 |
---- |
| 2057 |
|
| 2058 |
The :pep:`418` added new functions to the :mod:`time` module: |
| 2059 |
|
| 2060 |
* :func:`~time.get_clock_info`: Get information on a clock. |
| 2061 |
* :func:`~time.monotonic`: Monotonic clock (cannot go backward), not affected |
| 2062 |
by system clock updates. |
| 2063 |
* :func:`~time.perf_counter`: Performance counter with the highest available |
| 2064 |
resolution to measure a short duration. |
| 2065 |
* :func:`~time.process_time`: Sum of the system and user CPU time of the |
| 2066 |
current process. |
| 2067 |
|
| 2068 |
Other new functions: |
| 2069 |
|
| 2070 |
* :func:`~time.clock_getres`, :func:`~time.clock_gettime` and |
| 2071 |
:func:`~time.clock_settime` functions with :samp:`CLOCK_{xxx}` constants. |
| 2072 |
(Contributed by Victor Stinner in :issue:`10278`.) |
| 2073 |
|
| 2074 |
To improve cross platform consistency, :func:`~time.sleep` now raises a |
| 2075 |
:exc:`ValueError` when passed a negative sleep value. Previously this was an |
| 2076 |
error on posix, but produced an infinite sleep on Windows. |
| 2077 |
|
| 2078 |
|
| 2079 |
types |
| 2080 |
----- |
| 2081 |
|
| 2082 |
Add a new :class:`types.MappingProxyType` class: Read-only proxy of a mapping. |
| 2083 |
(:issue:`14386`) |
| 2084 |
|
| 2085 |
|
| 2086 |
The new functions :func:`types.new_class` and :func:`types.prepare_class` provide support |
| 2087 |
for :pep:`3115` compliant dynamic type creation. (:issue:`14588`) |
| 2088 |
|
| 2089 |
|
| 2090 |
unittest |
| 2091 |
-------- |
| 2092 |
|
| 2093 |
:meth:`.assertRaises`, :meth:`.assertRaisesRegex`, :meth:`.assertWarns`, and |
| 2094 |
:meth:`.assertWarnsRegex` now accept a keyword argument *msg* when used as |
| 2095 |
context managers. (Contributed by Ezio Melotti and Winston Ewert in |
| 2096 |
:issue:`10775`.) |
| 2097 |
|
| 2098 |
:meth:`unittest.TestCase.run` now returns the :class:`~unittest.TestResult` |
| 2099 |
object. |
| 2100 |
|
| 2101 |
|
| 2102 |
urllib |
| 2103 |
------ |
| 2104 |
|
| 2105 |
The :class:`~urllib.request.Request` class, now accepts a *method* argument |
| 2106 |
used by :meth:`~urllib.request.Request.get_method` to determine what HTTP method |
| 2107 |
should be used. For example, this will send a ``'HEAD'`` request:: |
| 2108 |
|
| 2109 |
>>> urlopen(Request('https://www.python.org', method='HEAD')) |
| 2110 |
|
| 2111 |
(:issue:`1673007`) |
| 2112 |
|
| 2113 |
|
| 2114 |
webbrowser |
| 2115 |
---------- |
| 2116 |
|
| 2117 |
The :mod:`webbrowser` module supports more "browsers": Google Chrome (named |
| 2118 |
:program:`chrome`, :program:`chromium`, :program:`chrome-browser` or |
| 2119 |
:program:`chromium-browser` depending on the version and operating system), |
| 2120 |
and the generic launchers :program:`xdg-open`, from the FreeDesktop.org |
| 2121 |
project, and :program:`gvfs-open`, which is the default URI handler for GNOME |
| 2122 |
3. (The former contributed by Arnaud Calmettes in :issue:`13620`, the latter |
| 2123 |
by Matthias Klose in :issue:`14493`.) |
| 2124 |
|
| 2125 |
|
| 2126 |
xml.etree.ElementTree |
| 2127 |
--------------------- |
| 2128 |
|
| 2129 |
The :mod:`xml.etree.ElementTree` module now imports its C accelerator by |
| 2130 |
default; there is no longer a need to explicitly import |
| 2131 |
:mod:`!xml.etree.cElementTree` (this module stays for backwards compatibility, |
| 2132 |
but is now deprecated). In addition, the ``iter`` family of methods of |
| 2133 |
:class:`~xml.etree.ElementTree.Element` has been optimized (rewritten in C). |
| 2134 |
The module's documentation has also been greatly improved with added examples |
| 2135 |
and a more detailed reference. |
| 2136 |
|
| 2137 |
|
| 2138 |
zlib |
| 2139 |
---- |
| 2140 |
|
| 2141 |
New attribute :attr:`zlib.Decompress.eof` makes it possible to distinguish |
| 2142 |
between a properly formed compressed stream and an incomplete or truncated one. |
| 2143 |
(Contributed by Nadeem Vawda in :issue:`12646`.) |
| 2144 |
|
| 2145 |
New attribute :const:`zlib.ZLIB_RUNTIME_VERSION` reports the version string of |
| 2146 |
the underlying ``zlib`` library that is loaded at runtime. (Contributed by |
| 2147 |
Torsten Landschoff in :issue:`12306`.) |
| 2148 |
|
| 2149 |
|
| 2150 |
Optimizations |
| 2151 |
============= |
| 2152 |
|
| 2153 |
Major performance enhancements have been added: |
| 2154 |
|
| 2155 |
* Thanks to :pep:`393`, some operations on Unicode strings have been optimized: |
| 2156 |
|
| 2157 |
* the memory footprint is divided by 2 to 4 depending on the text |
| 2158 |
* encode an ASCII string to UTF-8 doesn't need to encode characters anymore, |
| 2159 |
the UTF-8 representation is shared with the ASCII representation |
| 2160 |
* the UTF-8 encoder has been optimized |
| 2161 |
* repeating a single ASCII letter and getting a substring of an ASCII string |
| 2162 |
is 4 times faster |
| 2163 |
|
| 2164 |
* UTF-8 is now 2x to 4x faster. UTF-16 encoding is now up to 10x faster. |
| 2165 |
|
| 2166 |
(Contributed by Serhiy Storchaka, :issue:`14624`, :issue:`14738` and |
| 2167 |
:issue:`15026`.) |
| 2168 |
|
| 2169 |
|
| 2170 |
Build and C API Changes |
| 2171 |
======================= |
| 2172 |
|
| 2173 |
Changes to Python's build process and to the C API include: |
| 2174 |
|
| 2175 |
* New :pep:`3118` related function: |
| 2176 |
|
| 2177 |
* :c:func:`PyMemoryView_FromMemory` |
| 2178 |
|
| 2179 |
* :pep:`393` added new Unicode types, macros and functions: |
| 2180 |
|
| 2181 |
* High-level API: |
| 2182 |
|
| 2183 |
* :c:func:`PyUnicode_CopyCharacters` |
| 2184 |
* :c:func:`PyUnicode_FindChar` |
| 2185 |
* :c:func:`PyUnicode_GetLength`, :c:macro:`PyUnicode_GET_LENGTH` |
| 2186 |
* :c:func:`PyUnicode_New` |
| 2187 |
* :c:func:`PyUnicode_Substring` |
| 2188 |
* :c:func:`PyUnicode_ReadChar`, :c:func:`PyUnicode_WriteChar` |
| 2189 |
|
| 2190 |
* Low-level API: |
| 2191 |
|
| 2192 |
* :c:type:`Py_UCS1`, :c:type:`Py_UCS2`, :c:type:`Py_UCS4` types |
| 2193 |
* :c:type:`PyASCIIObject` and :c:type:`PyCompactUnicodeObject` structures |
| 2194 |
* :c:macro:`PyUnicode_READY` |
| 2195 |
* :c:func:`PyUnicode_FromKindAndData` |
| 2196 |
* :c:func:`PyUnicode_AsUCS4`, :c:func:`PyUnicode_AsUCS4Copy` |
| 2197 |
* :c:macro:`PyUnicode_DATA`, :c:macro:`PyUnicode_1BYTE_DATA`, |
| 2198 |
:c:macro:`PyUnicode_2BYTE_DATA`, :c:macro:`PyUnicode_4BYTE_DATA` |
| 2199 |
* :c:macro:`PyUnicode_KIND` with :c:enum:`!PyUnicode_Kind` enum: |
| 2200 |
:c:data:`!PyUnicode_WCHAR_KIND`, :c:data:`PyUnicode_1BYTE_KIND`, |
| 2201 |
:c:data:`PyUnicode_2BYTE_KIND`, :c:data:`PyUnicode_4BYTE_KIND` |
| 2202 |
* :c:macro:`PyUnicode_READ`, :c:macro:`PyUnicode_READ_CHAR`, :c:macro:`PyUnicode_WRITE` |
| 2203 |
* :c:macro:`PyUnicode_MAX_CHAR_VALUE` |
| 2204 |
|
| 2205 |
* :c:macro:`PyArg_ParseTuple` now accepts a :class:`bytearray` for the ``c`` |
| 2206 |
format (:issue:`12380`). |
| 2207 |
|
| 2208 |
|
| 2209 |
|
| 2210 |
Deprecated |
| 2211 |
========== |
| 2212 |
|
| 2213 |
Unsupported Operating Systems |
| 2214 |
----------------------------- |
| 2215 |
|
| 2216 |
OS/2 and VMS are no longer supported due to the lack of a maintainer. |
| 2217 |
|
| 2218 |
Windows 2000 and Windows platforms which set ``COMSPEC`` to ``command.com`` |
| 2219 |
are no longer supported due to maintenance burden. |
| 2220 |
|
| 2221 |
OSF support, which was deprecated in 3.2, has been completely removed. |
| 2222 |
|
| 2223 |
|
| 2224 |
Deprecated Python modules, functions and methods |
| 2225 |
------------------------------------------------ |
| 2226 |
|
| 2227 |
* Passing a non-empty string to ``object.__format__()`` is deprecated, and |
| 2228 |
will produce a :exc:`TypeError` in Python 3.4 (:issue:`9856`). |
| 2229 |
* The ``unicode_internal`` codec has been deprecated because of the |
| 2230 |
:pep:`393`, use UTF-8, UTF-16 (``utf-16-le`` or ``utf-16-be``), or UTF-32 |
| 2231 |
(``utf-32-le`` or ``utf-32-be``) |
| 2232 |
* :meth:`ftplib.FTP.nlst` and :meth:`ftplib.FTP.dir`: use |
| 2233 |
:meth:`ftplib.FTP.mlsd` |
| 2234 |
* :func:`!platform.popen`: use the :mod:`subprocess` module. Check especially |
| 2235 |
the :ref:`subprocess-replacements` section (:issue:`11377`). |
| 2236 |
* :issue:`13374`: The Windows bytes API has been deprecated in the :mod:`os` |
| 2237 |
module. Use Unicode filenames, instead of bytes filenames, to not depend on |
| 2238 |
the ANSI code page anymore and to support any filename. |
| 2239 |
* :issue:`13988`: The :mod:`!xml.etree.cElementTree` module is deprecated. The |
| 2240 |
accelerator is used automatically whenever available. |
| 2241 |
* The behaviour of :func:`!time.clock` depends on the platform: use the new |
| 2242 |
:func:`time.perf_counter` or :func:`time.process_time` function instead, |
| 2243 |
depending on your requirements, to have a well defined behaviour. |
| 2244 |
* The :func:`!os.stat_float_times` function is deprecated. |
| 2245 |
* :mod:`abc` module: |
| 2246 |
|
| 2247 |
* :class:`abc.abstractproperty` has been deprecated, use :class:`property` |
| 2248 |
with :func:`abc.abstractmethod` instead. |
| 2249 |
* :class:`abc.abstractclassmethod` has been deprecated, use |
| 2250 |
:class:`classmethod` with :func:`abc.abstractmethod` instead. |
| 2251 |
* :class:`abc.abstractstaticmethod` has been deprecated, use |
| 2252 |
:class:`staticmethod` with :func:`abc.abstractmethod` instead. |
| 2253 |
|
| 2254 |
* :mod:`importlib` package: |
| 2255 |
|
| 2256 |
* :meth:`importlib.abc.SourceLoader.path_mtime` is now deprecated in favour of |
| 2257 |
:meth:`importlib.abc.SourceLoader.path_stats` as bytecode files now store |
| 2258 |
both the modification time and size of the source file the bytecode file was |
| 2259 |
compiled from. |
| 2260 |
|
| 2261 |
|
| 2262 |
|
| 2263 |
|
| 2264 |
|
| 2265 |
Deprecated functions and types of the C API |
| 2266 |
------------------------------------------- |
| 2267 |
|
| 2268 |
The :c:type:`Py_UNICODE` has been deprecated by :pep:`393` and will be |
| 2269 |
removed in Python 4. All functions using this type are deprecated: |
| 2270 |
|
| 2271 |
Unicode functions and methods using :c:type:`Py_UNICODE` and |
| 2272 |
:c:expr:`Py_UNICODE*` types: |
| 2273 |
|
| 2274 |
* :c:macro:`!PyUnicode_FromUnicode`: use :c:func:`PyUnicode_FromWideChar` or |
| 2275 |
:c:func:`PyUnicode_FromKindAndData` |
| 2276 |
* :c:macro:`!PyUnicode_AS_UNICODE`, :c:func:`!PyUnicode_AsUnicode`, |
| 2277 |
:c:func:`!PyUnicode_AsUnicodeAndSize`: use :c:func:`PyUnicode_AsWideCharString` |
| 2278 |
* :c:macro:`!PyUnicode_AS_DATA`: use :c:macro:`PyUnicode_DATA` with |
| 2279 |
:c:macro:`PyUnicode_READ` and :c:macro:`PyUnicode_WRITE` |
| 2280 |
* :c:macro:`!PyUnicode_GET_SIZE`, :c:func:`!PyUnicode_GetSize`: use |
| 2281 |
:c:macro:`PyUnicode_GET_LENGTH` or :c:func:`PyUnicode_GetLength` |
| 2282 |
* :c:macro:`!PyUnicode_GET_DATA_SIZE`: use |
| 2283 |
``PyUnicode_GET_LENGTH(str) * PyUnicode_KIND(str)`` (only work on ready |
| 2284 |
strings) |
| 2285 |
* :c:func:`!PyUnicode_AsUnicodeCopy`: use :c:func:`PyUnicode_AsUCS4Copy` or |
| 2286 |
:c:func:`PyUnicode_AsWideCharString` |
| 2287 |
* :c:func:`!PyUnicode_GetMax` |
| 2288 |
|
| 2289 |
|
| 2290 |
Functions and macros manipulating Py_UNICODE* strings: |
| 2291 |
|
| 2292 |
* :c:macro:`!Py_UNICODE_strlen()`: use :c:func:`PyUnicode_GetLength` or |
| 2293 |
:c:macro:`PyUnicode_GET_LENGTH` |
| 2294 |
* :c:macro:`!Py_UNICODE_strcat()`: use :c:func:`PyUnicode_CopyCharacters` or |
| 2295 |
:c:func:`PyUnicode_FromFormat` |
| 2296 |
* :c:macro:`!Py_UNICODE_strcpy()`, :c:macro:`!Py_UNICODE_strncpy()`, |
| 2297 |
:c:macro:`!Py_UNICODE_COPY()`: use :c:func:`PyUnicode_CopyCharacters` or |
| 2298 |
:c:func:`PyUnicode_Substring` |
| 2299 |
* :c:macro:`!Py_UNICODE_strcmp()`: use :c:func:`PyUnicode_Compare` |
| 2300 |
* :c:macro:`!Py_UNICODE_strncmp()`: use :c:func:`PyUnicode_Tailmatch` |
| 2301 |
* :c:macro:`!Py_UNICODE_strchr()`, :c:macro:`!Py_UNICODE_strrchr()`: use |
| 2302 |
:c:func:`PyUnicode_FindChar` |
| 2303 |
* :c:macro:`!Py_UNICODE_FILL()`: use :c:func:`PyUnicode_Fill` |
| 2304 |
* :c:macro:`!Py_UNICODE_MATCH` |
| 2305 |
|
| 2306 |
Encoders: |
| 2307 |
|
| 2308 |
* :c:func:`!PyUnicode_Encode`: use :c:func:`!PyUnicode_AsEncodedObject` |
| 2309 |
* :c:func:`!PyUnicode_EncodeUTF7` |
| 2310 |
* :c:func:`!PyUnicode_EncodeUTF8`: use :c:func:`PyUnicode_AsUTF8` or |
| 2311 |
:c:func:`PyUnicode_AsUTF8String` |
| 2312 |
* :c:func:`!PyUnicode_EncodeUTF32` |
| 2313 |
* :c:func:`!PyUnicode_EncodeUTF16` |
| 2314 |
* :c:func:`!PyUnicode_EncodeUnicodeEscape` use |
| 2315 |
:c:func:`PyUnicode_AsUnicodeEscapeString` |
| 2316 |
* :c:func:`!PyUnicode_EncodeRawUnicodeEscape` use |
| 2317 |
:c:func:`PyUnicode_AsRawUnicodeEscapeString` |
| 2318 |
* :c:func:`!PyUnicode_EncodeLatin1`: use :c:func:`PyUnicode_AsLatin1String` |
| 2319 |
* :c:func:`!PyUnicode_EncodeASCII`: use :c:func:`PyUnicode_AsASCIIString` |
| 2320 |
* :c:func:`!PyUnicode_EncodeCharmap` |
| 2321 |
* :c:func:`!PyUnicode_TranslateCharmap` |
| 2322 |
* :c:func:`!PyUnicode_EncodeMBCS`: use :c:func:`PyUnicode_AsMBCSString` or |
| 2323 |
:c:func:`PyUnicode_EncodeCodePage` (with ``CP_ACP`` code_page) |
| 2324 |
* :c:func:`!PyUnicode_EncodeDecimal`, |
| 2325 |
:c:func:`!PyUnicode_TransformDecimalToASCII` |
| 2326 |
|
| 2327 |
|
| 2328 |
Deprecated features |
| 2329 |
------------------- |
| 2330 |
|
| 2331 |
The :mod:`array` module's ``'u'`` format code is now deprecated and will be |
| 2332 |
removed in Python 4 together with the rest of the (:c:type:`Py_UNICODE`) API. |
| 2333 |
|
| 2334 |
|
| 2335 |
Porting to Python 3.3 |
| 2336 |
===================== |
| 2337 |
|
| 2338 |
This section lists previously described changes and other bugfixes |
| 2339 |
that may require changes to your code. |
| 2340 |
|
| 2341 |
.. _portingpythoncode: |
| 2342 |
|
| 2343 |
Porting Python code |
| 2344 |
------------------- |
| 2345 |
|
| 2346 |
* Hash randomization is enabled by default. Set the :envvar:`PYTHONHASHSEED` |
| 2347 |
environment variable to ``0`` to disable hash randomization. See also the |
| 2348 |
:meth:`object.__hash__` method. |
| 2349 |
|
| 2350 |
* :issue:`12326`: On Linux, sys.platform doesn't contain the major version |
| 2351 |
anymore. It is now always 'linux', instead of 'linux2' or 'linux3' depending |
| 2352 |
on the Linux version used to build Python. Replace sys.platform == 'linux2' |
| 2353 |
with sys.platform.startswith('linux'), or directly sys.platform == 'linux' if |
| 2354 |
you don't need to support older Python versions. |
| 2355 |
|
| 2356 |
* :issue:`13847`, :issue:`14180`: :mod:`time` and :mod:`datetime`: |
| 2357 |
:exc:`OverflowError` is now raised instead of :exc:`ValueError` if a |
| 2358 |
timestamp is out of range. :exc:`OSError` is now raised if C functions |
| 2359 |
:c:func:`gmtime` or :c:func:`localtime` failed. |
| 2360 |
|
| 2361 |
* The default finders used by import now utilize a cache of what is contained |
| 2362 |
within a specific directory. If you create a Python source file or sourceless |
| 2363 |
bytecode file, make sure to call :func:`importlib.invalidate_caches` to clear |
| 2364 |
out the cache for the finders to notice the new file. |
| 2365 |
|
| 2366 |
* :exc:`ImportError` now uses the full name of the module that was attempted to |
| 2367 |
be imported. Doctests that check ImportErrors' message will need to be |
| 2368 |
updated to use the full name of the module instead of just the tail of the |
| 2369 |
name. |
| 2370 |
|
| 2371 |
* The *index* argument to :func:`__import__` now defaults to 0 instead of -1 |
| 2372 |
and no longer support negative values. It was an oversight when :pep:`328` was |
| 2373 |
implemented that the default value remained -1. If you need to continue to |
| 2374 |
perform a relative import followed by an absolute import, then perform the |
| 2375 |
relative import using an index of 1, followed by another import using an |
| 2376 |
index of 0. It is preferred, though, that you use |
| 2377 |
:func:`importlib.import_module` rather than call :func:`__import__` directly. |
| 2378 |
|
| 2379 |
* :func:`__import__` no longer allows one to use an index value other than 0 |
| 2380 |
for top-level modules. E.g. ``__import__('sys', level=1)`` is now an error. |
| 2381 |
|
| 2382 |
* Because :data:`sys.meta_path` and :data:`sys.path_hooks` now have finders on |
| 2383 |
them by default, you will most likely want to use :meth:`list.insert` instead |
| 2384 |
of :meth:`list.append` to add to those lists. |
| 2385 |
|
| 2386 |
* Because ``None`` is now inserted into :data:`sys.path_importer_cache`, if you |
| 2387 |
are clearing out entries in the dictionary of paths that do not have a |
| 2388 |
finder, you will need to remove keys paired with values of ``None`` **and** |
| 2389 |
:class:`!imp.NullImporter` to be backwards-compatible. This will lead to extra |
| 2390 |
overhead on older versions of Python that re-insert ``None`` into |
| 2391 |
:data:`sys.path_importer_cache` where it represents the use of implicit |
| 2392 |
finders, but semantically it should not change anything. |
| 2393 |
|
| 2394 |
* :class:`!importlib.abc.Finder` no longer specifies a ``find_module()`` abstract |
| 2395 |
method that must be implemented. If you were relying on subclasses to |
| 2396 |
implement that method, make sure to check for the method's existence first. |
| 2397 |
You will probably want to check for ``find_loader()`` first, though, in the |
| 2398 |
case of working with :term:`path entry finders <path entry finder>`. |
| 2399 |
|
| 2400 |
* :mod:`pkgutil` has been converted to use :mod:`importlib` internally. This |
| 2401 |
eliminates many edge cases where the old behaviour of the :pep:`302` import |
| 2402 |
emulation failed to match the behaviour of the real import system. The |
| 2403 |
import emulation itself is still present, but is now deprecated. The |
| 2404 |
:func:`pkgutil.iter_importers` and :func:`pkgutil.walk_packages` functions |
| 2405 |
special case the standard import hooks so they are still supported even |
| 2406 |
though they do not provide the non-standard ``iter_modules()`` method. |
| 2407 |
|
| 2408 |
* A longstanding RFC-compliance bug (:issue:`1079`) in the parsing done by |
| 2409 |
:func:`email.header.decode_header` has been fixed. Code that uses the |
| 2410 |
standard idiom to convert encoded headers into unicode |
| 2411 |
(``str(make_header(decode_header(h))``) will see no change, but code that |
| 2412 |
looks at the individual tuples returned by decode_header will see that |
| 2413 |
whitespace that precedes or follows ``ASCII`` sections is now included in the |
| 2414 |
``ASCII`` section. Code that builds headers using ``make_header`` should |
| 2415 |
also continue to work without change, since ``make_header`` continues to add |
| 2416 |
whitespace between ``ASCII`` and non-``ASCII`` sections if it is not already |
| 2417 |
present in the input strings. |
| 2418 |
|
| 2419 |
* :func:`email.utils.formataddr` now does the correct content transfer |
| 2420 |
encoding when passed non-``ASCII`` display names. Any code that depended on |
| 2421 |
the previous buggy behavior that preserved the non-``ASCII`` unicode in the |
| 2422 |
formatted output string will need to be changed (:issue:`1690608`). |
| 2423 |
|
| 2424 |
* :meth:`poplib.POP3.quit` may now raise protocol errors like all other |
| 2425 |
``poplib`` methods. Code that assumes ``quit`` does not raise |
| 2426 |
:exc:`poplib.error_proto` errors may need to be changed if errors on ``quit`` |
| 2427 |
are encountered by a particular application (:issue:`11291`). |
| 2428 |
|
| 2429 |
* The ``strict`` argument to :class:`email.parser.Parser`, deprecated since |
| 2430 |
Python 2.4, has finally been removed. |
| 2431 |
|
| 2432 |
* The deprecated method ``unittest.TestCase.assertSameElements`` has been |
| 2433 |
removed. |
| 2434 |
|
| 2435 |
* The deprecated variable ``time.accept2dyear`` has been removed. |
| 2436 |
|
| 2437 |
* The deprecated ``Context._clamp`` attribute has been removed from the |
| 2438 |
:mod:`decimal` module. It was previously replaced by the public attribute |
| 2439 |
:attr:`~decimal.Context.clamp`. (See :issue:`8540`.) |
| 2440 |
|
| 2441 |
* The undocumented internal helper class ``SSLFakeFile`` has been removed |
| 2442 |
from :mod:`smtplib`, since its functionality has long been provided directly |
| 2443 |
by :meth:`socket.socket.makefile`. |
| 2444 |
|
| 2445 |
* Passing a negative value to :func:`time.sleep` on Windows now raises an |
| 2446 |
error instead of sleeping forever. It has always raised an error on posix. |
| 2447 |
|
| 2448 |
* The ``ast.__version__`` constant has been removed. If you need to |
| 2449 |
make decisions affected by the AST version, use :data:`sys.version_info` |
| 2450 |
to make the decision. |
| 2451 |
|
| 2452 |
* Code that used to work around the fact that the :mod:`threading` module used |
| 2453 |
factory functions by subclassing the private classes will need to change to |
| 2454 |
subclass the now-public classes. |
| 2455 |
|
| 2456 |
* The undocumented debugging machinery in the threading module has been |
| 2457 |
removed, simplifying the code. This should have no effect on production |
| 2458 |
code, but is mentioned here in case any application debug frameworks were |
| 2459 |
interacting with it (:issue:`13550`). |
| 2460 |
|
| 2461 |
|
| 2462 |
Porting C code |
| 2463 |
-------------- |
| 2464 |
|
| 2465 |
* In the course of changes to the buffer API the undocumented |
| 2466 |
:c:member:`!smalltable` member of the |
| 2467 |
:c:type:`Py_buffer` structure has been removed and the |
| 2468 |
layout of the :c:type:`!PyMemoryViewObject` has changed. |
| 2469 |
|
| 2470 |
All extensions relying on the relevant parts in ``memoryobject.h`` |
| 2471 |
or ``object.h`` must be rebuilt. |
| 2472 |
|
| 2473 |
* Due to :ref:`PEP 393 <pep-393>`, the :c:type:`Py_UNICODE` type and all |
| 2474 |
functions using this type are deprecated (but will stay available for |
| 2475 |
at least five years). If you were using low-level Unicode APIs to |
| 2476 |
construct and access unicode objects and you want to benefit of the |
| 2477 |
memory footprint reduction provided by :pep:`393`, you have to convert |
| 2478 |
your code to the new :doc:`Unicode API <../c-api/unicode>`. |
| 2479 |
|
| 2480 |
However, if you only have been using high-level functions such as |
| 2481 |
:c:func:`PyUnicode_Concat()`, :c:func:`PyUnicode_Join` or |
| 2482 |
:c:func:`PyUnicode_FromFormat()`, your code will automatically take |
| 2483 |
advantage of the new unicode representations. |
| 2484 |
|
| 2485 |
* :c:func:`PyImport_GetMagicNumber` now returns ``-1`` upon failure. |
| 2486 |
|
| 2487 |
* As a negative value for the *level* argument to :func:`__import__` is no |
| 2488 |
longer valid, the same now holds for :c:func:`PyImport_ImportModuleLevel`. |
| 2489 |
This also means that the value of *level* used by |
| 2490 |
:c:func:`PyImport_ImportModuleEx` is now ``0`` instead of ``-1``. |
| 2491 |
|
| 2492 |
|
| 2493 |
Building C extensions |
| 2494 |
--------------------- |
| 2495 |
|
| 2496 |
* The range of possible file names for C extensions has been narrowed. |
| 2497 |
Very rarely used spellings have been suppressed: under POSIX, files |
| 2498 |
named ``xxxmodule.so``, ``xxxmodule.abi3.so`` and |
| 2499 |
``xxxmodule.cpython-*.so`` are no longer recognized as implementing |
| 2500 |
the ``xxx`` module. If you had been generating such files, you have |
| 2501 |
to switch to the other spellings (i.e., remove the ``module`` string |
| 2502 |
from the file names). |
| 2503 |
|
| 2504 |
(implemented in :issue:`14040`.) |
| 2505 |
|
| 2506 |
|
| 2507 |
Command Line Switch Changes |
| 2508 |
--------------------------- |
| 2509 |
|
| 2510 |
* The -Q command-line flag and related artifacts have been removed. Code |
| 2511 |
checking sys.flags.division_warning will need updating. |
| 2512 |
|
| 2513 |
(:issue:`10998`, contributed by Éric Araujo.) |
| 2514 |
|
| 2515 |
* When :program:`python` is started with :option:`-S`, ``import site`` |
| 2516 |
will no longer add site-specific paths to the module search paths. In |
| 2517 |
previous versions, it did. |
| 2518 |
|
| 2519 |
(:issue:`11591`, contributed by Carl Meyer with editions by Éric Araujo.) |