Debian Codegraph @

Doc/whatsnew/3.6.rst

raw
1 ****************************
2 What's New In Python 3.6
3 ****************************
4
5 :Editors: Elvis Pranskevichus <[email protected]>, Yury Selivanov <[email protected]>
6
7 .. Rules for maintenance:
8
9 * Anyone can add text to this document. Do not spend very much time
10 on the wording of your changes, because your text will probably
11 get rewritten to some degree.
12
13 * The maintainer will go through Misc/NEWS periodically and add
14 changes; it's therefore more important to add your changes to
15 Misc/NEWS than to this file.
16
17 * This is not a complete list of every single change; completeness
18 is the purpose of Misc/NEWS. Some changes I consider too small
19 or esoteric to include. If such a change is added to the text,
20 I'll just remove it. (This is another reason you shouldn't spend
21 too much time on writing your addition.)
22
23 * If you want to draw your new text to the attention of the
24 maintainer, add 'XXX' to the beginning of the paragraph or
25 section.
26
27 * It's OK to just add a fragmentary note about a change. For
28 example: "XXX Describe the transmogrify() function added to the
29 socket module." The maintainer will research the change and
30 write the necessary text.
31
32 * You can comment out your additions if you like, but it's not
33 necessary (especially when a final release is some months away).
34
35 * Credit the author of a patch or bugfix. Just the name is
36 sufficient; the e-mail address isn't necessary.
37
38 * It's helpful to add the bug/patch number as a comment:
39
40 XXX Describe the transmogrify() function added to the socket
41 module.
42 (Contributed by P.Y. Developer in :issue:`12345`.)
43
44 This saves the maintainer the effort of going through the Mercurial log
45 when researching a change.
46
47 This article explains the new features in Python 3.6, compared to 3.5.
48 Python 3.6 was released on December 23, 2016. See the
49 `changelog <https://docs.python.org/3.6/whatsnew/changelog.html>`_ for a full
50 list of changes.
51
52 .. seealso::
53
54 :pep:`494` - Python 3.6 Release Schedule
55
56
57 Summary -- Release highlights
58 =============================
59
60 New syntax features:
61
62 * :ref:`PEP 498 <whatsnew36-pep498>`, formatted string literals.
63
64 * :ref:`PEP 515 <whatsnew36-pep515>`, underscores in numeric literals.
65
66 * :ref:`PEP 526 <whatsnew36-pep526>`, syntax for variable annotations.
67
68 * :ref:`PEP 525 <whatsnew36-pep525>`, asynchronous generators.
69
70 * :ref:`PEP 530 <whatsnew36-pep530>`: asynchronous comprehensions.
71
72
73 New library modules:
74
75 * :mod:`secrets`: :ref:`PEP 506 -- Adding A Secrets Module To The Standard Library <whatsnew36-pep506>`.
76
77
78 CPython implementation improvements:
79
80 * The :ref:`dict <typesmapping>` type has been reimplemented to use
81 a :ref:`more compact representation <whatsnew36-compactdict>`
82 based on `a proposal by Raymond Hettinger
83 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_
84 and similar to the `PyPy dict implementation`_. This resulted in dictionaries
85 using 20% to 25% less memory when compared to Python 3.5.
86
87 * Customization of class creation has been simplified with the
88 :ref:`new protocol <whatsnew36-pep487>`.
89
90 * The class attribute definition order is
91 :ref:`now preserved <whatsnew36-pep520>`.
92
93 * The order of elements in ``**kwargs`` now
94 :ref:`corresponds to the order <whatsnew36-pep468>` in which keyword
95 arguments were passed to the function.
96
97 * DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>` has
98 been added.
99
100 * The new :ref:`PYTHONMALLOC <whatsnew36-pythonmalloc>` environment variable
101 can now be used to debug the interpreter memory allocation and access
102 errors.
103
104
105 Significant improvements in the standard library:
106
107 * The :mod:`asyncio` module has received new features, significant
108 usability and performance improvements, and a fair amount of bug fixes.
109 Starting with Python 3.6 the ``asyncio`` module is no longer provisional
110 and its API is considered stable.
111
112 * A new :ref:`file system path protocol <whatsnew36-pep519>` has been
113 implemented to support :term:`path-like objects <path-like object>`.
114 All standard library functions operating on paths have been updated to
115 work with the new protocol.
116
117 * The :mod:`datetime` module has gained support for
118 :ref:`Local Time Disambiguation <whatsnew36-pep495>`.
119
120 * The :mod:`typing` module received a number of
121 :ref:`improvements <whatsnew36-typing>`.
122
123 * The :mod:`tracemalloc` module has been significantly reworked
124 and is now used to provide better output for :exc:`ResourceWarning`
125 as well as provide better diagnostics for memory allocation errors.
126 See the :ref:`PYTHONMALLOC section <whatsnew36-pythonmalloc>` for more
127 information.
128
129
130 Security improvements:
131
132 * The new :mod:`secrets` module has been added to simplify the generation of
133 cryptographically strong pseudo-random numbers suitable for
134 managing secrets such as account authentication, tokens, and similar.
135
136 * On Linux, :func:`os.urandom` now blocks until the system urandom entropy
137 pool is initialized to increase the security. See the :pep:`524` for the
138 rationale.
139
140 * The :mod:`hashlib` and :mod:`ssl` modules now support OpenSSL 1.1.0.
141
142 * The default settings and feature set of the :mod:`ssl` module have been
143 improved.
144
145 * The :mod:`hashlib` module received support for the BLAKE2, SHA-3 and SHAKE
146 hash algorithms and the :func:`~hashlib.scrypt` key derivation function.
147
148
149 Windows improvements:
150
151 * :ref:`PEP 528 <whatsnew36-pep528>` and :ref:`PEP 529 <whatsnew36-pep529>`,
152 Windows filesystem and console encoding changed to UTF-8.
153
154 * The ``py.exe`` launcher, when used interactively, no longer prefers
155 Python 2 over Python 3 when the user doesn't specify a version (via
156 command line arguments or a config file). Handling of shebang lines
157 remains unchanged - "python" refers to Python 2 in that case.
158
159 * ``python.exe`` and ``pythonw.exe`` have been marked as long-path aware,
160 which means that the 260 character path limit may no longer apply.
161 See :ref:`removing the MAX_PATH limitation <max-path>` for details.
162
163 * A ``._pth`` file can be added to force isolated mode and fully specify
164 all search paths to avoid registry and environment lookup. See
165 :ref:`the documentation <windows_finding_modules>` for more information.
166
167 * A ``python36.zip`` file now works as a landmark to infer
168 :envvar:`PYTHONHOME`. See :ref:`the documentation <windows_finding_modules>` for
169 more information.
170
171
172 .. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
173
174
175 New Features
176 ============
177
178 .. _whatsnew36-pep498:
179
180 PEP 498: Formatted string literals
181 ----------------------------------
182
183 :pep:`498` introduces a new kind of string literals: *f-strings*, or
184 :ref:`formatted string literals <f-strings>`.
185
186 Formatted string literals are prefixed with ``'f'`` and are similar to
187 the format strings accepted by :meth:`str.format`. They contain replacement
188 fields surrounded by curly braces. The replacement fields are expressions,
189 which are evaluated at run time, and then formatted using the
190 :func:`format` protocol::
191
192 >>> name = "Fred"
193 >>> f"He said his name is {name}."
194 'He said his name is Fred.'
195 >>> width = 10
196 >>> precision = 4
197 >>> value = decimal.Decimal("12.34567")
198 >>> f"result: {value:{width}.{precision}}" # nested fields
199 'result: 12.35'
200
201 .. seealso::
202
203 :pep:`498` -- Literal String Interpolation.
204 PEP written and implemented by Eric V. Smith.
205
206 :ref:`Feature documentation <f-strings>`.
207
208
209 .. _whatsnew36-pep526:
210
211 PEP 526: Syntax for variable annotations
212 ----------------------------------------
213
214 :pep:`484` introduced the standard for type annotations of function
215 parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating
216 the types of variables including class variables and instance variables::
217
218 primes: List[int] = []
219
220 captain: str # Note: no initial value!
221
222 class Starship:
223 stats: Dict[str, int] = {}
224
225 Just as for function annotations, the Python interpreter does not attach any
226 particular meaning to variable annotations and only stores them in the
227 ``__annotations__`` attribute of a class or module.
228
229 In contrast to variable declarations in statically typed languages,
230 the goal of annotation syntax is to provide an easy way to specify structured
231 type metadata for third party tools and libraries via the abstract syntax tree
232 and the ``__annotations__`` attribute.
233
234 .. seealso::
235
236 :pep:`526` -- Syntax for variable annotations.
237 PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach,
238 and Guido van Rossum. Implemented by Ivan Levkivskyi.
239
240 Tools that use or will use the new syntax:
241 `mypy <https://www.mypy-lang.org/>`_,
242 `pytype <https://github.com/google/pytype>`_, PyCharm, etc.
243
244
245 .. _whatsnew36-pep515:
246
247 PEP 515: Underscores in Numeric Literals
248 ----------------------------------------
249
250 :pep:`515` adds the ability to use underscores in numeric literals for
251 improved readability. For example::
252
253 >>> 1_000_000_000_000_000
254 1000000000000000
255 >>> 0x_FF_FF_FF_FF
256 4294967295
257
258 Single underscores are allowed between digits and after any base
259 specifier. Leading, trailing, or multiple underscores in a row are not
260 allowed.
261
262 The :ref:`string formatting <formatspec>` language also now has support
263 for the ``'_'`` option to signal the use of an underscore for a thousands
264 separator for floating-point presentation types and for integer
265 presentation type ``'d'``. For integer presentation types ``'b'``,
266 ``'o'``, ``'x'``, and ``'X'``, underscores will be inserted every 4
267 digits::
268
269 >>> '{:_}'.format(1000000)
270 '1_000_000'
271 >>> '{:_x}'.format(0xFFFFFFFF)
272 'ffff_ffff'
273
274 .. seealso::
275
276 :pep:`515` -- Underscores in Numeric Literals
277 PEP written by Georg Brandl and Serhiy Storchaka.
278
279
280 .. _whatsnew36-pep525:
281
282 PEP 525: Asynchronous Generators
283 --------------------------------
284
285 :pep:`492` introduced support for native coroutines and ``async`` / ``await``
286 syntax to Python 3.5. A notable limitation of the Python 3.5 implementation
287 is that it was not possible to use ``await`` and ``yield`` in the same
288 function body. In Python 3.6 this restriction has been lifted, making it
289 possible to define *asynchronous generators*::
290
291 async def ticker(delay, to):
292 """Yield numbers from 0 to *to* every *delay* seconds."""
293 for i in range(to):
294 yield i
295 await asyncio.sleep(delay)
296
297 The new syntax allows for faster and more concise code.
298
299 .. seealso::
300
301 :pep:`525` -- Asynchronous Generators
302 PEP written and implemented by Yury Selivanov.
303
304
305 .. _whatsnew36-pep530:
306
307 PEP 530: Asynchronous Comprehensions
308 ------------------------------------
309
310 :pep:`530` adds support for using ``async for`` in list, set, dict
311 comprehensions and generator expressions::
312
313 result = [i async for i in aiter() if i % 2]
314
315 Additionally, ``await`` expressions are supported in all kinds
316 of comprehensions::
317
318 result = [await fun() for fun in funcs if await condition()]
319
320 .. seealso::
321
322 :pep:`530` -- Asynchronous Comprehensions
323 PEP written and implemented by Yury Selivanov.
324
325
326 .. _whatsnew36-pep487:
327
328 PEP 487: Simpler customization of class creation
329 ------------------------------------------------
330
331 It is now possible to customize subclass creation without using a metaclass.
332 The new ``__init_subclass__`` classmethod will be called on the base class
333 whenever a new subclass is created::
334
335 class PluginBase:
336 subclasses = []
337
338 def __init_subclass__(cls, **kwargs):
339 super().__init_subclass__(**kwargs)
340 cls.subclasses.append(cls)
341
342 class Plugin1(PluginBase):
343 pass
344
345 class Plugin2(PluginBase):
346 pass
347
348 In order to allow zero-argument :func:`super` calls to work correctly from
349 :meth:`~object.__init_subclass__` implementations, custom metaclasses must
350 ensure that the new ``__classcell__`` namespace entry is propagated to
351 ``type.__new__`` (as described in :ref:`class-object-creation`).
352
353 .. seealso::
354
355 :pep:`487` -- Simpler customization of class creation
356 PEP written and implemented by Martin Teichmann.
357
358 :ref:`Feature documentation <class-customization>`
359
360
361 .. _whatsnew36-pep487-descriptors:
362
363 PEP 487: Descriptor Protocol Enhancements
364 -----------------------------------------
365
366 :pep:`487` extends the descriptor protocol to include the new optional
367 :meth:`~object.__set_name__` method. Whenever a new class is defined, the new
368 method will be called on all descriptors included in the definition, providing
369 them with a reference to the class being defined and the name given to the
370 descriptor within the class namespace. In other words, instances of
371 descriptors can now know the attribute name of the descriptor in the
372 owner class::
373
374 class IntField:
375 def __get__(self, instance, owner):
376 return instance.__dict__[self.name]
377
378 def __set__(self, instance, value):
379 if not isinstance(value, int):
380 raise ValueError(f'expecting integer in {self.name}')
381 instance.__dict__[self.name] = value
382
383 # this is the new initializer:
384 def __set_name__(self, owner, name):
385 self.name = name
386
387 class Model:
388 int_field = IntField()
389
390
391 .. seealso::
392
393 :pep:`487` -- Simpler customization of class creation
394 PEP written and implemented by Martin Teichmann.
395
396 :ref:`Feature documentation <descriptors>`
397
398
399 .. _whatsnew36-pep519:
400
401 PEP 519: Adding a file system path protocol
402 -------------------------------------------
403
404 File system paths have historically been represented as :class:`str`
405 or :class:`bytes` objects. This has led to people who write code which
406 operate on file system paths to assume that such objects are only one
407 of those two types (an :class:`int` representing a file descriptor
408 does not count as that is not a file path). Unfortunately that
409 assumption prevents alternative object representations of file system
410 paths like :mod:`pathlib` from working with pre-existing code,
411 including Python's standard library.
412
413 To fix this situation, a new interface represented by
414 :class:`os.PathLike` has been defined. By implementing the
415 :meth:`~os.PathLike.__fspath__` method, an object signals that it
416 represents a path. An object can then provide a low-level
417 representation of a file system path as a :class:`str` or
418 :class:`bytes` object. This means an object is considered
419 :term:`path-like <path-like object>` if it implements
420 :class:`os.PathLike` or is a :class:`str` or :class:`bytes` object
421 which represents a file system path. Code can use :func:`os.fspath`,
422 :func:`os.fsdecode`, or :func:`os.fsencode` to explicitly get a
423 :class:`str` and/or :class:`bytes` representation of a path-like
424 object.
425
426 The built-in :func:`open` function has been updated to accept
427 :class:`os.PathLike` objects, as have all relevant functions in the
428 :mod:`os` and :mod:`os.path` modules, and most other functions and
429 classes in the standard library. The :class:`os.DirEntry` class
430 and relevant classes in :mod:`pathlib` have also been updated to
431 implement :class:`os.PathLike`.
432
433 The hope is that updating the fundamental functions for operating
434 on file system paths will lead to third-party code to implicitly
435 support all :term:`path-like objects <path-like object>` without any
436 code changes, or at least very minimal ones (e.g. calling
437 :func:`os.fspath` at the beginning of code before operating on a
438 path-like object).
439
440 Here are some examples of how the new interface allows for
441 :class:`pathlib.Path` to be used more easily and transparently with
442 pre-existing code::
443
444 >>> import pathlib
445 >>> with open(pathlib.Path("README")) as f:
446 ... contents = f.read()
447 ...
448 >>> import os.path
449 >>> os.path.splitext(pathlib.Path("some_file.txt"))
450 ('some_file', '.txt')
451 >>> os.path.join("/a/b", pathlib.Path("c"))
452 '/a/b/c'
453 >>> import os
454 >>> os.fspath(pathlib.Path("some_file.txt"))
455 'some_file.txt'
456
457 (Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle Zijlstra.)
458
459 .. seealso::
460
461 :pep:`519` -- Adding a file system path protocol
462 PEP written by Brett Cannon and Koos Zevenhoven.
463
464
465 .. _whatsnew36-pep495:
466
467 PEP 495: Local Time Disambiguation
468 ----------------------------------
469
470 In most world locations, there have been and will be times when local clocks
471 are moved back. In those times, intervals are introduced in which local
472 clocks show the same time twice in the same day. In these situations, the
473 information displayed on a local clock (or stored in a Python datetime
474 instance) is insufficient to identify a particular moment in time.
475
476 :pep:`495` adds the new *fold* attribute to instances of
477 :class:`datetime.datetime` and :class:`datetime.time` classes to differentiate
478 between two moments in time for which local times are the same::
479
480 >>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc)
481 >>> for i in range(4):
482 ... u = u0 + i*HOUR
483 ... t = u.astimezone(Eastern)
484 ... print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold)
485 ...
486 04:00:00 UTC = 00:00:00 EDT 0
487 05:00:00 UTC = 01:00:00 EDT 0
488 06:00:00 UTC = 01:00:00 EST 1
489 07:00:00 UTC = 02:00:00 EST 0
490
491 The values of the :attr:`fold <datetime.datetime.fold>` attribute have the
492 value ``0`` for all instances except those that represent the second
493 (chronologically) moment in time in an ambiguous case.
494
495 .. seealso::
496
497 :pep:`495` -- Local Time Disambiguation
498 PEP written by Alexander Belopolsky and Tim Peters, implementation
499 by Alexander Belopolsky.
500
501
502 .. _whatsnew36-pep529:
503
504 PEP 529: Change Windows filesystem encoding to UTF-8
505 ----------------------------------------------------
506
507 Representing filesystem paths is best performed with str (Unicode) rather than
508 bytes. However, there are some situations where using bytes is sufficient and
509 correct.
510
511 Prior to Python 3.6, data loss could result when using bytes paths on Windows.
512 With this change, using bytes to represent paths is now supported on Windows,
513 provided those bytes are encoded with the encoding returned by
514 :func:`sys.getfilesystemencoding`, which now defaults to ``'utf-8'``.
515
516 Applications that do not use str to represent paths should use
517 :func:`os.fsencode` and :func:`os.fsdecode` to ensure their bytes are
518 correctly encoded. To revert to the previous behaviour, set
519 :envvar:`PYTHONLEGACYWINDOWSFSENCODING` or call
520 :func:`sys._enablelegacywindowsfsencoding`.
521
522 See :pep:`529` for more information and discussion of code modifications that
523 may be required.
524
525
526 .. _whatsnew36-pep528:
527
528 PEP 528: Change Windows console encoding to UTF-8
529 -------------------------------------------------
530
531 The default console on Windows will now accept all Unicode characters and
532 provide correctly read str objects to Python code. ``sys.stdin``,
533 ``sys.stdout`` and ``sys.stderr`` now default to utf-8 encoding.
534
535 This change only applies when using an interactive console, and not when
536 redirecting files or pipes. To revert to the previous behaviour for interactive
537 console use, set :envvar:`PYTHONLEGACYWINDOWSSTDIO`.
538
539 .. seealso::
540
541 :pep:`528` -- Change Windows console encoding to UTF-8
542 PEP written and implemented by Steve Dower.
543
544
545 .. _whatsnew36-pep520:
546
547 PEP 520: Preserving Class Attribute Definition Order
548 ----------------------------------------------------
549
550 Attributes in a class definition body have a natural ordering: the same
551 order in which the names appear in the source. This order is now
552 preserved in the new class's :attr:`~type.__dict__` attribute.
553
554 Also, the effective default class *execution* namespace (returned from
555 :ref:`type.__prepare__() <prepare>`) is now an insertion-order-preserving
556 mapping.
557
558 .. seealso::
559
560 :pep:`520` -- Preserving Class Attribute Definition Order
561 PEP written and implemented by Eric Snow.
562
563
564 .. _whatsnew36-pep468:
565
566 PEP 468: Preserving Keyword Argument Order
567 ------------------------------------------
568
569 ``**kwargs`` in a function signature is now guaranteed to be an
570 insertion-order-preserving mapping.
571
572 .. seealso::
573
574 :pep:`468` -- Preserving Keyword Argument Order
575 PEP written and implemented by Eric Snow.
576
577
578 .. _whatsnew36-compactdict:
579
580 New :ref:`dict <typesmapping>` implementation
581 ---------------------------------------------
582
583 The :ref:`dict <typesmapping>` type now uses a "compact" representation
584 based on `a proposal by Raymond Hettinger
585 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_
586 which was `first implemented by PyPy
587 <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_.
588 The memory usage of the new :func:`dict` is between 20% and 25% smaller
589 compared to Python 3.5.
590
591 The order-preserving aspect of this new implementation is considered an
592 implementation detail and should not be relied upon (this may change in
593 the future, but it is desired to have this new dict implementation in
594 the language for a few releases before changing the language spec to mandate
595 order-preserving semantics for all current and future Python
596 implementations; this also helps preserve backwards-compatibility
597 with older versions of the language where random iteration order is
598 still in effect, e.g. Python 3.5).
599
600 (Contributed by INADA Naoki in :issue:`27350`. Idea
601 `originally suggested by Raymond Hettinger
602 <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.)
603
604
605 .. _whatsnew36-pep523:
606
607 PEP 523: Adding a frame evaluation API to CPython
608 -------------------------------------------------
609
610 While Python provides extensive support to customize how code
611 executes, one place it has not done so is in the evaluation of frame
612 objects. If you wanted some way to intercept frame evaluation in
613 Python there really wasn't any way without directly manipulating
614 function pointers for defined functions.
615
616 :pep:`523` changes this by providing an API to make frame
617 evaluation pluggable at the C level. This will allow for tools such
618 as debuggers and JITs to intercept frame evaluation before the
619 execution of Python code begins. This enables the use of alternative
620 evaluation implementations for Python code, tracking frame
621 evaluation, etc.
622
623 This API is not part of the limited C API and is marked as private to
624 signal that usage of this API is expected to be limited and only
625 applicable to very select, low-level use-cases. Semantics of the
626 API will change with Python as necessary.
627
628 .. seealso::
629
630 :pep:`523` -- Adding a frame evaluation API to CPython
631 PEP written by Brett Cannon and Dino Viehland.
632
633
634 .. _whatsnew36-pythonmalloc:
635
636 PYTHONMALLOC environment variable
637 ---------------------------------
638
639 The new :envvar:`PYTHONMALLOC` environment variable allows setting the Python
640 memory allocators and installing debug hooks.
641
642 It is now possible to install debug hooks on Python memory allocators on Python
643 compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks:
644
645 * Newly allocated memory is filled with the byte ``0xCB``
646 * Freed memory is filled with the byte ``0xDB``
647 * Detect violations of the Python memory allocator API. For example,
648 :c:func:`PyObject_Free` called on a memory block allocated by
649 :c:func:`PyMem_Malloc`.
650 * Detect writes before the start of a buffer (buffer underflows)
651 * Detect writes after the end of a buffer (buffer overflows)
652 * Check that the :term:`GIL <global interpreter lock>` is held when allocator
653 functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and
654 :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called.
655
656 Checking if the GIL is held is also a new feature of Python 3.6.
657
658 See the :c:func:`PyMem_SetupDebugHooks` function for debug hooks on Python
659 memory allocators.
660
661 It is now also possible to force the usage of the :c:func:`malloc` allocator of
662 the C library for all Python memory allocations using ``PYTHONMALLOC=malloc``.
663 This is helpful when using external memory debuggers like Valgrind on
664 a Python compiled in release mode.
665
666 On error, the debug hooks on Python memory allocators now use the
667 :mod:`tracemalloc` module to get the traceback where a memory block was
668 allocated.
669
670 Example of fatal error on buffer overflow using
671 ``python3.6 -X tracemalloc=5`` (store 5 frames in traces)::
672
673 Debug memory block at address p=0x7fbcd41666f8: API 'o'
674 4 bytes originally requested
675 The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
676 The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb):
677 at tail+0: 0x02 *** OUCH
678 at tail+1: 0xfb
679 at tail+2: 0xfb
680 at tail+3: 0xfb
681 at tail+4: 0xfb
682 at tail+5: 0xfb
683 at tail+6: 0xfb
684 at tail+7: 0xfb
685 The block was made by call #1233329 to debug malloc/realloc.
686 Data at p: 1a 2b 30 00
687
688 Memory block allocated at (most recent call first):
689 File "test/test_bytes.py", line 323
690 File "unittest/case.py", line 600
691 File "unittest/case.py", line 648
692 File "unittest/suite.py", line 122
693 File "unittest/suite.py", line 84
694
695 Fatal Python error: bad trailing pad byte
696
697 Current thread 0x00007fbcdbd32700 (most recent call first):
698 File "test/test_bytes.py", line 323 in test_hex
699 File "unittest/case.py", line 600 in run
700 File "unittest/case.py", line 648 in __call__
701 File "unittest/suite.py", line 122 in run
702 File "unittest/suite.py", line 84 in __call__
703 File "unittest/suite.py", line 122 in run
704 File "unittest/suite.py", line 84 in __call__
705 ...
706
707 (Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.)
708
709
710 .. _whatsnew36-tracing:
711
712 DTrace and SystemTap probing support
713 ------------------------------------
714
715 Python can now be built ``--with-dtrace`` which enables static markers
716 for the following events in the interpreter:
717
718 * function call/return
719
720 * garbage collection started/finished
721
722 * line of code executed.
723
724 This can be used to instrument running interpreters in production,
725 without the need to recompile specific :ref:`debug builds <debug-build>` or
726 providing application-specific profiling/debugging code.
727
728 More details in :ref:`instrumentation`.
729
730 The current implementation is tested on Linux and macOS. Additional
731 markers may be added in the future.
732
733 (Contributed by Ɓukasz Langa in :issue:`21590`, based on patches by
734 JesĂșs Cea AviĂłn, David Malcolm, and Nikhil Benesch.)
735
736
737 Other Language Changes
738 ======================
739
740 Some smaller changes made to the core Python language are:
741
742 * A ``global`` or ``nonlocal`` statement must now textually appear
743 before the first use of the affected name in the same scope.
744 Previously this was a :exc:`SyntaxWarning`.
745
746 * It is now possible to set a :ref:`special method <specialnames>` to
747 ``None`` to indicate that the corresponding operation is not available.
748 For example, if a class sets :meth:`~object.__iter__` to ``None``, the class
749 is not iterable.
750 (Contributed by Andrew Barnert and Ivan Levkivskyi in :issue:`25958`.)
751
752 * Long sequences of repeated traceback lines are now abbreviated as
753 ``"[Previous line repeated {count} more times]"`` (see
754 :ref:`whatsnew36-traceback` for an example).
755 (Contributed by Emanuel Barry in :issue:`26823`.)
756
757 * Import now raises the new exception :exc:`ModuleNotFoundError`
758 (subclass of :exc:`ImportError`) when it cannot find a module. Code
759 that currently checks for ImportError (in try-except) will still work.
760 (Contributed by Eric Snow in :issue:`15767`.)
761
762 * Class methods relying on zero-argument ``super()`` will now work correctly
763 when called from metaclass methods during class creation.
764 (Contributed by Martin Teichmann in :issue:`23722`.)
765
766
767 New Modules
768 ===========
769
770 .. _whatsnew36-pep506:
771
772 secrets
773 -------
774
775 The main purpose of the new :mod:`secrets` module is to provide an obvious way
776 to reliably generate cryptographically strong pseudo-random values suitable
777 for managing secrets, such as account authentication, tokens, and similar.
778
779 .. warning::
780
781 Note that the pseudo-random generators in the :mod:`random` module
782 should *NOT* be used for security purposes. Use :mod:`secrets`
783 on Python 3.6+ and :func:`os.urandom` on Python 3.5 and earlier.
784
785 .. seealso::
786
787 :pep:`506` -- Adding A Secrets Module To The Standard Library
788 PEP written and implemented by Steven D'Aprano.
789
790
791 Improved Modules
792 ================
793
794 array
795 -----
796
797 Exhausted iterators of :class:`array.array` will now stay exhausted even
798 if the iterated array is extended. This is consistent with the behavior
799 of other mutable sequences.
800
801 Contributed by Serhiy Storchaka in :issue:`26492`.
802
803 ast
804 ---
805
806 The new :class:`ast.Constant` AST node has been added. It can be used
807 by external AST optimizers for the purposes of constant folding.
808
809 Contributed by Victor Stinner in :issue:`26146`.
810
811
812 asyncio
813 -------
814
815 Starting with Python 3.6 the ``asyncio`` module is no longer provisional and its
816 API is considered stable.
817
818 Notable changes in the :mod:`asyncio` module since Python 3.5.0
819 (all backported to 3.5.x due to the provisional status):
820
821 * The :func:`~asyncio.get_event_loop` function has been changed to
822 always return the currently running loop when called from coroutines
823 and callbacks.
824 (Contributed by Yury Selivanov in :issue:`28613`.)
825
826 * The :func:`~asyncio.ensure_future` function and all functions that
827 use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`,
828 now accept all kinds of :term:`awaitable objects <awaitable>`.
829 (Contributed by Yury Selivanov.)
830
831 * New :func:`~asyncio.run_coroutine_threadsafe` function to submit
832 coroutines to event loops from other threads.
833 (Contributed by Vincent Michel.)
834
835 * New :meth:`Transport.is_closing() <asyncio.BaseTransport.is_closing>`
836 method to check if the transport is closing or closed.
837 (Contributed by Yury Selivanov.)
838
839 * The :meth:`loop.create_server() <asyncio.loop.create_server>`
840 method can now accept a list of hosts.
841 (Contributed by Yann Sionneau.)
842
843 * New :meth:`loop.create_future() <asyncio.loop.create_future>`
844 method to create Future objects. This allows alternative event
845 loop implementations, such as
846 `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
847 :class:`asyncio.Future` implementation.
848 (Contributed by Yury Selivanov in :issue:`27041`.)
849
850 * New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>`
851 method to get the current exception handler.
852 (Contributed by Yury Selivanov in :issue:`27040`.)
853
854 * New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>`
855 method to read data from the stream until a separator bytes
856 sequence appears.
857 (Contributed by Mark Korenberg.)
858
859 * The performance of :meth:`StreamReader.readexactly() <asyncio.StreamReader.readexactly>`
860 has been improved.
861 (Contributed by Mark Korenberg in :issue:`28370`.)
862
863 * The :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`
864 method is optimized to avoid calling the system ``getaddrinfo``
865 function if the address is already resolved.
866 (Contributed by A. Jesse Jiryu Davis.)
867
868 * The :meth:`loop.stop() <asyncio.loop.stop>`
869 method has been changed to stop the loop immediately after
870 the current iteration. Any new callbacks scheduled as a result
871 of the last iteration will be discarded.
872 (Contributed by Guido van Rossum in :issue:`25593`.)
873
874 * :meth:`Future.set_exception <asyncio.Future.set_exception>`
875 will now raise :exc:`TypeError` when passed an instance of
876 the :exc:`StopIteration` exception.
877 (Contributed by Chris Angelico in :issue:`26221`.)
878
879 * New :meth:`loop.connect_accepted_socket() <asyncio.loop.connect_accepted_socket>`
880 method to be used by servers that accept connections outside of asyncio,
881 but that use asyncio to handle them.
882 (Contributed by Jim Fulton in :issue:`27392`.)
883
884 * ``TCP_NODELAY`` flag is now set for all TCP transports by default.
885 (Contributed by Yury Selivanov in :issue:`27456`.)
886
887 * New :meth:`loop.shutdown_asyncgens() <asyncio.loop.shutdown_asyncgens>`
888 to properly close pending asynchronous generators before closing the
889 loop.
890 (Contributed by Yury Selivanov in :issue:`28003`.)
891
892 * :class:`Future <asyncio.Future>` and :class:`Task <asyncio.Task>`
893 classes now have an optimized C implementation which makes asyncio
894 code up to 30% faster.
895 (Contributed by Yury Selivanov and INADA Naoki in :issue:`26081`
896 and :issue:`28544`.)
897
898
899 binascii
900 --------
901
902 The :func:`~binascii.b2a_base64` function now accepts an optional *newline*
903 keyword argument to control whether the newline character is appended to the
904 return value.
905 (Contributed by Victor Stinner in :issue:`25357`.)
906
907
908 cmath
909 -----
910
911 The new :const:`cmath.tau` (*τ*) constant has been added.
912 (Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.)
913
914 New constants: :const:`cmath.inf` and :const:`cmath.nan` to
915 match :const:`math.inf` and :const:`math.nan`, and also :const:`cmath.infj`
916 and :const:`cmath.nanj` to match the format used by complex repr.
917 (Contributed by Mark Dickinson in :issue:`23229`.)
918
919
920 collections
921 -----------
922
923 The new :class:`~collections.abc.Collection` abstract base class has been
924 added to represent sized iterable container classes.
925 (Contributed by Ivan Levkivskyi, docs by Neil Girdhar in :issue:`27598`.)
926
927 The new :class:`~collections.abc.Reversible` abstract base class represents
928 iterable classes that also provide the :meth:`~object.__reversed__` method.
929 (Contributed by Ivan Levkivskyi in :issue:`25987`.)
930
931 The new :class:`~collections.abc.AsyncGenerator` abstract base class represents
932 asynchronous generators.
933 (Contributed by Yury Selivanov in :issue:`28720`.)
934
935 The :func:`~collections.namedtuple` function now accepts an optional
936 keyword argument *module*, which, when specified, is used for
937 the :attr:`~type.__module__` attribute of the returned named tuple class.
938 (Contributed by Raymond Hettinger in :issue:`17941`.)
939
940 The *verbose* and *rename* arguments for
941 :func:`~collections.namedtuple` are now keyword-only.
942 (Contributed by Raymond Hettinger in :issue:`25628`.)
943
944 Recursive :class:`collections.deque` instances can now be pickled.
945 (Contributed by Serhiy Storchaka in :issue:`26482`.)
946
947
948 concurrent.futures
949 ------------------
950
951 The :class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>`
952 class constructor now accepts an optional *thread_name_prefix* argument
953 to make it possible to customize the names of the threads created by the
954 pool.
955 (Contributed by Gregory P. Smith in :issue:`27664`.)
956
957
958 contextlib
959 ----------
960
961 The :class:`contextlib.AbstractContextManager` class has been added to
962 provide an abstract base class for context managers. It provides a
963 sensible default implementation for ``__enter__()`` which returns
964 ``self`` and leaves ``__exit__()`` an abstract method. A matching
965 class has been added to the :mod:`typing` module as
966 :class:`typing.ContextManager`.
967 (Contributed by Brett Cannon in :issue:`25609`.)
968
969
970 datetime
971 --------
972
973 The :class:`~datetime.datetime` and :class:`~datetime.time` classes have
974 the new :attr:`~datetime.time.fold` attribute used to disambiguate local time
975 when necessary. Many functions in the :mod:`datetime` have been
976 updated to support local time disambiguation.
977 See :ref:`Local Time Disambiguation <whatsnew36-pep495>` section for more
978 information.
979 (Contributed by Alexander Belopolsky in :issue:`24773`.)
980
981 The :meth:`datetime.strftime() <datetime.datetime.strftime>` and
982 :meth:`date.strftime() <datetime.date.strftime>` methods now support
983 ISO 8601 date directives ``%G``, ``%u`` and ``%V``.
984 (Contributed by Ashley Anderson in :issue:`12006`.)
985
986 The :func:`datetime.isoformat() <datetime.datetime.isoformat>` function
987 now accepts an optional *timespec* argument that specifies the number
988 of additional components of the time value to include.
989 (Contributed by Alessandro Cucci and Alexander Belopolsky in :issue:`19475`.)
990
991 The :meth:`datetime.combine() <datetime.datetime.combine>` now
992 accepts an optional *tzinfo* argument.
993 (Contributed by Alexander Belopolsky in :issue:`27661`.)
994
995
996 decimal
997 -------
998
999 New :meth:`Decimal.as_integer_ratio() <decimal.Decimal.as_integer_ratio>`
1000 method that returns a pair ``(n, d)`` of integers that represent the given
1001 :class:`~decimal.Decimal` instance as a fraction, in lowest terms and
1002 with a positive denominator::
1003
1004 >>> Decimal('-3.14').as_integer_ratio()
1005 (-157, 50)
1006
1007 (Contributed by Stefan Krah amd Mark Dickinson in :issue:`25928`.)
1008
1009
1010
1011 distutils
1012 ---------
1013
1014 The ``default_format`` attribute has been removed from
1015 ``distutils.command.sdist.sdist`` and the ``formats``
1016 attribute defaults to ``['gztar']``. Although not anticipated,
1017 any code relying on the presence of ``default_format`` may
1018 need to be adapted. See :issue:`27819` for more details.
1019
1020
1021 email
1022 -----
1023
1024 The new email API, enabled via the *policy* keyword to various constructors, is
1025 no longer provisional. The :mod:`email` documentation has been reorganized and
1026 rewritten to focus on the new API, while retaining the old documentation for
1027 the legacy API. (Contributed by R. David Murray in :issue:`24277`.)
1028
1029 The :mod:`email.mime` classes now all accept an optional *policy* keyword.
1030 (Contributed by Berker Peksag in :issue:`27331`.)
1031
1032 The :class:`~email.generator.DecodedGenerator` now supports the *policy*
1033 keyword.
1034
1035 There is a new :mod:`~email.policy` attribute,
1036 :attr:`~email.policy.Policy.message_factory`, that controls what class is used
1037 by default when the parser creates new message objects. For the
1038 :attr:`email.policy.compat32` policy this is :class:`~email.message.Message`,
1039 for the new policies it is :class:`~email.message.EmailMessage`.
1040 (Contributed by R. David Murray in :issue:`20476`.)
1041
1042
1043 encodings
1044 ---------
1045
1046 On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP``, and the ``'ansi'``
1047 alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page.
1048 (Contributed by Steve Dower in :issue:`27959`.)
1049
1050
1051 enum
1052 ----
1053
1054 Two new enumeration base classes have been added to the :mod:`enum` module:
1055 :class:`~enum.Flag` and :class:`~enum.IntFlag`. Both are used to define
1056 constants that can be combined using the bitwise operators.
1057 (Contributed by Ethan Furman in :issue:`23591`.)
1058
1059 Many standard library modules have been updated to use the
1060 :class:`~enum.IntFlag` class for their constants.
1061
1062 The new :class:`enum.auto` value can be used to assign values to enum
1063 members automatically::
1064
1065 >>> from enum import Enum, auto
1066 >>> class Color(Enum):
1067 ... red = auto()
1068 ... blue = auto()
1069 ... green = auto()
1070 ...
1071 >>> list(Color)
1072 [<Color.red: 1>, <Color.blue: 2>, <Color.green: 3>]
1073
1074
1075 faulthandler
1076 ------------
1077
1078 On Windows, the :mod:`faulthandler` module now installs a handler for Windows
1079 exceptions: see :func:`faulthandler.enable`. (Contributed by Victor Stinner in
1080 :issue:`23848`.)
1081
1082
1083 fileinput
1084 ---------
1085
1086 :func:`~fileinput.hook_encoded` now supports the *errors* argument.
1087 (Contributed by Joseph Hackman in :issue:`25788`.)
1088
1089
1090 hashlib
1091 -------
1092
1093 :mod:`hashlib` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
1094 (Contributed by Christian Heimes in :issue:`26470`.)
1095
1096 BLAKE2 hash functions were added to the module. :func:`~hashlib.blake2b`
1097 and :func:`~hashlib.blake2s` are always available and support the full
1098 feature set of BLAKE2.
1099 (Contributed by Christian Heimes in :issue:`26798` based on code by
1100 Dmitry Chestnykh and Samuel Neves. Documentation written by Dmitry Chestnykh.)
1101
1102 The SHA-3 hash functions :func:`~hashlib.sha3_224`, :func:`~hashlib.sha3_256`,
1103 :func:`~hashlib.sha3_384`, :func:`~hashlib.sha3_512`, and SHAKE hash functions
1104 :func:`~hashlib.shake_128` and :func:`~hashlib.shake_256` were added.
1105 (Contributed by Christian Heimes in :issue:`16113`. Keccak Code Package
1106 by Guido Bertoni, Joan Daemen, Michaël Peeters, Gilles Van Assche, and
1107 Ronny Van Keer.)
1108
1109 The password-based key derivation function :func:`~hashlib.scrypt` is now
1110 available with OpenSSL 1.1.0 and newer.
1111 (Contributed by Christian Heimes in :issue:`27928`.)
1112
1113 http.client
1114 -----------
1115
1116 :meth:`HTTPConnection.request() <http.client.HTTPConnection.request>` and
1117 :meth:`~http.client.HTTPConnection.endheaders` both now support
1118 chunked encoding request bodies.
1119 (Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.)
1120
1121
1122 idlelib and IDLE
1123 ----------------
1124
1125 The idlelib package is being modernized and refactored to make IDLE look and
1126 work better and to make the code easier to understand, test, and improve. Part
1127 of making IDLE look better, especially on Linux and Mac, is using ttk widgets,
1128 mostly in the dialogs. As a result, IDLE no longer runs with tcl/tk 8.4. It
1129 now requires tcl/tk 8.5 or 8.6. We recommend running the latest release of
1130 either.
1131
1132 'Modernizing' includes renaming and consolidation of idlelib modules. The
1133 renaming of files with partial uppercase names is similar to the renaming of,
1134 for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0. As a
1135 result, imports of idlelib files that worked in 3.5 will usually not work in
1136 3.6. At least a module name change will be needed (see idlelib/README.txt),
1137 sometimes more. (Name changes contributed by Al Swiegart and Terry Reedy in
1138 :issue:`24225`. Most idlelib patches since have been and will be part of the
1139 process.)
1140
1141 In compensation, the eventual result with be that some idlelib classes will be
1142 easier to use, with better APIs and docstrings explaining them. Additional
1143 useful information will be added to idlelib when available.
1144
1145 New in 3.6.2:
1146
1147 Multiple fixes for autocompletion. (Contributed by Louie Lu in :issue:`15786`.)
1148
1149 New in 3.6.3:
1150
1151 Module Browser (on the File menu, formerly called Class Browser),
1152 now displays nested functions and classes in addition to top-level
1153 functions and classes.
1154 (Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy
1155 in :issue:`1612262`.)
1156
1157 The IDLE features formerly implemented as extensions have been reimplemented
1158 as normal features. Their settings have been moved from the Extensions tab
1159 to other dialog tabs.
1160 (Contributed by Charles Wohlganger and Terry Jan Reedy in :issue:`27099`.)
1161
1162 The Settings dialog (Options, Configure IDLE) has been partly rewritten
1163 to improve both appearance and function.
1164 (Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.)
1165
1166 New in 3.6.4:
1167
1168 The font sample now includes a selection of non-Latin characters so that
1169 users can better see the effect of selecting a particular font.
1170 (Contributed by Terry Jan Reedy in :issue:`13802`.)
1171 The sample can be edited to include other characters.
1172 (Contributed by Serhiy Storchaka in :issue:`31860`.)
1173
1174 New in 3.6.6:
1175
1176 Editor code context option revised. Box displays all context lines up to
1177 maxlines. Clicking on a context line jumps the editor to that line. Context
1178 colors for custom themes is added to Highlights tab of Settings dialog.
1179 (Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`,
1180 :issue:`33768`, and :issue:`33679`.)
1181
1182 On Windows, a new API call tells Windows that tk scales for DPI. On Windows
1183 8.1+ or 10, with DPI compatibility properties of the Python binary
1184 unchanged, and a monitor resolution greater than 96 DPI, this should
1185 make text and lines sharper. It should otherwise have no effect.
1186 (Contributed by Terry Jan Reedy in :issue:`33656`.)
1187
1188 New in 3.6.7:
1189
1190 Output over N lines (50 by default) is squeezed down to a button.
1191 N can be changed in the PyShell section of the General page of the
1192 Settings dialog. Fewer, but possibly extra long, lines can be squeezed by
1193 right clicking on the output. Squeezed output can be expanded in place
1194 by double-clicking the button or into the clipboard or a separate window
1195 by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.)
1196
1197
1198 importlib
1199 ---------
1200
1201 Import now raises the new exception :exc:`ModuleNotFoundError`
1202 (subclass of :exc:`ImportError`) when it cannot find a module. Code
1203 that current checks for ``ImportError`` (in try-except) will still work.
1204 (Contributed by Eric Snow in :issue:`15767`.)
1205
1206 :class:`importlib.util.LazyLoader` now calls
1207 :meth:`~importlib.abc.Loader.create_module` on the wrapped loader, removing the
1208 restriction that :class:`importlib.machinery.BuiltinImporter` and
1209 :class:`importlib.machinery.ExtensionFileLoader` couldn't be used with
1210 :class:`importlib.util.LazyLoader`.
1211
1212 :func:`importlib.util.cache_from_source`,
1213 :func:`importlib.util.source_from_cache`, and
1214 :func:`importlib.util.spec_from_file_location` now accept a
1215 :term:`path-like object`.
1216
1217
1218 inspect
1219 -------
1220
1221 The :func:`inspect.signature() <inspect.signature>` function now reports the
1222 implicit ``.0`` parameters generated by the compiler for comprehension and
1223 generator expression scopes as if they were positional-only parameters called
1224 ``implicit0``. (Contributed by Jelle Zijlstra in :issue:`19611`.)
1225
1226 To reduce code churn when upgrading from Python 2.7 and the legacy
1227 :func:`!inspect.getargspec` API, the previously documented deprecation of
1228 :func:`inspect.getfullargspec` has been reversed. While this function is
1229 convenient for single/source Python 2/3 code bases, the richer
1230 :func:`inspect.signature` interface remains the recommended approach for new
1231 code. (Contributed by Nick Coghlan in :issue:`27172`)
1232
1233
1234 json
1235 ----
1236
1237 :func:`json.load` and :func:`json.loads` now support binary input. Encoded
1238 JSON should be represented using either UTF-8, UTF-16, or UTF-32.
1239 (Contributed by Serhiy Storchaka in :issue:`17909`.)
1240
1241
1242 logging
1243 -------
1244
1245 The new :meth:`WatchedFileHandler.reopenIfNeeded() <logging.handlers.WatchedFileHandler.reopenIfNeeded>`
1246 method has been added to add the ability to check if the log file needs to
1247 be reopened.
1248 (Contributed by Marian Horban in :issue:`24884`.)
1249
1250
1251 math
1252 ----
1253
1254 The tau (*τ*) constant has been added to the :mod:`math` and :mod:`cmath`
1255 modules.
1256 (Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.)
1257
1258
1259 multiprocessing
1260 ---------------
1261
1262 :ref:`Proxy Objects <multiprocessing-proxy_objects>` returned by
1263 :func:`multiprocessing.Manager` can now be nested.
1264 (Contributed by Davin Potts in :issue:`6766`.)
1265
1266
1267 os
1268 --
1269
1270 See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details on how the
1271 :mod:`os` and :mod:`os.path` modules now support
1272 :term:`path-like objects <path-like object>`.
1273
1274 :func:`~os.scandir` now supports :class:`bytes` paths on Windows.
1275
1276 A new :meth:`~os.scandir.close` method allows explicitly closing a
1277 :func:`~os.scandir` iterator. The :func:`~os.scandir` iterator now
1278 supports the :term:`context manager` protocol. If a :func:`!scandir`
1279 iterator is neither exhausted nor explicitly closed a :exc:`ResourceWarning`
1280 will be emitted in its destructor.
1281 (Contributed by Serhiy Storchaka in :issue:`25994`.)
1282
1283 On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
1284 is initialized to increase the security. See the :pep:`524` for the rationale.
1285
1286 The Linux ``getrandom()`` syscall (get random bytes) is now exposed as the new
1287 :func:`os.getrandom` function.
1288 (Contributed by Victor Stinner, part of the :pep:`524`)
1289
1290
1291 pathlib
1292 -------
1293
1294 :mod:`pathlib` now supports :term:`path-like objects <path-like object>`.
1295 (Contributed by Brett Cannon in :issue:`27186`.)
1296
1297 See the summary of :ref:`PEP 519 <whatsnew36-pep519>` for details.
1298
1299
1300 pdb
1301 ---
1302
1303 The :class:`~pdb.Pdb` class constructor has a new optional *readrc* argument
1304 to control whether ``.pdbrc`` files should be read.
1305
1306
1307 pickle
1308 ------
1309
1310 Objects that need ``__new__`` called with keyword arguments can now be pickled
1311 using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4.
1312 Protocol version 4 already supports this case. (Contributed by Serhiy
1313 Storchaka in :issue:`24164`.)
1314
1315
1316 pickletools
1317 -----------
1318
1319 :func:`pickletools.dis` now outputs the implicit memo index for the
1320 ``MEMOIZE`` opcode.
1321 (Contributed by Serhiy Storchaka in :issue:`25382`.)
1322
1323
1324 pydoc
1325 -----
1326
1327 The :mod:`pydoc` module has learned to respect the ``MANPAGER``
1328 environment variable.
1329 (Contributed by Matthias Klose in :issue:`8637`.)
1330
1331 :func:`help` and :mod:`pydoc` can now list named tuple fields in the
1332 order they were defined rather than alphabetically.
1333 (Contributed by Raymond Hettinger in :issue:`24879`.)
1334
1335
1336 random
1337 -------
1338
1339 The new :func:`~random.choices` function returns a list of elements of
1340 specified size from the given population with optional weights.
1341 (Contributed by Raymond Hettinger in :issue:`18844`.)
1342
1343
1344 re
1345 --
1346
1347 Added support of modifier spans in regular expressions. Examples:
1348 ``'(?i:p)ython'`` matches ``'python'`` and ``'Python'``, but not ``'PYTHON'``;
1349 ``'(?i)g(?-i:v)r'`` matches ``'GvR'`` and ``'gvr'``, but not ``'GVR'``.
1350 (Contributed by Serhiy Storchaka in :issue:`433028`.)
1351
1352 Match object groups can be accessed by ``__getitem__``, which is
1353 equivalent to ``group()``. So ``mo['name']`` is now equivalent to
1354 ``mo.group('name')``. (Contributed by Eric Smith in :issue:`24454`.)
1355
1356 :class:`~re.Match` objects now support
1357 :meth:`index-like objects <object.__index__>` as group
1358 indices.
1359 (Contributed by Jeroen Demeyer and Xiang Zhang in :issue:`27177`.)
1360
1361
1362 readline
1363 --------
1364
1365 Added :func:`~readline.set_auto_history` to enable or disable
1366 automatic addition of input to the history list. (Contributed by
1367 Tyler Crompton in :issue:`26870`.)
1368
1369
1370 rlcompleter
1371 -----------
1372
1373 Private and special attribute names now are omitted unless the prefix starts
1374 with underscores. A space or a colon is added after some completed keywords.
1375 (Contributed by Serhiy Storchaka in :issue:`25011` and :issue:`25209`.)
1376
1377
1378 shlex
1379 -----
1380
1381 The :class:`~shlex.shlex` has much
1382 :ref:`improved shell compatibility <improved-shell-compatibility>`
1383 through the new *punctuation_chars* argument to control which characters
1384 are treated as punctuation.
1385 (Contributed by Vinay Sajip in :issue:`1521950`.)
1386
1387
1388 site
1389 ----
1390
1391 When specifying paths to add to :data:`sys.path` in a ``.pth`` file,
1392 you may now specify file paths on top of directories (e.g. zip files).
1393 (Contributed by Wolfgang Langner in :issue:`26587`).
1394
1395
1396 sqlite3
1397 -------
1398
1399 :attr:`sqlite3.Cursor.lastrowid` now supports the ``REPLACE`` statement.
1400 (Contributed by Alex LordThorsen in :issue:`16864`.)
1401
1402
1403 socket
1404 ------
1405
1406 The :func:`~socket.socket.ioctl` function now supports the
1407 :const:`~socket.SIO_LOOPBACK_FAST_PATH` control code.
1408 (Contributed by Daniel Stokes in :issue:`26536`.)
1409
1410 The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``,
1411 ``SO_PROTOCOL``, ``SO_PEERSEC``, and ``SO_PASSSEC`` are now supported.
1412 (Contributed by Christian Heimes in :issue:`26907`.)
1413
1414 The :meth:`~socket.socket.setsockopt` now supports the
1415 ``setsockopt(level, optname, None, optlen: int)`` form.
1416 (Contributed by Christian Heimes in :issue:`27744`.)
1417
1418 The socket module now supports the address family
1419 :const:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``,
1420 ``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added.
1421 (Contributed by Christian Heimes in :issue:`27744` with support from
1422 Victor Stinner.)
1423
1424 New Linux constants ``TCP_USER_TIMEOUT`` and ``TCP_CONGESTION`` were added.
1425 (Contributed by Omar Sandoval, :issue:`26273`).
1426
1427
1428 socketserver
1429 ------------
1430
1431 Servers based on the :mod:`socketserver` module, including those
1432 defined in :mod:`http.server`, :mod:`xmlrpc.server` and
1433 :mod:`wsgiref.simple_server`, now support the :term:`context manager`
1434 protocol.
1435 (Contributed by Aviv Palivoda in :issue:`26404`.)
1436
1437 The :attr:`wfile <socketserver.DatagramRequestHandler.wfile>` attribute of
1438 :class:`~socketserver.StreamRequestHandler` classes now implements
1439 the :class:`io.BufferedIOBase` writable interface. In particular,
1440 calling :meth:`~io.BufferedIOBase.write` is now guaranteed to send the
1441 data in full. (Contributed by Martin Panter in :issue:`26721`.)
1442
1443
1444 ssl
1445 ---
1446
1447 :mod:`ssl` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2.
1448 (Contributed by Christian Heimes in :issue:`26470`.)
1449
1450 3DES has been removed from the default cipher suites and ChaCha20 Poly1305
1451 cipher suites have been added.
1452 (Contributed by Christian Heimes in :issue:`27850` and :issue:`27766`.)
1453
1454 :class:`~ssl.SSLContext` has better default configuration for options
1455 and ciphers.
1456 (Contributed by Christian Heimes in :issue:`28043`.)
1457
1458 SSL session can be copied from one client-side connection to another
1459 with the new :class:`~ssl.SSLSession` class. TLS session resumption can
1460 speed up the initial handshake, reduce latency and improve performance
1461 (Contributed by Christian Heimes in :issue:`19500` based on a draft by
1462 Alex Warhawk.)
1463
1464 The new :meth:`~ssl.SSLContext.get_ciphers` method can be used to
1465 get a list of enabled ciphers in order of cipher priority.
1466
1467 All constants and flags have been converted to :class:`~enum.IntEnum` and
1468 :class:`~enum.IntFlag`.
1469 (Contributed by Christian Heimes in :issue:`28025`.)
1470
1471 Server and client-side specific TLS protocols for :class:`~ssl.SSLContext`
1472 were added.
1473 (Contributed by Christian Heimes in :issue:`28085`.)
1474
1475 Added :attr:`ssl.SSLContext.post_handshake_auth` to enable and
1476 :meth:`ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3
1477 post-handshake authentication.
1478 (Contributed by Christian Heimes in :gh:`78851`.)
1479
1480 statistics
1481 ----------
1482
1483 A new :func:`~statistics.harmonic_mean` function has been added.
1484 (Contributed by Steven D'Aprano in :issue:`27181`.)
1485
1486
1487 struct
1488 ------
1489
1490 :mod:`struct` now supports IEEE 754 half-precision floats via the ``'e'``
1491 format specifier.
1492 (Contributed by Eli Stevens, Mark Dickinson in :issue:`11734`.)
1493
1494
1495 subprocess
1496 ----------
1497
1498 :class:`subprocess.Popen` destructor now emits a :exc:`ResourceWarning` warning
1499 if the child process is still running. Use the context manager protocol (``with
1500 proc: ...``) or explicitly call the :meth:`~subprocess.Popen.wait` method to
1501 read the exit status of the child process. (Contributed by Victor Stinner in
1502 :issue:`26741`.)
1503
1504 The :class:`subprocess.Popen` constructor and all functions that pass arguments
1505 through to it now accept *encoding* and *errors* arguments. Specifying either
1506 of these will enable text mode for the *stdin*, *stdout* and *stderr* streams.
1507 (Contributed by Steve Dower in :issue:`6135`.)
1508
1509
1510 sys
1511 ---
1512
1513 The new :func:`~sys.getfilesystemencodeerrors` function returns the name of
1514 the error mode used to convert between Unicode filenames and bytes filenames.
1515 (Contributed by Steve Dower in :issue:`27781`.)
1516
1517 On Windows the return value of the :func:`~sys.getwindowsversion` function
1518 now includes the *platform_version* field which contains the accurate major
1519 version, minor version and build number of the current operating system,
1520 rather than the version that is being emulated for the process
1521 (Contributed by Steve Dower in :issue:`27932`.)
1522
1523
1524 telnetlib
1525 ---------
1526
1527 :class:`!telnetlib.Telnet` is now a context manager (contributed by
1528 Stéphane Wirtel in :issue:`25485`).
1529
1530
1531 time
1532 ----
1533
1534 The :class:`~time.struct_time` attributes :attr:`!tm_gmtoff` and
1535 :attr:`!tm_zone` are now available on all platforms.
1536
1537
1538 timeit
1539 ------
1540
1541 The new :meth:`Timer.autorange() <timeit.Timer.autorange>` convenience
1542 method has been added to call :meth:`Timer.timeit() <timeit.Timer.timeit>`
1543 repeatedly so that the total run time is greater or equal to 200 milliseconds.
1544 (Contributed by Steven D'Aprano in :issue:`6422`.)
1545
1546 :mod:`timeit` now warns when there is substantial (4x) variance
1547 between best and worst times.
1548 (Contributed by Serhiy Storchaka in :issue:`23552`.)
1549
1550
1551 tkinter
1552 -------
1553
1554 Added methods :meth:`!Variable.trace_add`,
1555 :meth:`!Variable.trace_remove` and :meth:`!trace_info`
1556 in the :class:`!tkinter.Variable` class. They replace old methods
1557 :meth:`!trace_variable`, :meth:`!trace`,
1558 :meth:`!trace_vdelete` and
1559 :meth:`!trace_vinfo` that use obsolete Tcl commands and might
1560 not work in future versions of Tcl.
1561 (Contributed by Serhiy Storchaka in :issue:`22115`).
1562
1563
1564 .. _whatsnew36-traceback:
1565
1566 traceback
1567 ---------
1568
1569 Both the traceback module and the interpreter's builtin exception display now
1570 abbreviate long sequences of repeated lines in tracebacks as shown in the
1571 following example::
1572
1573 >>> def f(): f()
1574 ...
1575 >>> f()
1576 Traceback (most recent call last):
1577 File "<stdin>", line 1, in <module>
1578 File "<stdin>", line 1, in f
1579 File "<stdin>", line 1, in f
1580 File "<stdin>", line 1, in f
1581 [Previous line repeated 995 more times]
1582 RecursionError: maximum recursion depth exceeded
1583
1584 (Contributed by Emanuel Barry in :issue:`26823`.)
1585
1586
1587 tracemalloc
1588 -----------
1589
1590 The :mod:`tracemalloc` module now supports tracing memory allocations in
1591 multiple different address spaces.
1592
1593 The new :class:`~tracemalloc.DomainFilter` filter class has been added
1594 to filter block traces by their address space (domain).
1595
1596 (Contributed by Victor Stinner in :issue:`26588`.)
1597
1598
1599 .. _whatsnew36-typing:
1600
1601 typing
1602 ------
1603
1604 Since the :mod:`typing` module is :term:`provisional <provisional API>`,
1605 all changes introduced in Python 3.6 have also been
1606 backported to Python 3.5.x.
1607
1608 The :mod:`typing` module has a much improved support for generic type
1609 aliases. For example ``Dict[str, Tuple[S, T]]`` is now a valid
1610 type annotation.
1611 (Contributed by Guido van Rossum in `Github #195
1612 <https://github.com/python/typing/pull/195>`_.)
1613
1614 The :class:`typing.ContextManager` class has been added for
1615 representing :class:`contextlib.AbstractContextManager`.
1616 (Contributed by Brett Cannon in :issue:`25609`.)
1617
1618 The :class:`typing.Collection` class has been added for
1619 representing :class:`collections.abc.Collection`.
1620 (Contributed by Ivan Levkivskyi in :issue:`27598`.)
1621
1622 The :const:`typing.ClassVar` type construct has been added to
1623 mark class variables. As introduced in :pep:`526`, a variable annotation
1624 wrapped in ClassVar indicates that a given attribute is intended to be used as
1625 a class variable and should not be set on instances of that class.
1626 (Contributed by Ivan Levkivskyi in `Github #280
1627 <https://github.com/python/typing/pull/280>`_.)
1628
1629 A new :const:`~typing.TYPE_CHECKING` constant that is assumed to be
1630 ``True`` by the static type checkers, but is ``False`` at runtime.
1631 (Contributed by Guido van Rossum in `Github #230
1632 <https://github.com/python/typing/issues/230>`_.)
1633
1634 A new :func:`~typing.NewType` helper function has been added to create
1635 lightweight distinct types for annotations::
1636
1637 from typing import NewType
1638
1639 UserId = NewType('UserId', int)
1640 some_id = UserId(524313)
1641
1642 The static type checker will treat the new type as if it were a subclass
1643 of the original type. (Contributed by Ivan Levkivskyi in `Github #189
1644 <https://github.com/python/typing/issues/189>`_.)
1645
1646
1647 unicodedata
1648 -----------
1649
1650 The :mod:`unicodedata` module now uses data from `Unicode 9.0.0
1651 <https://unicode.org/versions/Unicode9.0.0/>`_.
1652 (Contributed by Benjamin Peterson.)
1653
1654
1655 unittest.mock
1656 -------------
1657
1658 The :class:`~unittest.mock.Mock` class has the following improvements:
1659
1660 * Two new methods, :meth:`Mock.assert_called()
1661 <unittest.mock.Mock.assert_called>` and :meth:`Mock.assert_called_once()
1662 <unittest.mock.Mock.assert_called_once>` to check if the mock object
1663 was called.
1664 (Contributed by Amit Saha in :issue:`26323`.)
1665
1666 * The :meth:`Mock.reset_mock() <unittest.mock.Mock.reset_mock>` method
1667 now has two optional keyword only arguments: *return_value* and
1668 *side_effect*.
1669 (Contributed by Kushal Das in :issue:`21271`.)
1670
1671
1672 urllib.request
1673 --------------
1674
1675 If a HTTP request has a file or iterable body (other than a
1676 bytes object) but no ``Content-Length`` header, rather than
1677 throwing an error, :class:`AbstractHTTPHandler <urllib.request.HTTPHandler>`
1678 now falls back to use chunked transfer encoding.
1679 (Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.)
1680
1681
1682 urllib.robotparser
1683 ------------------
1684
1685 :class:`~urllib.robotparser.RobotFileParser` now supports the ``Crawl-delay`` and
1686 ``Request-rate`` extensions.
1687 (Contributed by Nikolay Bogoychev in :issue:`16099`.)
1688
1689
1690 venv
1691 ----
1692
1693 :mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an
1694 alternative prefix for the virtual environment. (Proposed by Ɓukasz Balcerzak
1695 and ported to 3.6 by Stéphane Wirtel in :issue:`22829`.)
1696
1697
1698 warnings
1699 --------
1700
1701 A new optional *source* parameter has been added to the
1702 :func:`warnings.warn_explicit` function: the destroyed object which emitted a
1703 :exc:`ResourceWarning`. A *source* attribute has also been added to
1704 :class:`!warnings.WarningMessage` (contributed by Victor Stinner in
1705 :issue:`26568` and :issue:`26567`).
1706
1707 When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` module is now
1708 used to try to retrieve the traceback where the destroyed object was allocated.
1709
1710 Example with the script ``example.py``::
1711
1712 import warnings
1713
1714 def func():
1715 return open(__file__)
1716
1717 f = func()
1718 f = None
1719
1720 Output of the command ``python3.6 -Wd -X tracemalloc=5 example.py``::
1721
1722 example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'>
1723 f = None
1724 Object allocated at (most recent call first):
1725 File "example.py", lineno 4
1726 return open(__file__)
1727 File "example.py", lineno 6
1728 f = func()
1729
1730 The "Object allocated at" traceback is new and is only displayed if
1731 :mod:`tracemalloc` is tracing Python memory allocations and if the
1732 :mod:`warnings` module was already imported.
1733
1734
1735 winreg
1736 ------
1737
1738 Added the 64-bit integer type :data:`REG_QWORD <winreg.REG_QWORD>`.
1739 (Contributed by Clement Rouault in :issue:`23026`.)
1740
1741
1742 winsound
1743 --------
1744
1745 Allowed keyword arguments to be passed to :func:`Beep <winsound.Beep>`,
1746 :func:`MessageBeep <winsound.MessageBeep>`, and :func:`PlaySound
1747 <winsound.PlaySound>` (:issue:`27982`).
1748
1749
1750 xmlrpc.client
1751 -------------
1752
1753 The :mod:`xmlrpc.client` module now supports unmarshalling
1754 additional data types used by the Apache XML-RPC implementation
1755 for numerics and ``None``.
1756 (Contributed by Serhiy Storchaka in :issue:`26885`.)
1757
1758
1759 zipfile
1760 -------
1761
1762 A new :meth:`ZipInfo.from_file() <zipfile.ZipInfo.from_file>` class method
1763 allows making a :class:`~zipfile.ZipInfo` instance from a filesystem file.
1764 A new :meth:`ZipInfo.is_dir() <zipfile.ZipInfo.is_dir>` method can be used
1765 to check if the :class:`~zipfile.ZipInfo` instance represents a directory.
1766 (Contributed by Thomas Kluyver in :issue:`26039`.)
1767
1768 The :meth:`ZipFile.open() <zipfile.ZipFile.open>` method can now be used to
1769 write data into a ZIP file, as well as for extracting data.
1770 (Contributed by Thomas Kluyver in :issue:`26039`.)
1771
1772
1773 zlib
1774 ----
1775
1776 The :func:`~zlib.compress` and :func:`~zlib.decompress` functions now accept
1777 keyword arguments.
1778 (Contributed by Aviv Palivoda in :issue:`26243` and
1779 Xiang Zhang in :issue:`16764` respectively.)
1780
1781
1782 Optimizations
1783 =============
1784
1785 * The Python interpreter now uses a 16-bit wordcode instead of bytecode which
1786 made a number of opcode optimizations possible.
1787 (Contributed by Demur Rumed with input and reviews from
1788 Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.)
1789
1790 * The :class:`asyncio.Future` class now has an optimized C implementation.
1791 (Contributed by Yury Selivanov and INADA Naoki in :issue:`26081`.)
1792
1793 * The :class:`asyncio.Task` class now has an optimized
1794 C implementation. (Contributed by Yury Selivanov in :issue:`28544`.)
1795
1796 * Various implementation improvements in the :mod:`typing` module
1797 (such as caching of generic types) allow up to 30 times performance
1798 improvements and reduced memory footprint.
1799
1800 * The ASCII decoder is now up to 60 times as fast for error handlers
1801 ``surrogateescape``, ``ignore`` and ``replace`` (Contributed
1802 by Victor Stinner in :issue:`24870`).
1803
1804 * The ASCII and the Latin1 encoders are now up to 3 times as fast for the
1805 error handler ``surrogateescape``
1806 (Contributed by Victor Stinner in :issue:`25227`).
1807
1808 * The UTF-8 encoder is now up to 75 times as fast for error handlers
1809 ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass`` (Contributed
1810 by Victor Stinner in :issue:`25267`).
1811
1812 * The UTF-8 decoder is now up to 15 times as fast for error handlers
1813 ``ignore``, ``replace`` and ``surrogateescape`` (Contributed
1814 by Victor Stinner in :issue:`25301`).
1815
1816 * ``bytes % args`` is now up to 2 times faster. (Contributed by Victor Stinner
1817 in :issue:`25349`).
1818
1819 * ``bytearray % args`` is now between 2.5 and 5 times faster. (Contributed by
1820 Victor Stinner in :issue:`25399`).
1821
1822 * Optimize :meth:`bytes.fromhex` and :meth:`bytearray.fromhex`: they are now
1823 between 2x and 3.5x faster. (Contributed by Victor Stinner in :issue:`25401`).
1824
1825 * Optimize ``bytes.replace(b'', b'.')`` and ``bytearray.replace(b'', b'.')``:
1826 up to 80% faster. (Contributed by Josh Snider in :issue:`26574`).
1827
1828 * Allocator functions of the :c:func:`PyMem_Malloc` domain
1829 (:c:macro:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator
1830 <pymalloc>` instead of :c:func:`malloc` function of the C library. The
1831 pymalloc allocator is optimized for objects smaller or equal to 512 bytes
1832 with a short lifetime, and use :c:func:`malloc` for larger memory blocks.
1833 (Contributed by Victor Stinner in :issue:`26249`).
1834
1835 * :func:`pickle.load` and :func:`pickle.loads` are now up to 10% faster when
1836 deserializing many small objects (Contributed by Victor Stinner in
1837 :issue:`27056`).
1838
1839 * Passing :term:`keyword arguments <keyword argument>` to a function has an
1840 overhead in comparison with passing :term:`positional arguments
1841 <positional argument>`. Now in extension functions implemented with using
1842 Argument Clinic this overhead is significantly decreased.
1843 (Contributed by Serhiy Storchaka in :issue:`27574`).
1844
1845 * Optimized :func:`~glob.glob` and :func:`~glob.iglob` functions in the
1846 :mod:`glob` module; they are now about 3--6 times faster.
1847 (Contributed by Serhiy Storchaka in :issue:`25596`).
1848
1849 * Optimized globbing in :mod:`pathlib` by using :func:`os.scandir`;
1850 it is now about 1.5--4 times faster.
1851 (Contributed by Serhiy Storchaka in :issue:`26032`).
1852
1853 * :class:`xml.etree.ElementTree` parsing, iteration and deepcopy performance
1854 has been significantly improved.
1855 (Contributed by Serhiy Storchaka in :issue:`25638`, :issue:`25873`,
1856 and :issue:`25869`.)
1857
1858 * Creation of :class:`fractions.Fraction` instances from floats and
1859 decimals is now 2 to 3 times faster.
1860 (Contributed by Serhiy Storchaka in :issue:`25971`.)
1861
1862
1863 Build and C API Changes
1864 =======================
1865
1866 * Python now requires some C99 support in the toolchain to build.
1867 Most notably, Python now uses standard integer types and macros in
1868 place of custom macros like ``PY_LONG_LONG``.
1869 For more information, see :pep:`7` and :issue:`17884`.
1870
1871 * Cross-compiling CPython with the Android NDK and the Android API level set to
1872 21 (Android 5.0 Lollipop) or greater runs successfully. While Android is not
1873 yet a supported platform, the Python test suite runs on the Android emulator
1874 with only about 16 tests failures. See the Android meta-issue :issue:`26865`.
1875
1876 * The ``--enable-optimizations`` configure flag has been added. Turning it on
1877 will activate expensive optimizations like PGO.
1878 (Original patch by Alecsandru Patrascu of Intel in :issue:`26359`.)
1879
1880 * The :term:`GIL <global interpreter lock>` must now be held when allocator
1881 functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and
1882 :c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called.
1883
1884 * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data
1885 failed.
1886 (Contributed by Martin Panter in :issue:`5319`.)
1887
1888 * :c:func:`PyArg_ParseTupleAndKeywords` now supports :ref:`positional-only
1889 parameters <positional-only_parameter>`. Positional-only parameters are
1890 defined by empty names.
1891 (Contributed by Serhiy Storchaka in :issue:`26282`).
1892
1893 * ``PyTraceback_Print`` method now abbreviates long sequences of repeated lines
1894 as ``"[Previous line repeated {count} more times]"``.
1895 (Contributed by Emanuel Barry in :issue:`26823`.)
1896
1897 * The new :c:func:`PyErr_SetImportErrorSubclass` function allows for
1898 specifying a subclass of :exc:`ImportError` to raise.
1899 (Contributed by Eric Snow in :issue:`15767`.)
1900
1901 * The new :c:func:`PyErr_ResourceWarning` function can be used to generate
1902 a :exc:`ResourceWarning` providing the source of the resource allocation.
1903 (Contributed by Victor Stinner in :issue:`26567`.)
1904
1905 * The new :c:func:`PyOS_FSPath` function returns the file system
1906 representation of a :term:`path-like object`.
1907 (Contributed by Brett Cannon in :issue:`27186`.)
1908
1909 * The :c:func:`PyUnicode_FSConverter` and :c:func:`PyUnicode_FSDecoder`
1910 functions will now accept :term:`path-like objects <path-like object>`.
1911
1912
1913 Other Improvements
1914 ==================
1915
1916 * When :option:`--version` (short form: :option:`-V`) is supplied twice,
1917 Python prints :data:`sys.version` for detailed information.
1918
1919 .. code-block:: shell-session
1920
1921 $ ./python -VV
1922 Python 3.6.0b4+ (3.6:223967b49e49+, Nov 21 2016, 20:55:04)
1923 [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
1924
1925
1926 Deprecated
1927 ==========
1928
1929 New Keywords
1930 ------------
1931
1932 ``async`` and ``await`` are not recommended to be used as variable, class,
1933 function or module names. Introduced by :pep:`492` in Python 3.5, they will
1934 become proper keywords in Python 3.7. Starting in Python 3.6, the use of
1935 ``async`` or ``await`` as names will generate a :exc:`DeprecationWarning`.
1936
1937
1938 Deprecated Python behavior
1939 --------------------------
1940
1941 Raising the :exc:`StopIteration` exception inside a generator will now
1942 generate a :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError`
1943 in Python 3.7. See :ref:`whatsnew-pep-479` for details.
1944
1945 The :meth:`~object.__aiter__` method is now expected to return an asynchronous
1946 iterator directly instead of returning an awaitable as previously.
1947 Doing the former will trigger a :exc:`DeprecationWarning`. Backward
1948 compatibility will be removed in Python 3.7.
1949 (Contributed by Yury Selivanov in :issue:`27243`.)
1950
1951 A backslash-character pair that is not a valid escape sequence now generates
1952 a :exc:`DeprecationWarning`. Although this will eventually become a
1953 :exc:`SyntaxError`, that will not be for several Python releases.
1954 (Contributed by Emanuel Barry in :issue:`27364`.)
1955
1956 When performing a relative import, falling back on ``__name__`` and
1957 ``__path__`` from the calling module when ``__spec__`` or
1958 ``__package__`` are not defined now raises an :exc:`ImportWarning`.
1959 (Contributed by Rose Ames in :issue:`25791`.)
1960
1961
1962 Deprecated Python modules, functions and methods
1963 ------------------------------------------------
1964
1965 asynchat
1966 ~~~~~~~~
1967
1968 The :mod:`!asynchat` has been deprecated in favor of :mod:`asyncio`.
1969 (Contributed by Mariatta in :issue:`25002`.)
1970
1971
1972 asyncore
1973 ~~~~~~~~
1974
1975 The :mod:`!asyncore` has been deprecated in favor of :mod:`asyncio`.
1976 (Contributed by Mariatta in :issue:`25002`.)
1977
1978
1979 dbm
1980 ~~~
1981
1982 Unlike other :mod:`dbm` implementations, the :mod:`dbm.dumb` module
1983 creates databases with the ``'rw'`` mode and allows modifying the database
1984 opened with the ``'r'`` mode. This behavior is now deprecated and will
1985 be removed in 3.8.
1986 (Contributed by Serhiy Storchaka in :issue:`21708`.)
1987
1988
1989 distutils
1990 ~~~~~~~~~
1991
1992 The undocumented ``extra_path`` argument to the
1993 ``distutils.Distribution`` constructor is now considered deprecated
1994 and will raise a warning if set. Support for this parameter will be
1995 removed in a future Python release. See :issue:`27919` for details.
1996
1997
1998 grp
1999 ~~~
2000
2001 The support of non-integer arguments in :func:`~grp.getgrgid` has been
2002 deprecated.
2003 (Contributed by Serhiy Storchaka in :issue:`26129`.)
2004
2005
2006 importlib
2007 ~~~~~~~~~
2008
2009 The :meth:`importlib.machinery.SourceFileLoader.load_module` and
2010 :meth:`importlib.machinery.SourcelessFileLoader.load_module` methods
2011 are now deprecated. They were the only remaining implementations of
2012 :meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not
2013 been deprecated in previous versions of Python in favour of
2014 :meth:`importlib.abc.Loader.exec_module`.
2015
2016 The :class:`importlib.machinery.WindowsRegistryFinder` class is now
2017 deprecated. As of 3.6.0, it is still added to :data:`sys.meta_path` by
2018 default (on Windows), but this may change in future releases.
2019
2020 os
2021 ~~
2022
2023 Undocumented support of general :term:`bytes-like objects <bytes-like object>`
2024 as paths in :mod:`os` functions, :func:`compile` and similar functions is
2025 now deprecated.
2026 (Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.)
2027
2028 re
2029 ~~
2030
2031 Support for inline flags ``(?letters)`` in the middle of the regular
2032 expression has been deprecated and will be removed in a future Python
2033 version. Flags at the start of a regular expression are still allowed.
2034 (Contributed by Serhiy Storchaka in :issue:`22493`.)
2035
2036 ssl
2037 ~~~
2038
2039 OpenSSL 0.9.8, 1.0.0 and 1.0.1 are deprecated and no longer supported.
2040 In the future the :mod:`ssl` module will require at least OpenSSL 1.0.2 or
2041 1.1.0.
2042
2043 SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname``
2044 in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`,
2045 and :mod:`smtplib` have been deprecated in favor of ``context``.
2046 (Contributed by Christian Heimes in :issue:`28022`.)
2047
2048 A couple of protocols and functions of the :mod:`ssl` module are now
2049 deprecated. Some features will no longer be available in future versions
2050 of OpenSSL. Other features are deprecated in favor of a different API.
2051 (Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.)
2052
2053 tkinter
2054 ~~~~~~~
2055
2056 The :mod:`!tkinter.tix` module is now deprecated. :mod:`tkinter` users
2057 should use :mod:`tkinter.ttk` instead.
2058
2059 .. _whatsnew36-venv:
2060
2061 venv
2062 ~~~~
2063
2064 The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``.
2065 This prevents confusion as to what Python interpreter ``pyvenv`` is
2066 connected to and thus what Python interpreter will be used by the virtual
2067 environment. (Contributed by Brett Cannon in :issue:`25154`.)
2068
2069
2070 xml
2071 ---
2072
2073 * As mitigation against DTD and external entity retrieval, the
2074 :mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process
2075 external entities by default.
2076 (Contributed by Christian Heimes in :gh:`61441`.)
2077
2078
2079 Deprecated functions and types of the C API
2080 -------------------------------------------
2081
2082 Undocumented functions :c:func:`!PyUnicode_AsEncodedObject`,
2083 :c:func:`!PyUnicode_AsDecodedObject`, :c:func:`!PyUnicode_AsEncodedUnicode`
2084 and :c:func:`!PyUnicode_AsDecodedUnicode` are deprecated now.
2085 Use the :ref:`generic codec based API <codec-registry>` instead.
2086
2087
2088 Deprecated Build Options
2089 ------------------------
2090
2091 The ``--with-system-ffi`` configure flag is now on by default on non-macOS
2092 UNIX platforms. It may be disabled by using ``--without-system-ffi``, but
2093 using the flag is deprecated and will not be accepted in Python 3.7.
2094 macOS is unaffected by this change. Note that many OS distributors already
2095 use the ``--with-system-ffi`` flag when building their system Python.
2096
2097
2098 Removed
2099 =======
2100
2101 API and Feature Removals
2102 ------------------------
2103
2104 * Unknown escapes consisting of ``'\'`` and an ASCII letter in
2105 regular expressions will now cause an error. In replacement templates for
2106 :func:`re.sub` they are still allowed, but deprecated.
2107 The :const:`re.LOCALE` flag can now only be used with binary patterns.
2108
2109 * ``inspect.getmoduleinfo()`` was removed (was deprecated since CPython 3.3).
2110 :func:`inspect.getmodulename` should be used for obtaining the module
2111 name for a given path.
2112 (Contributed by Yury Selivanov in :issue:`13248`.)
2113
2114 * ``traceback.Ignore`` class and ``traceback.usage``, ``traceback.modname``,
2115 ``traceback.fullmodname``, ``traceback.find_lines_from_code``,
2116 ``traceback.find_lines``, ``traceback.find_strings``,
2117 ``traceback.find_executable_lines`` methods were removed from the
2118 :mod:`traceback` module. They were undocumented methods deprecated since
2119 Python 3.2 and equivalent functionality is available from private methods.
2120
2121 * The ``tk_menuBar()`` and ``tk_bindForTraversal()`` dummy methods in
2122 :mod:`tkinter` widget classes were removed (corresponding Tk commands
2123 were obsolete since Tk 4.0).
2124
2125 * The :meth:`~zipfile.ZipFile.open` method of the :class:`zipfile.ZipFile`
2126 class no longer supports the ``'U'`` mode (was deprecated since Python 3.4).
2127 Use :class:`io.TextIOWrapper` for reading compressed text files in
2128 :term:`universal newlines` mode.
2129
2130 * The undocumented ``IN``, ``CDROM``, ``DLFCN``, ``TYPES``, ``CDIO``, and
2131 ``STROPTS`` modules have been removed. They had been available in the
2132 platform specific ``Lib/plat-*/`` directories, but were chronically out of
2133 date, inconsistently available across platforms, and unmaintained. The
2134 script that created these modules is still available in the source
2135 distribution at `Tools/scripts/h2py.py
2136 <https://github.com/python/cpython/blob/v3.6.15/Tools/scripts/h2py.py>`_.
2137
2138 * The deprecated ``asynchat.fifo`` class has been removed.
2139
2140
2141 Porting to Python 3.6
2142 =====================
2143
2144 This section lists previously described changes and other bugfixes
2145 that may require changes to your code.
2146
2147 Changes in 'python' Command Behavior
2148 ------------------------------------
2149
2150 * The output of a special Python build with defined ``COUNT_ALLOCS``,
2151 ``SHOW_ALLOC_COUNT`` or ``SHOW_TRACK_COUNT`` macros is now off by
2152 default. It can be re-enabled using the ``-X showalloccount`` option.
2153 It now outputs to ``stderr`` instead of ``stdout``.
2154 (Contributed by Serhiy Storchaka in :issue:`23034`.)
2155
2156
2157 Changes in the Python API
2158 -------------------------
2159
2160 * :func:`open() <open>` will no longer allow combining the ``'U'`` mode flag
2161 with ``'+'``.
2162 (Contributed by Jeff Balogh and John O'Connor in :issue:`2091`.)
2163
2164 * :mod:`sqlite3` no longer implicitly commits an open transaction before DDL
2165 statements.
2166
2167 * On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool
2168 is initialized to increase the security.
2169
2170 * When :meth:`importlib.abc.Loader.exec_module` is defined,
2171 :meth:`importlib.abc.Loader.create_module` must also be defined.
2172
2173 * :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg**
2174 argument is not set. Previously only ``NULL`` was returned.
2175
2176 * The format of the :attr:`~codeobject.co_lnotab` attribute of code objects
2177 changed to support
2178 a negative line number delta. By default, Python does not emit bytecode with
2179 a negative line number delta. Functions using :attr:`frame.f_lineno`,
2180 ``PyFrame_GetLineNumber()`` or ``PyCode_Addr2Line()`` are not affected.
2181 Functions directly decoding :attr:`!co_lnotab` should be updated to use a signed
2182 8-bit integer type for the line number delta, but this is only required to
2183 support applications using a negative line number delta. See
2184 ``Objects/lnotab_notes.txt`` for the :attr:`!co_lnotab` format and how to decode
2185 it, and see the :pep:`511` for the rationale.
2186
2187 * The functions in the :mod:`compileall` module now return booleans instead
2188 of ``1`` or ``0`` to represent success or failure, respectively. Thanks to
2189 booleans being a subclass of integers, this should only be an issue if you
2190 were doing identity checks for ``1`` or ``0``. See :issue:`25768`.
2191
2192 * Reading the :attr:`!port` attribute of
2193 :func:`urllib.parse.urlsplit` and :func:`~urllib.parse.urlparse` results
2194 now raises :exc:`ValueError` for out-of-range values, rather than
2195 returning :const:`None`. See :issue:`20059`.
2196
2197 * The :mod:`!imp` module now raises a :exc:`DeprecationWarning` instead of
2198 :exc:`PendingDeprecationWarning`.
2199
2200 * The following modules have had missing APIs added to their
2201 :attr:`~module.__all__` attributes to match the documented APIs:
2202 :mod:`calendar`, :mod:`!cgi`, :mod:`csv`,
2203 :mod:`~xml.etree.ElementTree`, :mod:`enum`,
2204 :mod:`fileinput`, :mod:`ftplib`, :mod:`logging`, :mod:`mailbox`,
2205 :mod:`mimetypes`, :mod:`optparse`, :mod:`plistlib`, :mod:`!smtpd`,
2206 :mod:`subprocess`, :mod:`tarfile`, :mod:`threading` and
2207 :mod:`wave`. This means they will export new symbols when ``import *``
2208 is used.
2209 (Contributed by Joel Taddei and Jacek KoƂodziej in :issue:`23883`.)
2210
2211 * When performing a relative import, if ``__package__`` does not compare equal
2212 to ``__spec__.parent`` then :exc:`ImportWarning` is raised.
2213 (Contributed by Brett Cannon in :issue:`25791`.)
2214
2215 * When a relative import is performed and no parent package is known, then
2216 :exc:`ImportError` will be raised. Previously, :exc:`SystemError` could be
2217 raised. (Contributed by Brett Cannon in :issue:`18018`.)
2218
2219 * Servers based on the :mod:`socketserver` module, including those
2220 defined in :mod:`http.server`, :mod:`xmlrpc.server` and
2221 :mod:`wsgiref.simple_server`, now only catch exceptions derived
2222 from :exc:`Exception`. Therefore if a request handler raises
2223 an exception like :exc:`SystemExit` or :exc:`KeyboardInterrupt`,
2224 :meth:`~socketserver.BaseServer.handle_error` is no longer called, and
2225 the exception will stop a single-threaded server. (Contributed by
2226 Martin Panter in :issue:`23430`.)
2227
2228 * :func:`!spwd.getspnam` now raises a :exc:`PermissionError` instead of
2229 :exc:`KeyError` if the user doesn't have privileges.
2230
2231 * The :meth:`socket.socket.close` method now raises an exception if
2232 an error (e.g. ``EBADF``) was reported by the underlying system call.
2233 (Contributed by Martin Panter in :issue:`26685`.)
2234
2235 * The *decode_data* argument for the :class:`!smtpd.SMTPChannel` and
2236 :class:`!smtpd.SMTPServer` constructors is now ``False`` by default.
2237 This means that the argument passed to
2238 :meth:`!process_message` is now a bytes object by
2239 default, and :meth:`!process_message` will be passed keyword arguments.
2240 Code that has already been updated in accordance with the deprecation
2241 warning generated by 3.5 will not be affected.
2242
2243 * All optional arguments of the :func:`~json.dump`, :func:`~json.dumps`,
2244 :func:`~json.load` and :func:`~json.loads` functions and
2245 :class:`~json.JSONEncoder` and :class:`~json.JSONDecoder` class
2246 constructors in the :mod:`json` module are now :ref:`keyword-only
2247 <keyword-only_parameter>`.
2248 (Contributed by Serhiy Storchaka in :issue:`18726`.)
2249
2250 * Subclasses of :class:`type` which don't override ``type.__new__`` may no
2251 longer use the one-argument form to get the type of an object.
2252
2253 * As part of :pep:`487`, the handling of keyword arguments passed to
2254 :class:`type` (other than the metaclass hint, ``metaclass``) is now
2255 consistently delegated to :meth:`object.__init_subclass__`. This means that
2256 :meth:`type.__new__ <object.__new__>` and :meth:`type.__init__
2257 <object.__init__>` both now accept arbitrary keyword arguments,
2258 but :meth:`object.__init_subclass__` (which is called from
2259 :meth:`type.__new__ <object.__new__>`) will reject them by default.
2260 Custom metaclasses accepting additional keyword arguments will need to adjust
2261 their calls to :meth:`type.__new__ <object.__new__>`
2262 (whether direct or via :class:`super`) accordingly.
2263
2264 * In ``distutils.command.sdist.sdist``, the ``default_format``
2265 attribute has been removed and is no longer honored. Instead, the
2266 gzipped tarfile format is the default on all platforms and no
2267 platform-specific selection is made.
2268 In environments where distributions are
2269 built on Windows and zip distributions are required, configure
2270 the project with a ``setup.cfg`` file containing the following:
2271
2272 .. code-block:: ini
2273
2274 [sdist]
2275 formats=zip
2276
2277 This behavior has also been backported to earlier Python versions
2278 by Setuptools 26.0.0.
2279
2280 * In the :mod:`urllib.request` module and the
2281 :meth:`http.client.HTTPConnection.request` method, if no Content-Length
2282 header field has been specified and the request body is a file object,
2283 it is now sent with HTTP 1.1 chunked encoding. If a file object has to
2284 be sent to a HTTP 1.0 server, the Content-Length value now has to be
2285 specified by the caller.
2286 (Contributed by Demian Brecht and Rolf Krahl with tweaks from
2287 Martin Panter in :issue:`12319`.)
2288
2289 * The :class:`~csv.DictReader` now returns rows of type
2290 :class:`~collections.OrderedDict`.
2291 (Contributed by Steve Holden in :issue:`27842`.)
2292
2293 * The :const:`!crypt.METHOD_CRYPT` will no longer be added to ``crypt.methods``
2294 if unsupported by the platform.
2295 (Contributed by Victor Stinner in :issue:`25287`.)
2296
2297 * The *verbose* and *rename* arguments for
2298 :func:`~collections.namedtuple` are now keyword-only.
2299 (Contributed by Raymond Hettinger in :issue:`25628`.)
2300
2301 * On Linux, :func:`ctypes.util.find_library` now looks in
2302 ``LD_LIBRARY_PATH`` for shared libraries.
2303 (Contributed by Vinay Sajip in :issue:`9998`.)
2304
2305 * The :class:`imaplib.IMAP4` class now handles flags containing the
2306 ``']'`` character in messages sent from the server to improve
2307 real-world compatibility.
2308 (Contributed by Lita Cho in :issue:`21815`.)
2309
2310 * The :func:`mmap.mmap.write` function now returns the number
2311 of bytes written like other write methods.
2312 (Contributed by Jakub Stasiak in :issue:`26335`.)
2313
2314 * The :func:`pkgutil.iter_modules` and :func:`pkgutil.walk_packages`
2315 functions now return :class:`~pkgutil.ModuleInfo` named tuples.
2316 (Contributed by Ramchandra Apte in :issue:`17211`.)
2317
2318 * :func:`re.sub` now raises an error for invalid numerical group
2319 references in replacement templates even if the pattern is not
2320 found in the string. The error message for invalid group references
2321 now includes the group index and the position of the reference.
2322 (Contributed by SilentGhost, Serhiy Storchaka in :issue:`25953`.)
2323
2324 * :class:`zipfile.ZipFile` will now raise :exc:`NotImplementedError` for
2325 unrecognized compression values. Previously a plain :exc:`RuntimeError`
2326 was raised. Additionally, calling :class:`~zipfile.ZipFile` methods
2327 on a closed ZipFile or calling the :meth:`~zipfile.ZipFile.write` method
2328 on a ZipFile created with mode ``'r'`` will raise a :exc:`ValueError`.
2329 Previously, a :exc:`RuntimeError` was raised in those scenarios.
2330
2331 * when custom metaclasses are combined with zero-argument :func:`super` or
2332 direct references from methods to the implicit ``__class__`` closure
2333 variable, the implicit ``__classcell__`` namespace entry must now be passed
2334 up to ``type.__new__`` for initialisation. Failing to do so will result in
2335 a :exc:`DeprecationWarning` in Python 3.6 and a :exc:`RuntimeError` in
2336 Python 3.8.
2337
2338 * With the introduction of :exc:`ModuleNotFoundError`, import system consumers
2339 may start expecting import system replacements to raise that more specific
2340 exception when appropriate, rather than the less-specific :exc:`ImportError`.
2341 To provide future compatibility with such consumers, implementers of
2342 alternative import systems that completely replace :func:`__import__` will
2343 need to update their implementations to raise the new subclass when a module
2344 can't be found at all. Implementers of compliant plugins to the default
2345 import system shouldn't need to make any changes, as the default import
2346 system will raise the new subclass when appropriate.
2347
2348
2349 Changes in the C API
2350 --------------------
2351
2352 * The :c:func:`PyMem_Malloc` allocator family now uses the :ref:`pymalloc allocator
2353 <pymalloc>` rather than the system :c:func:`malloc`. Applications calling
2354 :c:func:`PyMem_Malloc` without holding the GIL can now crash. Set the
2355 :envvar:`PYTHONMALLOC` environment variable to ``debug`` to validate the
2356 usage of memory allocators in your application. See :issue:`26249`.
2357
2358 * :c:func:`Py_Exit` (and the main interpreter) now override the exit status
2359 with 120 if flushing buffered data failed. See :issue:`5319`.
2360
2361
2362 CPython bytecode changes
2363 ------------------------
2364
2365 There have been several major changes to the :term:`bytecode` in Python 3.6.
2366
2367 * The Python interpreter now uses a 16-bit wordcode instead of bytecode.
2368 (Contributed by Demur Rumed with input and reviews from
2369 Serhiy Storchaka and Victor Stinner in :issue:`26647` and :issue:`28050`.)
2370
2371 * The new :opcode:`!FORMAT_VALUE` and :opcode:`BUILD_STRING` opcodes as part
2372 of the :ref:`formatted string literal <whatsnew36-pep498>` implementation.
2373 (Contributed by Eric Smith in :issue:`25483` and
2374 Serhiy Storchaka in :issue:`27078`.)
2375
2376 * The new :opcode:`!BUILD_CONST_KEY_MAP` opcode to optimize the creation
2377 of dictionaries with constant keys.
2378 (Contributed by Serhiy Storchaka in :issue:`27140`.)
2379
2380 * The function call opcodes have been heavily reworked for better performance
2381 and simpler implementation.
2382 The :opcode:`MAKE_FUNCTION`, :opcode:`!CALL_FUNCTION`,
2383 :opcode:`!CALL_FUNCTION_KW` and :opcode:`!BUILD_MAP_UNPACK_WITH_CALL` opcodes
2384 have been modified, the new :opcode:`CALL_FUNCTION_EX` and
2385 :opcode:`!BUILD_TUPLE_UNPACK_WITH_CALL` have been added, and
2386 ``CALL_FUNCTION_VAR``, ``CALL_FUNCTION_VAR_KW`` and ``MAKE_CLOSURE`` opcodes
2387 have been removed.
2388 (Contributed by Demur Rumed in :issue:`27095`, and Serhiy Storchaka in
2389 :issue:`27213`, :issue:`28257`.)
2390
2391 * The new :opcode:`SETUP_ANNOTATIONS` and :opcode:`!STORE_ANNOTATION` opcodes
2392 have been added to support the new :term:`variable annotation` syntax.
2393 (Contributed by Ivan Levkivskyi in :issue:`27985`.)
2394
2395
2396 Notable changes in Python 3.6.2
2397 ===============================
2398
2399 New ``make regen-all`` build target
2400 -----------------------------------
2401
2402 To simplify cross-compilation, and to ensure that CPython can reliably be
2403 compiled without requiring an existing version of Python to already be
2404 available, the autotools-based build system no longer attempts to implicitly
2405 recompile generated files based on file modification times.
2406
2407 Instead, a new ``make regen-all`` command has been added to force regeneration
2408 of these files when desired (e.g. after an initial version of Python has
2409 already been built based on the pregenerated versions).
2410
2411 More selective regeneration targets are also defined - see
2412 :source:`Makefile.pre.in` for details.
2413
2414 (Contributed by Victor Stinner in :issue:`23404`.)
2415
2416 .. versionadded:: 3.6.2
2417
2418
2419 Removal of ``make touch`` build target
2420 --------------------------------------
2421
2422 The ``make touch`` build target previously used to request implicit regeneration
2423 of generated files by updating their modification times has been removed.
2424
2425 It has been replaced by the new ``make regen-all`` target.
2426
2427 (Contributed by Victor Stinner in :issue:`23404`.)
2428
2429 .. versionchanged:: 3.6.2
2430
2431
2432 Notable changes in Python 3.6.4
2433 ===============================
2434
2435 The ``PyExc_RecursionErrorInst`` singleton that was part of the public API
2436 has been removed as its members being never cleared may cause a segfault
2437 during finalization of the interpreter.
2438 (Contributed by Xavier de Gaye in :issue:`22898` and :issue:`30697`.)
2439
2440
2441 Notable changes in Python 3.6.5
2442 ===============================
2443
2444 The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE``
2445 locale to the ``LC_NUMERIC`` locale in some cases.
2446 (Contributed by Victor Stinner in :issue:`31900`.)
2447
2448
2449 Notable changes in Python 3.6.7
2450 ===============================
2451
2452 :mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process
2453 external entities by default. See also :gh:`61441`.
2454
2455 In 3.6.7 the :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token
2456 when provided with input that does not have a trailing new line. This behavior
2457 now matches what the C tokenizer does internally.
2458 (Contributed by Ammar Askar in :issue:`33899`.)
2459
2460 Notable changes in Python 3.6.10
2461 ================================
2462
2463 Due to significant security concerns, the *reuse_address* parameter of
2464 :meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is
2465 because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more
2466 details, see the documentation for ``loop.create_datagram_endpoint()``.
2467 (Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in
2468 :issue:`37228`.)
2469
2470 Notable changes in Python 3.6.13
2471 ================================
2472
2473 Earlier Python versions allowed using both ``;`` and ``&`` as
2474 query parameter separators in :func:`urllib.parse.parse_qs` and
2475 :func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with
2476 newer W3C recommendations, this has been changed to allow only a single
2477 separator key, with ``&`` as the default. This change also affects
2478 :func:`!cgi.parse` and :func:`!cgi.parse_multipart` as they use the affected
2479 functions internally. For more details, please see their respective
2480 documentation.
2481 (Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.)
2482
2483 Notable changes in Python 3.6.14
2484 ================================
2485
2486 A security fix alters the :class:`ftplib.FTP` behavior to not trust the
2487 IPv4 address sent from the remote server when setting up a passive data
2488 channel. We reuse the ftp server IP address instead. For unusual code
2489 requiring the old behavior, set a ``trust_server_pasv_ipv4_address``
2490 attribute on your FTP instance to ``True``. (See :gh:`87451`)
2491
2492 The presence of newline or tab characters in parts of a URL allows for some
2493 forms of attacks. Following the WHATWG specification that updates RFC 3986,
2494 ASCII newline ``\n``, ``\r`` and tab ``\t`` characters are stripped from the
2495 URL by the parser :func:`urllib.parse` preventing such attacks. The removal
2496 characters are controlled by a new module level variable
2497 ``urllib.parse._UNSAFE_URL_BYTES_TO_REMOVE``. (See :gh:`88048`)