Debian Codegraph @

Doc/whatsnew/3.8.rst

raw
1 ****************************
2 What's New In Python 3.8
3 ****************************
4
5 .. Rules for maintenance:
6
7 * Anyone can add text to this document. Do not spend very much time
8 on the wording of your changes, because your text will probably
9 get rewritten to some degree.
10
11 * The maintainer will go through Misc/NEWS periodically and add
12 changes; it's therefore more important to add your changes to
13 Misc/NEWS than to this file.
14
15 * This is not a complete list of every single change; completeness
16 is the purpose of Misc/NEWS. Some changes I consider too small
17 or esoteric to include. If such a change is added to the text,
18 I'll just remove it. (This is another reason you shouldn't spend
19 too much time on writing your addition.)
20
21 * If you want to draw your new text to the attention of the
22 maintainer, add 'XXX' to the beginning of the paragraph or
23 section.
24
25 * It's OK to just add a fragmentary note about a change. For
26 example: "XXX Describe the transmogrify() function added to the
27 socket module." The maintainer will research the change and
28 write the necessary text.
29
30 * You can comment out your additions if you like, but it's not
31 necessary (especially when a final release is some months away).
32
33 * Credit the author of a patch or bugfix. Just the name is
34 sufficient; the e-mail address isn't necessary.
35
36 * It's helpful to add the bug/patch number as a comment:
37
38 XXX Describe the transmogrify() function added to the socket
39 module.
40 (Contributed by P.Y. Developer in :issue:`12345`.)
41
42 This saves the maintainer the effort of going through the Git log
43 when researching a change.
44
45 :Editor: Raymond Hettinger
46
47 This article explains the new features in Python 3.8, compared to 3.7.
48 Python 3.8 was released on October 14, 2019.
49 For full details, see the :ref:`changelog <changelog>`.
50
51 .. testsetup::
52
53 from math import cos, radians
54 from unicodedata import normalize
55 import re
56 import math
57
58
59 Summary -- Release highlights
60 =============================
61
62 .. This section singles out the most important changes in Python 3.8.
63 Brevity is key.
64
65
66 .. PEP-sized items next.
67
68
69
70 New Features
71 ============
72
73 Assignment expressions
74 ----------------------
75
76 There is new syntax ``:=`` that assigns values to variables as part of a larger
77 expression. It is affectionately known as "the walrus operator" due to
78 its resemblance to `the eyes and tusks of a walrus
79 <https://en.wikipedia.org/wiki/Walrus#/media/File:Pacific_Walrus_-_Bull_(8247646168).jpg>`_.
80
81 In this example, the assignment expression helps avoid calling
82 :func:`len` twice::
83
84 if (n := len(a)) > 10:
85 print(f"List is too long ({n} elements, expected <= 10)")
86
87 A similar benefit arises during regular expression matching where
88 match objects are needed twice, once to test whether a match
89 occurred and another to extract a subgroup::
90
91 discount = 0.0
92 if (mo := re.search(r'(\d+)% discount', advertisement)):
93 discount = float(mo.group(1)) / 100.0
94
95 The operator is also useful with while-loops that compute
96 a value to test loop termination and then need that same
97 value again in the body of the loop::
98
99 # Loop over fixed length blocks
100 while (block := f.read(256)) != '':
101 process(block)
102
103 Another motivating use case arises in list comprehensions where
104 a value computed in a filtering condition is also needed in
105 the expression body::
106
107 [clean_name.title() for name in names
108 if (clean_name := normalize('NFC', name)) in allowed_names]
109
110 Try to limit use of the walrus operator to clean cases that reduce
111 complexity and improve readability.
112
113 See :pep:`572` for a full description.
114
115 (Contributed by Emily Morehouse in :issue:`35224`.)
116
117
118 Positional-only parameters
119 --------------------------
120
121 There is a new function parameter syntax ``/`` to indicate that some
122 function parameters must be specified positionally and cannot be used as
123 keyword arguments. This is the same notation shown by ``help()`` for C
124 functions annotated with Larry Hastings'
125 `Argument Clinic <https://devguide.python.org/development-tools/clinic/>`__ tool.
126
127 In the following example, parameters *a* and *b* are positional-only,
128 while *c* or *d* can be positional or keyword, and *e* or *f* are
129 required to be keywords::
130
131 def f(a, b, /, c, d, *, e, f):
132 print(a, b, c, d, e, f)
133
134 The following is a valid call::
135
136 f(10, 20, 30, d=40, e=50, f=60)
137
138 However, these are invalid calls::
139
140 f(10, b=20, c=30, d=40, e=50, f=60) # b cannot be a keyword argument
141 f(10, 20, 30, 40, 50, f=60) # e must be a keyword argument
142
143 One use case for this notation is that it allows pure Python functions
144 to fully emulate behaviors of existing C coded functions. For example,
145 the built-in :func:`divmod` function does not accept keyword arguments::
146
147 def divmod(a, b, /):
148 "Emulate the built in divmod() function"
149 return (a // b, a % b)
150
151 Another use case is to preclude keyword arguments when the parameter
152 name is not helpful. For example, the builtin :func:`len` function has
153 the signature ``len(obj, /)``. This precludes awkward calls such as::
154
155 len(obj='hello') # The "obj" keyword argument impairs readability
156
157 A further benefit of marking a parameter as positional-only is that it
158 allows the parameter name to be changed in the future without risk of
159 breaking client code. For example, in the :mod:`statistics` module, the
160 parameter name *dist* may be changed in the future. This was made
161 possible with the following function specification::
162
163 def quantiles(dist, /, *, n=4, method='exclusive')
164 ...
165
166 Since the parameters to the left of ``/`` are not exposed as possible
167 keywords, the parameters names remain available for use in ``**kwargs``::
168
169 >>> def f(a, b, /, **kwargs):
170 ... print(a, b, kwargs)
171 ...
172 >>> f(10, 20, a=1, b=2, c=3) # a and b are used in two ways
173 10 20 {'a': 1, 'b': 2, 'c': 3}
174
175 This greatly simplifies the implementation of functions and methods
176 that need to accept arbitrary keyword arguments. For example, here
177 is an excerpt from code in the :mod:`collections` module::
178
179 class Counter(dict):
180
181 def __init__(self, iterable=None, /, **kwds):
182 # Note "iterable" is a possible keyword argument
183
184 See :pep:`570` for a full description.
185
186 (Contributed by Pablo Galindo in :issue:`36540`.)
187
188 .. TODO: Pablo will sprint on docs at PyCon US 2019.
189
190
191 Parallel filesystem cache for compiled bytecode files
192 -----------------------------------------------------
193
194 The new :envvar:`PYTHONPYCACHEPREFIX` setting (also available as
195 :option:`-X` ``pycache_prefix``) configures the implicit bytecode
196 cache to use a separate parallel filesystem tree, rather than
197 the default ``__pycache__`` subdirectories within each source
198 directory.
199
200 The location of the cache is reported in :data:`sys.pycache_prefix`
201 (:const:`None` indicates the default location in ``__pycache__``
202 subdirectories).
203
204 (Contributed by Carl Meyer in :issue:`33499`.)
205
206
207 Debug build uses the same ABI as release build
208 -----------------------------------------------
209
210 The ABI of Python :ref:`debug builds <debug-build>` is now compatible with
211 Python release builds. On Unix, when Python is built in debug mode, it is now
212 possible to load C extensions built in release mode and C extensions built
213 using the stable ABI. The inverse is not true, as debug builds expose
214 additional symbols not available in release builds.
215
216 Defining the ``Py_DEBUG`` macro no longer implies the ``Py_TRACE_REFS`` macro,
217 which introduces the only ABI incompatibility. The ``Py_TRACE_REFS`` macro,
218 which adds the :func:`sys.getobjects` function and the :envvar:`PYTHONDUMPREFS`
219 environment variable, can be set using the new :option:`./configure
220 --with-trace-refs <--with-trace-refs>` build option.
221 (Contributed by Victor Stinner in :issue:`36465`.)
222
223 On Unix, C extensions are no longer linked to libpython except on Android
224 and Cygwin.
225 It is now possible
226 for a statically linked Python to load a C extension built using a shared
227 library Python.
228 (Contributed by Victor Stinner in :issue:`21536`.)
229
230 On Unix, when Python is built in debug mode, import now also looks for C
231 extensions compiled in release mode and for C extensions compiled with the
232 stable ABI.
233 (Contributed by Victor Stinner in :issue:`36722`.)
234
235 To embed Python into an application, a new ``--embed`` option must be passed to
236 ``python3-config --libs --embed`` to get ``-lpython3.8`` (link the application
237 to libpython). To support both 3.8 and older, try ``python3-config --libs
238 --embed`` first and fallback to ``python3-config --libs`` (without ``--embed``)
239 if the previous command fails.
240
241 Add a pkg-config ``python-3.8-embed`` module to embed Python into an
242 application: ``pkg-config python-3.8-embed --libs`` includes ``-lpython3.8``.
243 To support both 3.8 and older, try ``pkg-config python-X.Y-embed --libs`` first
244 and fallback to ``pkg-config python-X.Y --libs`` (without ``--embed``) if the
245 previous command fails (replace ``X.Y`` with the Python version).
246
247 On the other hand, ``pkg-config python3.8 --libs`` no longer contains
248 ``-lpython3.8``. C extensions must not be linked to libpython (except on
249 Android and Cygwin, whose cases are handled by the script);
250 this change is backward incompatible on purpose.
251 (Contributed by Victor Stinner in :issue:`36721`.)
252
253 .. _bpo-36817-whatsnew:
254
255 f-strings support ``=`` for self-documenting expressions and debugging
256 ----------------------------------------------------------------------
257
258 Added an ``=`` specifier to :term:`f-string`\s. An f-string such as
259 ``f'{expr=}'`` will expand to the text of the expression, an equal sign,
260 then the representation of the evaluated expression. For example:
261
262 >>> import datetime as dt
263 >>> user = 'eric_idle'
264 >>> member_since = dt.date(1975, 7, 31)
265 >>> f'{user=} {member_since=}'
266 "user='eric_idle' member_since=datetime.date(1975, 7, 31)"
267
268 The usual :ref:`f-string format specifiers <f-strings>` allow more
269 control over how the result of the expression is displayed::
270
271 >>> delta = dt.date.today() - member_since
272 >>> f'{user=!s} {delta.days=:,d}'
273 'user=eric_idle delta.days=16,075'
274
275 The ``=`` specifier will display the whole expression so that
276 calculations can be shown::
277
278 >>> print(f'{theta=} {cos(radians(theta))=:.3f}')
279 theta=30 cos(radians(theta))=0.866
280
281 (Contributed by Eric V. Smith and Larry Hastings in :issue:`36817`.)
282
283
284 PEP 578: Python Runtime Audit Hooks
285 -----------------------------------
286
287 The PEP adds an Audit Hook and Verified Open Hook. Both are available from
288 Python and native code, allowing applications and frameworks written in pure
289 Python code to take advantage of extra notifications, while also allowing
290 embedders or system administrators to deploy builds of Python where auditing is
291 always enabled.
292
293 See :pep:`578` for full details.
294
295
296 PEP 587: Python Initialization Configuration
297 --------------------------------------------
298
299 The :pep:`587` adds a new C API to configure the Python Initialization
300 providing finer control on the whole configuration and better error reporting.
301
302 New structures:
303
304 * :c:type:`PyConfig`
305 * :c:type:`PyPreConfig`
306 * :c:type:`PyStatus`
307 * :c:type:`PyWideStringList`
308
309 New functions:
310
311 * :c:func:`PyConfig_Clear`
312 * :c:func:`PyConfig_InitIsolatedConfig`
313 * :c:func:`PyConfig_InitPythonConfig`
314 * :c:func:`PyConfig_Read`
315 * :c:func:`PyConfig_SetArgv`
316 * :c:func:`PyConfig_SetBytesArgv`
317 * :c:func:`PyConfig_SetBytesString`
318 * :c:func:`PyConfig_SetString`
319 * :c:func:`PyPreConfig_InitIsolatedConfig`
320 * :c:func:`PyPreConfig_InitPythonConfig`
321 * :c:func:`PyStatus_Error`
322 * :c:func:`PyStatus_Exception`
323 * :c:func:`PyStatus_Exit`
324 * :c:func:`PyStatus_IsError`
325 * :c:func:`PyStatus_IsExit`
326 * :c:func:`PyStatus_NoMemory`
327 * :c:func:`PyStatus_Ok`
328 * :c:func:`PyWideStringList_Append`
329 * :c:func:`PyWideStringList_Insert`
330 * :c:func:`Py_BytesMain`
331 * :c:func:`Py_ExitStatusException`
332 * :c:func:`Py_InitializeFromConfig`
333 * :c:func:`Py_PreInitialize`
334 * :c:func:`Py_PreInitializeFromArgs`
335 * :c:func:`Py_PreInitializeFromBytesArgs`
336 * :c:func:`Py_RunMain`
337
338 This PEP also adds ``_PyRuntimeState.preconfig`` (:c:type:`PyPreConfig` type)
339 and ``PyInterpreterState.config`` (:c:type:`PyConfig` type) fields to these
340 internal structures. ``PyInterpreterState.config`` becomes the new
341 reference configuration, replacing global configuration variables and
342 other private variables.
343
344 See :ref:`Python Initialization Configuration <init-config>` for the
345 documentation.
346
347 See :pep:`587` for a full description.
348
349 (Contributed by Victor Stinner in :issue:`36763`.)
350
351
352 PEP 590: Vectorcall: a fast calling protocol for CPython
353 --------------------------------------------------------
354
355 :ref:`vectorcall` is added to the Python/C API.
356 It is meant to formalize existing optimizations which were already done
357 for various classes.
358 Any :ref:`static type <static-types>` implementing a callable can use this
359 protocol.
360
361 This is currently provisional.
362 The aim is to make it fully public in Python 3.9.
363
364 See :pep:`590` for a full description.
365
366 (Contributed by Jeroen Demeyer, Mark Shannon and Petr Viktorin in :issue:`36974`.)
367
368
369 Pickle protocol 5 with out-of-band data buffers
370 -----------------------------------------------
371
372 When :mod:`pickle` is used to transfer large data between Python processes
373 in order to take advantage of multi-core or multi-machine processing,
374 it is important to optimize the transfer by reducing memory copies, and
375 possibly by applying custom techniques such as data-dependent compression.
376
377 The :mod:`pickle` protocol 5 introduces support for out-of-band buffers
378 where :pep:`3118`-compatible data can be transmitted separately from the
379 main pickle stream, at the discretion of the communication layer.
380
381 See :pep:`574` for a full description.
382
383 (Contributed by Antoine Pitrou in :issue:`36785`.)
384
385
386 Other Language Changes
387 ======================
388
389 * A :keyword:`continue` statement was illegal in the :keyword:`finally` clause
390 due to a problem with the implementation. In Python 3.8 this restriction
391 was lifted.
392 (Contributed by Serhiy Storchaka in :issue:`32489`.)
393
394 * The :class:`bool`, :class:`int`, and :class:`fractions.Fraction` types
395 now have an :meth:`~int.as_integer_ratio` method like that found in
396 :class:`float` and :class:`decimal.Decimal`. This minor API extension
397 makes it possible to write ``numerator, denominator =
398 x.as_integer_ratio()`` and have it work across multiple numeric types.
399 (Contributed by Lisa Roach in :issue:`33073` and Raymond Hettinger in
400 :issue:`37819`.)
401
402 * Constructors of :class:`int`, :class:`float` and :class:`complex` will now
403 use the :meth:`~object.__index__` special method, if available and the
404 corresponding method :meth:`~object.__int__`, :meth:`~object.__float__`
405 or :meth:`~object.__complex__` is not available.
406 (Contributed by Serhiy Storchaka in :issue:`20092`.)
407
408 * Added support of :samp:`\\N\\{{name}\\}` escapes in :mod:`regular expressions <re>`::
409
410 >>> notice = 'Copyright © 2019'
411 >>> copyright_year_pattern = re.compile(r'\N{copyright sign}\s*(\d{4})')
412 >>> int(copyright_year_pattern.search(notice).group(1))
413 2019
414
415 (Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.)
416
417 * Dict and dictviews are now iterable in reversed insertion order using
418 :func:`reversed`. (Contributed by Rémi Lapeyre in :issue:`33462`.)
419
420 * The syntax allowed for keyword names in function calls was further
421 restricted. In particular, ``f((keyword)=arg)`` is no longer allowed. It was
422 never intended to permit more than a bare name on the left-hand side of a
423 keyword argument assignment term.
424 (Contributed by Benjamin Peterson in :issue:`34641`.)
425
426 * Generalized iterable unpacking in :keyword:`yield` and
427 :keyword:`return` statements no longer requires enclosing parentheses.
428 This brings the *yield* and *return* syntax into better agreement with
429 normal assignment syntax::
430
431 >>> def parse(family):
432 ... lastname, *members = family.split()
433 ... return lastname.upper(), *members
434 ...
435 >>> parse('simpsons homer marge bart lisa maggie')
436 ('SIMPSONS', 'homer', 'marge', 'bart', 'lisa', 'maggie')
437
438 (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.)
439
440 * When a comma is missed in code such as ``[(10, 20) (30, 40)]``, the
441 compiler displays a :exc:`SyntaxWarning` with a helpful suggestion.
442 This improves on just having a :exc:`TypeError` indicating that the
443 first tuple was not callable. (Contributed by Serhiy Storchaka in
444 :issue:`15248`.)
445
446 * Arithmetic operations between subclasses of :class:`datetime.date` or
447 :class:`datetime.datetime` and :class:`datetime.timedelta` objects now return
448 an instance of the subclass, rather than the base class. This also affects
449 the return type of operations whose implementation (directly or indirectly)
450 uses :class:`datetime.timedelta` arithmetic, such as
451 :meth:`~datetime.datetime.astimezone`.
452 (Contributed by Paul Ganssle in :issue:`32417`.)
453
454 * When the Python interpreter is interrupted by Ctrl-C (SIGINT) and the
455 resulting :exc:`KeyboardInterrupt` exception is not caught, the Python process
456 now exits via a SIGINT signal or with the correct exit code such that the
457 calling process can detect that it died due to a Ctrl-C. Shells on POSIX
458 and Windows use this to properly terminate scripts in interactive sessions.
459 (Contributed by Google via Gregory P. Smith in :issue:`1054041`.)
460
461 * Some advanced styles of programming require updating the
462 :class:`types.CodeType` object for an existing function. Since code
463 objects are immutable, a new code object needs to be created, one
464 that is modeled on the existing code object. With 19 parameters,
465 this was somewhat tedious. Now, the new ``replace()`` method makes
466 it possible to create a clone with a few altered parameters.
467
468 Here's an example that alters the :func:`statistics.mean` function to
469 prevent the *data* parameter from being used as a keyword argument::
470
471 >>> from statistics import mean
472 >>> mean(data=[10, 20, 90])
473 40
474 >>> mean.__code__ = mean.__code__.replace(co_posonlyargcount=1)
475 >>> mean(data=[10, 20, 90])
476 Traceback (most recent call last):
477 ...
478 TypeError: mean() got some positional-only arguments passed as keyword arguments: 'data'
479
480 (Contributed by Victor Stinner in :issue:`37032`.)
481
482 * For integers, the three-argument form of the :func:`pow` function now
483 permits the exponent to be negative in the case where the base is
484 relatively prime to the modulus. It then computes a modular inverse to
485 the base when the exponent is ``-1``, and a suitable power of that
486 inverse for other negative exponents. For example, to compute the
487 `modular multiplicative inverse
488 <https://en.wikipedia.org/wiki/Modular_multiplicative_inverse>`_ of 38
489 modulo 137, write::
490
491 >>> pow(38, -1, 137)
492 119
493 >>> 119 * 38 % 137
494 1
495
496 Modular inverses arise in the solution of `linear Diophantine
497 equations <https://en.wikipedia.org/wiki/Diophantine_equation>`_.
498 For example, to find integer solutions for ``4258𝑥 + 147𝑦 = 369``,
499 first rewrite as ``4258𝑥 ≡ 369 (mod 147)`` then solve:
500
501 >>> x = 369 * pow(4258, -1, 147) % 147
502 >>> y = (4258 * x - 369) // -147
503 >>> 4258 * x + 147 * y
504 369
505
506 (Contributed by Mark Dickinson in :issue:`36027`.)
507
508 * Dict comprehensions have been synced-up with dict literals so that the
509 key is computed first and the value second::
510
511 >>> # Dict comprehension
512 >>> cast = {input('role? '): input('actor? ') for i in range(2)}
513 role? King Arthur
514 actor? Chapman
515 role? Black Knight
516 actor? Cleese
517
518 >>> # Dict literal
519 >>> cast = {input('role? '): input('actor? ')}
520 role? Sir Robin
521 actor? Eric Idle
522
523 The guaranteed execution order is helpful with assignment expressions
524 because variables assigned in the key expression will be available in
525 the value expression::
526
527 >>> names = ['Martin von Löwis', 'Łukasz Langa', 'Walter Dörwald']
528 >>> {(n := normalize('NFC', name)).casefold() : n for name in names}
529 {'martin von löwis': 'Martin von Löwis',
530 'łukasz langa': 'Łukasz Langa',
531 'walter dörwald': 'Walter Dörwald'}
532
533 (Contributed by Jörn Heissler in :issue:`35224`.)
534
535 * The :meth:`object.__reduce__` method can now return a tuple from two to
536 six elements long. Formerly, five was the limit. The new, optional sixth
537 element is a callable with a ``(obj, state)`` signature. This allows the
538 direct control over the state-updating behavior of a specific object. If
539 not *None*, this callable will have priority over the object's
540 :meth:`~object.__setstate__` method.
541 (Contributed by Pierre Glaser and Olivier Grisel in :issue:`35900`.)
542
543 New Modules
544 ===========
545
546 * The new :mod:`importlib.metadata` module provides (provisional) support for
547 reading metadata from third-party packages. For example, it can extract an
548 installed package's version number, list of entry points, and more::
549
550 >>> # Note following example requires that the popular "requests"
551 >>> # package has been installed.
552 >>>
553 >>> from importlib.metadata import version, requires, files
554 >>> version('requests')
555 '2.22.0'
556 >>> list(requires('requests'))
557 ['chardet (<3.1.0,>=3.0.2)']
558 >>> list(files('requests'))[:5]
559 [PackagePath('requests-2.22.0.dist-info/INSTALLER'),
560 PackagePath('requests-2.22.0.dist-info/LICENSE'),
561 PackagePath('requests-2.22.0.dist-info/METADATA'),
562 PackagePath('requests-2.22.0.dist-info/RECORD'),
563 PackagePath('requests-2.22.0.dist-info/WHEEL')]
564
565 (Contributed by Barry Warsaw and Jason R. Coombs in :issue:`34632`.)
566
567
568 Improved Modules
569 ================
570
571 ast
572 ---
573
574 AST nodes now have ``end_lineno`` and ``end_col_offset`` attributes,
575 which give the precise location of the end of the node. (This only
576 applies to nodes that have ``lineno`` and ``col_offset`` attributes.)
577
578 New function :func:`ast.get_source_segment` returns the source code
579 for a specific AST node.
580
581 (Contributed by Ivan Levkivskyi in :issue:`33416`.)
582
583 The :func:`ast.parse` function has some new flags:
584
585 * ``type_comments=True`` causes it to return the text of :pep:`484` and
586 :pep:`526` type comments associated with certain AST nodes;
587
588 * ``mode='func_type'`` can be used to parse :pep:`484` "signature type
589 comments" (returned for function definition AST nodes);
590
591 * ``feature_version=(3, N)`` allows specifying an earlier Python 3
592 version. For example, ``feature_version=(3, 4)`` will treat
593 :keyword:`async` and :keyword:`await` as non-reserved words.
594
595 (Contributed by Guido van Rossum in :issue:`35766`.)
596
597
598 asyncio
599 -------
600
601 :func:`asyncio.run` has graduated from the provisional to stable API. This
602 function can be used to execute a :term:`coroutine` and return the result while
603 automatically managing the event loop. For example::
604
605 import asyncio
606
607 async def main():
608 await asyncio.sleep(0)
609 return 42
610
611 asyncio.run(main())
612
613 This is *roughly* equivalent to::
614
615 import asyncio
616
617 async def main():
618 await asyncio.sleep(0)
619 return 42
620
621 loop = asyncio.new_event_loop()
622 asyncio.set_event_loop(loop)
623 try:
624 loop.run_until_complete(main())
625 finally:
626 asyncio.set_event_loop(None)
627 loop.close()
628
629
630 The actual implementation is significantly more complex. Thus,
631 :func:`asyncio.run` should be the preferred way of running asyncio programs.
632
633 (Contributed by Yury Selivanov in :issue:`32314`.)
634
635 Running ``python -m asyncio`` launches a natively async REPL. This allows rapid
636 experimentation with code that has a top-level :keyword:`await`. There is no
637 longer a need to directly call ``asyncio.run()`` which would spawn a new event
638 loop on every invocation:
639
640 .. code-block:: none
641
642 $ python -m asyncio
643 asyncio REPL 3.8.0
644 Use "await" directly instead of "asyncio.run()".
645 Type "help", "copyright", "credits" or "license" for more information.
646 >>> import asyncio
647 >>> await asyncio.sleep(10, result='hello')
648 hello
649
650 (Contributed by Yury Selivanov in :issue:`37028`.)
651
652 The exception :class:`asyncio.CancelledError` now inherits from
653 :class:`BaseException` rather than :class:`Exception` and no longer inherits
654 from :class:`concurrent.futures.CancelledError`.
655 (Contributed by Yury Selivanov in :issue:`32528`.)
656
657 On Windows, the default event loop is now :class:`~asyncio.ProactorEventLoop`.
658 (Contributed by Victor Stinner in :issue:`34687`.)
659
660 :class:`~asyncio.ProactorEventLoop` now also supports UDP.
661 (Contributed by Adam Meily and Andrew Svetlov in :issue:`29883`.)
662
663 :class:`~asyncio.ProactorEventLoop` can now be interrupted by
664 :exc:`KeyboardInterrupt` ("CTRL+C").
665 (Contributed by Vladimir Matveev in :issue:`23057`.)
666
667 Added :meth:`asyncio.Task.get_coro` for getting the wrapped coroutine
668 within an :class:`asyncio.Task`.
669 (Contributed by Alex Grönholm in :issue:`36999`.)
670
671 Asyncio tasks can now be named, either by passing the ``name`` keyword
672 argument to :func:`asyncio.create_task` or
673 the :meth:`~asyncio.loop.create_task` event loop method, or by
674 calling the :meth:`~asyncio.Task.set_name` method on the task object. The
675 task name is visible in the ``repr()`` output of :class:`asyncio.Task` and
676 can also be retrieved using the :meth:`~asyncio.Task.get_name` method.
677 (Contributed by Alex Grönholm in :issue:`34270`.)
678
679 Added support for
680 `Happy Eyeballs <https://en.wikipedia.org/wiki/Happy_Eyeballs>`_ to
681 :func:`asyncio.loop.create_connection`. To specify the behavior, two new
682 parameters have been added: *happy_eyeballs_delay* and *interleave*. The Happy
683 Eyeballs algorithm improves responsiveness in applications that support IPv4
684 and IPv6 by attempting to simultaneously connect using both.
685 (Contributed by twisteroid ambassador in :issue:`33530`.)
686
687
688 builtins
689 --------
690
691 The :func:`compile` built-in has been improved to accept the
692 ``ast.PyCF_ALLOW_TOP_LEVEL_AWAIT`` flag. With this new flag passed,
693 :func:`compile` will allow top-level ``await``, ``async for`` and ``async with``
694 constructs that are usually considered invalid syntax. Asynchronous code object
695 marked with the ``CO_COROUTINE`` flag may then be returned.
696 (Contributed by Matthias Bussonnier in :issue:`34616`)
697
698
699 collections
700 -----------
701
702 The :meth:`~collections.somenamedtuple._asdict` method for
703 :func:`collections.namedtuple` now returns a :class:`dict` instead of a
704 :class:`collections.OrderedDict`. This works because regular dicts have
705 guaranteed ordering since Python 3.7. If the extra features of
706 :class:`OrderedDict` are required, the suggested remediation is to cast the
707 result to the desired type: ``OrderedDict(nt._asdict())``.
708 (Contributed by Raymond Hettinger in :issue:`35864`.)
709
710
711 cProfile
712 --------
713
714 The :class:`cProfile.Profile <profile.Profile>` class can now be used as a context manager.
715 Profile a block of code by running::
716
717 import cProfile
718
719 with cProfile.Profile() as profiler:
720 # code to be profiled
721 ...
722
723 (Contributed by Scott Sanderson in :issue:`29235`.)
724
725
726 csv
727 ---
728
729 The :class:`csv.DictReader` now returns instances of :class:`dict` instead of
730 a :class:`collections.OrderedDict`. The tool is now faster and uses less
731 memory while still preserving the field order.
732 (Contributed by Michael Selik in :issue:`34003`.)
733
734
735 curses
736 -------
737
738 Added a new variable holding structured version information for the
739 underlying ncurses library: :data:`~curses.ncurses_version`.
740 (Contributed by Serhiy Storchaka in :issue:`31680`.)
741
742
743 ctypes
744 ------
745
746 On Windows, :class:`~ctypes.CDLL` and subclasses now accept a *winmode* parameter
747 to specify flags for the underlying ``LoadLibraryEx`` call. The default flags are
748 set to only load DLL dependencies from trusted locations, including the path
749 where the DLL is stored (if a full or partial path is used to load the initial
750 DLL) and paths added by :func:`~os.add_dll_directory`.
751 (Contributed by Steve Dower in :issue:`36085`.)
752
753
754 datetime
755 --------
756
757 Added new alternate constructors :meth:`datetime.date.fromisocalendar` and
758 :meth:`datetime.datetime.fromisocalendar`, which construct :class:`~datetime.date` and
759 :class:`~datetime.datetime` objects respectively from ISO year, week number, and weekday;
760 these are the inverse of each class's ``isocalendar`` method.
761 (Contributed by Paul Ganssle in :issue:`36004`.)
762
763
764 functools
765 ---------
766
767 :func:`functools.lru_cache` can now be used as a straight decorator rather
768 than as a function returning a decorator. So both of these are now supported::
769
770 @lru_cache
771 def f(x):
772 ...
773
774 @lru_cache(maxsize=256)
775 def f(x):
776 ...
777
778 (Contributed by Raymond Hettinger in :issue:`36772`.)
779
780 Added a new :func:`functools.cached_property` decorator, for computed properties
781 cached for the life of the instance. ::
782
783 import functools
784 import statistics
785
786 class Dataset:
787 def __init__(self, sequence_of_numbers):
788 self.data = sequence_of_numbers
789
790 @functools.cached_property
791 def variance(self):
792 return statistics.variance(self.data)
793
794 (Contributed by Carl Meyer in :issue:`21145`)
795
796
797 Added a new :func:`functools.singledispatchmethod` decorator that converts
798 methods into :term:`generic functions <generic function>` using
799 :term:`single dispatch`::
800
801 from functools import singledispatchmethod
802 from contextlib import suppress
803
804 class TaskManager:
805
806 def __init__(self, tasks):
807 self.tasks = list(tasks)
808
809 @singledispatchmethod
810 def discard(self, value):
811 with suppress(ValueError):
812 self.tasks.remove(value)
813
814 @discard.register(list)
815 def _(self, tasks):
816 targets = set(tasks)
817 self.tasks = [x for x in self.tasks if x not in targets]
818
819 (Contributed by Ethan Smith in :issue:`32380`)
820
821 gc
822 --
823
824 :func:`~gc.get_objects` can now receive an optional *generation* parameter
825 indicating a generation to get objects from.
826 (Contributed by Pablo Galindo in :issue:`36016`.)
827
828
829 gettext
830 -------
831
832 Added :func:`~gettext.pgettext` and its variants.
833 (Contributed by Franz Glasner, Éric Araujo, and Cheryl Sabella in :issue:`2504`.)
834
835
836 gzip
837 ----
838
839 Added the *mtime* parameter to :func:`gzip.compress` for reproducible output.
840 (Contributed by Guo Ci Teo in :issue:`34898`.)
841
842 A :exc:`~gzip.BadGzipFile` exception is now raised instead of :exc:`OSError`
843 for certain types of invalid or corrupt gzip files.
844 (Contributed by Filip Gruszczyński, Michele Orrù, and Zackery Spytz in
845 :issue:`6584`.)
846
847
848 IDLE and idlelib
849 ----------------
850
851 Output over N lines (50 by default) is squeezed down to a button.
852 N can be changed in the PyShell section of the General page of the
853 Settings dialog. Fewer, but possibly extra long, lines can be squeezed by
854 right clicking on the output. Squeezed output can be expanded in place
855 by double-clicking the button or into the clipboard or a separate window
856 by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.)
857
858 Add "Run Customized" to the Run menu to run a module with customized
859 settings. Any command line arguments entered are added to sys.argv.
860 They also re-appear in the box for the next customized run. One can also
861 suppress the normal Shell main module restart. (Contributed by Cheryl
862 Sabella, Terry Jan Reedy, and others in :issue:`5680` and :issue:`37627`.)
863
864 Added optional line numbers for IDLE editor windows. Windows
865 open without line numbers unless set otherwise in the General
866 tab of the configuration dialog. Line numbers for an existing
867 window are shown and hidden in the Options menu.
868 (Contributed by Tal Einat and Saimadhav Heblikar in :issue:`17535`.)
869
870 OS native encoding is now used for converting between Python strings and Tcl
871 objects. This allows IDLE to work with emoji and other non-BMP characters.
872 These characters can be displayed or copied and pasted to or from the
873 clipboard. Converting strings from Tcl to Python and back now never fails.
874 (Many people worked on this for eight years but the problem was finally
875 solved by Serhiy Storchaka in :issue:`13153`.)
876
877 New in 3.8.1:
878
879 Add option to toggle cursor blink off. (Contributed by Zackery Spytz
880 in :issue:`4603`.)
881
882 Escape key now closes IDLE completion windows. (Contributed by Johnny
883 Najera in :issue:`38944`.)
884
885 The changes above have been backported to 3.7 maintenance releases.
886
887 Add keywords to module name completion list. (Contributed by Terry J.
888 Reedy in :issue:`37765`.)
889
890 inspect
891 -------
892
893 The :func:`inspect.getdoc` function can now find docstrings for ``__slots__``
894 if that attribute is a :class:`dict` where the values are docstrings.
895 This provides documentation options similar to what we already have
896 for :func:`property`, :func:`classmethod`, and :func:`staticmethod`::
897
898 class AudioClip:
899 __slots__ = {'bit_rate': 'expressed in kilohertz to one decimal place',
900 'duration': 'in seconds, rounded up to an integer'}
901 def __init__(self, bit_rate, duration):
902 self.bit_rate = round(bit_rate / 1000.0, 1)
903 self.duration = ceil(duration)
904
905 (Contributed by Raymond Hettinger in :issue:`36326`.)
906
907
908 io
909 --
910
911 In development mode (:option:`-X` ``env``) and in :ref:`debug build <debug-build>`, the
912 :class:`io.IOBase` finalizer now logs the exception if the ``close()`` method
913 fails. The exception is ignored silently by default in release build.
914 (Contributed by Victor Stinner in :issue:`18748`.)
915
916
917 itertools
918 ---------
919
920 The :func:`itertools.accumulate` function added an option *initial* keyword
921 argument to specify an initial value::
922
923 >>> from itertools import accumulate
924 >>> list(accumulate([10, 5, 30, 15], initial=1000))
925 [1000, 1010, 1015, 1045, 1060]
926
927 (Contributed by Lisa Roach in :issue:`34659`.)
928
929
930 json.tool
931 ---------
932
933 Add option ``--json-lines`` to parse every input line as a separate JSON object.
934 (Contributed by Weipeng Hong in :issue:`31553`.)
935
936
937 logging
938 -------
939
940 Added a *force* keyword argument to :func:`logging.basicConfig`.
941 When set to true, any existing handlers attached
942 to the root logger are removed and closed before carrying out the
943 configuration specified by the other arguments.
944
945 This solves a long-standing problem. Once a logger or *basicConfig()* had
946 been called, subsequent calls to *basicConfig()* were silently ignored.
947 This made it difficult to update, experiment with, or teach the various
948 logging configuration options using the interactive prompt or a Jupyter
949 notebook.
950
951 (Suggested by Raymond Hettinger, implemented by Donghee Na, and
952 reviewed by Vinay Sajip in :issue:`33897`.)
953
954
955 math
956 ----
957
958 Added new function :func:`math.dist` for computing Euclidean distance
959 between two points. (Contributed by Raymond Hettinger in :issue:`33089`.)
960
961 Expanded the :func:`math.hypot` function to handle multiple dimensions.
962 Formerly, it only supported the 2-D case.
963 (Contributed by Raymond Hettinger in :issue:`33089`.)
964
965 Added new function, :func:`math.prod`, as analogous function to :func:`sum`
966 that returns the product of a 'start' value (default: 1) times an iterable of
967 numbers::
968
969 >>> prior = 0.8
970 >>> likelihoods = [0.625, 0.84, 0.30]
971 >>> math.prod(likelihoods, start=prior)
972 0.126
973
974 (Contributed by Pablo Galindo in :issue:`35606`.)
975
976 Added two new combinatoric functions :func:`math.perm` and :func:`math.comb`::
977
978 >>> math.perm(10, 3) # Permutations of 10 things taken 3 at a time
979 720
980 >>> math.comb(10, 3) # Combinations of 10 things taken 3 at a time
981 120
982
983 (Contributed by Yash Aggarwal, Keller Fuchs, Serhiy Storchaka, and Raymond
984 Hettinger in :issue:`37128`, :issue:`37178`, and :issue:`35431`.)
985
986 Added a new function :func:`math.isqrt` for computing accurate integer square
987 roots without conversion to floating point. The new function supports
988 arbitrarily large integers. It is faster than ``floor(sqrt(n))`` but slower
989 than :func:`math.sqrt`::
990
991 >>> r = 650320427
992 >>> s = r ** 2
993 >>> isqrt(s - 1) # correct
994 650320426
995 >>> floor(sqrt(s - 1)) # incorrect
996 650320427
997
998 (Contributed by Mark Dickinson in :issue:`36887`.)
999
1000 The function :func:`math.factorial` no longer accepts arguments that are not
1001 int-like. (Contributed by Pablo Galindo in :issue:`33083`.)
1002
1003
1004 mmap
1005 ----
1006
1007 The :class:`mmap.mmap` class now has an :meth:`~mmap.mmap.madvise` method to
1008 access the ``madvise()`` system call.
1009 (Contributed by Zackery Spytz in :issue:`32941`.)
1010
1011
1012 multiprocessing
1013 ---------------
1014
1015 Added new :mod:`multiprocessing.shared_memory` module.
1016 (Contributed by Davin Potts in :issue:`35813`.)
1017
1018 On macOS, the *spawn* start method is now used by default.
1019 (Contributed by Victor Stinner in :issue:`33725`.)
1020
1021
1022 os
1023 --
1024
1025 Added new function :func:`~os.add_dll_directory` on Windows for providing
1026 additional search paths for native dependencies when importing extension
1027 modules or loading DLLs using :mod:`ctypes`.
1028 (Contributed by Steve Dower in :issue:`36085`.)
1029
1030 A new :func:`os.memfd_create` function was added to wrap the
1031 ``memfd_create()`` syscall.
1032 (Contributed by Zackery Spytz and Christian Heimes in :issue:`26836`.)
1033
1034 On Windows, much of the manual logic for handling reparse points (including
1035 symlinks and directory junctions) has been delegated to the operating system.
1036 Specifically, :func:`os.stat` will now traverse anything supported by the
1037 operating system, while :func:`os.lstat` will only open reparse points that
1038 identify as "name surrogates" while others are opened as for :func:`os.stat`.
1039 In all cases, :attr:`os.stat_result.st_mode` will only have ``S_IFLNK`` set for
1040 symbolic links and not other kinds of reparse points. To identify other kinds
1041 of reparse point, check the new :attr:`os.stat_result.st_reparse_tag` attribute.
1042
1043 On Windows, :func:`os.readlink` is now able to read directory junctions. Note
1044 that :func:`~os.path.islink` will return ``False`` for directory junctions,
1045 and so code that checks ``islink`` first will continue to treat junctions as
1046 directories, while code that handles errors from :func:`os.readlink` may now
1047 treat junctions as links.
1048
1049 (Contributed by Steve Dower in :issue:`37834`.)
1050
1051
1052 os.path
1053 -------
1054
1055 :mod:`os.path` functions that return a boolean result like
1056 :func:`~os.path.exists`, :func:`~os.path.lexists`, :func:`~os.path.isdir`,
1057 :func:`~os.path.isfile`, :func:`~os.path.islink`, and :func:`~os.path.ismount`
1058 now return ``False`` instead of raising :exc:`ValueError` or its subclasses
1059 :exc:`UnicodeEncodeError` and :exc:`UnicodeDecodeError` for paths that contain
1060 characters or bytes unrepresentable at the OS level.
1061 (Contributed by Serhiy Storchaka in :issue:`33721`.)
1062
1063 :func:`~os.path.expanduser` on Windows now prefers the :envvar:`USERPROFILE`
1064 environment variable and does not use :envvar:`HOME`, which is not normally set
1065 for regular user accounts.
1066 (Contributed by Anthony Sottile in :issue:`36264`.)
1067
1068 :func:`~os.path.isdir` on Windows no longer returns ``True`` for a link to a
1069 non-existent directory.
1070
1071 :func:`~os.path.realpath` on Windows now resolves reparse points, including
1072 symlinks and directory junctions.
1073
1074 (Contributed by Steve Dower in :issue:`37834`.)
1075
1076
1077 pathlib
1078 -------
1079
1080 :mod:`pathlib.Path` methods that return a boolean result like
1081 :meth:`~pathlib.Path.exists`, :meth:`~pathlib.Path.is_dir`,
1082 :meth:`~pathlib.Path.is_file`, :meth:`~pathlib.Path.is_mount`,
1083 :meth:`~pathlib.Path.is_symlink`, :meth:`~pathlib.Path.is_block_device`,
1084 :meth:`~pathlib.Path.is_char_device`, :meth:`~pathlib.Path.is_fifo`,
1085 :meth:`~pathlib.Path.is_socket` now return ``False`` instead of raising
1086 :exc:`ValueError` or its subclass :exc:`UnicodeEncodeError` for paths that
1087 contain characters unrepresentable at the OS level.
1088 (Contributed by Serhiy Storchaka in :issue:`33721`.)
1089
1090 Added :meth:`!pathlib.Path.link_to` which creates a hard link pointing
1091 to a path.
1092 (Contributed by Joannah Nanjekye in :issue:`26978`)
1093 Note that ``link_to`` was deprecated in 3.10 and removed in 3.12 in
1094 favor of a ``hardlink_to`` method added in 3.10 which matches the
1095 semantics of the existing ``symlink_to`` method.
1096
1097
1098 pickle
1099 ------
1100
1101 :mod:`pickle` extensions subclassing the C-optimized :class:`~pickle.Pickler`
1102 can now override the pickling logic of functions and classes by defining the
1103 special :meth:`~pickle.Pickler.reducer_override` method.
1104 (Contributed by Pierre Glaser and Olivier Grisel in :issue:`35900`.)
1105
1106
1107 plistlib
1108 --------
1109
1110 Added new :class:`plistlib.UID` and enabled support for reading and writing
1111 NSKeyedArchiver-encoded binary plists.
1112 (Contributed by Jon Janzen in :issue:`26707`.)
1113
1114
1115 pprint
1116 ------
1117
1118 The :mod:`pprint` module added a *sort_dicts* parameter to several functions.
1119 By default, those functions continue to sort dictionaries before rendering or
1120 printing. However, if *sort_dicts* is set to false, the dictionaries retain
1121 the order that keys were inserted. This can be useful for comparison to JSON
1122 inputs during debugging.
1123
1124 In addition, there is a convenience new function, :func:`pprint.pp` that is
1125 like :func:`pprint.pprint` but with *sort_dicts* defaulting to ``False``::
1126
1127 >>> from pprint import pprint, pp
1128 >>> d = dict(source='input.txt', operation='filter', destination='output.txt')
1129 >>> pp(d, width=40) # Original order
1130 {'source': 'input.txt',
1131 'operation': 'filter',
1132 'destination': 'output.txt'}
1133 >>> pprint(d, width=40) # Keys sorted alphabetically
1134 {'destination': 'output.txt',
1135 'operation': 'filter',
1136 'source': 'input.txt'}
1137
1138 (Contributed by Rémi Lapeyre in :issue:`30670`.)
1139
1140
1141 py_compile
1142 ----------
1143
1144 :func:`py_compile.compile` now supports silent mode.
1145 (Contributed by Joannah Nanjekye in :issue:`22640`.)
1146
1147
1148 shlex
1149 -----
1150
1151 The new :func:`shlex.join` function acts as the inverse of :func:`shlex.split`.
1152 (Contributed by Bo Bayles in :issue:`32102`.)
1153
1154
1155 shutil
1156 ------
1157
1158 :func:`shutil.copytree` now accepts a new ``dirs_exist_ok`` keyword argument.
1159 (Contributed by Josh Bronson in :issue:`20849`.)
1160
1161 :func:`shutil.make_archive` now defaults to the modern pax (POSIX.1-2001)
1162 format for new archives to improve portability and standards conformance,
1163 inherited from the corresponding change to the :mod:`tarfile` module.
1164 (Contributed by C.A.M. Gerlach in :issue:`30661`.)
1165
1166 :func:`shutil.rmtree` on Windows now removes directory junctions without
1167 recursively removing their contents first.
1168 (Contributed by Steve Dower in :issue:`37834`.)
1169
1170
1171 socket
1172 ------
1173
1174 Added :meth:`~socket.create_server` and :meth:`~socket.has_dualstack_ipv6`
1175 convenience functions to automate the necessary tasks usually involved when
1176 creating a server socket, including accepting both IPv4 and IPv6 connections
1177 on the same socket. (Contributed by Giampaolo Rodolà in :issue:`17561`.)
1178
1179 The :func:`socket.if_nameindex`, :func:`socket.if_nametoindex`, and
1180 :func:`socket.if_indextoname` functions have been implemented on Windows.
1181 (Contributed by Zackery Spytz in :issue:`37007`.)
1182
1183
1184 ssl
1185 ---
1186
1187 Added :attr:`~ssl.SSLContext.post_handshake_auth` to enable and
1188 :meth:`~ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3
1189 post-handshake authentication.
1190 (Contributed by Christian Heimes in :issue:`34670`.)
1191
1192
1193 statistics
1194 ----------
1195
1196 Added :func:`statistics.fmean` as a faster, floating-point variant of
1197 :func:`statistics.mean`. (Contributed by Raymond Hettinger and
1198 Steven D'Aprano in :issue:`35904`.)
1199
1200 Added :func:`statistics.geometric_mean`
1201 (Contributed by Raymond Hettinger in :issue:`27181`.)
1202
1203 Added :func:`statistics.multimode` that returns a list of the most
1204 common values. (Contributed by Raymond Hettinger in :issue:`35892`.)
1205
1206 Added :func:`statistics.quantiles` that divides data or a distribution
1207 in to equiprobable intervals (e.g. quartiles, deciles, or percentiles).
1208 (Contributed by Raymond Hettinger in :issue:`36546`.)
1209
1210 Added :class:`statistics.NormalDist`, a tool for creating
1211 and manipulating normal distributions of a random variable.
1212 (Contributed by Raymond Hettinger in :issue:`36018`.)
1213
1214 ::
1215
1216 >>> temperature_feb = NormalDist.from_samples([4, 12, -3, 2, 7, 14])
1217 >>> temperature_feb.mean
1218 6.0
1219 >>> temperature_feb.stdev
1220 6.356099432828281
1221
1222 >>> temperature_feb.cdf(3) # Chance of being under 3 degrees
1223 0.3184678262814532
1224 >>> # Relative chance of being 7 degrees versus 10 degrees
1225 >>> temperature_feb.pdf(7) / temperature_feb.pdf(10)
1226 1.2039930378537762
1227
1228 >>> el_niño = NormalDist(4, 2.5)
1229 >>> temperature_feb += el_niño # Add in a climate effect
1230 >>> temperature_feb
1231 NormalDist(mu=10.0, sigma=6.830080526611674)
1232
1233 >>> temperature_feb * (9/5) + 32 # Convert to Fahrenheit
1234 NormalDist(mu=50.0, sigma=12.294144947901014)
1235 >>> temperature_feb.samples(3) # Generate random samples
1236 [7.672102882379219, 12.000027119750287, 4.647488369766392]
1237
1238
1239 sys
1240 ---
1241
1242 Add new :func:`sys.unraisablehook` function which can be overridden to control
1243 how "unraisable exceptions" are handled. It is called when an exception has
1244 occurred but there is no way for Python to handle it. For example, when a
1245 destructor raises an exception or during garbage collection
1246 (:func:`gc.collect`).
1247 (Contributed by Victor Stinner in :issue:`36829`.)
1248
1249
1250 tarfile
1251 -------
1252
1253 The :mod:`tarfile` module now defaults to the modern pax (POSIX.1-2001)
1254 format for new archives, instead of the previous GNU-specific one.
1255 This improves cross-platform portability with a consistent encoding (UTF-8)
1256 in a standardized and extensible format, and offers several other benefits.
1257 (Contributed by C.A.M. Gerlach in :issue:`36268`.)
1258
1259
1260 threading
1261 ---------
1262
1263 Add a new :func:`threading.excepthook` function which handles uncaught
1264 :meth:`threading.Thread.run` exception. It can be overridden to control how
1265 uncaught :meth:`threading.Thread.run` exceptions are handled.
1266 (Contributed by Victor Stinner in :issue:`1230540`.)
1267
1268 Add a new :func:`threading.get_native_id` function and
1269 a :data:`~threading.Thread.native_id`
1270 attribute to the :class:`threading.Thread` class. These return the native
1271 integral Thread ID of the current thread assigned by the kernel.
1272 This feature is only available on certain platforms, see
1273 :func:`get_native_id <threading.get_native_id>` for more information.
1274 (Contributed by Jake Tesler in :issue:`36084`.)
1275
1276
1277 tokenize
1278 --------
1279
1280 The :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token when
1281 provided with input that does not have a trailing new line. This behavior
1282 now matches what the C tokenizer does internally.
1283 (Contributed by Ammar Askar in :issue:`33899`.)
1284
1285
1286 tkinter
1287 -------
1288
1289 Added methods :meth:`!selection_from`,
1290 :meth:`!selection_present`,
1291 :meth:`!selection_range` and
1292 :meth:`!selection_to`
1293 in the :class:`!tkinter.Spinbox` class.
1294 (Contributed by Juliette Monsel in :issue:`34829`.)
1295
1296 Added method :meth:`!moveto`
1297 in the :class:`!tkinter.Canvas` class.
1298 (Contributed by Juliette Monsel in :issue:`23831`.)
1299
1300 The :class:`!tkinter.PhotoImage` class now has
1301 :meth:`!transparency_get` and
1302 :meth:`!transparency_set` methods. (Contributed by
1303 Zackery Spytz in :issue:`25451`.)
1304
1305
1306 time
1307 ----
1308
1309 Added new clock :const:`~time.CLOCK_UPTIME_RAW` for macOS 10.12.
1310 (Contributed by Joannah Nanjekye in :issue:`35702`.)
1311
1312
1313 typing
1314 ------
1315
1316 The :mod:`typing` module incorporates several new features:
1317
1318 * A dictionary type with per-key types. See :pep:`589` and
1319 :class:`typing.TypedDict`.
1320 TypedDict uses only string keys. By default, every key is required
1321 to be present. Specify "total=False" to allow keys to be optional::
1322
1323 class Location(TypedDict, total=False):
1324 lat_long: tuple
1325 grid_square: str
1326 xy_coordinate: tuple
1327
1328 * Literal types. See :pep:`586` and :class:`typing.Literal`.
1329 Literal types indicate that a parameter or return value
1330 is constrained to one or more specific literal values::
1331
1332 def get_status(port: int) -> Literal['connected', 'disconnected']:
1333 ...
1334
1335 * "Final" variables, functions, methods and classes. See :pep:`591`,
1336 :class:`typing.Final` and :func:`typing.final`.
1337 The final qualifier instructs a static type checker to restrict
1338 subclassing, overriding, or reassignment::
1339
1340 pi: Final[float] = 3.1415926536
1341
1342 * Protocol definitions. See :pep:`544`, :class:`typing.Protocol` and
1343 :func:`typing.runtime_checkable`. Simple ABCs like
1344 :class:`typing.SupportsInt` are now ``Protocol`` subclasses.
1345
1346 * New protocol class :class:`typing.SupportsIndex`.
1347
1348 * New functions :func:`typing.get_origin` and :func:`typing.get_args`.
1349
1350
1351 unicodedata
1352 -----------
1353
1354 The :mod:`unicodedata` module has been upgraded to use the `Unicode 12.1.0
1355 <https://blog.unicode.org/2019/05/unicode-12-1-en.html>`_ release.
1356
1357 New function :func:`~unicodedata.is_normalized` can be used to verify a string
1358 is in a specific normal form, often much faster than by actually normalizing
1359 the string. (Contributed by Max Belanger, David Euresti, and Greg Price in
1360 :issue:`32285` and :issue:`37966`).
1361
1362
1363 unittest
1364 --------
1365
1366 Added :class:`~unittest.mock.AsyncMock` to support an asynchronous version of
1367 :class:`~unittest.mock.Mock`. Appropriate new assert functions for testing
1368 have been added as well.
1369 (Contributed by Lisa Roach in :issue:`26467`).
1370
1371 Added :func:`~unittest.addModuleCleanup` and
1372 :meth:`~unittest.TestCase.addClassCleanup` to unittest to support
1373 cleanups for :func:`~unittest.setUpModule` and
1374 :meth:`~unittest.TestCase.setUpClass`.
1375 (Contributed by Lisa Roach in :issue:`24412`.)
1376
1377 Several mock assert functions now also print a list of actual calls upon
1378 failure. (Contributed by Petter Strandmark in :issue:`35047`.)
1379
1380 :mod:`unittest` module gained support for coroutines to be used as test cases
1381 with :class:`unittest.IsolatedAsyncioTestCase`.
1382 (Contributed by Andrew Svetlov in :issue:`32972`.)
1383
1384 Example::
1385
1386 import unittest
1387
1388
1389 class TestRequest(unittest.IsolatedAsyncioTestCase):
1390
1391 async def asyncSetUp(self):
1392 self.connection = await AsyncConnection()
1393
1394 async def test_get(self):
1395 response = await self.connection.get("https://example.com")
1396 self.assertEqual(response.status_code, 200)
1397
1398 async def asyncTearDown(self):
1399 await self.connection.close()
1400
1401
1402 if __name__ == "__main__":
1403 unittest.main()
1404
1405
1406 venv
1407 ----
1408
1409 :mod:`venv` now includes an ``Activate.ps1`` script on all platforms for
1410 activating virtual environments under PowerShell Core 6.1.
1411 (Contributed by Brett Cannon in :issue:`32718`.)
1412
1413
1414 weakref
1415 -------
1416
1417 The proxy objects returned by :func:`weakref.proxy` now support the matrix
1418 multiplication operators ``@`` and ``@=`` in addition to the other
1419 numeric operators. (Contributed by Mark Dickinson in :issue:`36669`.)
1420
1421
1422 xml
1423 ---
1424
1425 As mitigation against DTD and external entity retrieval, the
1426 :mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process
1427 external entities by default.
1428 (Contributed by Christian Heimes in :issue:`17239`.)
1429
1430 The ``.find*()`` methods in the :mod:`xml.etree.ElementTree` module
1431 support wildcard searches like ``{*}tag`` which ignores the namespace
1432 and ``{namespace}*`` which returns all tags in the given namespace.
1433 (Contributed by Stefan Behnel in :issue:`28238`.)
1434
1435 The :mod:`xml.etree.ElementTree` module provides a new function
1436 :func:`~xml.etree.ElementTree.canonicalize` that implements C14N 2.0.
1437 (Contributed by Stefan Behnel in :issue:`13611`.)
1438
1439 The target object of :class:`xml.etree.ElementTree.XMLParser` can
1440 receive namespace declaration events through the new callback methods
1441 ``start_ns()`` and ``end_ns()``. Additionally, the
1442 :class:`xml.etree.ElementTree.TreeBuilder` target can be configured
1443 to process events about comments and processing instructions to include
1444 them in the generated tree.
1445 (Contributed by Stefan Behnel in :issue:`36676` and :issue:`36673`.)
1446
1447
1448 xmlrpc
1449 ------
1450
1451 :class:`xmlrpc.client.ServerProxy` now supports an optional *headers* keyword
1452 argument for a sequence of HTTP headers to be sent with each request. Among
1453 other things, this makes it possible to upgrade from default basic
1454 authentication to faster session authentication.
1455 (Contributed by Cédric Krier in :issue:`35153`.)
1456
1457
1458 Optimizations
1459 =============
1460
1461 * The :mod:`subprocess` module can now use the :func:`os.posix_spawn` function
1462 in some cases for better performance. Currently, it is only used on macOS
1463 and Linux (using glibc 2.24 or newer) if all these conditions are met:
1464
1465 * *close_fds* is false;
1466 * *preexec_fn*, *pass_fds*, *cwd* and *start_new_session* parameters
1467 are not set;
1468 * the *executable* path contains a directory.
1469
1470 (Contributed by Joannah Nanjekye and Victor Stinner in :issue:`35537`.)
1471
1472 * :func:`shutil.copyfile`, :func:`shutil.copy`, :func:`shutil.copy2`,
1473 :func:`shutil.copytree` and :func:`shutil.move` use platform-specific
1474 "fast-copy" syscalls on Linux and macOS in order to copy the file
1475 more efficiently.
1476 "fast-copy" means that the copying operation occurs within the kernel,
1477 avoiding the use of userspace buffers in Python as in
1478 "``outfd.write(infd.read())``".
1479 On Windows :func:`shutil.copyfile` uses a bigger default buffer size (1 MiB
1480 instead of 16 KiB) and a :func:`memoryview`-based variant of
1481 :func:`shutil.copyfileobj` is used.
1482 The speedup for copying a 512 MiB file within the same partition is about
1483 +26% on Linux, +50% on macOS and +40% on Windows. Also, much less CPU cycles
1484 are consumed.
1485 See :ref:`shutil-platform-dependent-efficient-copy-operations` section.
1486 (Contributed by Giampaolo Rodolà in :issue:`33671`.)
1487
1488 * :func:`shutil.copytree` uses :func:`os.scandir` function and all copy
1489 functions depending from it use cached :func:`os.stat` values. The speedup
1490 for copying a directory with 8000 files is around +9% on Linux, +20% on
1491 Windows and +30% on a Windows SMB share. Also the number of :func:`os.stat`
1492 syscalls is reduced by 38% making :func:`shutil.copytree` especially faster
1493 on network filesystems. (Contributed by Giampaolo Rodolà in :issue:`33695`.)
1494
1495 * The default protocol in the :mod:`pickle` module is now Protocol 4,
1496 first introduced in Python 3.4. It offers better performance and smaller
1497 size compared to Protocol 3 available since Python 3.0.
1498
1499 * Removed one :c:type:`Py_ssize_t` member from ``PyGC_Head``. All GC tracked
1500 objects (e.g. tuple, list, dict) size is reduced 4 or 8 bytes.
1501 (Contributed by Inada Naoki in :issue:`33597`.)
1502
1503 * :class:`uuid.UUID` now uses ``__slots__`` to reduce its memory footprint.
1504 (Contributed by Wouter Bolsterlee and Tal Einat in :issue:`30977`)
1505
1506 * Improved performance of :func:`operator.itemgetter` by 33%. Optimized
1507 argument handling and added a fast path for the common case of a single
1508 non-negative integer index into a tuple (which is the typical use case in
1509 the standard library). (Contributed by Raymond Hettinger in
1510 :issue:`35664`.)
1511
1512 * Sped-up field lookups in :func:`collections.namedtuple`. They are now more
1513 than two times faster, making them the fastest form of instance variable
1514 lookup in Python. (Contributed by Raymond Hettinger, Pablo Galindo, and
1515 Joe Jevnik, Serhiy Storchaka in :issue:`32492`.)
1516
1517 * The :class:`list` constructor does not overallocate the internal item buffer
1518 if the input iterable has a known length (the input implements ``__len__``).
1519 This makes the created list 12% smaller on average. (Contributed by
1520 Raymond Hettinger and Pablo Galindo in :issue:`33234`.)
1521
1522 * Doubled the speed of class variable writes. When a non-dunder attribute
1523 was updated, there was an unnecessary call to update slots.
1524 (Contributed by Stefan Behnel, Pablo Galindo Salgado, Raymond Hettinger,
1525 Neil Schemenauer, and Serhiy Storchaka in :issue:`36012`.)
1526
1527 * Reduced an overhead of converting arguments passed to many builtin functions
1528 and methods. This sped up calling some simple builtin functions and
1529 methods up to 20--50%. (Contributed by Serhiy Storchaka in :issue:`23867`,
1530 :issue:`35582` and :issue:`36127`.)
1531
1532 * ``LOAD_GLOBAL`` instruction now uses new "per opcode cache" mechanism.
1533 It is about 40% faster now. (Contributed by Yury Selivanov and Inada Naoki in
1534 :issue:`26219`.)
1535
1536
1537 Build and C API Changes
1538 =======================
1539
1540 * Default :data:`sys.abiflags` became an empty string: the ``m`` flag for
1541 pymalloc became useless (builds with and without pymalloc are ABI compatible)
1542 and so has been removed. (Contributed by Victor Stinner in :issue:`36707`.)
1543
1544 Example of changes:
1545
1546 * Only ``python3.8`` program is installed, ``python3.8m`` program is gone.
1547 * Only ``python3.8-config`` script is installed, ``python3.8m-config`` script
1548 is gone.
1549 * The ``m`` flag has been removed from the suffix of dynamic library
1550 filenames: extension modules in the standard library as well as those
1551 produced and installed by third-party packages, like those downloaded from
1552 PyPI. On Linux, for example, the Python 3.7 suffix
1553 ``.cpython-37m-x86_64-linux-gnu.so`` became
1554 ``.cpython-38-x86_64-linux-gnu.so`` in Python 3.8.
1555
1556 * The header files have been reorganized to better separate the different kinds
1557 of APIs:
1558
1559 * ``Include/*.h`` should be the portable public stable C API.
1560 * ``Include/cpython/*.h`` should be the unstable C API specific to CPython;
1561 public API, with some private API prefixed by ``_Py`` or ``_PY``.
1562 * ``Include/internal/*.h`` is the private internal C API very specific to
1563 CPython. This API comes with no backward compatibility warranty and should
1564 not be used outside CPython. It is only exposed for very specific needs
1565 like debuggers and profiles which has to access to CPython internals
1566 without calling functions. This API is now installed by ``make install``.
1567
1568 (Contributed by Victor Stinner in :issue:`35134` and :issue:`35081`,
1569 work initiated by Eric Snow in Python 3.7.)
1570
1571 * Some macros have been converted to static inline functions: parameter types
1572 and return type are well defined, they don't have issues specific to macros,
1573 variables have a local scopes. Examples:
1574
1575 * :c:func:`Py_INCREF`, :c:func:`Py_DECREF`
1576 * :c:func:`Py_XINCREF`, :c:func:`Py_XDECREF`
1577 * :c:macro:`!PyObject_INIT`, :c:macro:`!PyObject_INIT_VAR`
1578 * Private functions: :c:func:`!_PyObject_GC_TRACK`,
1579 :c:func:`!_PyObject_GC_UNTRACK`, :c:func:`!_Py_Dealloc`
1580
1581 (Contributed by Victor Stinner in :issue:`35059`.)
1582
1583 * The :c:func:`!PyByteArray_Init` and :c:func:`!PyByteArray_Fini` functions have
1584 been removed. They did nothing since Python 2.7.4 and Python 3.2.0, were
1585 excluded from the limited API (stable ABI), and were not documented.
1586 (Contributed by Victor Stinner in :issue:`35713`.)
1587
1588 * The result of :c:func:`PyExceptionClass_Name` is now of type
1589 ``const char *`` rather of ``char *``.
1590 (Contributed by Serhiy Storchaka in :issue:`33818`.)
1591
1592 * The duality of ``Modules/Setup.dist`` and ``Modules/Setup`` has been
1593 removed. Previously, when updating the CPython source tree, one had
1594 to manually copy ``Modules/Setup.dist`` (inside the source tree) to
1595 ``Modules/Setup`` (inside the build tree) in order to reflect any changes
1596 upstream. This was of a small benefit to packagers at the expense of
1597 a frequent annoyance to developers following CPython development, as
1598 forgetting to copy the file could produce build failures.
1599
1600 Now the build system always reads from ``Modules/Setup`` inside the source
1601 tree. People who want to customize that file are encouraged to maintain
1602 their changes in a git fork of CPython or as patch files, as they would do
1603 for any other change to the source tree.
1604
1605 (Contributed by Antoine Pitrou in :issue:`32430`.)
1606
1607 * Functions that convert Python number to C integer like
1608 :c:func:`PyLong_AsLong` and argument parsing functions like
1609 :c:func:`PyArg_ParseTuple` with integer converting format units like ``'i'``
1610 will now use the :meth:`~object.__index__` special method instead of
1611 :meth:`~object.__int__`, if available. The deprecation warning will be
1612 emitted for objects with the ``__int__()`` method but without the
1613 ``__index__()`` method (like :class:`~decimal.Decimal` and
1614 :class:`~fractions.Fraction`). :c:func:`PyNumber_Check` will now return
1615 ``1`` for objects implementing ``__index__()``.
1616 :c:func:`PyNumber_Long`, :c:func:`PyNumber_Float` and
1617 :c:func:`PyFloat_AsDouble` also now use the ``__index__()`` method if
1618 available.
1619 (Contributed by Serhiy Storchaka in :issue:`36048` and :issue:`20092`.)
1620
1621 * Heap-allocated type objects will now increase their reference count
1622 in :c:func:`PyObject_Init` (and its parallel macro ``PyObject_INIT``)
1623 instead of in :c:func:`PyType_GenericAlloc`. Types that modify instance
1624 allocation or deallocation may need to be adjusted.
1625 (Contributed by Eddie Elizondo in :issue:`35810`.)
1626
1627 * The new function :c:func:`!PyCode_NewWithPosOnlyArgs` allows to create
1628 code objects like :c:func:`!PyCode_New`, but with an extra *posonlyargcount*
1629 parameter for indicating the number of positional-only arguments.
1630 (Contributed by Pablo Galindo in :issue:`37221`.)
1631
1632 * :c:func:`!Py_SetPath` now sets :data:`sys.executable` to the program full
1633 path (:c:func:`Py_GetProgramFullPath`) rather than to the program name
1634 (:c:func:`Py_GetProgramName`).
1635 (Contributed by Victor Stinner in :issue:`38234`.)
1636
1637
1638 Deprecated
1639 ==========
1640
1641 * The distutils ``bdist_wininst`` command is now deprecated, use
1642 ``bdist_wheel`` (wheel packages) instead.
1643 (Contributed by Victor Stinner in :issue:`37481`.)
1644
1645 * Deprecated methods ``getchildren()`` and ``getiterator()`` in
1646 the :mod:`~xml.etree.ElementTree` module now emit a
1647 :exc:`DeprecationWarning` instead of :exc:`PendingDeprecationWarning`.
1648 They will be removed in Python 3.9.
1649 (Contributed by Serhiy Storchaka in :issue:`29209`.)
1650
1651 * Passing an object that is not an instance of
1652 :class:`concurrent.futures.ThreadPoolExecutor` to
1653 :meth:`loop.set_default_executor() <asyncio.loop.set_default_executor>` is
1654 deprecated and will be prohibited in Python 3.9.
1655 (Contributed by Elvis Pranskevichus in :issue:`34075`.)
1656
1657 * The :meth:`~object.__getitem__` methods of :class:`xml.dom.pulldom.DOMEventStream`,
1658 :class:`wsgiref.util.FileWrapper` and :class:`fileinput.FileInput` have been
1659 deprecated.
1660
1661 Implementations of these methods have been ignoring their *index* parameter,
1662 and returning the next item instead.
1663 (Contributed by Berker Peksag in :issue:`9372`.)
1664
1665 * The :class:`typing.NamedTuple` class has deprecated the ``_field_types``
1666 attribute in favor of the ``__annotations__`` attribute which has the same
1667 information. (Contributed by Raymond Hettinger in :issue:`36320`.)
1668
1669 * :mod:`ast` classes ``Num``, ``Str``, ``Bytes``, ``NameConstant`` and
1670 ``Ellipsis`` are considered deprecated and will be removed in future Python
1671 versions. :class:`~ast.Constant` should be used instead.
1672 (Contributed by Serhiy Storchaka in :issue:`32892`.)
1673
1674 * :class:`ast.NodeVisitor` methods ``visit_Num()``, ``visit_Str()``,
1675 ``visit_Bytes()``, ``visit_NameConstant()`` and ``visit_Ellipsis()`` are
1676 deprecated now and will not be called in future Python versions.
1677 Add the :meth:`~ast.NodeVisitor.visit_Constant` method to handle all
1678 constant nodes.
1679 (Contributed by Serhiy Storchaka in :issue:`36917`.)
1680
1681 * The :deco:`!asyncio.coroutine` :term:`decorator` is deprecated and will be
1682 removed in version 3.10. Instead of ``@asyncio.coroutine``, use
1683 :keyword:`async def` instead.
1684 (Contributed by Andrew Svetlov in :issue:`36921`.)
1685
1686 * In :mod:`asyncio`, the explicit passing of a *loop* argument has been
1687 deprecated and will be removed in version 3.10 for the following:
1688 :func:`asyncio.sleep`, :func:`asyncio.gather`, :func:`asyncio.shield`,
1689 :func:`asyncio.wait_for`, :func:`asyncio.wait`, :func:`asyncio.as_completed`,
1690 :class:`asyncio.Task`, :class:`asyncio.Lock`, :class:`asyncio.Event`,
1691 :class:`asyncio.Condition`, :class:`asyncio.Semaphore`,
1692 :class:`asyncio.BoundedSemaphore`, :class:`asyncio.Queue`,
1693 :func:`asyncio.create_subprocess_exec`, and
1694 :func:`asyncio.create_subprocess_shell`.
1695
1696 * The explicit passing of coroutine objects to :func:`asyncio.wait` has been
1697 deprecated and will be removed in version 3.11.
1698 (Contributed by Yury Selivanov in :issue:`34790`.)
1699
1700 * The following functions and methods are deprecated in the :mod:`gettext`
1701 module: :func:`!lgettext`, :func:`!ldgettext`,
1702 :func:`!lngettext` and :func:`!ldngettext`.
1703 They return encoded bytes, and it's possible that you will get unexpected
1704 Unicode-related exceptions if there are encoding problems with the
1705 translated strings. It's much better to use alternatives which return
1706 Unicode strings in Python 3. These functions have been broken for a long time.
1707
1708 Function :func:`!bind_textdomain_codeset`, methods
1709 :meth:`!NullTranslations.output_charset` and
1710 :meth:`!NullTranslations.set_output_charset`, and the *codeset*
1711 parameter of functions :func:`~gettext.translation` and
1712 :func:`~gettext.install` are also deprecated, since they are only used for
1713 the ``l*gettext()`` functions.
1714 (Contributed by Serhiy Storchaka in :issue:`33710`.)
1715
1716 * The :meth:`!isAlive` method of :class:`threading.Thread`
1717 has been deprecated.
1718 (Contributed by Donghee Na in :issue:`35283`.)
1719
1720 * Many builtin and extension functions that take integer arguments will
1721 now emit a deprecation warning for :class:`~decimal.Decimal`\ s,
1722 :class:`~fractions.Fraction`\ s and any other objects that can be converted
1723 to integers only with a loss (e.g. that have the :meth:`~object.__int__`
1724 method but do not have the :meth:`~object.__index__` method). In future
1725 version they will be errors.
1726 (Contributed by Serhiy Storchaka in :issue:`36048`.)
1727
1728 * Deprecated passing the following arguments as keyword arguments:
1729
1730 - *func* in :func:`functools.partialmethod`, :func:`weakref.finalize`,
1731 :meth:`profile.Profile.runcall`, :meth:`!cProfile.Profile.runcall`,
1732 :meth:`bdb.Bdb.runcall`, :meth:`trace.Trace.runfunc` and
1733 :func:`curses.wrapper`.
1734 - *function* in :meth:`unittest.TestCase.addCleanup`.
1735 - *fn* in the :meth:`~concurrent.futures.Executor.submit` method of
1736 :class:`concurrent.futures.ThreadPoolExecutor` and
1737 :class:`concurrent.futures.ProcessPoolExecutor`.
1738 - *callback* in :meth:`contextlib.ExitStack.callback`,
1739 :meth:`!contextlib.AsyncExitStack.callback` and
1740 :meth:`contextlib.AsyncExitStack.push_async_callback`.
1741 - *c* and *typeid* in the :meth:`!create`
1742 method of :class:`!multiprocessing.managers.Server` and
1743 :class:`!multiprocessing.managers.SharedMemoryServer`.
1744 - *obj* in :func:`weakref.finalize`.
1745
1746 In future releases of Python, they will be :ref:`positional-only
1747 <positional-only_parameter>`.
1748 (Contributed by Serhiy Storchaka in :issue:`36492`.)
1749
1750
1751 API and Feature Removals
1752 ========================
1753
1754 The following features and APIs have been removed from Python 3.8:
1755
1756 * Starting with Python 3.3, importing ABCs from :mod:`collections` was
1757 deprecated, and importing should be done from :mod:`collections.abc`. Being
1758 able to import from collections was marked for removal in 3.8, but has been
1759 delayed to 3.9. (See :gh:`81134`.)
1760
1761 * The :mod:`!macpath` module, deprecated in Python 3.7, has been removed.
1762 (Contributed by Victor Stinner in :issue:`35471`.)
1763
1764 * The function :func:`!platform.popen` has been removed, after having been
1765 deprecated since Python 3.3: use :func:`os.popen` instead.
1766 (Contributed by Victor Stinner in :issue:`35345`.)
1767
1768 * The function :func:`!time.clock` has been removed, after having been
1769 deprecated since Python 3.3: use :func:`time.perf_counter` or
1770 :func:`time.process_time` instead, depending
1771 on your requirements, to have well-defined behavior.
1772 (Contributed by Matthias Bussonnier in :issue:`36895`.)
1773
1774 * The ``pyvenv`` script has been removed in favor of ``python3.8 -m venv``
1775 to help eliminate confusion as to what Python interpreter the ``pyvenv``
1776 script is tied to. (Contributed by Brett Cannon in :issue:`25427`.)
1777
1778 * ``parse_qs``, ``parse_qsl``, and ``escape`` are removed from the :mod:`!cgi`
1779 module. They are deprecated in Python 3.2 or older. They should be imported
1780 from the ``urllib.parse`` and ``html`` modules instead.
1781
1782 * ``filemode`` function is removed from the :mod:`tarfile` module.
1783 It is not documented and deprecated since Python 3.3.
1784
1785 * The :class:`~xml.etree.ElementTree.XMLParser` constructor no longer accepts
1786 the *html* argument. It never had an effect and was deprecated in Python 3.4.
1787 All other parameters are now :ref:`keyword-only <keyword-only_parameter>`.
1788 (Contributed by Serhiy Storchaka in :issue:`29209`.)
1789
1790 * Removed the ``doctype()`` method of :class:`~xml.etree.ElementTree.XMLParser`.
1791 (Contributed by Serhiy Storchaka in :issue:`29209`.)
1792
1793 * "unicode_internal" codec is removed.
1794 (Contributed by Inada Naoki in :issue:`36297`.)
1795
1796 * The ``Cache`` and ``Statement`` objects of the :mod:`sqlite3` module are not
1797 exposed to the user.
1798 (Contributed by Aviv Palivoda in :issue:`30262`.)
1799
1800 * The ``bufsize`` keyword argument of :func:`fileinput.input` and
1801 :func:`fileinput.FileInput` which was ignored and deprecated since Python 3.6
1802 has been removed. :issue:`36952` (Contributed by Matthias Bussonnier.)
1803
1804 * The functions :func:`!sys.set_coroutine_wrapper` and
1805 :func:`!sys.get_coroutine_wrapper` deprecated in Python 3.7 have been removed;
1806 :issue:`36933` (Contributed by Matthias Bussonnier.)
1807
1808
1809 Porting to Python 3.8
1810 =====================
1811
1812 This section lists previously described changes and other bugfixes
1813 that may require changes to your code.
1814
1815
1816 Changes in Python behavior
1817 --------------------------
1818
1819 * Yield expressions (both ``yield`` and ``yield from`` clauses) are now disallowed
1820 in comprehensions and generator expressions (aside from the iterable expression
1821 in the leftmost :keyword:`!for` clause).
1822 (Contributed by Serhiy Storchaka in :issue:`10544`.)
1823
1824 * The compiler now produces a :exc:`SyntaxWarning` when identity checks
1825 (``is`` and ``is not``) are used with certain types of literals
1826 (e.g. strings, numbers). These can often work by accident in CPython,
1827 but are not guaranteed by the language spec. The warning advises users
1828 to use equality tests (``==`` and ``!=``) instead.
1829 (Contributed by Serhiy Storchaka in :issue:`34850`.)
1830
1831 * The CPython interpreter can swallow exceptions in some circumstances.
1832 In Python 3.8 this happens in fewer cases. In particular, exceptions
1833 raised when getting the attribute from the type dictionary are no longer
1834 ignored. (Contributed by Serhiy Storchaka in :issue:`35459`.)
1835
1836 * Removed ``__str__`` implementations from builtin types :class:`bool`,
1837 :class:`int`, :class:`float`, :class:`complex` and few classes from
1838 the standard library. They now inherit ``__str__()`` from :class:`object`.
1839 As result, defining the ``__repr__()`` method in the subclass of these
1840 classes will affect their string representation.
1841 (Contributed by Serhiy Storchaka in :issue:`36793`.)
1842
1843 * On AIX, :data:`sys.platform` doesn't contain the major version anymore.
1844 It is always ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since
1845 older Python versions include the version number, so it is recommended to
1846 always use ``sys.platform.startswith('aix')``.
1847 (Contributed by M. Felt in :issue:`36588`.)
1848
1849 * :c:func:`!PyEval_AcquireLock` and :c:func:`!PyEval_AcquireThread` now
1850 terminate the current thread if called while the interpreter is
1851 finalizing, making them consistent with :c:func:`PyEval_RestoreThread`,
1852 :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`. If this
1853 behavior is not desired, guard the call by checking :c:func:`!_Py_IsFinalizing`
1854 or :func:`sys.is_finalizing`.
1855 (Contributed by Joannah Nanjekye in :issue:`36475`.)
1856
1857
1858 Changes in the Python API
1859 -------------------------
1860
1861 * The :func:`os.getcwdb` function now uses the UTF-8 encoding on Windows,
1862 rather than the ANSI code page: see :pep:`529` for the rationale. The
1863 function is no longer deprecated on Windows.
1864 (Contributed by Victor Stinner in :issue:`37412`.)
1865
1866 * :class:`subprocess.Popen` can now use :func:`os.posix_spawn` in some cases
1867 for better performance. On Windows Subsystem for Linux and QEMU User
1868 Emulation, the :class:`~subprocess.Popen` constructor using
1869 :func:`os.posix_spawn` no longer raises an exception on errors like
1870 "missing program". Instead the child process fails with a
1871 non-zero :attr:`~subprocess.Popen.returncode`.
1872 (Contributed by Joannah Nanjekye and Victor Stinner in :issue:`35537`.)
1873
1874 * The *preexec_fn* argument of * :class:`subprocess.Popen` is no longer
1875 compatible with subinterpreters. The use of the parameter in a
1876 subinterpreter now raises :exc:`RuntimeError`.
1877 (Contributed by Eric Snow in :issue:`34651`, modified by Christian Heimes
1878 in :issue:`37951`.)
1879
1880 * The :meth:`imaplib.IMAP4.logout` method no longer silently ignores arbitrary
1881 exceptions.
1882 (Contributed by Victor Stinner in :issue:`36348`.)
1883
1884 * The function :func:`!platform.popen` has been removed, after having been deprecated since
1885 Python 3.3: use :func:`os.popen` instead.
1886 (Contributed by Victor Stinner in :issue:`35345`.)
1887
1888 * The :func:`statistics.mode` function no longer raises an exception
1889 when given multimodal data. Instead, it returns the first mode
1890 encountered in the input data. (Contributed by Raymond Hettinger
1891 in :issue:`35892`.)
1892
1893 * The :meth:`~tkinter.ttk.Treeview.selection` method of the
1894 :class:`tkinter.ttk.Treeview` class no longer takes arguments. Using it with
1895 arguments for changing the selection was deprecated in Python 3.6. Use
1896 specialized methods like :meth:`~tkinter.ttk.Treeview.selection_set` for
1897 changing the selection. (Contributed by Serhiy Storchaka in :issue:`31508`.)
1898
1899 * The :meth:`~xml.dom.minidom.Node.writexml`, :meth:`~xml.dom.minidom.Node.toxml`
1900 and :meth:`~xml.dom.minidom.Node.toprettyxml` methods of
1901 :mod:`xml.dom.minidom` and the :meth:`~xml.etree.ElementTree.ElementTree.write`
1902 method of :mod:`xml.etree.ElementTree` now preserve the attribute order
1903 specified by the user.
1904 (Contributed by Diego Rojas and Raymond Hettinger in :issue:`34160`.)
1905
1906 * A :mod:`dbm.dumb` database opened with flags ``'r'`` is now read-only.
1907 :func:`dbm.dumb.open` with flags ``'r'`` and ``'w'`` no longer creates
1908 a database if it does not exist.
1909 (Contributed by Serhiy Storchaka in :issue:`32749`.)
1910
1911 * The ``doctype()`` method defined in a subclass of
1912 :class:`~xml.etree.ElementTree.XMLParser` will no longer be called and will
1913 emit a :exc:`RuntimeWarning` instead of a :exc:`DeprecationWarning`.
1914 Define the :meth:`doctype() <xml.etree.ElementTree.TreeBuilder.doctype>`
1915 method on a target for handling an XML doctype declaration.
1916 (Contributed by Serhiy Storchaka in :issue:`29209`.)
1917
1918 * A :exc:`RuntimeError` is now raised when the custom metaclass doesn't
1919 provide the ``__classcell__`` entry in the namespace passed to
1920 ``type.__new__``. A :exc:`DeprecationWarning` was emitted in Python
1921 3.6--3.7. (Contributed by Serhiy Storchaka in :issue:`23722`.)
1922
1923 * The :class:`cProfile.Profile <profile.Profile>` class can now be used as
1924 a context manager. (Contributed by Scott Sanderson in :issue:`29235`.)
1925
1926 * :func:`shutil.copyfile`, :func:`shutil.copy`, :func:`shutil.copy2`,
1927 :func:`shutil.copytree` and :func:`shutil.move` use platform-specific
1928 "fast-copy" syscalls (see
1929 :ref:`shutil-platform-dependent-efficient-copy-operations` section).
1930
1931 * :func:`shutil.copyfile` default buffer size on Windows was changed from
1932 16 KiB to 1 MiB.
1933
1934 * The ``PyGC_Head`` struct has changed completely. All code that touched the
1935 struct member should be rewritten. (See :issue:`33597`.)
1936
1937 * The :c:type:`PyInterpreterState` struct has been moved into the "internal"
1938 header files (specifically Include/internal/pycore_pystate.h). An
1939 opaque ``PyInterpreterState`` is still available as part of the public
1940 API (and stable ABI). The docs indicate that none of the struct's
1941 fields are public, so we hope no one has been using them. However,
1942 if you do rely on one or more of those private fields and have no
1943 alternative then please open a BPO issue. We'll work on helping
1944 you adjust (possibly including adding accessor functions to the
1945 public API). (See :issue:`35886`.)
1946
1947 * The :meth:`mmap.flush() <mmap.mmap.flush>` method now returns ``None`` on
1948 success and raises an exception on error under all platforms. Previously,
1949 its behavior was platform-dependent: a nonzero value was returned on success;
1950 zero was returned on error under Windows. A zero value was returned on
1951 success; an exception was raised on error under Unix.
1952 (Contributed by Berker Peksag in :issue:`2122`.)
1953
1954 * :mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process
1955 external entities by default.
1956 (Contributed by Christian Heimes in :issue:`17239`.)
1957
1958 * Deleting a key from a read-only :mod:`dbm` database (:mod:`dbm.dumb`,
1959 :mod:`dbm.gnu` or :mod:`dbm.ndbm`) raises :attr:`!error` (:exc:`dbm.dumb.error`,
1960 :exc:`dbm.gnu.error` or :exc:`dbm.ndbm.error`) instead of :exc:`KeyError`.
1961 (Contributed by Xiang Zhang in :issue:`33106`.)
1962
1963 * Simplified AST for literals. All constants will be represented as
1964 :class:`ast.Constant` instances. Instantiating old classes ``Num``,
1965 ``Str``, ``Bytes``, ``NameConstant`` and ``Ellipsis`` will return
1966 an instance of ``Constant``.
1967 (Contributed by Serhiy Storchaka in :issue:`32892`.)
1968
1969 * :func:`~os.path.expanduser` on Windows now prefers the :envvar:`USERPROFILE`
1970 environment variable and does not use :envvar:`HOME`, which is not normally
1971 set for regular user accounts.
1972 (Contributed by Anthony Sottile in :issue:`36264`.)
1973
1974 * The exception :class:`asyncio.CancelledError` now inherits from
1975 :class:`BaseException` rather than :class:`Exception` and no longer inherits
1976 from :class:`concurrent.futures.CancelledError`.
1977 (Contributed by Yury Selivanov in :issue:`32528`.)
1978
1979 * The function :func:`asyncio.wait_for` now correctly waits for cancellation
1980 when using an instance of :class:`asyncio.Task`. Previously, upon reaching
1981 *timeout*, it was cancelled and immediately returned.
1982 (Contributed by Elvis Pranskevichus in :issue:`32751`.)
1983
1984 * The function :func:`asyncio.BaseTransport.get_extra_info` now returns a safe
1985 to use socket object when 'socket' is passed to the *name* parameter.
1986 (Contributed by Yury Selivanov in :issue:`37027`.)
1987
1988 * :class:`asyncio.BufferedProtocol` has graduated to the stable API.
1989
1990 .. _bpo-36085-whatsnew:
1991
1992 * DLL dependencies for extension modules and DLLs loaded with :mod:`ctypes` on
1993 Windows are now resolved more securely. Only the system paths, the directory
1994 containing the DLL or PYD file, and directories added with
1995 :func:`~os.add_dll_directory` are searched for load-time dependencies.
1996 Specifically, :envvar:`PATH` and the current working directory are no longer
1997 used, and modifications to these will no longer have any effect on normal DLL
1998 resolution. If your application relies on these mechanisms, you should check
1999 for :func:`~os.add_dll_directory` and if it exists, use it to add your DLLs
2000 directory while loading your library. Note that Windows 7 users will need to
2001 ensure that Windows Update KB2533623 has been installed (this is also verified
2002 by the installer).
2003 (Contributed by Steve Dower in :issue:`36085`.)
2004
2005 * The header files and functions related to pgen have been removed after its
2006 replacement by a pure Python implementation. (Contributed by Pablo Galindo
2007 in :issue:`36623`.)
2008
2009 * :class:`types.CodeType` has a new parameter in the second position of the
2010 constructor (*posonlyargcount*) to support positional-only arguments defined
2011 in :pep:`570`. The first argument (*argcount*) now represents the total
2012 number of positional arguments (including positional-only arguments). The new
2013 ``replace()`` method of :class:`types.CodeType` can be used to make the code
2014 future-proof.
2015
2016 * The parameter ``digestmod`` for :func:`hmac.new` no longer uses the MD5 digest
2017 by default.
2018
2019 Changes in the C API
2020 --------------------
2021
2022 * The :c:struct:`PyCompilerFlags` structure got a new *cf_feature_version*
2023 field. It should be initialized to ``PY_MINOR_VERSION``. The field is ignored
2024 by default, and is used if and only if ``PyCF_ONLY_AST`` flag is set in
2025 *cf_flags*.
2026 (Contributed by Guido van Rossum in :issue:`35766`.)
2027
2028 * The :c:func:`!PyEval_ReInitThreads` function has been removed from the C API.
2029 It should not be called explicitly: use :c:func:`PyOS_AfterFork_Child`
2030 instead.
2031 (Contributed by Victor Stinner in :issue:`36728`.)
2032
2033 * On Unix, C extensions are no longer linked to libpython except on Android
2034 and Cygwin. When Python is embedded, ``libpython`` must not be loaded with
2035 ``RTLD_LOCAL``, but ``RTLD_GLOBAL`` instead. Previously, using
2036 ``RTLD_LOCAL``, it was already not possible to load C extensions which
2037 were not linked to ``libpython``, like C extensions of the standard
2038 library built by the ``*shared*`` section of ``Modules/Setup``.
2039 (Contributed by Victor Stinner in :issue:`21536`.)
2040
2041 * Use of ``#`` variants of formats in parsing or building value (e.g.
2042 :c:func:`PyArg_ParseTuple`, :c:func:`Py_BuildValue`, :c:func:`PyObject_CallFunction`,
2043 etc.) without ``PY_SSIZE_T_CLEAN`` defined raises ``DeprecationWarning`` now.
2044 It will be removed in 3.10 or 4.0. Read :ref:`arg-parsing` for detail.
2045 (Contributed by Inada Naoki in :issue:`36381`.)
2046
2047 * Instances of heap-allocated types (such as those created with
2048 :c:func:`PyType_FromSpec`) hold a reference to their type object.
2049 Increasing the reference count of these type objects has been moved from
2050 :c:func:`PyType_GenericAlloc` to the more low-level functions,
2051 :c:func:`PyObject_Init` and :c:macro:`!PyObject_INIT`.
2052 This makes types created through :c:func:`PyType_FromSpec` behave like
2053 other classes in managed code.
2054
2055 :ref:`Statically allocated types <static-types>` are not affected.
2056
2057 For the vast majority of cases, there should be no side effect.
2058 However, types that manually increase the reference count after allocating
2059 an instance (perhaps to work around the bug) may now become immortal.
2060 To avoid this, these classes need to call Py_DECREF on the type object
2061 during instance deallocation.
2062
2063 To correctly port these types into 3.8, please apply the following
2064 changes:
2065
2066 * Remove :c:macro:`Py_INCREF` on the type object after allocating an
2067 instance - if any.
2068 This may happen after calling :c:macro:`PyObject_New`,
2069 :c:macro:`PyObject_NewVar`, :c:func:`PyObject_GC_New`,
2070 :c:func:`PyObject_GC_NewVar`, or any other custom allocator that uses
2071 :c:func:`PyObject_Init` or :c:macro:`!PyObject_INIT`.
2072
2073 Example:
2074
2075 .. code-block:: c
2076
2077 static foo_struct *
2078 foo_new(PyObject *type) {
2079 foo_struct *foo = PyObject_GC_New(foo_struct, (PyTypeObject *) type);
2080 if (foo == NULL)
2081 return NULL;
2082 #if PY_VERSION_HEX < 0x03080000
2083 // Workaround for Python issue 35810; no longer necessary in Python 3.8
2084 PY_INCREF(type)
2085 #endif
2086 return foo;
2087 }
2088
2089 * Ensure that all custom ``tp_dealloc`` functions of heap-allocated types
2090 decrease the type's reference count.
2091
2092 Example:
2093
2094 .. code-block:: c
2095
2096 static void
2097 foo_dealloc(foo_struct *instance) {
2098 PyObject *type = Py_TYPE(instance);
2099 PyObject_GC_Del(instance);
2100 #if PY_VERSION_HEX >= 0x03080000
2101 // This was not needed before Python 3.8 (Python issue 35810)
2102 Py_DECREF(type);
2103 #endif
2104 }
2105
2106 (Contributed by Eddie Elizondo in :issue:`35810`.)
2107
2108 * The :c:macro:`Py_DEPRECATED()` macro has been implemented for MSVC.
2109 The macro now must be placed before the symbol name.
2110
2111 Example:
2112
2113 .. code-block:: c
2114
2115 Py_DEPRECATED(3.8) PyAPI_FUNC(int) Py_OldFunction(void);
2116
2117 (Contributed by Zackery Spytz in :issue:`33407`.)
2118
2119 * The interpreter does not pretend to support binary compatibility of
2120 extension types across feature releases, anymore. A :c:type:`PyTypeObject`
2121 exported by a third-party extension module is supposed to have all the
2122 slots expected in the current Python version, including
2123 :c:member:`~PyTypeObject.tp_finalize` (:c:macro:`Py_TPFLAGS_HAVE_FINALIZE`
2124 is not checked anymore before reading :c:member:`~PyTypeObject.tp_finalize`).
2125
2126 (Contributed by Antoine Pitrou in :issue:`32388`.)
2127
2128 * The functions :c:func:`!PyNode_AddChild` and :c:func:`!PyParser_AddToken` now accept
2129 two additional ``int`` arguments *end_lineno* and *end_col_offset*.
2130
2131 * The :file:`libpython38.a` file to allow MinGW tools to link directly against
2132 :file:`python38.dll` is no longer included in the regular Windows distribution.
2133 If you require this file, it may be generated with the ``gendef`` and
2134 ``dlltool`` tools, which are part of the MinGW binutils package:
2135
2136 .. code-block:: shell
2137
2138 gendef - python38.dll > tmp.def
2139 dlltool --dllname python38.dll --def tmp.def --output-lib libpython38.a
2140
2141 The location of an installed :file:`pythonXY.dll` will depend on the
2142 installation options and the version and language of Windows. See
2143 :ref:`using-on-windows` for more information. The resulting library should be
2144 placed in the same directory as :file:`pythonXY.lib`, which is generally the
2145 :file:`libs` directory under your Python installation.
2146
2147 (Contributed by Steve Dower in :issue:`37351`.)
2148
2149
2150 CPython bytecode changes
2151 ------------------------
2152
2153 * The interpreter loop has been simplified by moving the logic of unrolling
2154 the stack of blocks into the compiler. The compiler emits now explicit
2155 instructions for adjusting the stack of values and calling the
2156 cleaning-up code for :keyword:`break`, :keyword:`continue` and
2157 :keyword:`return`.
2158
2159 Removed opcodes :opcode:`!BREAK_LOOP`, :opcode:`!CONTINUE_LOOP`,
2160 :opcode:`!SETUP_LOOP` and :opcode:`!SETUP_EXCEPT`. Added new opcodes
2161 :opcode:`!ROT_FOUR`, :opcode:`!BEGIN_FINALLY`, :opcode:`!CALL_FINALLY` and
2162 :opcode:`!POP_FINALLY`. Changed the behavior of :opcode:`!END_FINALLY`
2163 and :opcode:`!WITH_CLEANUP_START`.
2164
2165 (Contributed by Mark Shannon, Antoine Pitrou and Serhiy Storchaka in
2166 :issue:`17611`.)
2167
2168 * Added new opcode :opcode:`END_ASYNC_FOR` for handling exceptions raised
2169 when awaiting a next item in an :keyword:`async for` loop.
2170 (Contributed by Serhiy Storchaka in :issue:`33041`.)
2171
2172 * The :opcode:`MAP_ADD` now expects the value as the first element in the
2173 stack and the key as the second element. This change was made so the key
2174 is always evaluated before the value in dictionary comprehensions, as
2175 proposed by :pep:`572`. (Contributed by Jörn Heissler in :issue:`35224`.)
2176
2177
2178 Demos and Tools
2179 ---------------
2180
2181 Added a benchmark script for timing various ways to access variables:
2182 ``Tools/scripts/var_access_benchmark.py``.
2183 (Contributed by Raymond Hettinger in :issue:`35884`.)
2184
2185 Here's a summary of performance improvements since Python 3.3:
2186
2187 .. code-block:: none
2188
2189 Python version 3.3 3.4 3.5 3.6 3.7 3.8
2190 -------------- --- --- --- --- --- ---
2191
2192 Variable and attribute read access:
2193 read_local 4.0 7.1 7.1 5.4 5.1 3.9
2194 read_nonlocal 5.3 7.1 8.1 5.8 5.4 4.4
2195 read_global 13.3 15.5 19.0 14.3 13.6 7.6
2196 read_builtin 20.0 21.1 21.6 18.5 19.0 7.5
2197 read_classvar_from_class 20.5 25.6 26.5 20.7 19.5 18.4
2198 read_classvar_from_instance 18.5 22.8 23.5 18.8 17.1 16.4
2199 read_instancevar 26.8 32.4 33.1 28.0 26.3 25.4
2200 read_instancevar_slots 23.7 27.8 31.3 20.8 20.8 20.2
2201 read_namedtuple 68.5 73.8 57.5 45.0 46.8 18.4
2202 read_boundmethod 29.8 37.6 37.9 29.6 26.9 27.7
2203
2204 Variable and attribute write access:
2205 write_local 4.6 8.7 9.3 5.5 5.3 4.3
2206 write_nonlocal 7.3 10.5 11.1 5.6 5.5 4.7
2207 write_global 15.9 19.7 21.2 18.0 18.0 15.8
2208 write_classvar 81.9 92.9 96.0 104.6 102.1 39.2
2209 write_instancevar 36.4 44.6 45.8 40.0 38.9 35.5
2210 write_instancevar_slots 28.7 35.6 36.1 27.3 26.6 25.7
2211
2212 Data structure read access:
2213 read_list 19.2 24.2 24.5 20.8 20.8 19.0
2214 read_deque 19.9 24.7 25.5 20.2 20.6 19.8
2215 read_dict 19.7 24.3 25.7 22.3 23.0 21.0
2216 read_strdict 17.9 22.6 24.3 19.5 21.2 18.9
2217
2218 Data structure write access:
2219 write_list 21.2 27.1 28.5 22.5 21.6 20.0
2220 write_deque 23.8 28.7 30.1 22.7 21.8 23.5
2221 write_dict 25.9 31.4 33.3 29.3 29.2 24.7
2222 write_strdict 22.9 28.4 29.9 27.5 25.2 23.1
2223
2224 Stack (or queue) operations:
2225 list_append_pop 144.2 93.4 112.7 75.4 74.2 50.8
2226 deque_append_pop 30.4 43.5 57.0 49.4 49.2 42.5
2227 deque_append_popleft 30.8 43.7 57.3 49.7 49.7 42.8
2228
2229 Timing loop:
2230 loop_overhead 0.3 0.5 0.6 0.4 0.3 0.3
2231
2232 The benchmarks were measured on an
2233 `Intel® Core™ i7-4960HQ processor
2234 <https://ark.intel.com/content/www/us/en/ark/products/76088/intel-core-i7-4960hq-processor-6m-cache-up-to-3-80-ghz.html>`_
2235 running the macOS 64-bit builds found at
2236 `python.org <https://www.python.org/downloads/macos/>`_.
2237 The benchmark script displays timings in nanoseconds.
2238
2239
2240 Notable changes in Python 3.8.1
2241 ===============================
2242
2243 Due to significant security concerns, the *reuse_address* parameter of
2244 :meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is
2245 because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more
2246 details, see the documentation for ``loop.create_datagram_endpoint()``.
2247 (Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in
2248 :issue:`37228`.)
2249
2250 Notable changes in Python 3.8.2
2251 ===============================
2252
2253 Fixed a regression with the ``ignore`` callback of :func:`shutil.copytree`.
2254 The argument types are now str and List[str] again.
2255 (Contributed by Manuel Barkhau and Giampaolo Rodola in :gh:`83571`.)
2256
2257 Notable changes in Python 3.8.3
2258 ===============================
2259
2260 The constant values of future flags in the :mod:`__future__` module
2261 are updated in order to prevent collision with compiler flags. Previously
2262 ``PyCF_ALLOW_TOP_LEVEL_AWAIT`` was clashing with ``CO_FUTURE_DIVISION``.
2263 (Contributed by Batuhan Taskaya in :gh:`83743`)
2264
2265 Notable changes in Python 3.8.8
2266 ===============================
2267
2268 Earlier Python versions allowed using both ``;`` and ``&`` as
2269 query parameter separators in :func:`urllib.parse.parse_qs` and
2270 :func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with
2271 newer W3C recommendations, this has been changed to allow only a single
2272 separator key, with ``&`` as the default. This change also affects
2273 :func:`!cgi.parse` and :func:`!cgi.parse_multipart` as they use the affected
2274 functions internally. For more details, please see their respective
2275 documentation.
2276 (Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.)
2277
2278 Notable changes in Python 3.8.9
2279 ===============================
2280
2281 A security fix alters the :class:`ftplib.FTP` behavior to not trust the
2282 IPv4 address sent from the remote server when setting up a passive data
2283 channel. We reuse the ftp server IP address instead. For unusual code
2284 requiring the old behavior, set a ``trust_server_pasv_ipv4_address``
2285 attribute on your FTP instance to ``True``. (See :gh:`87451`)
2286
2287 Notable changes in Python 3.8.10
2288 ================================
2289
2290 macOS 11.0 (Big Sur) and Apple Silicon Mac support
2291 --------------------------------------------------
2292
2293 As of 3.8.10, Python now supports building and running on macOS 11
2294 (Big Sur) and on Apple Silicon Macs (based on the ``ARM64`` architecture).
2295 A new universal build variant, ``universal2``, is now available to natively
2296 support both ``ARM64`` and ``Intel 64`` in one set of executables.
2297 Note that support for "weaklinking", building binaries targeted for newer
2298 versions of macOS that will also run correctly on older versions by
2299 testing at runtime for missing features, is not included in this backport
2300 from Python 3.9; to support a range of macOS versions, continue to target
2301 for and build on the oldest version in the range.
2302
2303 (Originally contributed by Ronald Oussoren and Lawrence D'Anna in :gh:`85272`,
2304 with fixes by FX Coudert and Eli Rykoff, and backported to 3.8 by Maxime Bélanger
2305 and Ned Deily)
2306
2307 Notable changes in Python 3.8.10
2308 ================================
2309
2310 urllib.parse
2311 ------------
2312
2313 The presence of newline or tab characters in parts of a URL allows for some
2314 forms of attacks. Following the WHATWG specification that updates :rfc:`3986`,
2315 ASCII newline ``\n``, ``\r`` and tab ``\t`` characters are stripped from the
2316 URL by the parser in :mod:`urllib.parse` preventing such attacks. The removal
2317 characters are controlled by a new module level variable
2318 ``urllib.parse._UNSAFE_URL_BYTES_TO_REMOVE``. (See :issue:`43882`)
2319
2320
2321 Notable changes in Python 3.8.12
2322 ================================
2323
2324 Changes in the Python API
2325 -------------------------
2326
2327 Starting with Python 3.8.12 the :mod:`ipaddress` module no longer accepts
2328 any leading zeros in IPv4 address strings. Leading zeros are ambiguous and
2329 interpreted as octal notation by some libraries. For example the legacy
2330 function :func:`socket.inet_aton` treats leading zeros as octal notation.
2331 glibc implementation of modern :func:`~socket.inet_pton` does not accept
2332 any leading zeros.
2333
2334 (Originally contributed by Christian Heimes in :issue:`36384`, and backported
2335 to 3.8 by Achraf Merzouki.)
2336
2337 Notable security feature in 3.8.14
2338 ==================================
2339
2340 Converting between :class:`int` and :class:`str` in bases other than 2
2341 (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal)
2342 now raises a :exc:`ValueError` if the number of digits in string form is
2343 above a limit to avoid potential denial of service attacks due to the
2344 algorithmic complexity. This is a mitigation for :cve:`2020-10735`.
2345 This limit can be configured or disabled by environment variable, command
2346 line flag, or :mod:`sys` APIs. See the :ref:`integer string conversion
2347 length limitation <int_max_str_digits>` documentation. The default limit
2348 is 4300 digits in string form.
2349
2350 Notable changes in 3.8.17
2351 =========================
2352
2353 tarfile
2354 -------
2355
2356 * The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`,
2357 have a new a *filter* argument that allows limiting tar features than may be
2358 surprising or dangerous, such as creating files outside the destination
2359 directory.
2360 See :ref:`tarfile-extraction-filter` for details.
2361 In Python 3.12, use without the *filter* argument will show a
2362 :exc:`DeprecationWarning`.
2363 In Python 3.14, the default will switch to ``'data'``.
2364 (Contributed by Petr Viktorin in :pep:`706`.)