Debian Codegraph @

Doc/whatsnew/3.12.rst

raw
1
2 ****************************
3 What's New In Python 3.12
4 ****************************
5
6 :Editor: Adam Turner
7
8 .. Rules for maintenance:
9
10 * Anyone can add text to this document. Do not spend very much time
11 on the wording of your changes, because your text will probably
12 get rewritten to some degree.
13
14 * The maintainer will go through Misc/NEWS periodically and add
15 changes; it's therefore more important to add your changes to
16 Misc/NEWS than to this file.
17
18 * This is not a complete list of every single change; completeness
19 is the purpose of Misc/NEWS. Some changes I consider too small
20 or esoteric to include. If such a change is added to the text,
21 I'll just remove it. (This is another reason you shouldn't spend
22 too much time on writing your addition.)
23
24 * If you want to draw your new text to the attention of the
25 maintainer, add 'XXX' to the beginning of the paragraph or
26 section.
27
28 * It's OK to just add a fragmentary note about a change. For
29 example: "XXX Describe the transmogrify() function added to the
30 socket module." The maintainer will research the change and
31 write the necessary text.
32
33 * You can comment out your additions if you like, but it's not
34 necessary (especially when a final release is some months away).
35
36 * Credit the author of a patch or bugfix. Just the name is
37 sufficient; the e-mail address isn't necessary.
38
39 * It's helpful to add the issue number as a comment:
40
41 XXX Describe the transmogrify() function added to the socket
42 module.
43 (Contributed by P.Y. Developer in :gh:`12345`.)
44
45 This saves the maintainer the effort of going through the VCS log when
46 researching a change.
47
48 This article explains the new features in Python 3.12, compared to 3.11.
49 Python 3.12 was released on October 2, 2023.
50 For full details, see the :ref:`changelog <changelog>`.
51
52 .. seealso::
53
54 :pep:`693` -- Python 3.12 Release Schedule
55
56 Summary -- Release highlights
57 =============================
58
59 .. This section singles out the most important changes in Python 3.12.
60 Brevity is key.
61
62 Python 3.12 is a stable release of the Python programming language,
63 with a mix of changes to the language and the standard library.
64 The library changes focus on cleaning up deprecated APIs, usability, and correctness.
65 Of note, the :mod:`!distutils` package has been removed from the standard library.
66 Filesystem support in :mod:`os` and :mod:`pathlib` has seen a number of improvements,
67 and several modules have better performance.
68
69 The language changes focus on usability,
70 as :term:`f-strings <f-string>` have had many limitations removed
71 and 'Did you mean ...' suggestions continue to improve.
72 The new :ref:`type parameter syntax <whatsnew312-pep695>`
73 and :keyword:`type` statement improve ergonomics for using :term:`generic types
74 <generic type>` and :term:`type aliases <type alias>` with static type checkers.
75
76 This article doesn't attempt to provide a complete specification of all new features,
77 but instead gives a convenient overview.
78 For full details, you should refer to the documentation,
79 such as the :ref:`Library Reference <library-index>`
80 and :ref:`Language Reference <reference-index>`.
81 If you want to understand the complete implementation and design rationale for a change,
82 refer to the PEP for a particular new feature;
83 but note that PEPs usually are not kept up-to-date
84 once a feature has been fully implemented.
85
86 --------------
87
88 .. PEP-sized items next.
89
90 New syntax features:
91
92 * :ref:`PEP 695 <whatsnew312-pep695>`, type parameter syntax and the :keyword:`type` statement
93
94 New grammar features:
95
96 * :ref:`PEP 701 <whatsnew312-pep701>`, :term:`f-strings <f-string>` in the grammar
97
98 Interpreter improvements:
99
100 * :ref:`PEP 684 <whatsnew312-pep684>`, a unique per-interpreter :term:`GIL
101 <global interpreter lock>`
102 * :ref:`PEP 669 <whatsnew312-pep669>`, low impact monitoring
103 * `Improved 'Did you mean ...' suggestions <improved error messages_>`_
104 for :exc:`NameError`, :exc:`ImportError`, and :exc:`SyntaxError` exceptions
105
106 Python data model improvements:
107
108 * :ref:`PEP 688 <whatsnew312-pep688>`, using the :ref:`buffer protocol
109 <bufferobjects>` from Python
110
111 Significant improvements in the standard library:
112
113 * The :class:`pathlib.Path` class now supports subclassing
114 * The :mod:`os` module received several improvements for Windows support
115 * A :ref:`command-line interface <sqlite3-cli>` has been added to the
116 :mod:`sqlite3` module
117 * :func:`isinstance` checks against :func:`runtime-checkable protocols
118 <typing.runtime_checkable>` enjoy a speed up of between two and 20 times
119 * The :mod:`asyncio` package has had a number of performance improvements,
120 with some benchmarks showing a 75% speed up.
121 * A :ref:`command-line interface <uuid-cli>` has been added to the
122 :mod:`uuid` module
123 * Due to the changes in :ref:`PEP 701 <whatsnew312-pep701>`,
124 producing tokens via the :mod:`tokenize` module is up to 64% faster.
125
126 Security improvements:
127
128 * Replace the builtin :mod:`hashlib` implementations of
129 SHA1, SHA3, SHA2-384, SHA2-512, and MD5 with formally verified code from the
130 `HACL* <https://github.com/hacl-star/hacl-star/>`__ project.
131 These builtin implementations remain as fallbacks that are only used when
132 OpenSSL does not provide them.
133
134 C API improvements:
135
136 * :ref:`PEP 697 <whatsnew312-pep697>`, unstable C API tier
137 * :ref:`PEP 683 <whatsnew312-pep683>`, immortal objects
138
139 CPython implementation improvements:
140
141 * :ref:`PEP 709 <whatsnew312-pep709>`, comprehension inlining
142 * :ref:`CPython support <perf_profiling>` for the Linux ``perf`` profiler
143 * Implement stack overflow protection on supported platforms
144
145 New typing features:
146
147 * :ref:`PEP 692 <whatsnew312-pep692>`, using :class:`~typing.TypedDict` to
148 annotate :term:`**kwargs <argument>`
149 * :ref:`PEP 698 <whatsnew312-pep698>`, :func:`typing.override` decorator
150
151 Important deprecations, removals or restrictions:
152
153 * :pep:`623`: Remove ``wstr`` from Unicode objects in Python's C API,
154 reducing the size of every :class:`str` object by at least 8 bytes.
155
156 * :pep:`632`: Remove the :mod:`!distutils` package.
157 See :pep:`the migration guide <0632#migration-advice>`
158 for advice replacing the APIs it provided.
159 The third-party `Setuptools <https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html>`__
160 package continues to provide :mod:`!distutils`,
161 if you still require it in Python 3.12 and beyond.
162
163 * :gh:`95299`: Do not pre-install ``setuptools`` in virtual environments
164 created with :mod:`venv`.
165 This means that ``distutils``, ``setuptools``, ``pkg_resources``,
166 and ``easy_install`` will no longer available by default; to access these
167 run ``pip install setuptools`` in the :ref:`activated <venv-explanation>`
168 virtual environment.
169
170 * The :mod:`!asynchat`, :mod:`!asyncore`, and :mod:`!imp` modules have been
171 removed, along with several :class:`unittest.TestCase`
172 `method aliases <unittest-TestCase-removed-aliases_>`_.
173
174
175 New Features
176 ============
177
178 .. _whatsnew312-pep695:
179
180 PEP 695: Type Parameter Syntax
181 ------------------------------
182
183 Generic classes and functions under :pep:`484` were declared using a verbose syntax
184 that left the scope of type parameters unclear and required explicit declarations of
185 variance.
186
187 :pep:`695` introduces a new, more compact and explicit way to create
188 :ref:`generic classes <generic-classes>` and :ref:`functions <generic-functions>`::
189
190 def max[T](args: Iterable[T]) -> T:
191 ...
192
193 class list[T]:
194 def __getitem__(self, index: int, /) -> T:
195 ...
196
197 def append(self, element: T) -> None:
198 ...
199
200 In addition, the PEP introduces a new way to declare :ref:`type aliases <type-aliases>`
201 using the :keyword:`type` statement, which creates an instance of
202 :class:`~typing.TypeAliasType`::
203
204 type Point = tuple[float, float]
205
206 Type aliases can also be :ref:`generic <generic-type-aliases>`::
207
208 type Point[T] = tuple[T, T]
209
210 The new syntax allows declaring :class:`~typing.TypeVarTuple`
211 and :class:`~typing.ParamSpec` parameters, as well as :class:`~typing.TypeVar`
212 parameters with bounds or constraints::
213
214 type IntFunc[**P] = Callable[P, int] # ParamSpec
215 type LabeledTuple[*Ts] = tuple[str, *Ts] # TypeVarTuple
216 type HashableSequence[T: Hashable] = Sequence[T] # TypeVar with bound
217 type IntOrStrSequence[T: (int, str)] = Sequence[T] # TypeVar with constraints
218
219 The value of type aliases and the bound and constraints of type variables
220 created through this syntax are evaluated only on demand (see
221 :ref:`lazy evaluation <lazy-evaluation>`). This means type aliases are able to
222 refer to other types defined later in the file.
223
224 Type parameters declared through a type parameter list are visible within the
225 scope of the declaration and any nested scopes, but not in the outer scope. For
226 example, they can be used in the type annotations for the methods of a generic
227 class or in the class body. However, they cannot be used in the module scope after
228 the class is defined. See :ref:`type-params` for a detailed description of the
229 runtime semantics of type parameters.
230
231 In order to support these scoping semantics, a new kind of scope is introduced,
232 the :ref:`annotation scope <annotation-scopes>`. Annotation scopes behave for the
233 most part like function scopes, but interact differently with enclosing class scopes.
234 In Python 3.13, :term:`annotations <annotation>` will also be evaluated in
235 annotation scopes.
236
237 See :pep:`695` for more details.
238
239 (PEP written by Eric Traut. Implementation by Jelle Zijlstra, Eric Traut,
240 and others in :gh:`103764`.)
241
242 .. _whatsnew312-pep701:
243
244 PEP 701: Syntactic formalization of f-strings
245 ---------------------------------------------
246
247 :pep:`701` lifts some restrictions on the usage of :term:`f-strings <f-string>`.
248 Expression components inside f-strings can now be any valid Python expression,
249 including strings reusing the same quote as the containing f-string,
250 multi-line expressions, comments, backslashes, and unicode escape sequences.
251 Let's cover these in detail:
252
253 * Quote reuse: in Python 3.11, reusing the same quotes as the enclosing f-string
254 raises a :exc:`SyntaxError`, forcing the user to either use other available
255 quotes (like using double quotes or triple quotes if the f-string uses single
256 quotes). In Python 3.12, you can now do things like this:
257
258 >>> songs = ['Take me back to Eden', 'Alkaline', 'Ascensionism']
259 >>> f"This is the playlist: {", ".join(songs)}"
260 'This is the playlist: Take me back to Eden, Alkaline, Ascensionism'
261
262 Note that before this change there was no explicit limit in how f-strings can
263 be nested, but the fact that string quotes cannot be reused inside the
264 expression component of f-strings made it impossible to nest f-strings
265 arbitrarily. In fact, this is the most nested f-string that could be written:
266
267 >>> f"""{f'''{f'{f"{1+1}"}'}'''}"""
268 '2'
269
270 As now f-strings can contain any valid Python expression inside expression
271 components, it is now possible to nest f-strings arbitrarily:
272
273 >>> f"{f"{f"{f"{f"{f"{1+1}"}"}"}"}"}"
274 '2'
275
276 * Multi-line expressions and comments: In Python 3.11, f-string expressions
277 must be defined in a single line, even if the expression within the f-string
278 could normally span multiple lines
279 (like literal lists being defined over multiple lines),
280 making them harder to read. In Python 3.12 you can now define f-strings
281 spanning multiple lines, and add inline comments:
282
283 >>> f"This is the playlist: {", ".join([
284 ... 'Take me back to Eden', # My, my, those eyes like fire
285 ... 'Alkaline', # Not acid nor alkaline
286 ... 'Ascensionism' # Take to the broken skies at last
287 ... ])}"
288 'This is the playlist: Take me back to Eden, Alkaline, Ascensionism'
289
290 * Backslashes and unicode characters: before Python 3.12 f-string expressions
291 couldn't contain any ``\`` character. This also affected unicode :ref:`escape
292 sequences <escape-sequences>` (such as ``\N{snowman}``) as these contain
293 the ``\N`` part that previously could not be part of expression components of
294 f-strings. Now, you can define expressions like this:
295
296 >>> print(f"This is the playlist: {"\n".join(songs)}")
297 This is the playlist: Take me back to Eden
298 Alkaline
299 Ascensionism
300 >>> print(f"This is the playlist: {"\N{BLACK HEART SUIT}".join(songs)}")
301 This is the playlist: Take me back to Eden♥Alkaline♥Ascensionism
302
303 See :pep:`701` for more details.
304
305 As a positive side-effect of how this feature has been implemented (by parsing f-strings
306 with :pep:`the PEG parser <617>`), now error messages for f-strings are more precise
307 and include the exact location of the error. For example, in Python 3.11, the following
308 f-string raises a :exc:`SyntaxError`:
309
310 .. code-block:: python
311
312 >>> my_string = f"{x z y}" + f"{1 + 1}"
313 File "<stdin>", line 1
314 (x z y)
315 ^^^
316 SyntaxError: f-string: invalid syntax. Perhaps you forgot a comma?
317
318 but the error message doesn't include the exact location of the error within the line and
319 also has the expression artificially surrounded by parentheses. In Python 3.12, as f-strings
320 are parsed with the PEG parser, error messages can be more precise and show the entire line:
321
322 .. code-block:: python
323
324 >>> my_string = f"{x z y}" + f"{1 + 1}"
325 File "<stdin>", line 1
326 my_string = f"{x z y}" + f"{1 + 1}"
327 ^^^
328 SyntaxError: invalid syntax. Perhaps you forgot a comma?
329
330 (Contributed by Pablo Galindo, Batuhan Taskaya, Lysandros Nikolaou, Cristián
331 Maureira-Fredes and Marta Gómez in :gh:`102856`. PEP written by Pablo Galindo,
332 Batuhan Taskaya, Lysandros Nikolaou and Marta Gómez).
333
334 .. _whatsnew312-pep684:
335
336 PEP 684: A Per-Interpreter GIL
337 ------------------------------
338
339 :pep:`684` introduces a per-interpreter :term:`GIL <global interpreter lock>`,
340 so that sub-interpreters may now be created with a unique GIL per interpreter.
341 This allows Python programs to take full advantage of multiple CPU
342 cores. This is currently only available through the C-API,
343 though a Python API is :pep:`anticipated for 3.13 <554>`.
344
345 Use the new :c:func:`Py_NewInterpreterFromConfig` function to
346 create an interpreter with its own GIL:
347
348 .. code-block:: c
349
350 PyInterpreterConfig config = {
351 .check_multi_interp_extensions = 1,
352 .gil = PyInterpreterConfig_OWN_GIL,
353 };
354 PyThreadState *tstate = NULL;
355 PyStatus status = Py_NewInterpreterFromConfig(&tstate, &config);
356 if (PyStatus_Exception(status)) {
357 return -1;
358 }
359 /* The new interpreter is now active in the current thread. */
360
361 For further examples how to use the C-API for sub-interpreters with a
362 per-interpreter GIL, see ``Modules/_xxsubinterpretersmodule.c``.
363
364 (Contributed by Eric Snow in :gh:`104210`, etc.)
365
366 .. _whatsnew312-pep669:
367
368 PEP 669: Low impact monitoring for CPython
369 ------------------------------------------
370
371 :pep:`669` defines a new :mod:`API <sys.monitoring>` for profilers,
372 debuggers, and other tools to monitor events in CPython.
373 It covers a wide range of events, including calls,
374 returns, lines, exceptions, jumps, and more.
375 This means that you only pay for what you use, providing support
376 for near-zero overhead debuggers and coverage tools.
377 See :mod:`sys.monitoring` for details.
378
379 (Contributed by Mark Shannon in :gh:`103082`.)
380
381 .. _whatsnew312-pep688:
382
383 PEP 688: Making the buffer protocol accessible in Python
384 --------------------------------------------------------
385
386 :pep:`688` introduces a way to use the :ref:`buffer protocol <bufferobjects>`
387 from Python code. Classes that implement the :meth:`~object.__buffer__` method
388 are now usable as buffer types.
389
390 The new :class:`collections.abc.Buffer` ABC provides a standard
391 way to represent buffer objects, for example in type annotations.
392 The new :class:`inspect.BufferFlags` enum represents the flags that
393 can be used to customize buffer creation.
394 (Contributed by Jelle Zijlstra in :gh:`102500`.)
395
396 .. _whatsnew312-pep709:
397
398 PEP 709: Comprehension inlining
399 -------------------------------
400
401 Dictionary, list, and set comprehensions are now inlined, rather than creating a
402 new single-use function object for each execution of the comprehension. This
403 speeds up execution of a comprehension by up to two times.
404 See :pep:`709` for further details.
405
406 Comprehension iteration variables remain isolated and don't overwrite a
407 variable of the same name in the outer scope, nor are they visible after the
408 comprehension. Inlining does result in a few visible behavior changes:
409
410 * There is no longer a separate frame for the comprehension in tracebacks,
411 and tracing/profiling no longer shows the comprehension as a function call.
412 * The :mod:`symtable` module will no longer produce child symbol tables for each
413 comprehension; instead, the comprehension's locals will be included in the
414 parent function's symbol table.
415 * Calling :func:`locals` inside a comprehension now includes variables
416 from outside the comprehension, and no longer includes the synthetic ``.0``
417 variable for the comprehension "argument".
418 * A comprehension iterating directly over ``locals()`` (e.g. ``[k for k in
419 locals()]``) may see "RuntimeError: dictionary changed size during iteration"
420 when run under tracing (e.g. code coverage measurement). This is the same
421 behavior already seen in e.g. ``for k in locals():``. To avoid the error, first
422 create a list of keys to iterate over: ``keys = list(locals()); [k for k in
423 keys]``.
424
425 (Contributed by Carl Meyer and Vladimir Matveev in :pep:`709`.)
426
427 Improved Error Messages
428 -----------------------
429
430 * Modules from the standard library are now potentially suggested as part of
431 the error messages displayed by the interpreter when a :exc:`NameError` is
432 raised to the top level. (Contributed by Pablo Galindo in :gh:`98254`.)
433
434 >>> sys.version_info
435 Traceback (most recent call last):
436 File "<stdin>", line 1, in <module>
437 NameError: name 'sys' is not defined. Did you forget to import 'sys'?
438
439 * Improve the error suggestion for :exc:`NameError` exceptions for instances.
440 Now if a :exc:`NameError` is raised in a method and the instance has an
441 attribute that's exactly equal to the name in the exception, the suggestion
442 will include ``self.<NAME>`` instead of the closest match in the method
443 scope. (Contributed by Pablo Galindo in :gh:`99139`.)
444
445 >>> class A:
446 ... def __init__(self):
447 ... self.blech = 1
448 ...
449 ... def foo(self):
450 ... somethin = blech
451 ...
452 >>> A().foo()
453 Traceback (most recent call last):
454 File "<stdin>", line 1
455 somethin = blech
456 ^^^^^
457 NameError: name 'blech' is not defined. Did you mean: 'self.blech'?
458
459 * Improve the :exc:`SyntaxError` error message when the user types ``import x
460 from y`` instead of ``from y import x``. (Contributed by Pablo Galindo in :gh:`98931`.)
461
462 >>> import a.y.z from b.y.z
463 Traceback (most recent call last):
464 File "<stdin>", line 1
465 import a.y.z from b.y.z
466 ^^^^^^^^^^^^^^^^^^^^^^^
467 SyntaxError: Did you mean to use 'from ... import ...' instead?
468
469 * :exc:`ImportError` exceptions raised from failed ``from <module> import
470 <name>`` statements now include suggestions for the value of ``<name>`` based on the
471 available names in ``<module>``. (Contributed by Pablo Galindo in :gh:`91058`.)
472
473 >>> from collections import chainmap
474 Traceback (most recent call last):
475 File "<stdin>", line 1, in <module>
476 ImportError: cannot import name 'chainmap' from 'collections'. Did you mean: 'ChainMap'?
477
478
479 New Features Related to Type Hints
480 ==================================
481
482 This section covers major changes affecting :pep:`type hints <484>` and
483 the :mod:`typing` module.
484
485 .. _whatsnew312-pep692:
486
487 PEP 692: Using ``TypedDict`` for more precise ``**kwargs`` typing
488 -----------------------------------------------------------------
489
490 Typing ``**kwargs`` in a function signature as introduced by :pep:`484` allowed
491 for valid annotations only in cases where all of the ``**kwargs`` were of the
492 same type.
493
494 :pep:`692` specifies a more precise way of typing ``**kwargs`` by relying on
495 typed dictionaries::
496
497 from typing import TypedDict, Unpack
498
499 class Movie(TypedDict):
500 name: str
501 year: int
502
503 def foo(**kwargs: Unpack[Movie]): ...
504
505 See :pep:`692` for more details.
506
507 (Contributed by Franek Magiera in :gh:`103629`.)
508
509 .. _whatsnew312-pep698:
510
511 PEP 698: Override Decorator for Static Typing
512 ---------------------------------------------
513
514 A new decorator :func:`typing.override` has been added to the :mod:`typing`
515 module. It indicates to type checkers that the method is intended to override
516 a method in a superclass. This allows type checkers to catch mistakes where
517 a method that is intended to override something in a base class
518 does not in fact do so.
519
520 Example::
521
522 from typing import override
523
524 class Base:
525 def get_color(self) -> str:
526 return "blue"
527
528 class GoodChild(Base):
529 @override # ok: overrides Base.get_color
530 def get_color(self) -> str:
531 return "yellow"
532
533 class BadChild(Base):
534 @override # type checker error: does not override Base.get_color
535 def get_colour(self) -> str:
536 return "red"
537
538 See :pep:`698` for more details.
539
540 (Contributed by Steven Troxler in :gh:`101561`.)
541
542 Other Language Changes
543 ======================
544
545 * The parser now raises :exc:`SyntaxError` when parsing source code containing
546 null bytes. (Contributed by Pablo Galindo in :gh:`96670`.)
547
548 * A backslash-character pair that is not a valid escape sequence now generates
549 a :exc:`SyntaxWarning`, instead of :exc:`DeprecationWarning`.
550 For example, ``re.compile("\d+\.\d+")`` now emits a :exc:`SyntaxWarning`
551 (``"\d"`` is an invalid escape sequence, use raw strings for regular
552 expression: ``re.compile(r"\d+\.\d+")``).
553 In a future Python version, :exc:`SyntaxError` will eventually be raised,
554 instead of :exc:`SyntaxWarning`.
555 (Contributed by Victor Stinner in :gh:`98401`.)
556
557 * Octal escapes with value larger than ``0o377`` (ex: ``"\477"``), deprecated
558 in Python 3.11, now produce a :exc:`SyntaxWarning`, instead of
559 :exc:`DeprecationWarning`.
560 In a future Python version they will be eventually a :exc:`SyntaxError`.
561 (Contributed by Victor Stinner in :gh:`98401`.)
562
563 * Variables used in the target part of comprehensions that are not stored to
564 can now be used in assignment expressions (``:=``).
565 For example, in ``[(b := 1) for a, b.prop in some_iter]``, the assignment to
566 ``b`` is now allowed. Note that assigning to variables stored to in the target
567 part of comprehensions (like ``a``) is still disallowed, as per :pep:`572`.
568 (Contributed by Nikita Sobolev in :gh:`100581`.)
569
570 * Exceptions raised in a class or type's ``__set_name__`` method are no longer
571 wrapped by a :exc:`RuntimeError`. Context information is added to the
572 exception as a :pep:`678` note. (Contributed by Irit Katriel in :gh:`77757`.)
573
574 * When a ``try-except*`` construct handles the entire :exc:`ExceptionGroup`
575 and raises one other exception, that exception is no longer wrapped in an
576 :exc:`ExceptionGroup`. Also changed in version 3.11.4. (Contributed by Irit
577 Katriel in :gh:`103590`.)
578
579 * The Garbage Collector now runs only on the eval breaker mechanism of the
580 Python bytecode evaluation loop instead of object allocations. The GC can
581 also run when :c:func:`PyErr_CheckSignals` is called so C extensions that
582 need to run for a long time without executing any Python code also have a
583 chance to execute the GC periodically. (Contributed by Pablo Galindo in
584 :gh:`97922`.)
585
586 * All builtin and extension callables expecting boolean parameters now accept
587 arguments of any type instead of just :class:`bool` and :class:`int`.
588 (Contributed by Serhiy Storchaka in :gh:`60203`.)
589
590 * :class:`memoryview` now supports the half-float type (the "e" format code).
591 (Contributed by Donghee Na and Antoine Pitrou in :gh:`90751`.)
592
593 * :class:`slice` objects are now hashable, allowing them to be used as dict keys and
594 set items. (Contributed by Will Bradshaw, Furkan Onder, and Raymond Hettinger in :gh:`101264`.)
595
596 * :func:`sum` now uses Neumaier summation to improve accuracy and commutativity
597 when summing floats or mixed ints and floats.
598 (Contributed by Raymond Hettinger in :gh:`100425`.)
599
600 * :func:`ast.parse` now raises :exc:`SyntaxError` instead of :exc:`ValueError`
601 when parsing source code containing null bytes. (Contributed by Pablo Galindo
602 in :gh:`96670`.)
603
604 * The extraction methods in :mod:`tarfile`, and :func:`shutil.unpack_archive`,
605 have a new a *filter* argument that allows limiting tar features than may be
606 surprising or dangerous, such as creating files outside the destination
607 directory.
608 See :ref:`tarfile extraction filters <tarfile-extraction-filter>` for details.
609 In Python 3.14, the default will switch to ``'data'``.
610 (Contributed by Petr Viktorin in :pep:`706`.)
611
612 * :class:`types.MappingProxyType` instances are now hashable if the underlying
613 mapping is hashable.
614 (Contributed by Serhiy Storchaka in :gh:`87995`.)
615
616 * Add :ref:`support for the perf profiler <perf_profiling>` through the new
617 environment variable :envvar:`PYTHONPERFSUPPORT`
618 and command-line option :option:`-X perf <-X>`,
619 as well as the new :func:`sys.activate_stack_trampoline`,
620 :func:`sys.deactivate_stack_trampoline`,
621 and :func:`sys.is_stack_trampoline_active` functions.
622 (Design by Pablo Galindo. Contributed by Pablo Galindo and Christian Heimes
623 with contributions from Gregory P. Smith [Google] and Mark Shannon
624 in :gh:`96123`.)
625
626
627 New Modules
628 ===========
629
630 * None.
631
632
633 Improved Modules
634 ================
635
636 array
637 -----
638
639 * The :class:`array.array` class now supports subscripting, making it a
640 :term:`generic type`. (Contributed by Jelle Zijlstra in :gh:`98658`.)
641
642 asyncio
643 -------
644
645 * The performance of writing to sockets in :mod:`asyncio` has been
646 significantly improved. ``asyncio`` now avoids unnecessary copying when
647 writing to sockets and uses :meth:`~socket.socket.sendmsg` if the platform
648 supports it. (Contributed by Kumar Aditya in :gh:`91166`.)
649
650 * Add :func:`asyncio.eager_task_factory` and :func:`asyncio.create_eager_task_factory`
651 functions to allow opting an event loop in to eager task execution,
652 making some use-cases 2x to 5x faster.
653 (Contributed by Jacob Bower & Itamar Oren in :gh:`102853`, :gh:`104140`, and :gh:`104138`)
654
655 * On Linux, :mod:`asyncio` uses :class:`!asyncio.PidfdChildWatcher` by default
656 if :func:`os.pidfd_open` is available and functional instead of
657 :class:`!asyncio.ThreadedChildWatcher`.
658 (Contributed by Kumar Aditya in :gh:`98024`.)
659
660 * The event loop now uses the best available child watcher for each platform
661 (:class:`!asyncio.PidfdChildWatcher` if supported and
662 :class:`!asyncio.ThreadedChildWatcher` otherwise), so manually
663 configuring a child watcher is not recommended.
664 (Contributed by Kumar Aditya in :gh:`94597`.)
665
666 * Add *loop_factory* parameter to :func:`asyncio.run` to allow specifying
667 a custom event loop factory.
668 (Contributed by Kumar Aditya in :gh:`99388`.)
669
670 * Add C implementation of :func:`asyncio.current_task` for 4x-6x speedup.
671 (Contributed by Itamar Oren and Pranav Thulasiram Bhat in :gh:`100344`.)
672
673 * :func:`asyncio.iscoroutine` now returns ``False`` for generators as
674 :mod:`asyncio` does not support legacy generator-based coroutines.
675 (Contributed by Kumar Aditya in :gh:`102748`.)
676
677 * :func:`asyncio.wait` and :func:`asyncio.as_completed` now accepts generators
678 yielding tasks.
679 (Contributed by Kumar Aditya in :gh:`78530`.)
680
681 calendar
682 --------
683
684 * Add enums :data:`calendar.Month` and :data:`calendar.Day`
685 defining months of the year and days of the week.
686 (Contributed by Prince Roshan in :gh:`103636`.)
687
688 csv
689 ---
690
691 * Add :const:`csv.QUOTE_NOTNULL` and :const:`csv.QUOTE_STRINGS` flags to
692 provide finer grained control of ``None`` and empty strings by
693 :class:`~csv.reader` and :class:`~csv.writer` objects.
694
695 dis
696 ---
697
698 * Pseudo instruction opcodes (which are used by the compiler but
699 do not appear in executable bytecode) are now exposed in the
700 :mod:`dis` module.
701 :opcode:`HAVE_ARGUMENT` is still relevant to real opcodes,
702 but it is not useful for pseudo instructions. Use the new
703 :data:`dis.hasarg` collection instead.
704 (Contributed by Irit Katriel in :gh:`94216`.)
705
706 * Add the :data:`dis.hasexc` collection to signify instructions that set
707 an exception handler. (Contributed by Irit Katriel in :gh:`94216`.)
708
709 fractions
710 ---------
711
712 * Objects of type :class:`fractions.Fraction` now support float-style
713 formatting. (Contributed by Mark Dickinson in :gh:`100161`.)
714
715 importlib.resources
716 -------------------
717
718 * :func:`importlib.resources.as_file` now supports resource directories.
719 (Contributed by Jason R. Coombs in :gh:`97930`.)
720
721 * Rename first parameter of :func:`importlib.resources.files` to *anchor*.
722 (Contributed by Jason R. Coombs in :gh:`100598`.)
723
724 inspect
725 -------
726
727 * Add :func:`inspect.markcoroutinefunction` to mark sync functions that return
728 a :term:`coroutine` for use with :func:`inspect.iscoroutinefunction`.
729 (Contributed by Carlton Gibson in :gh:`99247`.)
730
731 * Add :func:`inspect.getasyncgenstate` and :func:`inspect.getasyncgenlocals`
732 for determining the current state of asynchronous generators.
733 (Contributed by Thomas Krennwallner in :gh:`79940`.)
734
735 * The performance of :func:`inspect.getattr_static` has been considerably
736 improved. Most calls to the function should be at least 2x faster than they
737 were in Python 3.11. (Contributed by Alex Waygood in :gh:`103193`.)
738
739 itertools
740 ---------
741
742 * Add :func:`itertools.batched` for collecting into even-sized
743 tuples where the last batch may be shorter than the rest.
744 (Contributed by Raymond Hettinger in :gh:`98363`.)
745
746 math
747 ----
748
749 * Add :func:`math.sumprod` for computing a sum of products.
750 (Contributed by Raymond Hettinger in :gh:`100485`.)
751
752 * Extend :func:`math.nextafter` to include a *steps* argument
753 for moving up or down multiple steps at a time. (Contributed by
754 Matthias Goergens, Mark Dickinson, and Raymond Hettinger in :gh:`94906`.)
755
756 os
757 --
758
759 * Add :const:`os.PIDFD_NONBLOCK` to open a file descriptor
760 for a process with :func:`os.pidfd_open` in non-blocking mode.
761 (Contributed by Kumar Aditya in :gh:`93312`.)
762
763 * :class:`os.DirEntry` now includes an :meth:`os.DirEntry.is_junction`
764 method to check if the entry is a junction.
765 (Contributed by Charles Machalow in :gh:`99547`.)
766
767 * Add :func:`os.listdrives`, :func:`os.listvolumes` and :func:`os.listmounts`
768 functions on Windows for enumerating drives, volumes and mount points.
769 (Contributed by Steve Dower in :gh:`102519`.)
770
771 * :func:`os.stat` and :func:`os.lstat` are now more accurate on Windows.
772 The ``st_birthtime`` field will now be filled with the creation time
773 of the file, and ``st_ctime`` is deprecated but still contains the
774 creation time (but in the future will return the last metadata change,
775 for consistency with other platforms). ``st_dev`` may be up to 64 bits
776 and ``st_ino`` up to 128 bits depending on your file system, and
777 ``st_rdev`` is always set to zero rather than incorrect values.
778 Both functions may be significantly faster on newer releases of
779 Windows. (Contributed by Steve Dower in :gh:`99726`.)
780
781 os.path
782 -------
783
784 * Add :func:`os.path.isjunction` to check if a given path is a junction.
785 (Contributed by Charles Machalow in :gh:`99547`.)
786
787 * Add :func:`os.path.splitroot` to split a path into a triad
788 ``(drive, root, tail)``. (Contributed by Barney Gale in :gh:`101000`.)
789
790 pathlib
791 -------
792
793 * Add support for subclassing :class:`pathlib.PurePath` and
794 :class:`pathlib.Path`, plus their Posix- and Windows-specific variants.
795 Subclasses may override the :meth:`pathlib.PurePath.with_segments` method
796 to pass information between path instances.
797
798 * Add :meth:`pathlib.Path.walk` for walking the directory trees and generating
799 all file or directory names within them, similar to :func:`os.walk`.
800 (Contributed by Stanislav Zmiev in :gh:`90385`.)
801
802 * Add *walk_up* optional parameter to :meth:`pathlib.PurePath.relative_to`
803 to allow the insertion of ``..`` entries in the result; this behavior is
804 more consistent with :func:`os.path.relpath`.
805 (Contributed by Domenico Ragusa in :gh:`84538`.)
806
807 * Add :meth:`pathlib.Path.is_junction` as a proxy to :func:`os.path.isjunction`.
808 (Contributed by Charles Machalow in :gh:`99547`.)
809
810 * Add *case_sensitive* optional parameter to :meth:`pathlib.Path.glob`,
811 :meth:`pathlib.Path.rglob` and :meth:`pathlib.PurePath.match` for matching
812 the path's case sensitivity, allowing for more precise control over the matching process.
813
814 platform
815 --------
816
817 * Add support for detecting Windows 11 and Windows Server releases past 2012.
818 Previously, lookups on Windows Server platforms newer than Windows Server 2012
819 and on Windows 11 would return ``Windows-10``.
820 (Contributed by Steve Dower in :gh:`89545`.)
821
822 pdb
823 ---
824
825 * Add convenience variables to hold values temporarily for debug session
826 and provide quick access to values like the current frame or the return
827 value.
828 (Contributed by Tian Gao in :gh:`103693`.)
829
830 random
831 ------
832
833 * Add :func:`random.binomialvariate`.
834 (Contributed by Raymond Hettinger in :gh:`81620`.)
835
836 * Add a default of ``lambd=1.0`` to :func:`random.expovariate`.
837 (Contributed by Raymond Hettinger in :gh:`100234`.)
838
839 shutil
840 ------
841
842 * :func:`shutil.make_archive` now passes the *root_dir* argument to custom
843 archivers which support it.
844 In this case it no longer temporarily changes the current working directory
845 of the process to *root_dir* to perform archiving.
846 (Contributed by Serhiy Storchaka in :gh:`74696`.)
847
848 * :func:`shutil.rmtree` now accepts a new argument *onexc* which is an
849 error handler like *onerror* but which expects an exception instance
850 rather than a *(typ, val, tb)* triplet. *onerror* is deprecated.
851 (Contributed by Irit Katriel in :gh:`102828`.)
852
853 * :func:`shutil.which` now consults the *PATHEXT* environment variable to
854 find matches within *PATH* on Windows even when the given *cmd* includes
855 a directory component.
856 (Contributed by Charles Machalow in :gh:`103179`.)
857
858 :func:`shutil.which` will call ``NeedCurrentDirectoryForExePathW`` when
859 querying for executables on Windows to determine if the current working
860 directory should be prepended to the search path.
861 (Contributed by Charles Machalow in :gh:`103179`.)
862
863 :func:`shutil.which` will return a path matching the *cmd* with a component
864 from ``PATHEXT`` prior to a direct match elsewhere in the search path on
865 Windows.
866 (Contributed by Charles Machalow in :gh:`103179`.)
867
868 sqlite3
869 -------
870
871 * Add a :ref:`command-line interface <sqlite3-cli>`.
872 (Contributed by Erlend E. Aasland in :gh:`77617`.)
873
874 * Add the :attr:`sqlite3.Connection.autocommit` attribute
875 to :class:`sqlite3.Connection`
876 and the *autocommit* parameter to :func:`sqlite3.connect`
877 to control :pep:`249`-compliant
878 :ref:`transaction handling <sqlite3-transaction-control-autocommit>`.
879 (Contributed by Erlend E. Aasland in :gh:`83638`.)
880
881 * Add *entrypoint* keyword-only parameter to
882 :meth:`sqlite3.Connection.load_extension`,
883 for overriding the SQLite extension entry point.
884 (Contributed by Erlend E. Aasland in :gh:`103015`.)
885
886 * Add :meth:`sqlite3.Connection.getconfig` and
887 :meth:`sqlite3.Connection.setconfig` to :class:`sqlite3.Connection`
888 to make configuration changes to a database connection.
889 (Contributed by Erlend E. Aasland in :gh:`103489`.)
890
891 statistics
892 ----------
893
894 * Extend :func:`statistics.correlation` to include as a ``ranked`` method
895 for computing the Spearman correlation of ranked data.
896 (Contributed by Raymond Hettinger in :gh:`95861`.)
897
898 sys
899 ---
900
901 * Add the :mod:`sys.monitoring` namespace to expose the new :ref:`PEP 669
902 <whatsnew312-pep669>` monitoring API.
903 (Contributed by Mark Shannon in :gh:`103082`.)
904
905 * Add :func:`sys.activate_stack_trampoline` and
906 :func:`sys.deactivate_stack_trampoline` for activating and deactivating
907 stack profiler trampolines,
908 and :func:`sys.is_stack_trampoline_active` for querying if stack profiler
909 trampolines are active.
910 (Contributed by Pablo Galindo and Christian Heimes
911 with contributions from Gregory P. Smith [Google] and Mark Shannon
912 in :gh:`96123`.)
913
914 * Add :data:`sys.last_exc` which holds the last unhandled exception that
915 was raised (for post-mortem debugging use cases). Deprecate the
916 three fields that have the same information in its legacy form:
917 :data:`sys.last_type`, :data:`sys.last_value` and :data:`sys.last_traceback`.
918 (Contributed by Irit Katriel in :gh:`102778`.)
919
920 * :func:`sys._current_exceptions` now returns a mapping from thread-id to an
921 exception instance, rather than to a ``(typ, exc, tb)`` tuple.
922 (Contributed by Irit Katriel in :gh:`103176`.)
923
924 * :func:`sys.setrecursionlimit` and :func:`sys.getrecursionlimit`.
925 The recursion limit now applies only to Python code. Builtin functions do
926 not use the recursion limit, but are protected by a different mechanism
927 that prevents recursion from causing a virtual machine crash.
928
929 tempfile
930 --------
931
932 * The :class:`tempfile.NamedTemporaryFile` function has a new optional parameter
933 *delete_on_close* (Contributed by Evgeny Zorin in :gh:`58451`.)
934 * :func:`tempfile.mkdtemp` now always returns an absolute path, even if the
935 argument provided to the *dir* parameter is a relative path.
936
937 threading
938 ---------
939
940 * Add :func:`threading.settrace_all_threads` and
941 :func:`threading.setprofile_all_threads` that allow to set tracing and
942 profiling functions in all running threads in addition to the calling one.
943 (Contributed by Pablo Galindo in :gh:`93503`.)
944
945 tkinter
946 -------
947
948 * ``tkinter.Canvas.coords()`` now flattens its arguments.
949 It now accepts not only coordinates as separate arguments
950 (``x1, y1, x2, y2, ...``) and a sequence of coordinates
951 (``[x1, y1, x2, y2, ...]``), but also coordinates grouped in pairs
952 (``(x1, y1), (x2, y2), ...`` and ``[(x1, y1), (x2, y2), ...]``),
953 like ``create_*()`` methods.
954 (Contributed by Serhiy Storchaka in :gh:`94473`.)
955
956 tokenize
957 --------
958
959 * The :mod:`tokenize` module includes the changes introduced in :pep:`701`.
960 (Contributed by Marta Gómez Macías and Pablo Galindo in :gh:`102856`.)
961 See :ref:`whatsnew312-porting-to-python312` for more information on the
962 changes to the :mod:`tokenize` module.
963
964 types
965 -----
966
967 * Add :func:`types.get_original_bases` to allow for further introspection of
968 :ref:`user-defined-generics` when subclassed. (Contributed by
969 James Hilton-Balfe and Alex Waygood in :gh:`101827`.)
970
971 .. _whatsnew-typing-py312:
972
973 typing
974 ------
975
976 * :func:`isinstance` checks against
977 :func:`runtime-checkable protocols <typing.runtime_checkable>` now use
978 :func:`inspect.getattr_static` rather than :func:`hasattr` to lookup whether
979 attributes exist. This means that descriptors and :meth:`~object.__getattr__`
980 methods are no longer unexpectedly evaluated during ``isinstance()`` checks
981 against runtime-checkable protocols. However, it may also mean that some
982 objects which used to be considered instances of a runtime-checkable protocol
983 may no longer be considered instances of that protocol on Python 3.12+, and
984 vice versa. Most users are unlikely to be affected by this change.
985 (Contributed by Alex Waygood in :gh:`102433`.)
986
987 * The members of a runtime-checkable protocol are now considered "frozen" at
988 runtime as soon as the class has been created. Monkey-patching attributes
989 onto a runtime-checkable protocol will still work, but will have no impact on
990 :func:`isinstance` checks comparing objects to the protocol. For example::
991
992 >>> from typing import Protocol, runtime_checkable
993 >>> @runtime_checkable
994 ... class HasX(Protocol):
995 ... x = 1
996 ...
997 >>> class Foo: ...
998 ...
999 >>> f = Foo()
1000 >>> isinstance(f, HasX)
1001 False
1002 >>> f.x = 1
1003 >>> isinstance(f, HasX)
1004 True
1005 >>> HasX.y = 2
1006 >>> isinstance(f, HasX) # unchanged, even though HasX now also has a "y" attribute
1007 True
1008
1009 This change was made in order to speed up ``isinstance()`` checks against
1010 runtime-checkable protocols.
1011
1012 * The performance profile of :func:`isinstance` checks against
1013 :func:`runtime-checkable protocols <typing.runtime_checkable>` has changed
1014 significantly. Most ``isinstance()`` checks against protocols with only a few
1015 members should be at least 2x faster than in 3.11, and some may be 20x
1016 faster or more. However, ``isinstance()`` checks against protocols with many
1017 members may be slower than in Python 3.11. (Contributed by Alex
1018 Waygood in :gh:`74690` and :gh:`103193`.)
1019
1020 * All :data:`typing.TypedDict` and :data:`typing.NamedTuple` classes now have the
1021 ``__orig_bases__`` attribute. (Contributed by Adrian Garcia Badaracco in
1022 :gh:`103699`.)
1023
1024 * Add ``frozen_default`` parameter to :func:`typing.dataclass_transform`.
1025 (Contributed by Erik De Bonte in :gh:`99957`.)
1026
1027 unicodedata
1028 -----------
1029
1030 * The Unicode database has been updated to version 15.0.0. (Contributed by
1031 Benjamin Peterson in :gh:`96734`).
1032
1033 unittest
1034 --------
1035
1036 Add a ``--durations`` command line option, showing the N slowest test cases::
1037
1038 python3 -m unittest --durations=3 lib.tests.test_threading
1039 .....
1040 Slowest test durations
1041 ----------------------------------------------------------------------
1042 1.210s test_timeout (Lib.test.test_threading.BarrierTests)
1043 1.003s test_default_timeout (Lib.test.test_threading.BarrierTests)
1044 0.518s test_timeout (Lib.test.test_threading.EventTests)
1045
1046 (0.000 durations hidden. Use -v to show these durations.)
1047 ----------------------------------------------------------------------
1048 Ran 158 tests in 9.869s
1049
1050 OK (skipped=3)
1051
1052 (Contributed by Giampaolo Rodola in :gh:`48330`)
1053
1054 uuid
1055 ----
1056
1057 * Add a :ref:`command-line interface <uuid-cli>`.
1058 (Contributed by Adam Chhina in :gh:`88597`.)
1059
1060
1061 Optimizations
1062 =============
1063
1064 * Remove ``wstr`` and ``wstr_length`` members from Unicode objects.
1065 It reduces object size by 8 or 16 bytes on 64bit platform. (:pep:`623`)
1066 (Contributed by Inada Naoki in :gh:`92536`.)
1067
1068 * Add experimental support for using the BOLT binary optimizer in the build
1069 process, which improves performance by 1-5%.
1070 (Contributed by Kevin Modzelewski in :gh:`90536` and tuned by Donghee Na in :gh:`101525`)
1071
1072 * Speed up the regular expression substitution (functions :func:`re.sub` and
1073 :func:`re.subn` and corresponding :class:`!re.Pattern` methods) for
1074 replacement strings containing group references by 2--3 times.
1075 (Contributed by Serhiy Storchaka in :gh:`91524`.)
1076
1077 * Speed up :class:`asyncio.Task` creation by deferring expensive string formatting.
1078 (Contributed by Itamar Oren in :gh:`103793`.)
1079
1080 * The :func:`tokenize.tokenize` and :func:`tokenize.generate_tokens` functions are
1081 up to 64% faster as a side effect of the changes required to cover :pep:`701` in
1082 the :mod:`tokenize` module. (Contributed by Marta Gómez Macías and Pablo Galindo
1083 in :gh:`102856`.)
1084
1085 * Speed up :func:`super` method calls and attribute loads via the
1086 new :opcode:`LOAD_SUPER_ATTR` instruction. (Contributed by Carl Meyer and
1087 Vladimir Matveev in :gh:`103497`.)
1088
1089
1090 CPython bytecode changes
1091 ========================
1092
1093 * Remove the :opcode:`!LOAD_METHOD` instruction. It has been merged into
1094 :opcode:`LOAD_ATTR`. :opcode:`LOAD_ATTR` will now behave like the old
1095 :opcode:`!LOAD_METHOD` instruction if the low bit of its oparg is set.
1096 (Contributed by Ken Jin in :gh:`93429`.)
1097
1098 * Remove the :opcode:`!JUMP_IF_FALSE_OR_POP` and :opcode:`!JUMP_IF_TRUE_OR_POP`
1099 instructions. (Contributed by Irit Katriel in :gh:`102859`.)
1100
1101 * Remove the :opcode:`!PRECALL` instruction. (Contributed by Mark Shannon in
1102 :gh:`92925`.)
1103
1104 * Add the :opcode:`BINARY_SLICE` and :opcode:`STORE_SLICE` instructions.
1105 (Contributed by Mark Shannon in :gh:`94163`.)
1106
1107 * Add the :opcode:`CALL_INTRINSIC_1` instructions.
1108 (Contributed by Mark Shannon in :gh:`99005`.)
1109
1110 * Add the :opcode:`CALL_INTRINSIC_2` instruction.
1111 (Contributed by Irit Katriel in :gh:`101799`.)
1112
1113 * Add the :opcode:`CLEANUP_THROW` instruction.
1114 (Contributed by Brandt Bucher in :gh:`90997`.)
1115
1116 * Add the :opcode:`!END_SEND` instruction.
1117 (Contributed by Mark Shannon in :gh:`103082`.)
1118
1119 * Add the :opcode:`LOAD_FAST_AND_CLEAR` instruction as part of the
1120 implementation of :pep:`709`. (Contributed by Carl Meyer in :gh:`101441`.)
1121
1122 * Add the :opcode:`LOAD_FAST_CHECK` instruction.
1123 (Contributed by Dennis Sweeney in :gh:`93143`.)
1124
1125 * Add the :opcode:`LOAD_FROM_DICT_OR_DEREF`, :opcode:`LOAD_FROM_DICT_OR_GLOBALS`,
1126 and :opcode:`LOAD_LOCALS` opcodes as part of the implementation of :pep:`695`.
1127 Remove the :opcode:`!LOAD_CLASSDEREF` opcode, which can be replaced with
1128 :opcode:`LOAD_LOCALS` plus :opcode:`LOAD_FROM_DICT_OR_DEREF`. (Contributed
1129 by Jelle Zijlstra in :gh:`103764`.)
1130
1131 * Add the :opcode:`LOAD_SUPER_ATTR` instruction. (Contributed by Carl Meyer and
1132 Vladimir Matveev in :gh:`103497`.)
1133
1134 * Add the ``RETURN_CONST`` instruction. (Contributed by Wenyang Wang in :gh:`101632`.)
1135
1136 Demos and Tools
1137 ===============
1138
1139 * Remove the ``Tools/demo/`` directory which contained old demo scripts. A copy
1140 can be found in the `old-demos project
1141 <https://github.com/gvanrossum/old-demos>`_.
1142 (Contributed by Victor Stinner in :gh:`97681`.)
1143
1144 * Remove outdated example scripts of the ``Tools/scripts/`` directory.
1145 A copy can be found in the `old-demos project
1146 <https://github.com/gvanrossum/old-demos>`_.
1147 (Contributed by Victor Stinner in :gh:`97669`.)
1148
1149
1150 Deprecated
1151 ==========
1152
1153 * :mod:`argparse`: The *type*, *choices*, and *metavar* parameters
1154 of :class:`!argparse.BooleanOptionalAction` are deprecated
1155 and will be removed in 3.14.
1156 (Contributed by Nikita Sobolev in :gh:`92248`.)
1157
1158 * :mod:`ast`: The following :mod:`ast` features have been deprecated in documentation since
1159 Python 3.8, now cause a :exc:`DeprecationWarning` to be emitted at runtime
1160 when they are accessed or used, and will be removed in Python 3.14:
1161
1162 * :class:`!ast.Num`
1163 * :class:`!ast.Str`
1164 * :class:`!ast.Bytes`
1165 * :class:`!ast.NameConstant`
1166 * :class:`!ast.Ellipsis`
1167
1168 Use :class:`ast.Constant` instead.
1169 (Contributed by Serhiy Storchaka in :gh:`90953`.)
1170
1171 * :mod:`asyncio`:
1172
1173 * The child watcher classes :class:`!asyncio.MultiLoopChildWatcher`,
1174 :class:`!asyncio.FastChildWatcher`, :class:`!asyncio.AbstractChildWatcher`
1175 and :class:`!asyncio.SafeChildWatcher` are deprecated and
1176 will be removed in Python 3.14.
1177 (Contributed by Kumar Aditya in :gh:`94597`.)
1178
1179 * :func:`!asyncio.set_child_watcher`, :func:`!asyncio.get_child_watcher`,
1180 :meth:`!asyncio.AbstractEventLoopPolicy.set_child_watcher` and
1181 :meth:`!asyncio.AbstractEventLoopPolicy.get_child_watcher` are deprecated
1182 and will be removed in Python 3.14.
1183 (Contributed by Kumar Aditya in :gh:`94597`.)
1184
1185 * The :meth:`~asyncio.get_event_loop` method of the
1186 default event loop policy now emits a :exc:`DeprecationWarning` if there
1187 is no current event loop set and it decides to create one.
1188 (Contributed by Serhiy Storchaka and Guido van Rossum in :gh:`100160`.)
1189
1190 * :mod:`calendar`: ``calendar.January`` and ``calendar.February`` constants are deprecated and
1191 replaced by :data:`calendar.JANUARY` and :data:`calendar.FEBRUARY`.
1192 (Contributed by Prince Roshan in :gh:`103636`.)
1193
1194 * :mod:`collections.abc`: Deprecated :class:`collections.abc.ByteString`.
1195
1196 Use ``isinstance(obj, collections.abc.Buffer)`` to test if ``obj`` implements
1197 the :ref:`buffer protocol <bufferobjects>` at runtime. For use in type
1198 annotations, either use :class:`~collections.abc.Buffer` or a union
1199 that explicitly specifies the types your code supports (e.g.,
1200 ``bytes | bytearray | memoryview``).
1201
1202 :class:`!ByteString` was originally intended to be an abstract class that
1203 would serve as a supertype of both :class:`bytes` and :class:`bytearray`.
1204 However, since the ABC never had any methods, knowing that an object was an
1205 instance of :class:`!ByteString` never actually told you anything useful
1206 about the object. Other common buffer types such as :class:`memoryview` were
1207 also never understood as subtypes of :class:`!ByteString` (either at
1208 runtime or by static type checkers).
1209
1210 See :pep:`PEP 688 <688#current-options>` for more details.
1211 (Contributed by Shantanu Jain in :gh:`91896`.)
1212
1213 * :mod:`datetime`: :class:`datetime.datetime`'s :meth:`~datetime.datetime.utcnow` and
1214 :meth:`~datetime.datetime.utcfromtimestamp` are deprecated and will be
1215 removed in a future version. Instead, use timezone-aware objects to represent
1216 datetimes in UTC: respectively, call :meth:`~datetime.datetime.now` and
1217 :meth:`~datetime.datetime.fromtimestamp` with the *tz* parameter set to
1218 :const:`datetime.UTC`.
1219 (Contributed by Paul Ganssle in :gh:`103857`.)
1220
1221 * :mod:`email`: Deprecate the *isdst* parameter in :func:`email.utils.localtime`.
1222 (Contributed by Alan Williams in :gh:`72346`.)
1223
1224 * :mod:`importlib.abc`: Deprecated the following classes, scheduled for removal in
1225 Python 3.14:
1226
1227 * :class:`!importlib.abc.ResourceReader`
1228 * :class:`!importlib.abc.Traversable`
1229 * :class:`!importlib.abc.TraversableResources`
1230
1231 Use :mod:`importlib.resources.abc` classes instead:
1232
1233 * :class:`importlib.resources.abc.Traversable`
1234 * :class:`importlib.resources.abc.TraversableResources`
1235
1236 (Contributed by Jason R. Coombs and Hugo van Kemenade in :gh:`93963`.)
1237
1238 * :mod:`itertools`: Deprecate the support for copy, deepcopy, and pickle operations,
1239 which is undocumented, inefficient, historically buggy, and inconsistent.
1240 This will be removed in 3.14 for a significant reduction in code
1241 volume and maintenance burden.
1242 (Contributed by Raymond Hettinger in :gh:`101588`.)
1243
1244 * :mod:`multiprocessing`: In Python 3.14, the default :mod:`multiprocessing`
1245 start method will change to a safer one on Linux, BSDs,
1246 and other non-macOS POSIX platforms where ``'fork'`` is currently
1247 the default (:gh:`84559`). Adding a runtime warning about this was deemed too
1248 disruptive as the majority of code is not expected to care. Use the
1249 :func:`~multiprocessing.get_context` or
1250 :func:`~multiprocessing.set_start_method` APIs to explicitly specify when
1251 your code *requires* ``'fork'``. See :ref:`contexts and start methods
1252 <multiprocessing-start-methods>`.
1253
1254 * :mod:`pkgutil`: :func:`!pkgutil.find_loader` and :func:`!pkgutil.get_loader`
1255 are deprecated and will be removed in Python 3.14;
1256 use :func:`importlib.util.find_spec` instead.
1257 (Contributed by Nikita Sobolev in :gh:`97850`.)
1258
1259 * :mod:`pty`: The module has two undocumented ``master_open()`` and ``slave_open()``
1260 functions that have been deprecated since Python 2 but only gained a
1261 proper :exc:`DeprecationWarning` in 3.12. Remove them in 3.14.
1262 (Contributed by Soumendra Ganguly and Gregory P. Smith in :gh:`85984`.)
1263
1264 * :mod:`os`:
1265
1266 * The ``st_ctime`` fields return by :func:`os.stat` and :func:`os.lstat` on
1267 Windows are deprecated. In a future release, they will contain the last
1268 metadata change time, consistent with other platforms. For now, they still
1269 contain the creation time, which is also available in the new ``st_birthtime``
1270 field. (Contributed by Steve Dower in :gh:`99726`.)
1271
1272 * On POSIX platforms, :func:`os.fork` can now raise a
1273 :exc:`DeprecationWarning` when it can detect being called from a
1274 multithreaded process. There has always been a fundamental incompatibility
1275 with the POSIX platform when doing so. Even if such code *appeared* to work.
1276 We added the warning to raise awareness as issues encountered by code doing
1277 this are becoming more frequent. See the :func:`os.fork` documentation for
1278 more details along with `this discussion on fork being incompatible with threads
1279 <https://discuss.python.org/t/concerns-regarding-deprecation-of-fork-with-alive-threads/33555>`_ for *why* we're now surfacing this
1280 longstanding platform compatibility problem to developers.
1281
1282 When this warning appears due to usage of :mod:`multiprocessing` or
1283 :mod:`concurrent.futures` the fix is to use a different
1284 :mod:`multiprocessing` start method such as ``"spawn"`` or ``"forkserver"``.
1285
1286 * :mod:`shutil`: The *onerror* argument of :func:`shutil.rmtree` is deprecated;
1287 use *onexc* instead. (Contributed by Irit Katriel in :gh:`102828`.)
1288
1289 * :mod:`sqlite3`:
1290
1291 * :ref:`default adapters and converters
1292 <sqlite3-default-converters>` are now deprecated.
1293 Instead, use the :ref:`sqlite3-adapter-converter-recipes`
1294 and tailor them to your needs.
1295 (Contributed by Erlend E. Aasland in :gh:`90016`.)
1296
1297 * In :meth:`~sqlite3.Cursor.execute`, :exc:`DeprecationWarning` is now emitted
1298 when :ref:`named placeholders <sqlite3-placeholders>` are used together with
1299 parameters supplied as a :term:`sequence` instead of as a :class:`dict`.
1300 Starting from Python 3.14, using named placeholders with parameters supplied
1301 as a sequence will raise a :exc:`~sqlite3.ProgrammingError`.
1302 (Contributed by Erlend E. Aasland in :gh:`101698`.)
1303
1304 * :mod:`sys`: The :data:`sys.last_type`, :data:`sys.last_value` and :data:`sys.last_traceback`
1305 fields are deprecated. Use :data:`sys.last_exc` instead.
1306 (Contributed by Irit Katriel in :gh:`102778`.)
1307
1308 * :mod:`tarfile`: Extracting tar archives without specifying *filter* is deprecated until
1309 Python 3.14, when ``'data'`` filter will become the default.
1310 See :ref:`tarfile-extraction-filter` for details.
1311
1312 * :mod:`typing`:
1313
1314 * :class:`typing.Hashable` and :class:`typing.Sized`, aliases for
1315 :class:`collections.abc.Hashable` and :class:`collections.abc.Sized` respectively, are
1316 deprecated. (:gh:`94309`.)
1317
1318 * :class:`typing.ByteString`, deprecated since Python 3.9, now causes a
1319 :exc:`DeprecationWarning` to be emitted when it is used.
1320 (Contributed by Alex Waygood in :gh:`91896`.)
1321
1322 * :mod:`xml.etree.ElementTree`: The module now emits :exc:`DeprecationWarning`
1323 when testing the truth value of an :class:`xml.etree.ElementTree.Element`.
1324 Before, the Python implementation emitted :exc:`FutureWarning`, and the C
1325 implementation emitted nothing.
1326 (Contributed by Jacob Walls in :gh:`83122`.)
1327
1328 * The 3-arg signatures (type, value, traceback) of :meth:`coroutine throw()
1329 <coroutine.throw>`, :meth:`generator throw() <generator.throw>` and
1330 :meth:`async generator throw() <agen.athrow>` are deprecated and
1331 may be removed in a future version of Python. Use the single-arg versions
1332 of these functions instead. (Contributed by Ofey Chan in :gh:`89874`.)
1333
1334 * :exc:`DeprecationWarning` is now raised when :attr:`~module.__package__` on a
1335 module differs from
1336 :attr:`__spec__.parent <importlib.machinery.ModuleSpec.parent>` (previously
1337 it was :exc:`ImportWarning`).
1338 (Contributed by Brett Cannon in :gh:`65961`.)
1339
1340 * Setting :attr:`~module.__package__` or :attr:`~module.__cached__` on a
1341 module is deprecated, and will cease to be set or taken into consideration by
1342 the import system in Python 3.14. (Contributed by Brett Cannon in :gh:`65961`.)
1343
1344 * The bitwise inversion operator (``~``) on bool is deprecated. It will throw an
1345 error in Python 3.16. Use ``not`` for logical negation of bools instead.
1346 In the rare case that you really need the bitwise inversion of the underlying
1347 ``int``, convert to int explicitly: ``~int(x)``. (Contributed by Tim Hoffmann
1348 in :gh:`103487`.)
1349
1350 * Accessing :attr:`~codeobject.co_lnotab` on code objects was deprecated in
1351 Python 3.10 via :pep:`626`,
1352 but it only got a proper :exc:`DeprecationWarning` in 3.12.
1353 May be removed in 3.15.
1354 (Contributed by Nikita Sobolev in :gh:`101866`.)
1355
1356 .. include:: ../deprecations/pending-removal-in-3.13.rst
1357
1358 .. include:: ../deprecations/pending-removal-in-3.14.rst
1359
1360 .. include:: ../deprecations/pending-removal-in-3.15.rst
1361
1362 .. include:: ../deprecations/pending-removal-in-3.16.rst
1363
1364 .. include:: ../deprecations/pending-removal-in-3.17.rst
1365
1366 .. include:: ../deprecations/pending-removal-in-future.rst
1367
1368 .. _whatsnew312-removed:
1369
1370 Removed
1371 =======
1372
1373 asynchat and asyncore
1374 ---------------------
1375
1376 * These two modules have been removed
1377 according to the schedule in :pep:`594`,
1378 having been deprecated in Python 3.6.
1379 Use :mod:`asyncio` instead.
1380 (Contributed by Nikita Sobolev in :gh:`96580`.)
1381
1382 configparser
1383 ------------
1384
1385 * Several names deprecated in the :mod:`configparser` way back in 3.2 have
1386 been removed per :gh:`89336`:
1387
1388 * :class:`configparser.ParsingError` no longer has a ``filename`` attribute
1389 or argument. Use the ``source`` attribute and argument instead.
1390 * :mod:`configparser` no longer has a ``SafeConfigParser`` class. Use the
1391 shorter :class:`~configparser.ConfigParser` name instead.
1392 * :class:`configparser.ConfigParser` no longer has a ``readfp`` method.
1393 Use :meth:`~configparser.ConfigParser.read_file` instead.
1394
1395 .. _whatsnew312-removed-distutils:
1396
1397 distutils
1398 ---------
1399
1400 * Remove the :py:mod:`!distutils` package. It was deprecated in Python 3.10 by
1401 :pep:`632` "Deprecate distutils module". For projects still using
1402 ``distutils`` and cannot be updated to something else, the ``setuptools``
1403 project can be installed: it still provides ``distutils``.
1404 (Contributed by Victor Stinner in :gh:`92584`.)
1405
1406 ensurepip
1407 ---------
1408
1409 * Remove the bundled setuptools wheel from :mod:`ensurepip`,
1410 and stop installing setuptools in environments created by :mod:`venv`.
1411
1412 ``pip (>= 22.1)`` does not require setuptools to be installed in the
1413 environment. ``setuptools``-based (and ``distutils``-based) packages
1414 can still be used with ``pip install``, since pip will provide
1415 ``setuptools`` in the build environment it uses for building a
1416 package.
1417
1418 ``easy_install``, ``pkg_resources``, ``setuptools`` and ``distutils``
1419 are no longer provided by default in environments created with
1420 ``venv`` or bootstrapped with ``ensurepip``, since they are part of
1421 the ``setuptools`` package. For projects relying on these at runtime,
1422 the ``setuptools`` project should be declared as a dependency and
1423 installed separately (typically, using pip).
1424
1425 (Contributed by Pradyun Gedam in :gh:`95299`.)
1426
1427 enum
1428 ----
1429
1430 * Remove :mod:`enum`'s ``EnumMeta.__getattr__``, which is no longer needed for
1431 enum attribute access.
1432 (Contributed by Ethan Furman in :gh:`95083`.)
1433
1434 ftplib
1435 ------
1436
1437 * Remove :mod:`ftplib`'s ``FTP_TLS.ssl_version`` class attribute: use the
1438 *context* parameter instead.
1439 (Contributed by Victor Stinner in :gh:`94172`.)
1440
1441 gzip
1442 ----
1443
1444 * Remove the ``filename`` attribute of :mod:`gzip`'s :class:`gzip.GzipFile`,
1445 deprecated since Python 2.6, use the :attr:`~gzip.GzipFile.name` attribute
1446 instead. In write mode, the ``filename`` attribute added ``'.gz'`` file
1447 extension if it was not present.
1448 (Contributed by Victor Stinner in :gh:`94196`.)
1449
1450 hashlib
1451 -------
1452
1453 * Remove the pure Python implementation of :mod:`hashlib`'s
1454 :func:`hashlib.pbkdf2_hmac`, deprecated in Python 3.10. Python 3.10 and
1455 newer requires OpenSSL 1.1.1 (:pep:`644`): this OpenSSL version provides
1456 a C implementation of :func:`~hashlib.pbkdf2_hmac` which is faster.
1457 (Contributed by Victor Stinner in :gh:`94199`.)
1458
1459 importlib
1460 ---------
1461
1462 * Many previously deprecated cleanups in :mod:`importlib` have now been
1463 completed:
1464
1465 * References to, and support for :meth:`!module_repr` has been removed.
1466 (Contributed by Barry Warsaw in :gh:`97850`.)
1467
1468 * ``importlib.util.set_package``, ``importlib.util.set_loader`` and
1469 ``importlib.util.module_for_loader`` have all been removed. (Contributed by
1470 Brett Cannon and Nikita Sobolev in :gh:`65961` and :gh:`97850`.)
1471
1472 * Support for ``find_loader()`` and ``find_module()`` APIs have been
1473 removed. (Contributed by Barry Warsaw in :gh:`98040`.)
1474
1475 * ``importlib.abc.Finder``, ``pkgutil.ImpImporter``, and ``pkgutil.ImpLoader``
1476 have been removed. (Contributed by Barry Warsaw in :gh:`98040`.)
1477
1478 .. _whatsnew312-removed-imp:
1479
1480 imp
1481 ---
1482
1483 * The :mod:`!imp` module has been removed. (Contributed by Barry Warsaw in
1484 :gh:`98040`.)
1485
1486 To migrate, consult the following correspondence table:
1487
1488 ================================= =======================================
1489 imp importlib
1490 ================================= =======================================
1491 ``imp.NullImporter`` Insert ``None`` into ``sys.path_importer_cache``
1492 ``imp.cache_from_source()`` :func:`importlib.util.cache_from_source`
1493 ``imp.find_module()`` :func:`importlib.util.find_spec`
1494 ``imp.get_magic()`` :const:`importlib.util.MAGIC_NUMBER`
1495 ``imp.get_suffixes()`` :const:`importlib.machinery.SOURCE_SUFFIXES`, :const:`importlib.machinery.EXTENSION_SUFFIXES`, and :const:`importlib.machinery.BYTECODE_SUFFIXES`
1496 ``imp.get_tag()`` :attr:`sys.implementation.cache_tag <sys.implementation>`
1497 ``imp.load_module()`` :func:`importlib.import_module`
1498 ``imp.new_module(name)`` ``types.ModuleType(name)``
1499 ``imp.reload()`` :func:`importlib.reload`
1500 ``imp.source_from_cache()`` :func:`importlib.util.source_from_cache`
1501 ``imp.load_source()`` *See below*
1502 ================================= =======================================
1503
1504 Replace ``imp.load_source()`` with::
1505
1506 import importlib.util
1507 import importlib.machinery
1508
1509 def load_source(modname, filename):
1510 loader = importlib.machinery.SourceFileLoader(modname, filename)
1511 spec = importlib.util.spec_from_file_location(modname, filename, loader=loader)
1512 module = importlib.util.module_from_spec(spec)
1513 # The module is always executed and not cached in sys.modules.
1514 # Uncomment the following line to cache the module.
1515 # sys.modules[module.__name__] = module
1516 loader.exec_module(module)
1517 return module
1518
1519 * Remove :mod:`!imp` functions and attributes with no replacements:
1520
1521 * Undocumented functions:
1522
1523 * ``imp.init_builtin()``
1524 * ``imp.load_compiled()``
1525 * ``imp.load_dynamic()``
1526 * ``imp.load_package()``
1527
1528 * ``imp.lock_held()``, ``imp.acquire_lock()``, ``imp.release_lock()``:
1529 the locking scheme has changed in Python 3.3 to per-module locks.
1530 * ``imp.find_module()`` constants: ``SEARCH_ERROR``, ``PY_SOURCE``,
1531 ``PY_COMPILED``, ``C_EXTENSION``, ``PY_RESOURCE``, ``PKG_DIRECTORY``,
1532 ``C_BUILTIN``, ``PY_FROZEN``, ``PY_CODERESOURCE``, ``IMP_HOOK``.
1533
1534 io
1535 --
1536
1537 * Remove :mod:`io`'s ``io.OpenWrapper`` and ``_pyio.OpenWrapper``, deprecated in Python
1538 3.10: just use :func:`open` instead. The :func:`open` (:func:`io.open`)
1539 function is a built-in function. Since Python 3.10, :func:`!_pyio.open` is
1540 also a static method.
1541 (Contributed by Victor Stinner in :gh:`94169`.)
1542
1543 locale
1544 ------
1545
1546 * Remove :mod:`locale`'s :func:`!locale.format` function, deprecated in Python 3.7:
1547 use :func:`locale.format_string` instead.
1548 (Contributed by Victor Stinner in :gh:`94226`.)
1549
1550 smtpd
1551 -----
1552
1553 * The ``smtpd`` module has been removed according to the schedule in :pep:`594`,
1554 having been deprecated in Python 3.4.7 and 3.5.4.
1555 Use the :pypi:`aiosmtpd` PyPI module or any other
1556 :mod:`asyncio`-based server instead.
1557 (Contributed by Oleg Iarygin in :gh:`93243`.)
1558
1559 sqlite3
1560 -------
1561
1562 * The following undocumented :mod:`sqlite3` features, deprecated in Python
1563 3.10, are now removed:
1564
1565 * ``sqlite3.enable_shared_cache()``
1566 * ``sqlite3.OptimizedUnicode``
1567
1568 If a shared cache must be used, open the database in URI mode using the
1569 ``cache=shared`` query parameter.
1570
1571 The ``sqlite3.OptimizedUnicode`` text factory has been an alias for
1572 :class:`str` since Python 3.3. Code that previously set the text factory to
1573 ``OptimizedUnicode`` can either use ``str`` explicitly, or rely on the
1574 default value which is also ``str``.
1575
1576 (Contributed by Erlend E. Aasland in :gh:`92548`.)
1577
1578 ssl
1579 ---
1580
1581 * Remove :mod:`ssl`'s :func:`!ssl.RAND_pseudo_bytes` function, deprecated in Python 3.6:
1582 use :func:`os.urandom` or :func:`ssl.RAND_bytes` instead.
1583 (Contributed by Victor Stinner in :gh:`94199`.)
1584
1585 * Remove the :func:`!ssl.match_hostname` function.
1586 It was deprecated in Python 3.7. OpenSSL performs
1587 hostname matching since Python 3.7, Python no longer uses the
1588 :func:`!ssl.match_hostname` function.
1589 (Contributed by Victor Stinner in :gh:`94199`.)
1590
1591 * Remove the :func:`!ssl.wrap_socket` function, deprecated in Python 3.7:
1592 instead, create a :class:`ssl.SSLContext` object and call its
1593 :class:`ssl.SSLContext.wrap_socket` method. Any package that still uses
1594 :func:`!ssl.wrap_socket` is broken and insecure. The function neither sends a
1595 SNI TLS extension nor validates the server hostname. Code is subject to :cwe:`295`
1596 (Improper Certificate Validation).
1597 (Contributed by Victor Stinner in :gh:`94199`.)
1598
1599 unittest
1600 --------
1601
1602 * Remove many long-deprecated :mod:`unittest` features:
1603
1604 .. _unittest-TestCase-removed-aliases:
1605
1606 * A number of :class:`~unittest.TestCase` method aliases:
1607
1608 ============================ =============================== ===============
1609 Deprecated alias Method Name Deprecated in
1610 ============================ =============================== ===============
1611 ``failUnless`` :meth:`.assertTrue` 3.1
1612 ``failIf`` :meth:`.assertFalse` 3.1
1613 ``failUnlessEqual`` :meth:`.assertEqual` 3.1
1614 ``failIfEqual`` :meth:`.assertNotEqual` 3.1
1615 ``failUnlessAlmostEqual`` :meth:`.assertAlmostEqual` 3.1
1616 ``failIfAlmostEqual`` :meth:`.assertNotAlmostEqual` 3.1
1617 ``failUnlessRaises`` :meth:`.assertRaises` 3.1
1618 ``assert_`` :meth:`.assertTrue` 3.2
1619 ``assertEquals`` :meth:`.assertEqual` 3.2
1620 ``assertNotEquals`` :meth:`.assertNotEqual` 3.2
1621 ``assertAlmostEquals`` :meth:`.assertAlmostEqual` 3.2
1622 ``assertNotAlmostEquals`` :meth:`.assertNotAlmostEqual` 3.2
1623 ``assertRegexpMatches`` :meth:`.assertRegex` 3.2
1624 ``assertRaisesRegexp`` :meth:`.assertRaisesRegex` 3.2
1625 ``assertNotRegexpMatches`` :meth:`.assertNotRegex` 3.5
1626 ============================ =============================== ===============
1627
1628 You can use https://github.com/isidentical/teyit to automatically modernise
1629 your unit tests.
1630
1631 * Undocumented and broken :class:`~unittest.TestCase` method
1632 ``assertDictContainsSubset`` (deprecated in Python 3.2).
1633
1634 * Undocumented :meth:`TestLoader.loadTestsFromModule
1635 <unittest.TestLoader.loadTestsFromModule>` parameter *use_load_tests*
1636 (deprecated and ignored since Python 3.5).
1637
1638 * An alias of the :class:`~unittest.TextTestResult` class:
1639 ``_TextTestResult`` (deprecated in Python 3.2).
1640
1641 (Contributed by Serhiy Storchaka in :gh:`89325`.)
1642
1643 webbrowser
1644 ----------
1645
1646 * Remove support for obsolete browsers from :mod:`webbrowser`.
1647 The removed browsers include: Grail, Mosaic, Netscape, Galeon, Skipstone,
1648 Iceape, Firebird, and Firefox versions 35 and below (:gh:`102871`).
1649
1650 xml.etree.ElementTree
1651 ---------------------
1652
1653 * Remove the ``ElementTree.Element.copy()`` method of the
1654 pure Python implementation, deprecated in Python 3.10, use the
1655 :func:`copy.copy` function instead. The C implementation of :mod:`xml.etree.ElementTree`
1656 has no ``copy()`` method, only a ``__copy__()`` method.
1657 (Contributed by Victor Stinner in :gh:`94383`.)
1658
1659 zipimport
1660 ---------
1661
1662 * Remove :mod:`zipimport`'s ``find_loader()`` and ``find_module()`` methods,
1663 deprecated in Python 3.10: use the ``find_spec()`` method instead. See
1664 :pep:`451` for the rationale.
1665 (Contributed by Victor Stinner in :gh:`94379`.)
1666
1667 Others
1668 ------
1669
1670 * Remove the ``suspicious`` rule from the documentation :file:`Makefile` and
1671 :file:`Doc/tools/rstlint.py`, both in favor of `sphinx-lint
1672 <https://github.com/sphinx-contrib/sphinx-lint>`_.
1673 (Contributed by Julien Palard in :gh:`98179`.)
1674
1675 * Remove the *keyfile* and *certfile* parameters from the
1676 :mod:`ftplib`, :mod:`imaplib`, :mod:`poplib` and :mod:`smtplib` modules,
1677 and the *key_file*, *cert_file* and *check_hostname* parameters from the
1678 :mod:`http.client` module,
1679 all deprecated since Python 3.6. Use the *context* parameter
1680 (*ssl_context* in :mod:`imaplib`) instead.
1681 (Contributed by Victor Stinner in :gh:`94172`.)
1682
1683 * Remove ``Jython`` compatibility hacks from several stdlib modules and tests.
1684 (Contributed by Nikita Sobolev in :gh:`99482`.)
1685
1686 * Remove ``_use_broken_old_ctypes_structure_semantics_`` flag
1687 from :mod:`ctypes` module.
1688 (Contributed by Nikita Sobolev in :gh:`99285`.)
1689
1690
1691 .. _whatsnew312-porting-to-python312:
1692
1693 Porting to Python 3.12
1694 ======================
1695
1696 This section lists previously described changes and other bugfixes
1697 that may require changes to your code.
1698
1699 Changes in the Python API
1700 -------------------------
1701
1702 * More strict rules are now applied for numerical group references and
1703 group names in regular expressions.
1704 Only sequence of ASCII digits is now accepted as a numerical reference.
1705 The group name in bytes patterns and replacement strings can now only
1706 contain ASCII letters and digits and underscore.
1707 (Contributed by Serhiy Storchaka in :gh:`91760`.)
1708
1709 * Remove ``randrange()`` functionality deprecated since Python 3.10. Formerly,
1710 ``randrange(10.0)`` losslessly converted to ``randrange(10)``. Now, it raises a
1711 :exc:`TypeError`. Also, the exception raised for non-integer values such as
1712 ``randrange(10.5)`` or ``randrange('10')`` has been changed from :exc:`ValueError` to
1713 :exc:`TypeError`. This also prevents bugs where ``randrange(1e25)`` would silently
1714 select from a larger range than ``randrange(10**25)``.
1715 (Originally suggested by Serhiy Storchaka :gh:`86388`.)
1716
1717 * :class:`argparse.ArgumentParser` changed encoding and error handler
1718 for reading arguments from file (e.g. ``fromfile_prefix_chars`` option)
1719 from default text encoding (e.g. :func:`locale.getpreferredencoding(False) <locale.getpreferredencoding>`)
1720 to :term:`filesystem encoding and error handler`.
1721 Argument files should be encoded in UTF-8 instead of ANSI Codepage on Windows.
1722
1723 * Remove the ``asyncore``-based ``smtpd`` module deprecated in Python 3.4.7
1724 and 3.5.4. A recommended replacement is the
1725 :mod:`asyncio`-based :pypi:`aiosmtpd` PyPI module.
1726
1727 * :func:`shlex.split`: Passing ``None`` for *s* argument now raises an
1728 exception, rather than reading :data:`sys.stdin`. The feature was deprecated
1729 in Python 3.9.
1730 (Contributed by Victor Stinner in :gh:`94352`.)
1731
1732 * The :mod:`os` module no longer accepts bytes-like paths, like
1733 :class:`bytearray` and :class:`memoryview` types: only the exact
1734 :class:`bytes` type is accepted for bytes strings.
1735 (Contributed by Victor Stinner in :gh:`98393`.)
1736
1737 * :func:`syslog.openlog` and :func:`syslog.closelog` now fail if used in subinterpreters.
1738 :func:`syslog.syslog` may still be used in subinterpreters,
1739 but now only if :func:`syslog.openlog` has already been called in the main interpreter.
1740 These new restrictions do not apply to the main interpreter,
1741 so only a very small set of users might be affected.
1742 This change helps with interpreter isolation. Furthermore, :mod:`syslog` is a wrapper
1743 around process-global resources, which are best managed from the main interpreter.
1744 (Contributed by Donghee Na in :gh:`99127`.)
1745
1746 * The undocumented locking behavior of :func:`~functools.cached_property`
1747 is removed, because it locked across all instances of the class, leading to high
1748 lock contention. This means that a cached property getter function could now run
1749 more than once for a single instance, if two threads race. For most simple
1750 cached properties (e.g. those that are idempotent and simply calculate a value
1751 based on other attributes of the instance) this will be fine. If
1752 synchronization is needed, implement locking within the cached property getter
1753 function or around multi-threaded access points.
1754
1755 * :func:`sys._current_exceptions` now returns a mapping from thread-id to an
1756 exception instance, rather than to a ``(typ, exc, tb)`` tuple.
1757 (Contributed by Irit Katriel in :gh:`103176`.)
1758
1759 * When extracting tar files using :mod:`tarfile` or
1760 :func:`shutil.unpack_archive`, pass the *filter* argument to limit features
1761 that may be surprising or dangerous.
1762 See :ref:`tarfile-extraction-filter` for details.
1763
1764 * The output of the :func:`tokenize.tokenize` and :func:`tokenize.generate_tokens`
1765 functions is now changed due to the changes introduced in :pep:`701`. This
1766 means that ``STRING`` tokens are not emitted any more for f-strings and the
1767 tokens described in :pep:`701` are now produced instead: ``FSTRING_START``,
1768 ``FSTRING_MIDDLE`` and ``FSTRING_END`` are now emitted for f-string "string"
1769 parts in addition to the appropriate tokens for the tokenization in the
1770 expression components. For example for the f-string ``f"start {1+1} end"``
1771 the old version of the tokenizer emitted::
1772
1773 1,0-1,18: STRING 'f"start {1+1} end"'
1774
1775 while the new version emits::
1776
1777 1,0-1,2: FSTRING_START 'f"'
1778 1,2-1,8: FSTRING_MIDDLE 'start '
1779 1,8-1,9: OP '{'
1780 1,9-1,10: NUMBER '1'
1781 1,10-1,11: OP '+'
1782 1,11-1,12: NUMBER '1'
1783 1,12-1,13: OP '}'
1784 1,13-1,17: FSTRING_MIDDLE ' end'
1785 1,17-1,18: FSTRING_END '"'
1786
1787 Additionally, there may be some minor behavioral changes as a consequence of the
1788 changes required to support :pep:`701`. Some of these changes include:
1789
1790 * The ``type`` attribute of the tokens emitted when tokenizing some invalid Python
1791 characters such as ``!`` has changed from ``ERRORTOKEN`` to ``OP``.
1792
1793 * Incomplete single-line strings now also raise :exc:`tokenize.TokenError` as incomplete
1794 multiline strings do.
1795
1796 * Some incomplete or invalid Python code now raises :exc:`tokenize.TokenError` instead of
1797 returning arbitrary ``ERRORTOKEN`` tokens when tokenizing it.
1798
1799 * Mixing tabs and spaces as indentation in the same file is not supported anymore and will
1800 raise a :exc:`TabError`.
1801
1802 * The :mod:`threading` module now expects the :mod:`!_thread` module to have
1803 an ``_is_main_interpreter`` attribute. It is a function with no
1804 arguments that returns ``True`` if the current interpreter is the
1805 main interpreter.
1806
1807 Any library or application that provides a custom ``_thread`` module
1808 should provide ``_is_main_interpreter()``.
1809 (See :gh:`112826`.)
1810
1811 Build Changes
1812 =============
1813
1814 * Python no longer uses :file:`setup.py` to build shared C extension modules.
1815 Build parameters like headers and libraries are detected in ``configure``
1816 script. Extensions are built by :file:`Makefile`. Most extensions use
1817 ``pkg-config`` and fall back to manual detection.
1818 (Contributed by Christian Heimes in :gh:`93939`.)
1819
1820 * ``va_start()`` with two parameters, like ``va_start(args, format),``
1821 is now required to build Python.
1822 ``va_start()`` is no longer called with a single parameter.
1823 (Contributed by Kumar Aditya in :gh:`93207`.)
1824
1825 * CPython now uses the ThinLTO option as the default link time optimization policy
1826 if the Clang compiler accepts the flag.
1827 (Contributed by Donghee Na in :gh:`89536`.)
1828
1829 * Add ``COMPILEALL_OPTS`` variable in :file:`Makefile` to override :mod:`compileall`
1830 options (default: ``-j0``) in ``make install``. Also merged the 3
1831 ``compileall`` commands into a single command to build .pyc files for all
1832 optimization levels (0, 1, 2) at once.
1833 (Contributed by Victor Stinner in :gh:`99289`.)
1834
1835 * Add platform triplets for 64-bit LoongArch:
1836
1837 * loongarch64-linux-gnusf
1838 * loongarch64-linux-gnuf32
1839 * loongarch64-linux-gnu
1840
1841 (Contributed by Zhang Na in :gh:`90656`.)
1842
1843 * ``PYTHON_FOR_REGEN`` now require Python 3.10 or newer.
1844
1845 * Autoconf 2.71 and aclocal 1.16.4 is now required to regenerate
1846 :file:`configure`.
1847 (Contributed by Christian Heimes in :gh:`89886`.)
1848
1849 * Windows builds and macOS installers from python.org now use OpenSSL 3.0.
1850
1851
1852 C API Changes
1853 =============
1854
1855 New Features
1856 ------------
1857
1858 .. _whatsnew312-pep697:
1859
1860 * :pep:`697`: Introduce the :ref:`Unstable C API tier <unstable-c-api>`,
1861 intended for low-level tools like debuggers and JIT compilers.
1862 This API may change in each minor release of CPython without deprecation
1863 warnings.
1864 Its contents are marked by the ``PyUnstable_`` prefix in names.
1865
1866 Code object constructors:
1867
1868 - ``PyUnstable_Code_New()`` (renamed from ``PyCode_New``)
1869 - ``PyUnstable_Code_NewWithPosOnlyArgs()`` (renamed from ``PyCode_NewWithPosOnlyArgs``)
1870
1871 Extra storage for code objects (:pep:`523`):
1872
1873 - ``PyUnstable_Eval_RequestCodeExtraIndex()`` (renamed from ``_PyEval_RequestCodeExtraIndex``)
1874 - ``PyUnstable_Code_GetExtra()`` (renamed from ``_PyCode_GetExtra``)
1875 - ``PyUnstable_Code_SetExtra()`` (renamed from ``_PyCode_SetExtra``)
1876
1877 The original names will continue to be available until the respective
1878 API changes.
1879
1880 (Contributed by Petr Viktorin in :gh:`101101`.)
1881
1882 * :pep:`697`: Add an API for extending types whose instance memory layout is
1883 opaque:
1884
1885 - :c:member:`PyType_Spec.basicsize` can be zero or negative to specify
1886 inheriting or extending the base class size.
1887 - :c:func:`PyObject_GetTypeData` and :c:func:`PyType_GetTypeDataSize`
1888 added to allow access to subclass-specific instance data.
1889 - :c:macro:`Py_TPFLAGS_ITEMS_AT_END` and :c:func:`PyObject_GetItemData`
1890 added to allow safely extending certain variable-sized types, including
1891 :c:var:`PyType_Type`.
1892 - :c:macro:`Py_RELATIVE_OFFSET` added to allow defining
1893 :c:type:`members <PyMemberDef>` in terms of a subclass-specific struct.
1894
1895 (Contributed by Petr Viktorin in :gh:`103509`.)
1896
1897 * Add the new :ref:`limited C API <limited-c-api>` function :c:func:`PyType_FromMetaclass`,
1898 which generalizes the existing :c:func:`PyType_FromModuleAndSpec` using
1899 an additional metaclass argument.
1900 (Contributed by Wenzel Jakob in :gh:`93012`.)
1901
1902 * API for creating objects that can be called using
1903 :ref:`the vectorcall protocol <vectorcall>` was added to the
1904 :ref:`Limited API <stable>`:
1905
1906 * :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL`
1907 * :c:func:`PyVectorcall_NARGS`
1908 * :c:func:`PyVectorcall_Call`
1909 * :c:type:`vectorcallfunc`
1910
1911 The :c:macro:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class
1912 when the class's :py:meth:`~object.__call__` method is reassigned.
1913 This makes vectorcall safe to use with mutable types (i.e. heap types
1914 without the immutable flag, :c:macro:`Py_TPFLAGS_IMMUTABLETYPE`).
1915 Mutable types that do not override :c:member:`~PyTypeObject.tp_call` now
1916 inherit the ``Py_TPFLAGS_HAVE_VECTORCALL`` flag.
1917 (Contributed by Petr Viktorin in :gh:`93274`.)
1918
1919 The :c:macro:`Py_TPFLAGS_MANAGED_DICT` and :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF`
1920 flags have been added. This allows extensions classes to support object
1921 :attr:`~object.__dict__` and weakrefs with less bookkeeping,
1922 using less memory and with faster access.
1923
1924 * API for performing calls using
1925 :ref:`the vectorcall protocol <vectorcall>` was added to the
1926 :ref:`Limited API <stable>`:
1927
1928 * :c:func:`PyObject_Vectorcall`
1929 * :c:func:`PyObject_VectorcallMethod`
1930 * :c:macro:`PY_VECTORCALL_ARGUMENTS_OFFSET`
1931
1932 This means that both the incoming and outgoing ends of the vector call
1933 protocol are now available in the :ref:`Limited API <stable>`. (Contributed
1934 by Wenzel Jakob in :gh:`98586`.)
1935
1936 * Add two new public functions,
1937 :c:func:`PyEval_SetProfileAllThreads` and
1938 :c:func:`PyEval_SetTraceAllThreads`, that allow to set tracing and profiling
1939 functions in all running threads in addition to the calling one. (Contributed
1940 by Pablo Galindo in :gh:`93503`.)
1941
1942 * Add new function :c:func:`PyFunction_SetVectorcall` to the C API
1943 which sets the vectorcall field of a given :c:type:`PyFunctionObject`.
1944 (Contributed by Andrew Frost in :gh:`92257`.)
1945
1946 * The C API now permits registering callbacks via :c:func:`PyDict_AddWatcher`,
1947 :c:func:`PyDict_Watch` and related APIs to be called whenever a dictionary
1948 is modified. This is intended for use by optimizing interpreters, JIT
1949 compilers, or debuggers.
1950 (Contributed by Carl Meyer in :gh:`91052`.)
1951
1952 * Add :c:func:`PyType_AddWatcher` and :c:func:`PyType_Watch` API to register
1953 callbacks to receive notification on changes to a type.
1954 (Contributed by Carl Meyer in :gh:`91051`.)
1955
1956 * Add :c:func:`PyCode_AddWatcher` and :c:func:`PyCode_ClearWatcher`
1957 APIs to register callbacks to receive notification on creation and
1958 destruction of code objects.
1959 (Contributed by Itamar Oren in :gh:`91054`.)
1960
1961 * Add :c:func:`PyFrame_GetVar` and :c:func:`PyFrame_GetVarString` functions to
1962 get a frame variable by its name.
1963 (Contributed by Victor Stinner in :gh:`91248`.)
1964
1965 * Add :c:func:`PyErr_GetRaisedException` and :c:func:`PyErr_SetRaisedException`
1966 for saving and restoring the current exception.
1967 These functions return and accept a single exception object,
1968 rather than the triple arguments of the now-deprecated
1969 :c:func:`PyErr_Fetch` and :c:func:`PyErr_Restore`.
1970 This is less error prone and a bit more efficient.
1971 (Contributed by Mark Shannon in :gh:`101578`.)
1972
1973 * Add ``_PyErr_ChainExceptions1``, which takes an exception instance,
1974 to replace the legacy-API ``_PyErr_ChainExceptions``, which is now
1975 deprecated. (Contributed by Mark Shannon in :gh:`101578`.)
1976
1977 * Add :c:func:`PyException_GetArgs` and :c:func:`PyException_SetArgs`
1978 as convenience functions for retrieving and modifying
1979 the :attr:`~BaseException.args` passed to the exception's constructor.
1980 (Contributed by Mark Shannon in :gh:`101578`.)
1981
1982 * Add :c:func:`PyErr_DisplayException`, which takes an exception instance,
1983 to replace the legacy-api :c:func:`!PyErr_Display`. (Contributed by
1984 Irit Katriel in :gh:`102755`).
1985
1986 .. _whatsnew312-pep683:
1987
1988 * :pep:`683`: Introduce *Immortal Objects*, which allows objects
1989 to bypass reference counts, and related changes to the C-API:
1990
1991 - ``_Py_IMMORTAL_REFCNT``: The reference count that defines an object
1992 as immortal.
1993 - ``_Py_IsImmortal`` Checks if an object has the immortal reference count.
1994 - ``PyObject_HEAD_INIT`` This will now initialize reference count to
1995 ``_Py_IMMORTAL_REFCNT`` when used with ``Py_BUILD_CORE``.
1996 - ``SSTATE_INTERNED_IMMORTAL`` An identifier for interned unicode objects
1997 that are immortal.
1998 - ``SSTATE_INTERNED_IMMORTAL_STATIC`` An identifier for interned unicode
1999 objects that are immortal and static
2000 - ``sys.getunicodeinternedsize`` This returns the total number of unicode
2001 objects that have been interned. This is now needed for :file:`refleak.py` to
2002 correctly track reference counts and allocated blocks
2003
2004 (Contributed by Eddie Elizondo in :gh:`84436`.)
2005
2006 * :pep:`684`: Add the new :c:func:`Py_NewInterpreterFromConfig`
2007 function and :c:type:`PyInterpreterConfig`, which may be used
2008 to create sub-interpreters with their own GILs.
2009 (See :ref:`whatsnew312-pep684` for more info.)
2010 (Contributed by Eric Snow in :gh:`104110`.)
2011
2012 * In the limited C API version 3.12, :c:func:`Py_INCREF` and
2013 :c:func:`Py_DECREF` functions are now implemented as opaque function calls to
2014 hide implementation details.
2015 (Contributed by Victor Stinner in :gh:`105387`.)
2016
2017 Porting to Python 3.12
2018 ----------------------
2019
2020 * Legacy Unicode APIs based on ``Py_UNICODE*`` representation has been removed.
2021 Please migrate to APIs based on UTF-8 or ``wchar_t*``.
2022
2023 * Argument parsing functions like :c:func:`PyArg_ParseTuple` doesn't support
2024 ``Py_UNICODE*`` based format (e.g. ``u``, ``Z``) anymore. Please migrate
2025 to other formats for Unicode like ``s``, ``z``, ``es``, and ``U``.
2026
2027 * ``tp_weaklist`` for all static builtin types is always ``NULL``.
2028 This is an internal-only field on ``PyTypeObject``
2029 but we're pointing out the change in case someone happens to be
2030 accessing the field directly anyway. To avoid breakage, consider
2031 using the existing public C-API instead, or, if necessary, the
2032 (internal-only) ``_PyObject_GET_WEAKREFS_LISTPTR()`` macro.
2033
2034 * This internal-only :c:member:`PyTypeObject.tp_subclasses` may now not be
2035 a valid object pointer. Its type was changed to :c:expr:`void *` to
2036 reflect this. We mention this in case someone happens to be accessing the
2037 internal-only field directly.
2038
2039 To get a list of subclasses, call the Python method
2040 :py:meth:`~type.__subclasses__` (using :c:func:`PyObject_CallMethod`,
2041 for example).
2042
2043 * Add support of more formatting options (left aligning, octals, uppercase
2044 hexadecimals, :c:type:`intmax_t`, :c:type:`ptrdiff_t`, :c:type:`wchar_t` C
2045 strings, variable width and precision) in :c:func:`PyUnicode_FromFormat` and
2046 :c:func:`PyUnicode_FromFormatV`.
2047 (Contributed by Serhiy Storchaka in :gh:`98836`.)
2048
2049 * An unrecognized format character in :c:func:`PyUnicode_FromFormat` and
2050 :c:func:`PyUnicode_FromFormatV` now sets a :exc:`SystemError`.
2051 In previous versions it caused all the rest of the format string to be
2052 copied as-is to the result string, and any extra arguments discarded.
2053 (Contributed by Serhiy Storchaka in :gh:`95781`.)
2054
2055 * Fix wrong sign placement in :c:func:`PyUnicode_FromFormat` and
2056 :c:func:`PyUnicode_FromFormatV`.
2057 (Contributed by Philip Georgi in :gh:`95504`.)
2058
2059 * Extension classes wanting to add a :attr:`~object.__dict__` or weak reference slot
2060 should use :c:macro:`Py_TPFLAGS_MANAGED_DICT` and
2061 :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` instead of ``tp_dictoffset`` and
2062 ``tp_weaklistoffset``, respectively.
2063 The use of ``tp_dictoffset`` and ``tp_weaklistoffset`` is still
2064 supported, but does not fully support multiple inheritance
2065 (:gh:`95589`), and performance may be worse.
2066 Classes declaring :c:macro:`Py_TPFLAGS_MANAGED_DICT` must call
2067 :c:func:`!_PyObject_VisitManagedDict` and :c:func:`!_PyObject_ClearManagedDict`
2068 to traverse and clear their instance's dictionaries.
2069 To clear weakrefs, call :c:func:`PyObject_ClearWeakRefs`, as before.
2070
2071 * The :c:func:`PyUnicode_FSDecoder` function no longer accepts bytes-like
2072 paths, like :class:`bytearray` and :class:`memoryview` types: only the exact
2073 :class:`bytes` type is accepted for bytes strings.
2074 (Contributed by Victor Stinner in :gh:`98393`.)
2075
2076 * The :c:macro:`Py_CLEAR`, :c:macro:`Py_SETREF` and :c:macro:`Py_XSETREF`
2077 macros now only evaluate their arguments once. If an argument has side
2078 effects, these side effects are no longer duplicated.
2079 (Contributed by Victor Stinner in :gh:`98724`.)
2080
2081 * The interpreter's error indicator is now always normalized. This means
2082 that :c:func:`PyErr_SetObject`, :c:func:`PyErr_SetString` and the other
2083 functions that set the error indicator now normalize the exception
2084 before storing it. (Contributed by Mark Shannon in :gh:`101578`.)
2085
2086 * ``_Py_RefTotal`` is no longer authoritative and only kept around
2087 for ABI compatibility. Note that it is an internal global and only
2088 available on debug builds. If you happen to be using it then you'll
2089 need to start using ``_Py_GetGlobalRefTotal()``.
2090
2091 * The following functions now select an appropriate metaclass for the newly
2092 created type:
2093
2094 * :c:func:`PyType_FromSpec`
2095 * :c:func:`PyType_FromSpecWithBases`
2096 * :c:func:`PyType_FromModuleAndSpec`
2097
2098 Creating classes whose metaclass overrides :c:member:`~PyTypeObject.tp_new`
2099 is deprecated, and in Python 3.14+ it will be disallowed.
2100 Note that these functions ignore ``tp_new`` of the metaclass, possibly
2101 allowing incomplete initialization.
2102
2103 Note that :c:func:`PyType_FromMetaclass` (added in Python 3.12)
2104 already disallows creating classes whose metaclass overrides ``tp_new``
2105 (:meth:`~object.__new__` in Python).
2106
2107 Since ``tp_new`` overrides almost everything ``PyType_From*`` functions do,
2108 the two are incompatible with each other.
2109 The existing behavior -- ignoring the metaclass for several steps
2110 of type creation -- is unsafe in general, since (meta)classes assume that
2111 ``tp_new`` was called.
2112 There is no simple general workaround. One of the following may work for you:
2113
2114 - If you control the metaclass, avoid using ``tp_new`` in it:
2115
2116 - If initialization can be skipped, it can be done in
2117 :c:member:`~PyTypeObject.tp_init` instead.
2118 - If the metaclass doesn't need to be instantiated from Python,
2119 set its ``tp_new`` to ``NULL`` using
2120 the :c:macro:`Py_TPFLAGS_DISALLOW_INSTANTIATION` flag.
2121 This makes it acceptable for ``PyType_From*`` functions.
2122
2123 - Avoid ``PyType_From*`` functions: if you don't need C-specific features
2124 (slots or setting the instance size), create types by :ref:`calling <call>`
2125 the metaclass.
2126
2127 - If you *know* the ``tp_new`` can be skipped safely, filter the deprecation
2128 warning out using :func:`warnings.catch_warnings` from Python.
2129
2130 * :c:var:`PyOS_InputHook` and :c:var:`PyOS_ReadlineFunctionPointer` are no
2131 longer called in :ref:`subinterpreters <sub-interpreter-support>`. This is
2132 because clients generally rely on process-wide global state (since these
2133 callbacks have no way of recovering extension module state).
2134
2135 This also avoids situations where extensions may find themselves running in a
2136 subinterpreter that they don't support (or haven't yet been loaded in). See
2137 :gh:`104668` for more info.
2138
2139 * :c:struct:`PyLongObject` has had its internals changed for better performance.
2140 Although the internals of :c:struct:`PyLongObject` are private, they are used
2141 by some extension modules.
2142 The internal fields should no longer be accessed directly, instead the API
2143 functions beginning ``PyLong_...`` should be used instead.
2144 Two new *unstable* API functions are provided for efficient access to the
2145 value of :c:struct:`PyLongObject`\s which fit into a single machine word:
2146
2147 * :c:func:`PyUnstable_Long_IsCompact`
2148 * :c:func:`PyUnstable_Long_CompactValue`
2149
2150 * Custom allocators, set via :c:func:`PyMem_SetAllocator`, are now
2151 required to be thread-safe, regardless of memory domain. Allocators
2152 that don't have their own state, including "hooks", are not affected.
2153 If your custom allocator is not already thread-safe and you need
2154 guidance then please create a new GitHub issue
2155 and CC ``@ericsnowcurrently``.
2156
2157 Deprecated
2158 ----------
2159
2160 * In accordance with :pep:`699`, the ``ma_version_tag`` field in :c:type:`PyDictObject`
2161 is deprecated for extension modules. Accessing this field will generate a compiler
2162 warning at compile time. This field will be removed in Python 3.14.
2163 (Contributed by Ramvikrams and Kumar Aditya in :gh:`101193`. PEP by Ken Jin.)
2164
2165 * Deprecate global configuration variable:
2166
2167 * :c:var:`Py_DebugFlag`: use :c:member:`PyConfig.parser_debug`
2168 * :c:var:`Py_VerboseFlag`: use :c:member:`PyConfig.verbose`
2169 * :c:var:`Py_QuietFlag`: use :c:member:`PyConfig.quiet`
2170 * :c:var:`Py_InteractiveFlag`: use :c:member:`PyConfig.interactive`
2171 * :c:var:`Py_InspectFlag`: use :c:member:`PyConfig.inspect`
2172 * :c:var:`Py_OptimizeFlag`: use :c:member:`PyConfig.optimization_level`
2173 * :c:var:`Py_NoSiteFlag`: use :c:member:`PyConfig.site_import`
2174 * :c:var:`Py_BytesWarningFlag`: use :c:member:`PyConfig.bytes_warning`
2175 * :c:var:`Py_FrozenFlag`: use :c:member:`PyConfig.pathconfig_warnings`
2176 * :c:var:`Py_IgnoreEnvironmentFlag`: use :c:member:`PyConfig.use_environment`
2177 * :c:var:`Py_DontWriteBytecodeFlag`: use :c:member:`PyConfig.write_bytecode`
2178 * :c:var:`Py_NoUserSiteDirectory`: use :c:member:`PyConfig.user_site_directory`
2179 * :c:var:`Py_UnbufferedStdioFlag`: use :c:member:`PyConfig.buffered_stdio`
2180 * :c:var:`Py_HashRandomizationFlag`: use :c:member:`PyConfig.use_hash_seed`
2181 and :c:member:`PyConfig.hash_seed`
2182 * :c:var:`Py_IsolatedFlag`: use :c:member:`PyConfig.isolated`
2183 * :c:var:`Py_LegacyWindowsFSEncodingFlag`: use :c:member:`PyPreConfig.legacy_windows_fs_encoding`
2184 * :c:var:`Py_LegacyWindowsStdioFlag`: use :c:member:`PyConfig.legacy_windows_stdio`
2185 * :c:var:`!Py_FileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding`
2186 * :c:var:`!Py_HasFileSystemDefaultEncoding`: use :c:member:`PyConfig.filesystem_encoding`
2187 * :c:var:`!Py_FileSystemDefaultEncodeErrors`: use :c:member:`PyConfig.filesystem_errors`
2188 * :c:var:`!Py_UTF8Mode`: use :c:member:`PyPreConfig.utf8_mode` (see :c:func:`Py_PreInitialize`)
2189
2190 The :c:func:`Py_InitializeFromConfig` API should be used with
2191 :c:type:`PyConfig` instead.
2192 (Contributed by Victor Stinner in :gh:`77782`.)
2193
2194 * Creating :c:data:`immutable types <Py_TPFLAGS_IMMUTABLETYPE>` with mutable
2195 bases is deprecated and will be disabled in Python 3.14. (:gh:`95388`)
2196
2197 * The :file:`structmember.h` header is deprecated, though it continues to be
2198 available and there are no plans to remove it.
2199
2200 Its contents are now available just by including :file:`Python.h`,
2201 with a ``Py`` prefix added if it was missing:
2202
2203 - :c:struct:`PyMemberDef`, :c:func:`PyMember_GetOne` and
2204 :c:func:`PyMember_SetOne`
2205 - Type macros like :c:macro:`Py_T_INT`, :c:macro:`Py_T_DOUBLE`, etc.
2206 (previously ``T_INT``, ``T_DOUBLE``, etc.)
2207 - The flags :c:macro:`Py_READONLY` (previously ``READONLY``) and
2208 :c:macro:`Py_AUDIT_READ` (previously all uppercase)
2209
2210 Several items are not exposed from :file:`Python.h`:
2211
2212 - :c:macro:`T_OBJECT` (use :c:macro:`Py_T_OBJECT_EX`)
2213 - :c:macro:`T_NONE` (previously undocumented, and pretty quirky)
2214 - The macro ``WRITE_RESTRICTED`` which does nothing.
2215 - The macros ``RESTRICTED`` and ``READ_RESTRICTED``, equivalents of
2216 :c:macro:`Py_AUDIT_READ`.
2217 - In some configurations, ``<stddef.h>`` is not included from :file:`Python.h`.
2218 It should be included manually when using ``offsetof()``.
2219
2220 The deprecated header continues to provide its original
2221 contents under the original names.
2222 Your old code can stay unchanged, unless the extra include and non-namespaced
2223 macros bother you greatly.
2224
2225 (Contributed in :gh:`47146` by Petr Viktorin, based on
2226 earlier work by Alexander Belopolsky and Matthias Braun.)
2227
2228 * :c:func:`PyErr_Fetch` and :c:func:`PyErr_Restore` are deprecated.
2229 Use :c:func:`PyErr_GetRaisedException` and
2230 :c:func:`PyErr_SetRaisedException` instead.
2231 (Contributed by Mark Shannon in :gh:`101578`.)
2232
2233 * :c:func:`!PyErr_Display` is deprecated. Use :c:func:`PyErr_DisplayException`
2234 instead. (Contributed by Irit Katriel in :gh:`102755`).
2235
2236 * ``_PyErr_ChainExceptions`` is deprecated. Use ``_PyErr_ChainExceptions1``
2237 instead. (Contributed by Irit Katriel in :gh:`102192`.)
2238
2239 * Using :c:func:`PyType_FromSpec`, :c:func:`PyType_FromSpecWithBases`
2240 or :c:func:`PyType_FromModuleAndSpec` to create a class whose metaclass
2241 overrides :c:member:`~PyTypeObject.tp_new` is deprecated.
2242 Call the metaclass instead.
2243
2244 .. Add deprecations above alphabetically, not here at the end.
2245
2246 .. include:: ../deprecations/c-api-pending-removal-in-3.14.rst
2247
2248 .. include:: ../deprecations/c-api-pending-removal-in-3.15.rst
2249
2250 .. include:: ../deprecations/c-api-pending-removal-in-3.16.rst
2251
2252 .. include:: ../deprecations/c-api-pending-removal-in-future.rst
2253
2254 Removed
2255 -------
2256
2257 * Remove the :file:`token.h` header file. There was never any public tokenizer C
2258 API. The :file:`token.h` header file was only designed to be used by Python
2259 internals.
2260 (Contributed by Victor Stinner in :gh:`92651`.)
2261
2262 * Legacy Unicode APIs have been removed. See :pep:`623` for detail.
2263
2264 * :c:macro:`!PyUnicode_WCHAR_KIND`
2265 * :c:func:`!PyUnicode_AS_UNICODE`
2266 * :c:func:`!PyUnicode_AsUnicode`
2267 * :c:func:`!PyUnicode_AsUnicodeAndSize`
2268 * :c:func:`!PyUnicode_AS_DATA`
2269 * :c:func:`!PyUnicode_FromUnicode`
2270 * :c:func:`!PyUnicode_GET_SIZE`
2271 * :c:func:`!PyUnicode_GetSize`
2272 * :c:func:`!PyUnicode_GET_DATA_SIZE`
2273
2274 * Remove the ``PyUnicode_InternImmortal()`` function macro.
2275 (Contributed by Victor Stinner in :gh:`85858`.)