| 1 |
**************************** |
| 2 |
What's New in Python 2.5 |
| 3 |
**************************** |
| 4 |
|
| 5 |
:Author: A.M. Kuchling |
| 6 |
|
| 7 |
.. |release| replace:: 1.01 |
| 8 |
|
| 9 |
.. $Id: whatsnew25.tex 56611 2007-07-29 08:26:10Z georg.brandl $ |
| 10 |
.. Fix XXX comments |
| 11 |
|
| 12 |
This article explains the new features in Python 2.5. The final release of |
| 13 |
Python 2.5 is scheduled for August 2006; :pep:`356` describes the planned |
| 14 |
release schedule. Python 2.5 was released on September 19, 2006. |
| 15 |
|
| 16 |
The changes in Python 2.5 are an interesting mix of language and library |
| 17 |
improvements. The library enhancements will be more important to Python's user |
| 18 |
community, I think, because several widely useful packages were added. New |
| 19 |
modules include ElementTree for XML processing (:mod:`xml.etree`), |
| 20 |
the SQLite database module (:mod:`sqlite`), and the :mod:`ctypes` |
| 21 |
module for calling C functions. |
| 22 |
|
| 23 |
The language changes are of middling significance. Some pleasant new features |
| 24 |
were added, but most of them aren't features that you'll use every day. |
| 25 |
Conditional expressions were finally added to the language using a novel syntax; |
| 26 |
see section :ref:`pep-308`. The new ':keyword:`with`' statement will make |
| 27 |
writing cleanup code easier (section :ref:`pep-343`). Values can now be passed |
| 28 |
into generators (section :ref:`pep-342`). Imports are now visible as either |
| 29 |
absolute or relative (section :ref:`pep-328`). Some corner cases of exception |
| 30 |
handling are handled better (section :ref:`pep-341`). All these improvements |
| 31 |
are worthwhile, but they're improvements to one specific language feature or |
| 32 |
another; none of them are broad modifications to Python's semantics. |
| 33 |
|
| 34 |
As well as the language and library additions, other improvements and bugfixes |
| 35 |
were made throughout the source tree. A search through the SVN change logs |
| 36 |
finds there were 353 patches applied and 458 bugs fixed between Python 2.4 and |
| 37 |
2.5. (Both figures are likely to be underestimates.) |
| 38 |
|
| 39 |
This article doesn't try to be a complete specification of the new features; |
| 40 |
instead changes are briefly introduced using helpful examples. For full |
| 41 |
details, you should always refer to the documentation for Python 2.5 at |
| 42 |
https://docs.python.org. If you want to understand the complete implementation |
| 43 |
and design rationale, refer to the PEP for a particular new feature. |
| 44 |
|
| 45 |
Comments, suggestions, and error reports for this document are welcome; please |
| 46 |
e-mail them to the author or open a bug in the Python bug tracker. |
| 47 |
|
| 48 |
.. ====================================================================== |
| 49 |
|
| 50 |
|
| 51 |
.. _pep-308: |
| 52 |
|
| 53 |
PEP 308: Conditional Expressions |
| 54 |
================================ |
| 55 |
|
| 56 |
For a long time, people have been requesting a way to write conditional |
| 57 |
expressions, which are expressions that return value A or value B depending on |
| 58 |
whether a Boolean value is true or false. A conditional expression lets you |
| 59 |
write a single assignment statement that has the same effect as the following:: |
| 60 |
|
| 61 |
if condition: |
| 62 |
x = true_value |
| 63 |
else: |
| 64 |
x = false_value |
| 65 |
|
| 66 |
There have been endless tedious discussions of syntax on both python-dev and |
| 67 |
comp.lang.python. A vote was even held that found the majority of voters wanted |
| 68 |
conditional expressions in some form, but there was no syntax that was preferred |
| 69 |
by a clear majority. Candidates included C's ``cond ? true_v : false_v``, ``if |
| 70 |
cond then true_v else false_v``, and 16 other variations. |
| 71 |
|
| 72 |
Guido van Rossum eventually chose a surprising syntax:: |
| 73 |
|
| 74 |
x = true_value if condition else false_value |
| 75 |
|
| 76 |
Evaluation is still lazy as in existing Boolean expressions, so the order of |
| 77 |
evaluation jumps around a bit. The *condition* expression in the middle is |
| 78 |
evaluated first, and the *true_value* expression is evaluated only if the |
| 79 |
condition was true. Similarly, the *false_value* expression is only evaluated |
| 80 |
when the condition is false. |
| 81 |
|
| 82 |
This syntax may seem strange and backwards; why does the condition go in the |
| 83 |
*middle* of the expression, and not in the front as in C's ``c ? x : y``? The |
| 84 |
decision was checked by applying the new syntax to the modules in the standard |
| 85 |
library and seeing how the resulting code read. In many cases where a |
| 86 |
conditional expression is used, one value seems to be the 'common case' and one |
| 87 |
value is an 'exceptional case', used only on rarer occasions when the condition |
| 88 |
isn't met. The conditional syntax makes this pattern a bit more obvious:: |
| 89 |
|
| 90 |
contents = ((doc + '\n') if doc else '') |
| 91 |
|
| 92 |
I read the above statement as meaning "here *contents* is usually assigned a |
| 93 |
value of ``doc+'\n'``; sometimes *doc* is empty, in which special case an empty |
| 94 |
string is returned." I doubt I will use conditional expressions very often |
| 95 |
where there isn't a clear common and uncommon case. |
| 96 |
|
| 97 |
There was some discussion of whether the language should require surrounding |
| 98 |
conditional expressions with parentheses. The decision was made to *not* |
| 99 |
require parentheses in the Python language's grammar, but as a matter of style I |
| 100 |
think you should always use them. Consider these two statements:: |
| 101 |
|
| 102 |
# First version -- no parens |
| 103 |
level = 1 if logging else 0 |
| 104 |
|
| 105 |
# Second version -- with parens |
| 106 |
level = (1 if logging else 0) |
| 107 |
|
| 108 |
In the first version, I think a reader's eye might group the statement into |
| 109 |
'level = 1', 'if logging', 'else 0', and think that the condition decides |
| 110 |
whether the assignment to *level* is performed. The second version reads |
| 111 |
better, in my opinion, because it makes it clear that the assignment is always |
| 112 |
performed and the choice is being made between two values. |
| 113 |
|
| 114 |
Another reason for including the brackets: a few odd combinations of list |
| 115 |
comprehensions and lambdas could look like incorrect conditional expressions. |
| 116 |
See :pep:`308` for some examples. If you put parentheses around your |
| 117 |
conditional expressions, you won't run into this case. |
| 118 |
|
| 119 |
|
| 120 |
.. seealso:: |
| 121 |
|
| 122 |
:pep:`308` - Conditional Expressions |
| 123 |
PEP written by Guido van Rossum and Raymond D. Hettinger; implemented by Thomas |
| 124 |
Wouters. |
| 125 |
|
| 126 |
.. ====================================================================== |
| 127 |
|
| 128 |
|
| 129 |
.. _pep-309: |
| 130 |
|
| 131 |
PEP 309: Partial Function Application |
| 132 |
===================================== |
| 133 |
|
| 134 |
The :mod:`functools` module is intended to contain tools for functional-style |
| 135 |
programming. |
| 136 |
|
| 137 |
One useful tool in this module is the :func:`partial` function. For programs |
| 138 |
written in a functional style, you'll sometimes want to construct variants of |
| 139 |
existing functions that have some of the parameters filled in. Consider a |
| 140 |
Python function ``f(a, b, c)``; you could create a new function ``g(b, c)`` that |
| 141 |
was equivalent to ``f(1, b, c)``. This is called "partial function |
| 142 |
application". |
| 143 |
|
| 144 |
:func:`partial` takes the arguments ``(function, arg1, arg2, ... kwarg1=value1, |
| 145 |
kwarg2=value2)``. The resulting object is callable, so you can just call it to |
| 146 |
invoke *function* with the filled-in arguments. |
| 147 |
|
| 148 |
Here's a small but realistic example:: |
| 149 |
|
| 150 |
import functools |
| 151 |
|
| 152 |
def log (message, subsystem): |
| 153 |
"Write the contents of 'message' to the specified subsystem." |
| 154 |
print '%s: %s' % (subsystem, message) |
| 155 |
... |
| 156 |
|
| 157 |
server_log = functools.partial(log, subsystem='server') |
| 158 |
server_log('Unable to open socket') |
| 159 |
|
| 160 |
Here's another example, from a program that uses PyGTK. Here a context-sensitive |
| 161 |
pop-up menu is being constructed dynamically. The callback provided |
| 162 |
for the menu option is a partially applied version of the :meth:`open_item` |
| 163 |
method, where the first argument has been provided. :: |
| 164 |
|
| 165 |
... |
| 166 |
class Application: |
| 167 |
def open_item(self, path): |
| 168 |
... |
| 169 |
def init (self): |
| 170 |
open_func = functools.partial(self.open_item, item_path) |
| 171 |
popup_menu.append( ("Open", open_func, 1) ) |
| 172 |
|
| 173 |
Another function in the :mod:`functools` module is the |
| 174 |
``update_wrapper(wrapper, wrapped)`` function that helps you write |
| 175 |
well-behaved decorators. :func:`update_wrapper` copies the name, module, and |
| 176 |
docstring attribute to a wrapper function so that tracebacks inside the wrapped |
| 177 |
function are easier to understand. For example, you might write:: |
| 178 |
|
| 179 |
def my_decorator(f): |
| 180 |
def wrapper(*args, **kwds): |
| 181 |
print 'Calling decorated function' |
| 182 |
return f(*args, **kwds) |
| 183 |
functools.update_wrapper(wrapper, f) |
| 184 |
return wrapper |
| 185 |
|
| 186 |
:func:`wraps` is a decorator that can be used inside your own decorators to copy |
| 187 |
the wrapped function's information. An alternate version of the previous |
| 188 |
example would be:: |
| 189 |
|
| 190 |
def my_decorator(f): |
| 191 |
@functools.wraps(f) |
| 192 |
def wrapper(*args, **kwds): |
| 193 |
print 'Calling decorated function' |
| 194 |
return f(*args, **kwds) |
| 195 |
return wrapper |
| 196 |
|
| 197 |
|
| 198 |
.. seealso:: |
| 199 |
|
| 200 |
:pep:`309` - Partial Function Application |
| 201 |
PEP proposed and written by Peter Harris; implemented by Hye-Shik Chang and Nick |
| 202 |
Coghlan, with adaptations by Raymond Hettinger. |
| 203 |
|
| 204 |
.. ====================================================================== |
| 205 |
|
| 206 |
|
| 207 |
.. _pep-314: |
| 208 |
|
| 209 |
PEP 314: Metadata for Python Software Packages v1.1 |
| 210 |
=================================================== |
| 211 |
|
| 212 |
Some simple dependency support was added to Distutils. The :func:`setup` |
| 213 |
function now has ``requires``, ``provides``, and ``obsoletes`` keyword |
| 214 |
parameters. When you build a source distribution using the ``sdist`` command, |
| 215 |
the dependency information will be recorded in the :file:`PKG-INFO` file. |
| 216 |
|
| 217 |
Another new keyword parameter is ``download_url``, which should be set to a URL |
| 218 |
for the package's source code. This means it's now possible to look up an entry |
| 219 |
in the package index, determine the dependencies for a package, and download the |
| 220 |
required packages. :: |
| 221 |
|
| 222 |
VERSION = '1.0' |
| 223 |
setup(name='PyPackage', |
| 224 |
version=VERSION, |
| 225 |
requires=['numarray', 'zlib (>=1.1.4)'], |
| 226 |
obsoletes=['OldPackage'] |
| 227 |
download_url=('http://www.example.com/pypackage/dist/pkg-%s.tar.gz' |
| 228 |
% VERSION), |
| 229 |
) |
| 230 |
|
| 231 |
Another new enhancement to the Python package index at |
| 232 |
https://pypi.org is storing source and binary archives for a |
| 233 |
package. The new :command:`upload` Distutils command will upload a package to |
| 234 |
the repository. |
| 235 |
|
| 236 |
Before a package can be uploaded, you must be able to build a distribution using |
| 237 |
the :command:`sdist` Distutils command. Once that works, you can run ``python |
| 238 |
setup.py upload`` to add your package to the PyPI archive. Optionally you can |
| 239 |
GPG-sign the package by supplying the :option:`!--sign` and :option:`!--identity` |
| 240 |
options. |
| 241 |
|
| 242 |
Package uploading was implemented by Martin von Löwis and Richard Jones. |
| 243 |
|
| 244 |
|
| 245 |
.. seealso:: |
| 246 |
|
| 247 |
:pep:`314` - Metadata for Python Software Packages v1.1 |
| 248 |
PEP proposed and written by A.M. Kuchling, Richard Jones, and Fred Drake; |
| 249 |
implemented by Richard Jones and Fred Drake. |
| 250 |
|
| 251 |
.. ====================================================================== |
| 252 |
|
| 253 |
|
| 254 |
.. _pep-328: |
| 255 |
|
| 256 |
PEP 328: Absolute and Relative Imports |
| 257 |
====================================== |
| 258 |
|
| 259 |
The simpler part of :pep:`328` was implemented in Python 2.4: parentheses could now |
| 260 |
be used to enclose the names imported from a module using the ``from ... import |
| 261 |
...`` statement, making it easier to import many different names. |
| 262 |
|
| 263 |
The more complicated part has been implemented in Python 2.5: importing a module |
| 264 |
can be specified to use absolute or package-relative imports. The plan is to |
| 265 |
move toward making absolute imports the default in future versions of Python. |
| 266 |
|
| 267 |
Let's say you have a package directory like this:: |
| 268 |
|
| 269 |
pkg/ |
| 270 |
pkg/__init__.py |
| 271 |
pkg/main.py |
| 272 |
pkg/string.py |
| 273 |
|
| 274 |
This defines a package named :mod:`pkg` containing the :mod:`pkg.main` and |
| 275 |
:mod:`pkg.string` submodules. |
| 276 |
|
| 277 |
Consider the code in the :file:`main.py` module. What happens if it executes |
| 278 |
the statement ``import string``? In Python 2.4 and earlier, it will first look |
| 279 |
in the package's directory to perform a relative import, finds |
| 280 |
:file:`pkg/string.py`, imports the contents of that file as the |
| 281 |
:mod:`pkg.string` module, and that module is bound to the name ``string`` in the |
| 282 |
:mod:`pkg.main` module's namespace. |
| 283 |
|
| 284 |
That's fine if :mod:`pkg.string` was what you wanted. But what if you wanted |
| 285 |
Python's standard :mod:`string` module? There's no clean way to ignore |
| 286 |
:mod:`pkg.string` and look for the standard module; generally you had to look at |
| 287 |
the contents of ``sys.modules``, which is slightly unclean. Holger Krekel's |
| 288 |
:mod:`py.std` package provides a tidier way to perform imports from the standard |
| 289 |
library, ``import py; py.std.string.join()``, but that package isn't available |
| 290 |
on all Python installations. |
| 291 |
|
| 292 |
Reading code which relies on relative imports is also less clear, because a |
| 293 |
reader may be confused about which module, :mod:`string` or :mod:`pkg.string`, |
| 294 |
is intended to be used. Python users soon learned not to duplicate the names of |
| 295 |
standard library modules in the names of their packages' submodules, but you |
| 296 |
can't protect against having your submodule's name being used for a new module |
| 297 |
added in a future version of Python. |
| 298 |
|
| 299 |
In Python 2.5, you can switch :keyword:`import`'s behaviour to absolute imports |
| 300 |
using a ``from __future__ import absolute_import`` directive. This absolute-import |
| 301 |
behaviour will become the default in a future version (probably Python |
| 302 |
2.7). Once absolute imports are the default, ``import string`` will always |
| 303 |
find the standard library's version. It's suggested that users should begin |
| 304 |
using absolute imports as much as possible, so it's preferable to begin writing |
| 305 |
``from pkg import string`` in your code. |
| 306 |
|
| 307 |
Relative imports are still possible by adding a leading period to the module |
| 308 |
name when using the ``from ... import`` form:: |
| 309 |
|
| 310 |
# Import names from pkg.string |
| 311 |
from .string import name1, name2 |
| 312 |
# Import pkg.string |
| 313 |
from . import string |
| 314 |
|
| 315 |
This imports the :mod:`string` module relative to the current package, so in |
| 316 |
:mod:`pkg.main` this will import *name1* and *name2* from :mod:`pkg.string`. |
| 317 |
Additional leading periods perform the relative import starting from the parent |
| 318 |
of the current package. For example, code in the :mod:`A.B.C` module can do:: |
| 319 |
|
| 320 |
from . import D # Imports A.B.D |
| 321 |
from .. import E # Imports A.E |
| 322 |
from ..F import G # Imports A.F.G |
| 323 |
|
| 324 |
Leading periods cannot be used with the ``import modname`` form of the import |
| 325 |
statement, only the ``from ... import`` form. |
| 326 |
|
| 327 |
|
| 328 |
.. seealso:: |
| 329 |
|
| 330 |
:pep:`328` - Imports: Multi-Line and Absolute/Relative |
| 331 |
PEP written by Aahz; implemented by Thomas Wouters. |
| 332 |
|
| 333 |
https://pylib.readthedocs.io/ |
| 334 |
The py library by Holger Krekel, which contains the :mod:`py.std` package. |
| 335 |
|
| 336 |
.. ====================================================================== |
| 337 |
|
| 338 |
|
| 339 |
.. _pep-338: |
| 340 |
|
| 341 |
PEP 338: Executing Modules as Scripts |
| 342 |
===================================== |
| 343 |
|
| 344 |
The :option:`-m` switch added in Python 2.4 to execute a module as a script |
| 345 |
gained a few more abilities. Instead of being implemented in C code inside the |
| 346 |
Python interpreter, the switch now uses an implementation in a new module, |
| 347 |
:mod:`runpy`. |
| 348 |
|
| 349 |
The :mod:`runpy` module implements a more sophisticated import mechanism so that |
| 350 |
it's now possible to run modules in a package such as :mod:`pychecker.checker`. |
| 351 |
The module also supports alternative import mechanisms such as the |
| 352 |
:mod:`zipimport` module. This means you can add a .zip archive's path to |
| 353 |
``sys.path`` and then use the :option:`-m` switch to execute code from the |
| 354 |
archive. |
| 355 |
|
| 356 |
|
| 357 |
.. seealso:: |
| 358 |
|
| 359 |
:pep:`338` - Executing modules as scripts |
| 360 |
PEP written and implemented by Nick Coghlan. |
| 361 |
|
| 362 |
.. ====================================================================== |
| 363 |
|
| 364 |
|
| 365 |
.. _pep-341: |
| 366 |
|
| 367 |
PEP 341: Unified try/except/finally |
| 368 |
=================================== |
| 369 |
|
| 370 |
Until Python 2.5, the :keyword:`try` statement came in two flavours. You could |
| 371 |
use a :keyword:`finally` block to ensure that code is always executed, or one or |
| 372 |
more :keyword:`except` blocks to catch specific exceptions. You couldn't |
| 373 |
combine both :keyword:`!except` blocks and a :keyword:`!finally` block, because |
| 374 |
generating the right bytecode for the combined version was complicated and it |
| 375 |
wasn't clear what the semantics of the combined statement should be. |
| 376 |
|
| 377 |
Guido van Rossum spent some time working with Java, which does support the |
| 378 |
equivalent of combining :keyword:`except` blocks and a :keyword:`finally` block, |
| 379 |
and this clarified what the statement should mean. In Python 2.5, you can now |
| 380 |
write:: |
| 381 |
|
| 382 |
try: |
| 383 |
block-1 ... |
| 384 |
except Exception1: |
| 385 |
handler-1 ... |
| 386 |
except Exception2: |
| 387 |
handler-2 ... |
| 388 |
else: |
| 389 |
else-block |
| 390 |
finally: |
| 391 |
final-block |
| 392 |
|
| 393 |
The code in *block-1* is executed. If the code raises an exception, the various |
| 394 |
:keyword:`except` blocks are tested: if the exception is of class |
| 395 |
:class:`Exception1`, *handler-1* is executed; otherwise if it's of class |
| 396 |
:class:`Exception2`, *handler-2* is executed, and so forth. If no exception is |
| 397 |
raised, the *else-block* is executed. |
| 398 |
|
| 399 |
No matter what happened previously, the *final-block* is executed once the code |
| 400 |
block is complete and any raised exceptions handled. Even if there's an error in |
| 401 |
an exception handler or the *else-block* and a new exception is raised, the code |
| 402 |
in the *final-block* is still run. |
| 403 |
|
| 404 |
|
| 405 |
.. seealso:: |
| 406 |
|
| 407 |
:pep:`341` - Unifying try-except and try-finally |
| 408 |
PEP written by Georg Brandl; implementation by Thomas Lee. |
| 409 |
|
| 410 |
.. ====================================================================== |
| 411 |
|
| 412 |
|
| 413 |
.. _pep-342: |
| 414 |
|
| 415 |
PEP 342: New Generator Features |
| 416 |
=============================== |
| 417 |
|
| 418 |
Python 2.5 adds a simple way to pass values *into* a generator. As introduced in |
| 419 |
Python 2.3, generators only produce output; once a generator's code was invoked |
| 420 |
to create an iterator, there was no way to pass any new information into the |
| 421 |
function when its execution is resumed. Sometimes the ability to pass in some |
| 422 |
information would be useful. Hackish solutions to this include making the |
| 423 |
generator's code look at a global variable and then changing the global |
| 424 |
variable's value, or passing in some mutable object that callers then modify. |
| 425 |
|
| 426 |
To refresh your memory of basic generators, here's a simple example:: |
| 427 |
|
| 428 |
def counter (maximum): |
| 429 |
i = 0 |
| 430 |
while i < maximum: |
| 431 |
yield i |
| 432 |
i += 1 |
| 433 |
|
| 434 |
When you call ``counter(10)``, the result is an iterator that returns the values |
| 435 |
from 0 up to 9. On encountering the :keyword:`yield` statement, the iterator |
| 436 |
returns the provided value and suspends the function's execution, preserving the |
| 437 |
local variables. Execution resumes on the following call to the iterator's |
| 438 |
:meth:`next` method, picking up after the :keyword:`!yield` statement. |
| 439 |
|
| 440 |
In Python 2.3, :keyword:`yield` was a statement; it didn't return any value. In |
| 441 |
2.5, :keyword:`!yield` is now an expression, returning a value that can be |
| 442 |
assigned to a variable or otherwise operated on:: |
| 443 |
|
| 444 |
val = (yield i) |
| 445 |
|
| 446 |
I recommend that you always put parentheses around a :keyword:`yield` expression |
| 447 |
when you're doing something with the returned value, as in the above example. |
| 448 |
The parentheses aren't always necessary, but it's easier to always add them |
| 449 |
instead of having to remember when they're needed. |
| 450 |
|
| 451 |
(:pep:`342` explains the exact rules, which are that a |
| 452 |
:keyword:`yield`\ -expression must always be parenthesized except when it |
| 453 |
occurs at the top-level |
| 454 |
expression on the right-hand side of an assignment. This means you can write |
| 455 |
``val = yield i`` but have to use parentheses when there's an operation, as in |
| 456 |
``val = (yield i) + 12``.) |
| 457 |
|
| 458 |
Values are sent into a generator by calling its ``send(value)`` method. The |
| 459 |
generator's code is then resumed and the :keyword:`yield` expression returns the |
| 460 |
specified *value*. If the regular :meth:`next` method is called, the |
| 461 |
:keyword:`!yield` returns :const:`None`. |
| 462 |
|
| 463 |
Here's the previous example, modified to allow changing the value of the |
| 464 |
internal counter. :: |
| 465 |
|
| 466 |
def counter (maximum): |
| 467 |
i = 0 |
| 468 |
while i < maximum: |
| 469 |
val = (yield i) |
| 470 |
# If value provided, change counter |
| 471 |
if val is not None: |
| 472 |
i = val |
| 473 |
else: |
| 474 |
i += 1 |
| 475 |
|
| 476 |
And here's an example of changing the counter:: |
| 477 |
|
| 478 |
>>> it = counter(10) |
| 479 |
>>> print it.next() |
| 480 |
0 |
| 481 |
>>> print it.next() |
| 482 |
1 |
| 483 |
>>> print it.send(8) |
| 484 |
8 |
| 485 |
>>> print it.next() |
| 486 |
9 |
| 487 |
>>> print it.next() |
| 488 |
Traceback (most recent call last): |
| 489 |
File "t.py", line 15, in ? |
| 490 |
print it.next() |
| 491 |
StopIteration |
| 492 |
|
| 493 |
:keyword:`yield` will usually return :const:`None`, so you should always check |
| 494 |
for this case. Don't just use its value in expressions unless you're sure that |
| 495 |
the :meth:`send` method will be the only method used to resume your generator |
| 496 |
function. |
| 497 |
|
| 498 |
In addition to :meth:`send`, there are two other new methods on generators: |
| 499 |
|
| 500 |
* ``throw(type, value=None, traceback=None)`` is used to raise an exception |
| 501 |
inside the generator; the exception is raised by the :keyword:`yield` expression |
| 502 |
where the generator's execution is paused. |
| 503 |
|
| 504 |
* :meth:`close` raises a new :exc:`GeneratorExit` exception inside the generator |
| 505 |
to terminate the iteration. On receiving this exception, the generator's code |
| 506 |
must either raise :exc:`GeneratorExit` or :exc:`StopIteration`. Catching the |
| 507 |
:exc:`GeneratorExit` exception and returning a value is illegal and will trigger |
| 508 |
a :exc:`RuntimeError`; if the function raises some other exception, that |
| 509 |
exception is propagated to the caller. :meth:`close` will also be called by |
| 510 |
Python's garbage collector when the generator is garbage-collected. |
| 511 |
|
| 512 |
If you need to run cleanup code when a :exc:`GeneratorExit` occurs, I suggest |
| 513 |
using a ``try: ... finally:`` suite instead of catching :exc:`GeneratorExit`. |
| 514 |
|
| 515 |
The cumulative effect of these changes is to turn generators from one-way |
| 516 |
producers of information into both producers and consumers. |
| 517 |
|
| 518 |
Generators also become *coroutines*, a more generalized form of subroutines. |
| 519 |
Subroutines are entered at one point and exited at another point (the top of the |
| 520 |
function, and a :keyword:`return` statement), but coroutines can be entered, |
| 521 |
exited, and resumed at many different points (the :keyword:`yield` statements). |
| 522 |
We'll have to figure out patterns for using coroutines effectively in Python. |
| 523 |
|
| 524 |
The addition of the :meth:`close` method has one side effect that isn't obvious. |
| 525 |
:meth:`close` is called when a generator is garbage-collected, so this means the |
| 526 |
generator's code gets one last chance to run before the generator is destroyed. |
| 527 |
This last chance means that ``try...finally`` statements in generators can now |
| 528 |
be guaranteed to work; the :keyword:`finally` clause will now always get a |
| 529 |
chance to run. The syntactic restriction that you couldn't mix :keyword:`yield` |
| 530 |
statements with a ``try...finally`` suite has therefore been removed. This |
| 531 |
seems like a minor bit of language trivia, but using generators and |
| 532 |
``try...finally`` is actually necessary in order to implement the |
| 533 |
:keyword:`with` statement described by :pep:`343`. I'll look at this new statement |
| 534 |
in the following section. |
| 535 |
|
| 536 |
Another even more esoteric effect of this change: previously, the |
| 537 |
:attr:`gi_frame` attribute of a generator was always a frame object. It's now |
| 538 |
possible for :attr:`gi_frame` to be ``None`` once the generator has been |
| 539 |
exhausted. |
| 540 |
|
| 541 |
|
| 542 |
.. seealso:: |
| 543 |
|
| 544 |
:pep:`342` - Coroutines via Enhanced Generators |
| 545 |
PEP written by Guido van Rossum and Phillip J. Eby; implemented by Phillip J. |
| 546 |
Eby. Includes examples of some fancier uses of generators as coroutines. |
| 547 |
|
| 548 |
Earlier versions of these features were proposed in :pep:`288` by Raymond |
| 549 |
Hettinger and :pep:`325` by Samuele Pedroni. |
| 550 |
|
| 551 |
https://en.wikipedia.org/wiki/Coroutine |
| 552 |
The Wikipedia entry for coroutines. |
| 553 |
|
| 554 |
https://web.archive.org/web/20160321211320/http://www.sidhe.org/~dan/blog/archives/000178.html |
| 555 |
An explanation of coroutines from a Perl point of view, written by Dan Sugalski. |
| 556 |
|
| 557 |
.. ====================================================================== |
| 558 |
|
| 559 |
|
| 560 |
.. _pep-343: |
| 561 |
|
| 562 |
PEP 343: The 'with' statement |
| 563 |
============================= |
| 564 |
|
| 565 |
The ':keyword:`with`' statement clarifies code that previously would use |
| 566 |
``try...finally`` blocks to ensure that clean-up code is executed. In this |
| 567 |
section, I'll discuss the statement as it will commonly be used. In the next |
| 568 |
section, I'll examine the implementation details and show how to write objects |
| 569 |
for use with this statement. |
| 570 |
|
| 571 |
The ':keyword:`with`' statement is a new control-flow structure whose basic |
| 572 |
structure is:: |
| 573 |
|
| 574 |
with expression [as variable]: |
| 575 |
with-block |
| 576 |
|
| 577 |
The expression is evaluated, and it should result in an object that supports the |
| 578 |
context management protocol (that is, has :meth:`~object.__enter__` and :meth:`~object.__exit__` |
| 579 |
methods. |
| 580 |
|
| 581 |
The object's :meth:`~object.__enter__` is called before *with-block* is executed and |
| 582 |
therefore can run set-up code. It also may return a value that is bound to the |
| 583 |
name *variable*, if given. (Note carefully that *variable* is *not* assigned |
| 584 |
the result of *expression*.) |
| 585 |
|
| 586 |
After execution of the *with-block* is finished, the object's :meth:`~object.__exit__` |
| 587 |
method is called, even if the block raised an exception, and can therefore run |
| 588 |
clean-up code. |
| 589 |
|
| 590 |
To enable the statement in Python 2.5, you need to add the following directive |
| 591 |
to your module:: |
| 592 |
|
| 593 |
from __future__ import with_statement |
| 594 |
|
| 595 |
The statement will always be enabled in Python 2.6. |
| 596 |
|
| 597 |
Some standard Python objects now support the context management protocol and can |
| 598 |
be used with the ':keyword:`with`' statement. File objects are one example:: |
| 599 |
|
| 600 |
with open('/etc/passwd', 'r') as f: |
| 601 |
for line in f: |
| 602 |
print line |
| 603 |
... more processing code ... |
| 604 |
|
| 605 |
After this statement has executed, the file object in *f* will have been |
| 606 |
automatically closed, even if the :keyword:`for` loop raised an exception |
| 607 |
part-way through the block. |
| 608 |
|
| 609 |
.. note:: |
| 610 |
|
| 611 |
In this case, *f* is the same object created by :func:`open`, because |
| 612 |
:meth:`~object.__enter__` returns *self*. |
| 613 |
|
| 614 |
The :mod:`threading` module's locks and condition variables also support the |
| 615 |
':keyword:`with`' statement:: |
| 616 |
|
| 617 |
lock = threading.Lock() |
| 618 |
with lock: |
| 619 |
# Critical section of code |
| 620 |
... |
| 621 |
|
| 622 |
The lock is acquired before the block is executed and always released once the |
| 623 |
block is complete. |
| 624 |
|
| 625 |
The new :func:`localcontext` function in the :mod:`decimal` module makes it easy |
| 626 |
to save and restore the current decimal context, which encapsulates the desired |
| 627 |
precision and rounding characteristics for computations:: |
| 628 |
|
| 629 |
from decimal import Decimal, Context, localcontext |
| 630 |
|
| 631 |
# Displays with default precision of 28 digits |
| 632 |
v = Decimal('578') |
| 633 |
print v.sqrt() |
| 634 |
|
| 635 |
with localcontext(Context(prec=16)): |
| 636 |
# All code in this block uses a precision of 16 digits. |
| 637 |
# The original context is restored on exiting the block. |
| 638 |
print v.sqrt() |
| 639 |
|
| 640 |
|
| 641 |
.. _new-25-context-managers: |
| 642 |
|
| 643 |
Writing Context Managers |
| 644 |
------------------------ |
| 645 |
|
| 646 |
Under the hood, the ':keyword:`with`' statement is fairly complicated. Most |
| 647 |
people will only use ':keyword:`!with`' in company with existing objects and |
| 648 |
don't need to know these details, so you can skip the rest of this section if |
| 649 |
you like. Authors of new objects will need to understand the details of the |
| 650 |
underlying implementation and should keep reading. |
| 651 |
|
| 652 |
A high-level explanation of the context management protocol is: |
| 653 |
|
| 654 |
* The expression is evaluated and should result in an object called a "context |
| 655 |
manager". The context manager must have :meth:`~object.__enter__` and :meth:`~object.__exit__` |
| 656 |
methods. |
| 657 |
|
| 658 |
* The context manager's :meth:`~object.__enter__` method is called. The value returned |
| 659 |
is assigned to *VAR*. If no ``'as VAR'`` clause is present, the value is simply |
| 660 |
discarded. |
| 661 |
|
| 662 |
* The code in *BLOCK* is executed. |
| 663 |
|
| 664 |
* If *BLOCK* raises an exception, the ``__exit__(type, value, traceback)`` |
| 665 |
is called with the exception details, the same values returned by |
| 666 |
:func:`sys.exc_info`. The method's return value controls whether the exception |
| 667 |
is re-raised: any false value re-raises the exception, and ``True`` will result |
| 668 |
in suppressing it. You'll only rarely want to suppress the exception, because |
| 669 |
if you do the author of the code containing the ':keyword:`with`' statement will |
| 670 |
never realize anything went wrong. |
| 671 |
|
| 672 |
* If *BLOCK* didn't raise an exception, the :meth:`~object.__exit__` method is still |
| 673 |
called, but *type*, *value*, and *traceback* are all ``None``. |
| 674 |
|
| 675 |
Let's think through an example. I won't present detailed code but will only |
| 676 |
sketch the methods necessary for a database that supports transactions. |
| 677 |
|
| 678 |
(For people unfamiliar with database terminology: a set of changes to the |
| 679 |
database are grouped into a transaction. Transactions can be either committed, |
| 680 |
meaning that all the changes are written into the database, or rolled back, |
| 681 |
meaning that the changes are all discarded and the database is unchanged. See |
| 682 |
any database textbook for more information.) |
| 683 |
|
| 684 |
Let's assume there's an object representing a database connection. Our goal will |
| 685 |
be to let the user write code like this:: |
| 686 |
|
| 687 |
db_connection = DatabaseConnection() |
| 688 |
with db_connection as cursor: |
| 689 |
cursor.execute('insert into ...') |
| 690 |
cursor.execute('delete from ...') |
| 691 |
# ... more operations ... |
| 692 |
|
| 693 |
The transaction should be committed if the code in the block runs flawlessly or |
| 694 |
rolled back if there's an exception. Here's the basic interface for |
| 695 |
:class:`DatabaseConnection` that I'll assume:: |
| 696 |
|
| 697 |
class DatabaseConnection: |
| 698 |
# Database interface |
| 699 |
def cursor (self): |
| 700 |
"Returns a cursor object and starts a new transaction" |
| 701 |
def commit (self): |
| 702 |
"Commits current transaction" |
| 703 |
def rollback (self): |
| 704 |
"Rolls back current transaction" |
| 705 |
|
| 706 |
The :meth:`~object.__enter__` method is pretty easy, having only to start a new |
| 707 |
transaction. For this application the resulting cursor object would be a useful |
| 708 |
result, so the method will return it. The user can then add ``as cursor`` to |
| 709 |
their ':keyword:`with`' statement to bind the cursor to a variable name. :: |
| 710 |
|
| 711 |
class DatabaseConnection: |
| 712 |
... |
| 713 |
def __enter__ (self): |
| 714 |
# Code to start a new transaction |
| 715 |
cursor = self.cursor() |
| 716 |
return cursor |
| 717 |
|
| 718 |
The :meth:`~object.__exit__` method is the most complicated because it's where most of |
| 719 |
the work has to be done. The method has to check if an exception occurred. If |
| 720 |
there was no exception, the transaction is committed. The transaction is rolled |
| 721 |
back if there was an exception. |
| 722 |
|
| 723 |
In the code below, execution will just fall off the end of the function, |
| 724 |
returning the default value of ``None``. ``None`` is false, so the exception |
| 725 |
will be re-raised automatically. If you wished, you could be more explicit and |
| 726 |
add a :keyword:`return` statement at the marked location. :: |
| 727 |
|
| 728 |
class DatabaseConnection: |
| 729 |
... |
| 730 |
def __exit__ (self, type, value, tb): |
| 731 |
if tb is None: |
| 732 |
# No exception, so commit |
| 733 |
self.commit() |
| 734 |
else: |
| 735 |
# Exception occurred, so rollback. |
| 736 |
self.rollback() |
| 737 |
# return False |
| 738 |
|
| 739 |
|
| 740 |
.. _contextlibmod: |
| 741 |
|
| 742 |
The contextlib module |
| 743 |
--------------------- |
| 744 |
|
| 745 |
The new :mod:`contextlib` module provides some functions and a decorator that |
| 746 |
are useful for writing objects for use with the ':keyword:`with`' statement. |
| 747 |
|
| 748 |
The decorator is called :func:`contextmanager`, and lets you write a single |
| 749 |
generator function instead of defining a new class. The generator should yield |
| 750 |
exactly one value. The code up to the :keyword:`yield` will be executed as the |
| 751 |
:meth:`~object.__enter__` method, and the value yielded will be the method's return |
| 752 |
value that will get bound to the variable in the ':keyword:`with`' statement's |
| 753 |
:keyword:`!as` clause, if any. The code after the :keyword:`yield` will be |
| 754 |
executed in the :meth:`~object.__exit__` method. Any exception raised in the block will |
| 755 |
be raised by the :keyword:`!yield` statement. |
| 756 |
|
| 757 |
Our database example from the previous section could be written using this |
| 758 |
decorator as:: |
| 759 |
|
| 760 |
from contextlib import contextmanager |
| 761 |
|
| 762 |
@contextmanager |
| 763 |
def db_transaction (connection): |
| 764 |
cursor = connection.cursor() |
| 765 |
try: |
| 766 |
yield cursor |
| 767 |
except: |
| 768 |
connection.rollback() |
| 769 |
raise |
| 770 |
else: |
| 771 |
connection.commit() |
| 772 |
|
| 773 |
db = DatabaseConnection() |
| 774 |
with db_transaction(db) as cursor: |
| 775 |
... |
| 776 |
|
| 777 |
The :mod:`contextlib` module also has a ``nested(mgr1, mgr2, ...)`` function |
| 778 |
that combines a number of context managers so you don't need to write nested |
| 779 |
':keyword:`with`' statements. In this example, the single ':keyword:`!with`' |
| 780 |
statement both starts a database transaction and acquires a thread lock:: |
| 781 |
|
| 782 |
lock = threading.Lock() |
| 783 |
with nested (db_transaction(db), lock) as (cursor, locked): |
| 784 |
... |
| 785 |
|
| 786 |
Finally, the ``closing(object)`` function returns *object* so that it can be |
| 787 |
bound to a variable, and calls ``object.close`` at the end of the block. :: |
| 788 |
|
| 789 |
import urllib, sys |
| 790 |
from contextlib import closing |
| 791 |
|
| 792 |
with closing(urllib.urlopen('http://www.yahoo.com')) as f: |
| 793 |
for line in f: |
| 794 |
sys.stdout.write(line) |
| 795 |
|
| 796 |
|
| 797 |
.. seealso:: |
| 798 |
|
| 799 |
:pep:`343` - The "with" statement |
| 800 |
PEP written by Guido van Rossum and Nick Coghlan; implemented by Mike Bland, |
| 801 |
Guido van Rossum, and Neal Norwitz. The PEP shows the code generated for a |
| 802 |
':keyword:`with`' statement, which can be helpful in learning how the statement |
| 803 |
works. |
| 804 |
|
| 805 |
The documentation for the :mod:`contextlib` module. |
| 806 |
|
| 807 |
.. ====================================================================== |
| 808 |
|
| 809 |
|
| 810 |
.. _pep-352: |
| 811 |
|
| 812 |
PEP 352: Exceptions as New-Style Classes |
| 813 |
======================================== |
| 814 |
|
| 815 |
Exception classes can now be new-style classes, not just classic classes, and |
| 816 |
the built-in :exc:`Exception` class and all the standard built-in exceptions |
| 817 |
(:exc:`NameError`, :exc:`ValueError`, etc.) are now new-style classes. |
| 818 |
|
| 819 |
The inheritance hierarchy for exceptions has been rearranged a bit. In 2.5, the |
| 820 |
inheritance relationships are:: |
| 821 |
|
| 822 |
BaseException # New in Python 2.5 |
| 823 |
|- KeyboardInterrupt |
| 824 |
|- SystemExit |
| 825 |
|- Exception |
| 826 |
|- (all other current built-in exceptions) |
| 827 |
|
| 828 |
This rearrangement was done because people often want to catch all exceptions |
| 829 |
that indicate program errors. :exc:`KeyboardInterrupt` and :exc:`SystemExit` |
| 830 |
aren't errors, though, and usually represent an explicit action such as the user |
| 831 |
hitting :kbd:`Control-C` or code calling :func:`sys.exit`. A bare ``except:`` will |
| 832 |
catch all exceptions, so you commonly need to list :exc:`KeyboardInterrupt` and |
| 833 |
:exc:`SystemExit` in order to re-raise them. The usual pattern is:: |
| 834 |
|
| 835 |
try: |
| 836 |
... |
| 837 |
except (KeyboardInterrupt, SystemExit): |
| 838 |
raise |
| 839 |
except: |
| 840 |
# Log error... |
| 841 |
# Continue running program... |
| 842 |
|
| 843 |
In Python 2.5, you can now write ``except Exception`` to achieve the same |
| 844 |
result, catching all the exceptions that usually indicate errors but leaving |
| 845 |
:exc:`KeyboardInterrupt` and :exc:`SystemExit` alone. As in previous versions, |
| 846 |
a bare ``except:`` still catches all exceptions. |
| 847 |
|
| 848 |
The goal for Python 3.0 is to require any class raised as an exception to derive |
| 849 |
from :exc:`BaseException` or some descendant of :exc:`BaseException`, and future |
| 850 |
releases in the Python 2.x series may begin to enforce this constraint. |
| 851 |
Therefore, I suggest you begin making all your exception classes derive from |
| 852 |
:exc:`Exception` now. It's been suggested that the bare ``except:`` form should |
| 853 |
be removed in Python 3.0, but Guido van Rossum hasn't decided whether to do this |
| 854 |
or not. |
| 855 |
|
| 856 |
Raising of strings as exceptions, as in the statement ``raise "Error |
| 857 |
occurred"``, is deprecated in Python 2.5 and will trigger a warning. The aim is |
| 858 |
to be able to remove the string-exception feature in a few releases. |
| 859 |
|
| 860 |
|
| 861 |
.. seealso:: |
| 862 |
|
| 863 |
:pep:`352` - Required Superclass for Exceptions |
| 864 |
PEP written by Brett Cannon and Guido van Rossum; implemented by Brett Cannon. |
| 865 |
|
| 866 |
.. ====================================================================== |
| 867 |
|
| 868 |
|
| 869 |
.. _pep-353: |
| 870 |
|
| 871 |
PEP 353: Using ssize_t as the index type |
| 872 |
======================================== |
| 873 |
|
| 874 |
A wide-ranging change to Python's C API, using a new :c:type:`Py_ssize_t` type |
| 875 |
definition instead of :c:expr:`int`, will permit the interpreter to handle more |
| 876 |
data on 64-bit platforms. This change doesn't affect Python's capacity on 32-bit |
| 877 |
platforms. |
| 878 |
|
| 879 |
Various pieces of the Python interpreter used C's :c:expr:`int` type to store |
| 880 |
sizes or counts; for example, the number of items in a list or tuple were stored |
| 881 |
in an :c:expr:`int`. The C compilers for most 64-bit platforms still define |
| 882 |
:c:expr:`int` as a 32-bit type, so that meant that lists could only hold up to |
| 883 |
``2**31 - 1`` = 2147483647 items. (There are actually a few different |
| 884 |
programming models that 64-bit C compilers can use -- see |
| 885 |
https://unix.org/version2/whatsnew/lp64_wp.html for a discussion -- but the |
| 886 |
most commonly available model leaves :c:expr:`int` as 32 bits.) |
| 887 |
|
| 888 |
A limit of 2147483647 items doesn't really matter on a 32-bit platform because |
| 889 |
you'll run out of memory before hitting the length limit. Each list item |
| 890 |
requires space for a pointer, which is 4 bytes, plus space for a |
| 891 |
:c:type:`PyObject` representing the item. 2147483647\*4 is already more bytes |
| 892 |
than a 32-bit address space can contain. |
| 893 |
|
| 894 |
It's possible to address that much memory on a 64-bit platform, however. The |
| 895 |
pointers for a list that size would only require 16 GiB of space, so it's not |
| 896 |
unreasonable that Python programmers might construct lists that large. |
| 897 |
Therefore, the Python interpreter had to be changed to use some type other than |
| 898 |
:c:expr:`int`, and this will be a 64-bit type on 64-bit platforms. The change |
| 899 |
will cause incompatibilities on 64-bit machines, so it was deemed worth making |
| 900 |
the transition now, while the number of 64-bit users is still relatively small. |
| 901 |
(In 5 or 10 years, we may *all* be on 64-bit machines, and the transition would |
| 902 |
be more painful then.) |
| 903 |
|
| 904 |
This change most strongly affects authors of C extension modules. Python |
| 905 |
strings and container types such as lists and tuples now use |
| 906 |
:c:type:`Py_ssize_t` to store their size. Functions such as |
| 907 |
:c:func:`PyList_Size` now return :c:type:`Py_ssize_t`. Code in extension modules |
| 908 |
may therefore need to have some variables changed to :c:type:`Py_ssize_t`. |
| 909 |
|
| 910 |
The :c:func:`PyArg_ParseTuple` and :c:func:`Py_BuildValue` functions have a new |
| 911 |
conversion code, ``n``, for :c:type:`Py_ssize_t`. :c:func:`PyArg_ParseTuple`'s |
| 912 |
``s#`` and ``t#`` still output :c:expr:`int` by default, but you can define the |
| 913 |
macro :c:macro:`PY_SSIZE_T_CLEAN` before including :file:`Python.h` to make |
| 914 |
them return :c:type:`Py_ssize_t`. |
| 915 |
|
| 916 |
:pep:`353` has a section on conversion guidelines that extension authors should |
| 917 |
read to learn about supporting 64-bit platforms. |
| 918 |
|
| 919 |
|
| 920 |
.. seealso:: |
| 921 |
|
| 922 |
:pep:`353` - Using ssize_t as the index type |
| 923 |
PEP written and implemented by Martin von Löwis. |
| 924 |
|
| 925 |
.. ====================================================================== |
| 926 |
|
| 927 |
|
| 928 |
.. _pep-357: |
| 929 |
|
| 930 |
PEP 357: The '__index__' method |
| 931 |
=============================== |
| 932 |
|
| 933 |
The NumPy developers had a problem that could only be solved by adding a new |
| 934 |
special method, :meth:`__index__`. When using slice notation, as in |
| 935 |
``[start:stop:step]``, the values of the *start*, *stop*, and *step* indexes |
| 936 |
must all be either integers or long integers. NumPy defines a variety of |
| 937 |
specialized integer types corresponding to unsigned and signed integers of 8, |
| 938 |
16, 32, and 64 bits, but there was no way to signal that these types could be |
| 939 |
used as slice indexes. |
| 940 |
|
| 941 |
Slicing can't just use the existing :meth:`__int__` method because that method |
| 942 |
is also used to implement coercion to integers. If slicing used |
| 943 |
:meth:`__int__`, floating-point numbers would also become legal slice indexes |
| 944 |
and that's clearly an undesirable behaviour. |
| 945 |
|
| 946 |
Instead, a new special method called :meth:`__index__` was added. It takes no |
| 947 |
arguments and returns an integer giving the slice index to use. For example:: |
| 948 |
|
| 949 |
class C: |
| 950 |
def __index__ (self): |
| 951 |
return self.value |
| 952 |
|
| 953 |
The return value must be either a Python integer or long integer. The |
| 954 |
interpreter will check that the type returned is correct, and raises a |
| 955 |
:exc:`TypeError` if this requirement isn't met. |
| 956 |
|
| 957 |
A corresponding :c:member:`~PyNumberMethods.nb_index` slot was added to the C-level |
| 958 |
:c:type:`PyNumberMethods` structure to let C extensions implement this protocol. |
| 959 |
``PyNumber_Index(obj)`` can be used in extension code to call the |
| 960 |
:meth:`__index__` function and retrieve its result. |
| 961 |
|
| 962 |
|
| 963 |
.. seealso:: |
| 964 |
|
| 965 |
:pep:`357` - Allowing Any Object to be Used for Slicing |
| 966 |
PEP written and implemented by Travis Oliphant. |
| 967 |
|
| 968 |
.. ====================================================================== |
| 969 |
|
| 970 |
|
| 971 |
.. _other-lang: |
| 972 |
|
| 973 |
Other Language Changes |
| 974 |
====================== |
| 975 |
|
| 976 |
Here are all of the changes that Python 2.5 makes to the core Python language. |
| 977 |
|
| 978 |
* The :class:`dict` type has a new hook for letting subclasses provide a default |
| 979 |
value when a key isn't contained in the dictionary. When a key isn't found, the |
| 980 |
dictionary's ``__missing__(key)`` method will be called. This hook is used |
| 981 |
to implement the new :class:`defaultdict` class in the :mod:`collections` |
| 982 |
module. The following example defines a dictionary that returns zero for any |
| 983 |
missing key:: |
| 984 |
|
| 985 |
class zerodict (dict): |
| 986 |
def __missing__ (self, key): |
| 987 |
return 0 |
| 988 |
|
| 989 |
d = zerodict({1:1, 2:2}) |
| 990 |
print d[1], d[2] # Prints 1, 2 |
| 991 |
print d[3], d[4] # Prints 0, 0 |
| 992 |
|
| 993 |
* Both 8-bit and Unicode strings have new ``partition(sep)`` and |
| 994 |
``rpartition(sep)`` methods that simplify a common use case. |
| 995 |
|
| 996 |
The ``find(S)`` method is often used to get an index which is then used to |
| 997 |
slice the string and obtain the pieces that are before and after the separator. |
| 998 |
``partition(sep)`` condenses this pattern into a single method call that |
| 999 |
returns a 3-tuple containing the substring before the separator, the separator |
| 1000 |
itself, and the substring after the separator. If the separator isn't found, |
| 1001 |
the first element of the tuple is the entire string and the other two elements |
| 1002 |
are empty. ``rpartition(sep)`` also returns a 3-tuple but starts searching |
| 1003 |
from the end of the string; the ``r`` stands for 'reverse'. |
| 1004 |
|
| 1005 |
Some examples:: |
| 1006 |
|
| 1007 |
>>> ('http://www.python.org').partition('://') |
| 1008 |
('http', '://', 'www.python.org') |
| 1009 |
>>> ('file:/usr/share/doc/index.html').partition('://') |
| 1010 |
('file:/usr/share/doc/index.html', '', '') |
| 1011 |
>>> (u'Subject: a quick question').partition(':') |
| 1012 |
(u'Subject', u':', u' a quick question') |
| 1013 |
>>> 'www.python.org'.rpartition('.') |
| 1014 |
('www.python', '.', 'org') |
| 1015 |
>>> 'www.python.org'.rpartition(':') |
| 1016 |
('', '', 'www.python.org') |
| 1017 |
|
| 1018 |
(Implemented by Fredrik Lundh following a suggestion by Raymond Hettinger.) |
| 1019 |
|
| 1020 |
* The :meth:`startswith` and :meth:`endswith` methods of string types now accept |
| 1021 |
tuples of strings to check for. :: |
| 1022 |
|
| 1023 |
def is_image_file (filename): |
| 1024 |
return filename.endswith(('.gif', '.jpg', '.tiff')) |
| 1025 |
|
| 1026 |
(Implemented by Georg Brandl following a suggestion by Tom Lynn.) |
| 1027 |
|
| 1028 |
.. RFE #1491485 |
| 1029 |
|
| 1030 |
* The :func:`min` and :func:`max` built-in functions gained a ``key`` keyword |
| 1031 |
parameter analogous to the ``key`` argument for :meth:`sort`. This parameter |
| 1032 |
supplies a function that takes a single argument and is called for every value |
| 1033 |
in the list; :func:`min`/:func:`max` will return the element with the |
| 1034 |
smallest/largest return value from this function. For example, to find the |
| 1035 |
longest string in a list, you can do:: |
| 1036 |
|
| 1037 |
L = ['medium', 'longest', 'short'] |
| 1038 |
# Prints 'longest' |
| 1039 |
print max(L, key=len) |
| 1040 |
# Prints 'short', because lexicographically 'short' has the largest value |
| 1041 |
print max(L) |
| 1042 |
|
| 1043 |
(Contributed by Steven Bethard and Raymond Hettinger.) |
| 1044 |
|
| 1045 |
* Two new built-in functions, :func:`any` and :func:`all`, evaluate whether an |
| 1046 |
iterator contains any true or false values. :func:`any` returns :const:`True` |
| 1047 |
if any value returned by the iterator is true; otherwise it will return |
| 1048 |
:const:`False`. :func:`all` returns :const:`True` only if all of the values |
| 1049 |
returned by the iterator evaluate as true. (Suggested by Guido van Rossum, and |
| 1050 |
implemented by Raymond Hettinger.) |
| 1051 |
|
| 1052 |
* The result of a class's :meth:`__hash__` method can now be either a long |
| 1053 |
integer or a regular integer. If a long integer is returned, the hash of that |
| 1054 |
value is taken. In earlier versions the hash value was required to be a |
| 1055 |
regular integer, but in 2.5 the :func:`id` built-in was changed to always |
| 1056 |
return non-negative numbers, and users often seem to use ``id(self)`` in |
| 1057 |
:meth:`__hash__` methods (though this is discouraged). |
| 1058 |
|
| 1059 |
.. Bug #1536021 |
| 1060 |
|
| 1061 |
* ASCII is now the default encoding for modules. It's now a syntax error if a |
| 1062 |
module contains string literals with 8-bit characters but doesn't have an |
| 1063 |
encoding declaration. In Python 2.4 this triggered a warning, not a syntax |
| 1064 |
error. See :pep:`263` for how to declare a module's encoding; for example, you |
| 1065 |
might add a line like this near the top of the source file:: |
| 1066 |
|
| 1067 |
# -*- coding: latin1 -*- |
| 1068 |
|
| 1069 |
* A new warning, :class:`UnicodeWarning`, is triggered when you attempt to |
| 1070 |
compare a Unicode string and an 8-bit string that can't be converted to Unicode |
| 1071 |
using the default ASCII encoding. The result of the comparison is false:: |
| 1072 |
|
| 1073 |
>>> chr(128) == unichr(128) # Can't convert chr(128) to Unicode |
| 1074 |
__main__:1: UnicodeWarning: Unicode equal comparison failed |
| 1075 |
to convert both arguments to Unicode - interpreting them |
| 1076 |
as being unequal |
| 1077 |
False |
| 1078 |
>>> chr(127) == unichr(127) # chr(127) can be converted |
| 1079 |
True |
| 1080 |
|
| 1081 |
Previously this would raise a :class:`UnicodeDecodeError` exception, but in 2.5 |
| 1082 |
this could result in puzzling problems when accessing a dictionary. If you |
| 1083 |
looked up ``unichr(128)`` and ``chr(128)`` was being used as a key, you'd get a |
| 1084 |
:class:`UnicodeDecodeError` exception. Other changes in 2.5 resulted in this |
| 1085 |
exception being raised instead of suppressed by the code in :file:`dictobject.c` |
| 1086 |
that implements dictionaries. |
| 1087 |
|
| 1088 |
Raising an exception for such a comparison is strictly correct, but the change |
| 1089 |
might have broken code, so instead :class:`UnicodeWarning` was introduced. |
| 1090 |
|
| 1091 |
(Implemented by Marc-André Lemburg.) |
| 1092 |
|
| 1093 |
* One error that Python programmers sometimes make is forgetting to include an |
| 1094 |
:file:`__init__.py` module in a package directory. Debugging this mistake can be |
| 1095 |
confusing, and usually requires running Python with the :option:`-v` switch to |
| 1096 |
log all the paths searched. In Python 2.5, a new :exc:`ImportWarning` warning is |
| 1097 |
triggered when an import would have picked up a directory as a package but no |
| 1098 |
:file:`__init__.py` was found. This warning is silently ignored by default; |
| 1099 |
provide the :option:`-Wd <-W>` option when running the Python executable to display |
| 1100 |
the warning message. (Implemented by Thomas Wouters.) |
| 1101 |
|
| 1102 |
* The list of base classes in a class definition can now be empty. As an |
| 1103 |
example, this is now legal:: |
| 1104 |
|
| 1105 |
class C(): |
| 1106 |
pass |
| 1107 |
|
| 1108 |
(Implemented by Brett Cannon.) |
| 1109 |
|
| 1110 |
.. ====================================================================== |
| 1111 |
|
| 1112 |
|
| 1113 |
.. _25interactive: |
| 1114 |
|
| 1115 |
Interactive Interpreter Changes |
| 1116 |
------------------------------- |
| 1117 |
|
| 1118 |
In the interactive interpreter, ``quit`` and ``exit`` have long been strings so |
| 1119 |
that new users get a somewhat helpful message when they try to quit:: |
| 1120 |
|
| 1121 |
>>> quit |
| 1122 |
'Use Ctrl-D (i.e. EOF) to exit.' |
| 1123 |
|
| 1124 |
In Python 2.5, ``quit`` and ``exit`` are now objects that still produce string |
| 1125 |
representations of themselves, but are also callable. Newbies who try ``quit()`` |
| 1126 |
or ``exit()`` will now exit the interpreter as they expect. (Implemented by |
| 1127 |
Georg Brandl.) |
| 1128 |
|
| 1129 |
The Python executable now accepts the standard long options :option:`--help` |
| 1130 |
and :option:`--version`; on Windows, it also accepts the :option:`/? <-?>` option |
| 1131 |
for displaying a help message. (Implemented by Georg Brandl.) |
| 1132 |
|
| 1133 |
.. ====================================================================== |
| 1134 |
|
| 1135 |
|
| 1136 |
.. _opts: |
| 1137 |
|
| 1138 |
Optimizations |
| 1139 |
------------- |
| 1140 |
|
| 1141 |
Several of the optimizations were developed at the NeedForSpeed sprint, an event |
| 1142 |
held in Reykjavik, Iceland, from May 21--28 2006. The sprint focused on speed |
| 1143 |
enhancements to the CPython implementation and was funded by EWT LLC with local |
| 1144 |
support from CCP Games. Those optimizations added at this sprint are specially |
| 1145 |
marked in the following list. |
| 1146 |
|
| 1147 |
* When they were introduced in Python 2.4, the built-in :class:`set` and |
| 1148 |
:class:`frozenset` types were built on top of Python's dictionary type. In 2.5 |
| 1149 |
the internal data structure has been customized for implementing sets, and as a |
| 1150 |
result sets will use a third less memory and are somewhat faster. (Implemented |
| 1151 |
by Raymond Hettinger.) |
| 1152 |
|
| 1153 |
* The speed of some Unicode operations, such as finding substrings, string |
| 1154 |
splitting, and character map encoding and decoding, has been improved. |
| 1155 |
(Substring search and splitting improvements were added by Fredrik Lundh and |
| 1156 |
Andrew Dalke at the NeedForSpeed sprint. Character maps were improved by Walter |
| 1157 |
Dörwald and Martin von Löwis.) |
| 1158 |
|
| 1159 |
.. Patch 1313939, 1359618 |
| 1160 |
|
| 1161 |
* The ``long(str, base)`` function is now faster on long digit strings |
| 1162 |
because fewer intermediate results are calculated. The peak is for strings of |
| 1163 |
around 800--1000 digits where the function is 6 times faster. (Contributed by |
| 1164 |
Alan McIntyre and committed at the NeedForSpeed sprint.) |
| 1165 |
|
| 1166 |
.. Patch 1442927 |
| 1167 |
|
| 1168 |
* It's now illegal to mix iterating over a file with ``for line in file`` and |
| 1169 |
calling the file object's :meth:`read`/:meth:`readline`/:meth:`readlines` |
| 1170 |
methods. Iteration uses an internal buffer and the :meth:`!read\*` methods |
| 1171 |
don't use that buffer. Instead they would return the data following the |
| 1172 |
buffer, causing the data to appear out of order. Mixing iteration and these |
| 1173 |
methods will now trigger a :exc:`ValueError` from the :meth:`!read\*` method. |
| 1174 |
(Implemented by Thomas Wouters.) |
| 1175 |
|
| 1176 |
.. Patch 1397960 |
| 1177 |
|
| 1178 |
* The :mod:`struct` module now compiles structure format strings into an |
| 1179 |
internal representation and caches this representation, yielding a 20% speedup. |
| 1180 |
(Contributed by Bob Ippolito at the NeedForSpeed sprint.) |
| 1181 |
|
| 1182 |
* The :mod:`re` module got a 1 or 2% speedup by switching to Python's allocator |
| 1183 |
functions instead of the system's :c:func:`malloc` and :c:func:`free`. |
| 1184 |
(Contributed by Jack Diederich at the NeedForSpeed sprint.) |
| 1185 |
|
| 1186 |
* The code generator's peephole optimizer now performs simple constant folding |
| 1187 |
in expressions. If you write something like ``a = 2+3``, the code generator |
| 1188 |
will do the arithmetic and produce code corresponding to ``a = 5``. (Proposed |
| 1189 |
and implemented by Raymond Hettinger.) |
| 1190 |
|
| 1191 |
* Function calls are now faster because code objects now keep the most recently |
| 1192 |
finished frame (a "zombie frame") in an internal field of the code object, |
| 1193 |
reusing it the next time the code object is invoked. (Original patch by Michael |
| 1194 |
Hudson, modified by Armin Rigo and Richard Jones; committed at the NeedForSpeed |
| 1195 |
sprint.) Frame objects are also slightly smaller, which may improve cache |
| 1196 |
locality and reduce memory usage a bit. (Contributed by Neal Norwitz.) |
| 1197 |
|
| 1198 |
.. Patch 876206 |
| 1199 |
.. Patch 1337051 |
| 1200 |
|
| 1201 |
* Python's built-in exceptions are now new-style classes, a change that speeds |
| 1202 |
up instantiation considerably. Exception handling in Python 2.5 is therefore |
| 1203 |
about 30% faster than in 2.4. (Contributed by Richard Jones, Georg Brandl and |
| 1204 |
Sean Reifschneider at the NeedForSpeed sprint.) |
| 1205 |
|
| 1206 |
* Importing now caches the paths tried, recording whether they exist or not so |
| 1207 |
that the interpreter makes fewer :c:func:`open` and :c:func:`stat` calls on |
| 1208 |
startup. (Contributed by Martin von Löwis and Georg Brandl.) |
| 1209 |
|
| 1210 |
.. Patch 921466 |
| 1211 |
|
| 1212 |
.. ====================================================================== |
| 1213 |
|
| 1214 |
|
| 1215 |
.. _25modules: |
| 1216 |
|
| 1217 |
New, Improved, and Removed Modules |
| 1218 |
================================== |
| 1219 |
|
| 1220 |
The standard library received many enhancements and bug fixes in Python 2.5. |
| 1221 |
Here's a partial list of the most notable changes, sorted alphabetically by |
| 1222 |
module name. Consult the :file:`Misc/NEWS` file in the source tree for a more |
| 1223 |
complete list of changes, or look through the SVN logs for all the details. |
| 1224 |
|
| 1225 |
* The :mod:`!audioop` module now supports the a-LAW encoding, and the code for |
| 1226 |
u-LAW encoding has been improved. (Contributed by Lars Immisch.) |
| 1227 |
|
| 1228 |
* The :mod:`codecs` module gained support for incremental codecs. The |
| 1229 |
:func:`codec.lookup` function now returns a :class:`CodecInfo` instance instead |
| 1230 |
of a tuple. :class:`CodecInfo` instances behave like a 4-tuple to preserve |
| 1231 |
backward compatibility but also have the attributes :attr:`encode`, |
| 1232 |
:attr:`decode`, :attr:`incrementalencoder`, :attr:`incrementaldecoder`, |
| 1233 |
:attr:`streamwriter`, and :attr:`streamreader`. Incremental codecs can receive |
| 1234 |
input and produce output in multiple chunks; the output is the same as if the |
| 1235 |
entire input was fed to the non-incremental codec. See the :mod:`codecs` module |
| 1236 |
documentation for details. (Designed and implemented by Walter Dörwald.) |
| 1237 |
|
| 1238 |
.. Patch 1436130 |
| 1239 |
|
| 1240 |
* The :mod:`collections` module gained a new type, :class:`defaultdict`, that |
| 1241 |
subclasses the standard :class:`dict` type. The new type mostly behaves like a |
| 1242 |
dictionary but constructs a default value when a key isn't present, |
| 1243 |
automatically adding it to the dictionary for the requested key value. |
| 1244 |
|
| 1245 |
The first argument to :class:`defaultdict`'s constructor is a factory function |
| 1246 |
that gets called whenever a key is requested but not found. This factory |
| 1247 |
function receives no arguments, so you can use built-in type constructors such |
| 1248 |
as :func:`list` or :func:`int`. For example, you can make an index of words |
| 1249 |
based on their initial letter like this:: |
| 1250 |
|
| 1251 |
words = """Nel mezzo del cammin di nostra vita |
| 1252 |
mi ritrovai per una selva oscura |
| 1253 |
che la diritta via era smarrita""".lower().split() |
| 1254 |
|
| 1255 |
index = defaultdict(list) |
| 1256 |
|
| 1257 |
for w in words: |
| 1258 |
init_letter = w[0] |
| 1259 |
index[init_letter].append(w) |
| 1260 |
|
| 1261 |
Printing ``index`` results in the following output:: |
| 1262 |
|
| 1263 |
defaultdict(<type 'list'>, {'c': ['cammin', 'che'], 'e': ['era'], |
| 1264 |
'd': ['del', 'di', 'diritta'], 'm': ['mezzo', 'mi'], |
| 1265 |
'l': ['la'], 'o': ['oscura'], 'n': ['nel', 'nostra'], |
| 1266 |
'p': ['per'], 's': ['selva', 'smarrita'], |
| 1267 |
'r': ['ritrovai'], 'u': ['una'], 'v': ['vita', 'via']} |
| 1268 |
|
| 1269 |
(Contributed by Guido van Rossum.) |
| 1270 |
|
| 1271 |
* The :class:`deque` double-ended queue type supplied by the :mod:`collections` |
| 1272 |
module now has a ``remove(value)`` method that removes the first occurrence |
| 1273 |
of *value* in the queue, raising :exc:`ValueError` if the value isn't found. |
| 1274 |
(Contributed by Raymond Hettinger.) |
| 1275 |
|
| 1276 |
* New module: The :mod:`contextlib` module contains helper functions for use |
| 1277 |
with the new ':keyword:`with`' statement. See section :ref:`contextlibmod` |
| 1278 |
for more about this module. |
| 1279 |
|
| 1280 |
* New module: The :mod:`cProfile` module is a C implementation of the existing |
| 1281 |
:mod:`profile` module that has much lower overhead. The module's interface is |
| 1282 |
the same as :mod:`profile`: you run ``cProfile.run('main()')`` to profile a |
| 1283 |
function, can save profile data to a file, etc. It's not yet known if the |
| 1284 |
Hotshot profiler, which is also written in C but doesn't match the |
| 1285 |
:mod:`profile` module's interface, will continue to be maintained in future |
| 1286 |
versions of Python. (Contributed by Armin Rigo.) |
| 1287 |
|
| 1288 |
Also, the :mod:`pstats` module for analyzing the data measured by the profiler |
| 1289 |
now supports directing the output to any file object by supplying a *stream* |
| 1290 |
argument to the :class:`Stats` constructor. (Contributed by Skip Montanaro.) |
| 1291 |
|
| 1292 |
* The :mod:`csv` module, which parses files in comma-separated value format, |
| 1293 |
received several enhancements and a number of bugfixes. You can now set the |
| 1294 |
maximum size in bytes of a field by calling the |
| 1295 |
``csv.field_size_limit(new_limit)`` function; omitting the *new_limit* |
| 1296 |
argument will return the currently set limit. The :class:`reader` class now has |
| 1297 |
a :attr:`line_num` attribute that counts the number of physical lines read from |
| 1298 |
the source; records can span multiple physical lines, so :attr:`line_num` is not |
| 1299 |
the same as the number of records read. |
| 1300 |
|
| 1301 |
The CSV parser is now stricter about multi-line quoted fields. Previously, if a |
| 1302 |
line ended within a quoted field without a terminating newline character, a |
| 1303 |
newline would be inserted into the returned field. This behavior caused problems |
| 1304 |
when reading files that contained carriage return characters within fields, so |
| 1305 |
the code was changed to return the field without inserting newlines. As a |
| 1306 |
consequence, if newlines embedded within fields are important, the input should |
| 1307 |
be split into lines in a manner that preserves the newline characters. |
| 1308 |
|
| 1309 |
(Contributed by Skip Montanaro and Andrew McNamara.) |
| 1310 |
|
| 1311 |
* The :class:`~datetime.datetime` class in the :mod:`datetime` module now has a |
| 1312 |
``strptime(string, format)`` method for parsing date strings, contributed |
| 1313 |
by Josh Spoerri. It uses the same format characters as :func:`time.strptime` and |
| 1314 |
:func:`time.strftime`:: |
| 1315 |
|
| 1316 |
import datetime as dt |
| 1317 |
|
| 1318 |
ts = dt.datetime.strptime('10:13:15 2006-03-07', |
| 1319 |
'%H:%M:%S %Y-%m-%d') |
| 1320 |
|
| 1321 |
* The :meth:`SequenceMatcher.get_matching_blocks` method in the :mod:`difflib` |
| 1322 |
module now guarantees to return a minimal list of blocks describing matching |
| 1323 |
subsequences. Previously, the algorithm would occasionally break a block of |
| 1324 |
matching elements into two list entries. (Enhancement by Tim Peters.) |
| 1325 |
|
| 1326 |
* The :mod:`doctest` module gained a ``SKIP`` option that keeps an example from |
| 1327 |
being executed at all. This is intended for code snippets that are usage |
| 1328 |
examples intended for the reader and aren't actually test cases. |
| 1329 |
|
| 1330 |
An *encoding* parameter was added to the :func:`testfile` function and the |
| 1331 |
:class:`DocFileSuite` class to specify the file's encoding. This makes it |
| 1332 |
easier to use non-ASCII characters in tests contained within a docstring. |
| 1333 |
(Contributed by Bjorn Tillenius.) |
| 1334 |
|
| 1335 |
.. Patch 1080727 |
| 1336 |
|
| 1337 |
* The :mod:`email` package has been updated to version 4.0. (Contributed by |
| 1338 |
Barry Warsaw.) |
| 1339 |
|
| 1340 |
.. XXX need to provide some more detail here |
| 1341 |
|
| 1342 |
.. index:: |
| 1343 |
single: universal newlines; What's new |
| 1344 |
|
| 1345 |
* The :mod:`fileinput` module was made more flexible. Unicode filenames are now |
| 1346 |
supported, and a *mode* parameter that defaults to ``"r"`` was added to the |
| 1347 |
:func:`input` function to allow opening files in binary or :term:`universal |
| 1348 |
newlines` mode. Another new parameter, *openhook*, lets you use a function |
| 1349 |
other than :func:`open` to open the input files. Once you're iterating over |
| 1350 |
the set of files, the :class:`FileInput` object's new :meth:`~fileinput.fileno` returns |
| 1351 |
the file descriptor for the currently opened file. (Contributed by Georg |
| 1352 |
Brandl.) |
| 1353 |
|
| 1354 |
* In the :mod:`gc` module, the new :func:`get_count` function returns a 3-tuple |
| 1355 |
containing the current collection counts for the three GC generations. This is |
| 1356 |
accounting information for the garbage collector; when these counts reach a |
| 1357 |
specified threshold, a garbage collection sweep will be made. The existing |
| 1358 |
:func:`gc.collect` function now takes an optional *generation* argument of 0, 1, |
| 1359 |
or 2 to specify which generation to collect. (Contributed by Barry Warsaw.) |
| 1360 |
|
| 1361 |
* The :func:`nsmallest` and :func:`nlargest` functions in the :mod:`heapq` |
| 1362 |
module now support a ``key`` keyword parameter similar to the one provided by |
| 1363 |
the :func:`min`/:func:`max` functions and the :meth:`sort` methods. For |
| 1364 |
example:: |
| 1365 |
|
| 1366 |
>>> import heapq |
| 1367 |
>>> L = ["short", 'medium', 'longest', 'longer still'] |
| 1368 |
>>> heapq.nsmallest(2, L) # Return two lowest elements, lexicographically |
| 1369 |
['longer still', 'longest'] |
| 1370 |
>>> heapq.nsmallest(2, L, key=len) # Return two shortest elements |
| 1371 |
['short', 'medium'] |
| 1372 |
|
| 1373 |
(Contributed by Raymond Hettinger.) |
| 1374 |
|
| 1375 |
* The :func:`itertools.islice` function now accepts ``None`` for the start and |
| 1376 |
step arguments. This makes it more compatible with the attributes of slice |
| 1377 |
objects, so that you can now write the following:: |
| 1378 |
|
| 1379 |
s = slice(5) # Create slice object |
| 1380 |
itertools.islice(iterable, s.start, s.stop, s.step) |
| 1381 |
|
| 1382 |
(Contributed by Raymond Hettinger.) |
| 1383 |
|
| 1384 |
* The :func:`format` function in the :mod:`locale` module has been modified and |
| 1385 |
two new functions were added, :func:`format_string` and :func:`currency`. |
| 1386 |
|
| 1387 |
The :func:`format` function's *val* parameter could previously be a string as |
| 1388 |
long as no more than one %char specifier appeared; now the parameter must be |
| 1389 |
exactly one %char specifier with no surrounding text. An optional *monetary* |
| 1390 |
parameter was also added which, if ``True``, will use the locale's rules for |
| 1391 |
formatting currency in placing a separator between groups of three digits. |
| 1392 |
|
| 1393 |
To format strings with multiple %char specifiers, use the new |
| 1394 |
:func:`format_string` function that works like :func:`format` but also supports |
| 1395 |
mixing %char specifiers with arbitrary text. |
| 1396 |
|
| 1397 |
A new :func:`currency` function was also added that formats a number according |
| 1398 |
to the current locale's settings. |
| 1399 |
|
| 1400 |
(Contributed by Georg Brandl.) |
| 1401 |
|
| 1402 |
.. Patch 1180296 |
| 1403 |
|
| 1404 |
* The :mod:`mailbox` module underwent a massive rewrite to add the capability to |
| 1405 |
modify mailboxes in addition to reading them. A new set of classes that include |
| 1406 |
:class:`mbox`, :class:`MH`, and :class:`Maildir` are used to read mailboxes, and |
| 1407 |
have an ``add(message)`` method to add messages, ``remove(key)`` to |
| 1408 |
remove messages, and :meth:`lock`/:meth:`unlock` to lock/unlock the mailbox. |
| 1409 |
The following example converts a maildir-format mailbox into an mbox-format |
| 1410 |
one:: |
| 1411 |
|
| 1412 |
import mailbox |
| 1413 |
|
| 1414 |
# 'factory=None' uses email.Message.Message as the class representing |
| 1415 |
# individual messages. |
| 1416 |
src = mailbox.Maildir('maildir', factory=None) |
| 1417 |
dest = mailbox.mbox('/tmp/mbox') |
| 1418 |
|
| 1419 |
for msg in src: |
| 1420 |
dest.add(msg) |
| 1421 |
|
| 1422 |
(Contributed by Gregory K. Johnson. Funding was provided by Google's 2005 |
| 1423 |
Summer of Code.) |
| 1424 |
|
| 1425 |
* New module: the :mod:`!msilib` module allows creating Microsoft Installer |
| 1426 |
:file:`.msi` files and CAB files. Some support for reading the :file:`.msi` |
| 1427 |
database is also included. (Contributed by Martin von Löwis.) |
| 1428 |
|
| 1429 |
* The :mod:`!nis` module now supports accessing domains other than the system |
| 1430 |
default domain by supplying a *domain* argument to the :func:`!nis.match` and |
| 1431 |
:func:`!nis.maps` functions. (Contributed by Ben Bell.) |
| 1432 |
|
| 1433 |
* The :mod:`operator` module's :func:`itemgetter` and :func:`attrgetter` |
| 1434 |
functions now support multiple fields. A call such as |
| 1435 |
``operator.attrgetter('a', 'b')`` will return a function that retrieves the |
| 1436 |
:attr:`a` and :attr:`b` attributes. Combining this new feature with the |
| 1437 |
:meth:`sort` method's ``key`` parameter lets you easily sort lists using |
| 1438 |
multiple fields. (Contributed by Raymond Hettinger.) |
| 1439 |
|
| 1440 |
* The :mod:`optparse` module was updated to version 1.5.1 of the Optik library. |
| 1441 |
The :class:`OptionParser` class gained an :attr:`epilog` attribute, a string |
| 1442 |
that will be printed after the help message, and a :meth:`destroy` method to |
| 1443 |
break reference cycles created by the object. (Contributed by Greg Ward.) |
| 1444 |
|
| 1445 |
* The :mod:`os` module underwent several changes. The :attr:`stat_float_times` |
| 1446 |
variable now defaults to true, meaning that :func:`os.stat` will now return time |
| 1447 |
values as floats. (This doesn't necessarily mean that :func:`os.stat` will |
| 1448 |
return times that are precise to fractions of a second; not all systems support |
| 1449 |
such precision.) |
| 1450 |
|
| 1451 |
Constants named :const:`os.SEEK_SET`, :const:`os.SEEK_CUR`, and |
| 1452 |
:const:`os.SEEK_END` have been added; these are the parameters to the |
| 1453 |
:func:`os.lseek` function. Two new constants for locking are |
| 1454 |
:const:`os.O_SHLOCK` and :const:`os.O_EXLOCK`. |
| 1455 |
|
| 1456 |
Two new functions, :func:`wait3` and :func:`wait4`, were added. They're similar |
| 1457 |
the :func:`waitpid` function which waits for a child process to exit and returns |
| 1458 |
a tuple of the process ID and its exit status, but :func:`wait3` and |
| 1459 |
:func:`wait4` return additional information. :func:`wait3` doesn't take a |
| 1460 |
process ID as input, so it waits for any child process to exit and returns a |
| 1461 |
3-tuple of *process-id*, *exit-status*, *resource-usage* as returned from the |
| 1462 |
:func:`resource.getrusage` function. ``wait4(pid)`` does take a process ID. |
| 1463 |
(Contributed by Chad J. Schroeder.) |
| 1464 |
|
| 1465 |
On FreeBSD, the :func:`os.stat` function now returns times with nanosecond |
| 1466 |
resolution, and the returned object now has :attr:`st_gen` and |
| 1467 |
:attr:`st_birthtime`. The :attr:`st_flags` attribute is also available, if the |
| 1468 |
platform supports it. (Contributed by Antti Louko and Diego Pettenò.) |
| 1469 |
|
| 1470 |
.. (Patch 1180695, 1212117) |
| 1471 |
|
| 1472 |
* The Python debugger provided by the :mod:`pdb` module can now store lists of |
| 1473 |
commands to execute when a breakpoint is reached and execution stops. Once |
| 1474 |
breakpoint #1 has been created, enter ``commands 1`` and enter a series of |
| 1475 |
commands to be executed, finishing the list with ``end``. The command list can |
| 1476 |
include commands that resume execution, such as ``continue`` or ``next``. |
| 1477 |
(Contributed by Grégoire Dooms.) |
| 1478 |
|
| 1479 |
.. Patch 790710 |
| 1480 |
|
| 1481 |
* The :mod:`pickle` and :mod:`!cPickle` modules no longer accept a return value |
| 1482 |
of ``None`` from the :meth:`~object.__reduce__` method; the method must return a tuple |
| 1483 |
of arguments instead. The ability to return ``None`` was deprecated in Python |
| 1484 |
2.4, so this completes the removal of the feature. |
| 1485 |
|
| 1486 |
* The :mod:`pkgutil` module, containing various utility functions for finding |
| 1487 |
packages, was enhanced to support :pep:`302`'s import hooks and now also works for |
| 1488 |
packages stored in ZIP-format archives. (Contributed by Phillip J. Eby.) |
| 1489 |
|
| 1490 |
* The pybench benchmark suite by Marc-André Lemburg is now included in the |
| 1491 |
:file:`Tools/pybench` directory. The pybench suite is an improvement on the |
| 1492 |
commonly used :file:`pystone.py` program because pybench provides a more |
| 1493 |
detailed measurement of the interpreter's speed. It times particular operations |
| 1494 |
such as function calls, tuple slicing, method lookups, and numeric operations, |
| 1495 |
instead of performing many different operations and reducing the result to a |
| 1496 |
single number as :file:`pystone.py` does. |
| 1497 |
|
| 1498 |
* The :mod:`pyexpat` module now uses version 2.0 of the Expat parser. |
| 1499 |
(Contributed by Trent Mick.) |
| 1500 |
|
| 1501 |
* The :class:`~queue.Queue` class provided by the :mod:`Queue` module gained two new |
| 1502 |
methods. :meth:`join` blocks until all items in the queue have been retrieved |
| 1503 |
and all processing work on the items have been completed. Worker threads call |
| 1504 |
the other new method, :meth:`task_done`, to signal that processing for an item |
| 1505 |
has been completed. (Contributed by Raymond Hettinger.) |
| 1506 |
|
| 1507 |
* The old :mod:`regex` and :mod:`regsub` modules, which have been deprecated |
| 1508 |
ever since Python 2.0, have finally been deleted. Other deleted modules: |
| 1509 |
:mod:`statcache`, :mod:`tzparse`, :mod:`whrandom`. |
| 1510 |
|
| 1511 |
* Also deleted: the :file:`lib-old` directory, which includes ancient modules |
| 1512 |
such as :mod:`dircmp` and :mod:`ni`, was removed. :file:`lib-old` wasn't on the |
| 1513 |
default ``sys.path``, so unless your programs explicitly added the directory to |
| 1514 |
``sys.path``, this removal shouldn't affect your code. |
| 1515 |
|
| 1516 |
* The :mod:`rlcompleter` module is no longer dependent on importing the |
| 1517 |
:mod:`readline` module and therefore now works on non-Unix platforms. (Patch |
| 1518 |
from Robert Kiendl.) |
| 1519 |
|
| 1520 |
.. Patch #1472854 |
| 1521 |
|
| 1522 |
* The :mod:`SimpleXMLRPCServer <xmlrpc.server>` and :mod:`DocXMLRPCServer <xmlrpc.server>` classes now have a |
| 1523 |
:attr:`rpc_paths` attribute that constrains XML-RPC operations to a limited set |
| 1524 |
of URL paths; the default is to allow only ``'/'`` and ``'/RPC2'``. Setting |
| 1525 |
:attr:`rpc_paths` to ``None`` or an empty tuple disables this path checking. |
| 1526 |
|
| 1527 |
.. Bug #1473048 |
| 1528 |
|
| 1529 |
* The :mod:`socket` module now supports :const:`AF_NETLINK` sockets on Linux, |
| 1530 |
thanks to a patch from Philippe Biondi. Netlink sockets are a Linux-specific |
| 1531 |
mechanism for communications between a user-space process and kernel code; an |
| 1532 |
introductory article about them is at https://www.linuxjournal.com/article/7356. |
| 1533 |
In Python code, netlink addresses are represented as a tuple of 2 integers, |
| 1534 |
``(pid, group_mask)``. |
| 1535 |
|
| 1536 |
Two new methods on socket objects, ``recv_into(buffer)`` and |
| 1537 |
``recvfrom_into(buffer)``, store the received data in an object that |
| 1538 |
supports the buffer protocol instead of returning the data as a string. This |
| 1539 |
means you can put the data directly into an array or a memory-mapped file. |
| 1540 |
|
| 1541 |
Socket objects also gained :meth:`getfamily`, :meth:`gettype`, and |
| 1542 |
:meth:`getproto` accessor methods to retrieve the family, type, and protocol |
| 1543 |
values for the socket. |
| 1544 |
|
| 1545 |
* New module: the :mod:`!spwd` module provides functions for accessing the shadow |
| 1546 |
password database on systems that support shadow passwords. |
| 1547 |
|
| 1548 |
* The :mod:`struct` is now faster because it compiles format strings into |
| 1549 |
:class:`Struct` objects with :meth:`pack` and :meth:`unpack` methods. This is |
| 1550 |
similar to how the :mod:`re` module lets you create compiled regular expression |
| 1551 |
objects. You can still use the module-level :func:`pack` and :func:`unpack` |
| 1552 |
functions; they'll create :class:`Struct` objects and cache them. Or you can |
| 1553 |
use :class:`Struct` instances directly:: |
| 1554 |
|
| 1555 |
s = struct.Struct('ih3s') |
| 1556 |
|
| 1557 |
data = s.pack(1972, 187, 'abc') |
| 1558 |
year, number, name = s.unpack(data) |
| 1559 |
|
| 1560 |
You can also pack and unpack data to and from buffer objects directly using the |
| 1561 |
``pack_into(buffer, offset, v1, v2, ...)`` and ``unpack_from(buffer, |
| 1562 |
offset)`` methods. This lets you store data directly into an array or a |
| 1563 |
memory-mapped file. |
| 1564 |
|
| 1565 |
(:class:`Struct` objects were implemented by Bob Ippolito at the NeedForSpeed |
| 1566 |
sprint. Support for buffer objects was added by Martin Blais, also at the |
| 1567 |
NeedForSpeed sprint.) |
| 1568 |
|
| 1569 |
* The Python developers switched from CVS to Subversion during the 2.5 |
| 1570 |
development process. Information about the exact build version is available as |
| 1571 |
the ``sys.subversion`` variable, a 3-tuple of ``(interpreter-name, branch-name, |
| 1572 |
revision-range)``. For example, at the time of writing my copy of 2.5 was |
| 1573 |
reporting ``('CPython', 'trunk', '45313:45315')``. |
| 1574 |
|
| 1575 |
This information is also available to C extensions via the |
| 1576 |
:c:func:`Py_GetBuildInfo` function that returns a string of build information |
| 1577 |
like this: ``"trunk:45355:45356M, Apr 13 2006, 07:42:19"``. (Contributed by |
| 1578 |
Barry Warsaw.) |
| 1579 |
|
| 1580 |
* Another new function, :func:`sys._current_frames`, returns the current stack |
| 1581 |
frames for all running threads as a dictionary mapping thread identifiers to the |
| 1582 |
topmost stack frame currently active in that thread at the time the function is |
| 1583 |
called. (Contributed by Tim Peters.) |
| 1584 |
|
| 1585 |
* The :class:`TarFile` class in the :mod:`tarfile` module now has an |
| 1586 |
:meth:`extractall` method that extracts all members from the archive into the |
| 1587 |
current working directory. It's also possible to set a different directory as |
| 1588 |
the extraction target, and to unpack only a subset of the archive's members. |
| 1589 |
|
| 1590 |
The compression used for a tarfile opened in stream mode can now be autodetected |
| 1591 |
using the mode ``'r|*'``. (Contributed by Lars Gustäbel.) |
| 1592 |
|
| 1593 |
.. patch 918101 |
| 1594 |
|
| 1595 |
* The :mod:`threading` module now lets you set the stack size used when new |
| 1596 |
threads are created. The ``stack_size([*size*])`` function returns the |
| 1597 |
currently configured stack size, and supplying the optional *size* parameter |
| 1598 |
sets a new value. Not all platforms support changing the stack size, but |
| 1599 |
Windows, POSIX threading, and OS/2 all do. (Contributed by Andrew MacIntyre.) |
| 1600 |
|
| 1601 |
.. Patch 1454481 |
| 1602 |
|
| 1603 |
* The :mod:`unicodedata` module has been updated to use version 4.1.0 of the |
| 1604 |
Unicode character database. Version 3.2.0 is required by some specifications, |
| 1605 |
so it's still available as :data:`unicodedata.ucd_3_2_0`. |
| 1606 |
|
| 1607 |
* New module: the :mod:`uuid` module generates universally unique identifiers |
| 1608 |
(UUIDs) according to :rfc:`4122`. The RFC defines several different UUID |
| 1609 |
versions that are generated from a starting string, from system properties, or |
| 1610 |
purely randomly. This module contains a :class:`UUID` class and functions |
| 1611 |
named :func:`uuid1`, :func:`uuid3`, :func:`uuid4`, and :func:`uuid5` to |
| 1612 |
generate different versions of UUID. (Version 2 UUIDs are not specified in |
| 1613 |
:rfc:`4122` and are not supported by this module.) :: |
| 1614 |
|
| 1615 |
>>> import uuid |
| 1616 |
>>> # make a UUID based on the host ID and current time |
| 1617 |
>>> uuid.uuid1() |
| 1618 |
UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') |
| 1619 |
|
| 1620 |
>>> # make a UUID using an MD5 hash of a namespace UUID and a name |
| 1621 |
>>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') |
| 1622 |
UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') |
| 1623 |
|
| 1624 |
>>> # make a random UUID |
| 1625 |
>>> uuid.uuid4() |
| 1626 |
UUID('16fd2706-8baf-433b-82eb-8c7fada847da') |
| 1627 |
|
| 1628 |
>>> # make a UUID using a SHA-1 hash of a namespace UUID and a name |
| 1629 |
>>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') |
| 1630 |
UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') |
| 1631 |
|
| 1632 |
(Contributed by Ka-Ping Yee.) |
| 1633 |
|
| 1634 |
* The :mod:`weakref` module's :class:`WeakKeyDictionary` and |
| 1635 |
:class:`WeakValueDictionary` types gained new methods for iterating over the |
| 1636 |
weak references contained in the dictionary. :meth:`iterkeyrefs` and |
| 1637 |
:meth:`keyrefs` methods were added to :class:`WeakKeyDictionary`, and |
| 1638 |
:meth:`itervaluerefs` and :meth:`valuerefs` were added to |
| 1639 |
:class:`WeakValueDictionary`. (Contributed by Fred L. Drake, Jr.) |
| 1640 |
|
| 1641 |
* The :mod:`webbrowser` module received a number of enhancements. It's now |
| 1642 |
usable as a script with ``python -m webbrowser``, taking a URL as the argument; |
| 1643 |
there are a number of switches to control the behaviour (:option:`!-n` for a new |
| 1644 |
browser window, :option:`!-t` for a new tab). New module-level functions, |
| 1645 |
:func:`open_new` and :func:`open_new_tab`, were added to support this. The |
| 1646 |
module's :func:`open` function supports an additional feature, an *autoraise* |
| 1647 |
parameter that signals whether to raise the open window when possible. A number |
| 1648 |
of additional browsers were added to the supported list such as Firefox, Opera, |
| 1649 |
Konqueror, and elinks. (Contributed by Oleg Broytmann and Georg Brandl.) |
| 1650 |
|
| 1651 |
.. Patch #754022 |
| 1652 |
|
| 1653 |
* The :mod:`xmlrpclib <xmlrpc.client>` module now supports returning :class:`~datetime.datetime` objects |
| 1654 |
for the XML-RPC date type. Supply ``use_datetime=True`` to the :func:`~xmlrpc.client.loads` |
| 1655 |
function or the :class:`!Unmarshaller` class to enable this feature. (Contributed |
| 1656 |
by Skip Montanaro.) |
| 1657 |
|
| 1658 |
.. Patch 1120353 |
| 1659 |
|
| 1660 |
* The :mod:`zipfile` module now supports the ZIP64 version of the format, |
| 1661 |
meaning that a .zip archive can now be larger than 4 GiB and can contain |
| 1662 |
individual files larger than 4 GiB. (Contributed by Ronald Oussoren.) |
| 1663 |
|
| 1664 |
.. Patch 1446489 |
| 1665 |
|
| 1666 |
* The :mod:`zlib` module's :class:`Compress` and :class:`Decompress` objects now |
| 1667 |
support a :meth:`copy` method that makes a copy of the object's internal state |
| 1668 |
and returns a new :class:`Compress` or :class:`Decompress` object. |
| 1669 |
(Contributed by Chris AtLee.) |
| 1670 |
|
| 1671 |
.. Patch 1435422 |
| 1672 |
|
| 1673 |
.. ====================================================================== |
| 1674 |
|
| 1675 |
|
| 1676 |
.. _module-ctypes: |
| 1677 |
|
| 1678 |
The ctypes package |
| 1679 |
------------------ |
| 1680 |
|
| 1681 |
The :mod:`ctypes` package, written by Thomas Heller, has been added to the |
| 1682 |
standard library. :mod:`ctypes` lets you call arbitrary functions in shared |
| 1683 |
libraries or DLLs. Long-time users may remember the :mod:`!dl` module, which |
| 1684 |
provides functions for loading shared libraries and calling functions in them. |
| 1685 |
The :mod:`ctypes` package is much fancier. |
| 1686 |
|
| 1687 |
To load a shared library or DLL, you must create an instance of the |
| 1688 |
:class:`CDLL` class and provide the name or path of the shared library or DLL. |
| 1689 |
Once that's done, you can call arbitrary functions by accessing them as |
| 1690 |
attributes of the :class:`CDLL` object. :: |
| 1691 |
|
| 1692 |
import ctypes |
| 1693 |
|
| 1694 |
libc = ctypes.CDLL('libc.so.6') |
| 1695 |
result = libc.printf("Line of output\n") |
| 1696 |
|
| 1697 |
Type constructors for the various C types are provided: :func:`c_int`, |
| 1698 |
:func:`c_float`, :func:`c_double`, :func:`c_char_p` (equivalent to :c:expr:`char |
| 1699 |
\*`), and so forth. Unlike Python's types, the C versions are all mutable; you |
| 1700 |
can assign to their :attr:`value` attribute to change the wrapped value. Python |
| 1701 |
integers and strings will be automatically converted to the corresponding C |
| 1702 |
types, but for other types you must call the correct type constructor. (And I |
| 1703 |
mean *must*; getting it wrong will often result in the interpreter crashing |
| 1704 |
with a segmentation fault.) |
| 1705 |
|
| 1706 |
You shouldn't use :func:`c_char_p` with a Python string when the C function will |
| 1707 |
be modifying the memory area, because Python strings are supposed to be |
| 1708 |
immutable; breaking this rule will cause puzzling bugs. When you need a |
| 1709 |
modifiable memory area, use :func:`create_string_buffer`:: |
| 1710 |
|
| 1711 |
s = "this is a string" |
| 1712 |
buf = ctypes.create_string_buffer(s) |
| 1713 |
libc.strfry(buf) |
| 1714 |
|
| 1715 |
C functions are assumed to return integers, but you can set the :attr:`restype` |
| 1716 |
attribute of the function object to change this:: |
| 1717 |
|
| 1718 |
>>> libc.atof('2.71828') |
| 1719 |
-1783957616 |
| 1720 |
>>> libc.atof.restype = ctypes.c_double |
| 1721 |
>>> libc.atof('2.71828') |
| 1722 |
2.71828 |
| 1723 |
|
| 1724 |
:mod:`ctypes` also provides a wrapper for Python's C API as the |
| 1725 |
``ctypes.pythonapi`` object. This object does *not* release the global |
| 1726 |
interpreter lock before calling a function, because the lock must be held when |
| 1727 |
calling into the interpreter's code. There's a :class:`~ctypes.py_object` type |
| 1728 |
constructor that will create a :c:expr:`PyObject *` pointer. A simple usage:: |
| 1729 |
|
| 1730 |
import ctypes |
| 1731 |
|
| 1732 |
d = {} |
| 1733 |
ctypes.pythonapi.PyObject_SetItem(ctypes.py_object(d), |
| 1734 |
ctypes.py_object("abc"), ctypes.py_object(1)) |
| 1735 |
# d is now {'abc', 1}. |
| 1736 |
|
| 1737 |
Don't forget to use :func:`~ctypes.py_object`; if it's omitted you end up with a |
| 1738 |
segmentation fault. |
| 1739 |
|
| 1740 |
:mod:`ctypes` has been around for a while, but people still write and |
| 1741 |
distribution hand-coded extension modules because you can't rely on |
| 1742 |
:mod:`ctypes` being present. Perhaps developers will begin to write Python |
| 1743 |
wrappers atop a library accessed through :mod:`ctypes` instead of extension |
| 1744 |
modules, now that :mod:`ctypes` is included with core Python. |
| 1745 |
|
| 1746 |
|
| 1747 |
.. seealso:: |
| 1748 |
|
| 1749 |
https://web.archive.org/web/20180410025338/http://starship.python.net/crew/theller/ctypes/ |
| 1750 |
The pre-stdlib ctypes web page, with a tutorial, reference, and FAQ. |
| 1751 |
|
| 1752 |
The documentation for the :mod:`ctypes` module. |
| 1753 |
|
| 1754 |
.. ====================================================================== |
| 1755 |
|
| 1756 |
|
| 1757 |
.. _module-etree: |
| 1758 |
|
| 1759 |
The ElementTree package |
| 1760 |
----------------------- |
| 1761 |
|
| 1762 |
A subset of Fredrik Lundh's ElementTree library for processing XML has been |
| 1763 |
added to the standard library as :mod:`xml.etree`. The available modules are |
| 1764 |
:mod:`ElementTree`, :mod:`ElementPath`, and :mod:`ElementInclude` from |
| 1765 |
ElementTree 1.2.6. The :mod:`cElementTree` accelerator module is also |
| 1766 |
included. |
| 1767 |
|
| 1768 |
The rest of this section will provide a brief overview of using ElementTree. |
| 1769 |
Full documentation for ElementTree is available at |
| 1770 |
https://web.archive.org/web/20201124024954/http://effbot.org/zone/element-index.htm. |
| 1771 |
|
| 1772 |
ElementTree represents an XML document as a tree of element nodes. The text |
| 1773 |
content of the document is stored as the :attr:`text` and :attr:`tail` |
| 1774 |
attributes of (This is one of the major differences between ElementTree and |
| 1775 |
the Document Object Model; in the DOM there are many different types of node, |
| 1776 |
including :class:`TextNode`.) |
| 1777 |
|
| 1778 |
The most commonly used parsing function is :func:`parse`, that takes either a |
| 1779 |
string (assumed to contain a filename) or a file-like object and returns an |
| 1780 |
:class:`ElementTree` instance:: |
| 1781 |
|
| 1782 |
from xml.etree import ElementTree as ET |
| 1783 |
|
| 1784 |
tree = ET.parse('ex-1.xml') |
| 1785 |
|
| 1786 |
feed = urllib.urlopen( |
| 1787 |
'http://planet.python.org/rss10.xml') |
| 1788 |
tree = ET.parse(feed) |
| 1789 |
|
| 1790 |
Once you have an :class:`ElementTree` instance, you can call its :meth:`getroot` |
| 1791 |
method to get the root :class:`Element` node. |
| 1792 |
|
| 1793 |
There's also an :func:`XML` function that takes a string literal and returns an |
| 1794 |
:class:`Element` node (not an :class:`ElementTree`). This function provides a |
| 1795 |
tidy way to incorporate XML fragments, approaching the convenience of an XML |
| 1796 |
literal:: |
| 1797 |
|
| 1798 |
svg = ET.XML("""<svg width="10px" version="1.0"> |
| 1799 |
</svg>""") |
| 1800 |
svg.set('height', '320px') |
| 1801 |
svg.append(elem1) |
| 1802 |
|
| 1803 |
Each XML element supports some dictionary-like and some list-like access |
| 1804 |
methods. Dictionary-like operations are used to access attribute values, and |
| 1805 |
list-like operations are used to access child nodes. |
| 1806 |
|
| 1807 |
+-------------------------------+--------------------------------------------+ |
| 1808 |
| Operation | Result | |
| 1809 |
+===============================+============================================+ |
| 1810 |
| ``elem[n]`` | Returns n'th child element. | |
| 1811 |
+-------------------------------+--------------------------------------------+ |
| 1812 |
| ``elem[m:n]`` | Returns list of m'th through n'th child | |
| 1813 |
| | elements. | |
| 1814 |
+-------------------------------+--------------------------------------------+ |
| 1815 |
| ``len(elem)`` | Returns number of child elements. | |
| 1816 |
+-------------------------------+--------------------------------------------+ |
| 1817 |
| ``list(elem)`` | Returns list of child elements. | |
| 1818 |
+-------------------------------+--------------------------------------------+ |
| 1819 |
| ``elem.append(elem2)`` | Adds *elem2* as a child. | |
| 1820 |
+-------------------------------+--------------------------------------------+ |
| 1821 |
| ``elem.insert(index, elem2)`` | Inserts *elem2* at the specified location. | |
| 1822 |
+-------------------------------+--------------------------------------------+ |
| 1823 |
| ``del elem[n]`` | Deletes n'th child element. | |
| 1824 |
+-------------------------------+--------------------------------------------+ |
| 1825 |
| ``elem.keys()`` | Returns list of attribute names. | |
| 1826 |
+-------------------------------+--------------------------------------------+ |
| 1827 |
| ``elem.get(name)`` | Returns value of attribute *name*. | |
| 1828 |
+-------------------------------+--------------------------------------------+ |
| 1829 |
| ``elem.set(name, value)`` | Sets new value for attribute *name*. | |
| 1830 |
+-------------------------------+--------------------------------------------+ |
| 1831 |
| ``elem.attrib`` | Retrieves the dictionary containing | |
| 1832 |
| | attributes. | |
| 1833 |
+-------------------------------+--------------------------------------------+ |
| 1834 |
| ``del elem.attrib[name]`` | Deletes attribute *name*. | |
| 1835 |
+-------------------------------+--------------------------------------------+ |
| 1836 |
|
| 1837 |
Comments and processing instructions are also represented as :class:`Element` |
| 1838 |
nodes. To check if a node is a comment or processing instructions:: |
| 1839 |
|
| 1840 |
if elem.tag is ET.Comment: |
| 1841 |
... |
| 1842 |
elif elem.tag is ET.ProcessingInstruction: |
| 1843 |
... |
| 1844 |
|
| 1845 |
To generate XML output, you should call the :meth:`ElementTree.write` method. |
| 1846 |
Like :func:`parse`, it can take either a string or a file-like object:: |
| 1847 |
|
| 1848 |
# Encoding is US-ASCII |
| 1849 |
tree.write('output.xml') |
| 1850 |
|
| 1851 |
# Encoding is UTF-8 |
| 1852 |
f = open('output.xml', 'w') |
| 1853 |
tree.write(f, encoding='utf-8') |
| 1854 |
|
| 1855 |
(Caution: the default encoding used for output is ASCII. For general XML work, |
| 1856 |
where an element's name may contain arbitrary Unicode characters, ASCII isn't a |
| 1857 |
very useful encoding because it will raise an exception if an element's name |
| 1858 |
contains any characters with values greater than 127. Therefore, it's best to |
| 1859 |
specify a different encoding such as UTF-8 that can handle any Unicode |
| 1860 |
character.) |
| 1861 |
|
| 1862 |
This section is only a partial description of the ElementTree interfaces. Please |
| 1863 |
read the package's official documentation for more details. |
| 1864 |
|
| 1865 |
|
| 1866 |
.. seealso:: |
| 1867 |
|
| 1868 |
https://web.archive.org/web/20201124024954/http://effbot.org/zone/element-index.htm |
| 1869 |
Official documentation for ElementTree. |
| 1870 |
|
| 1871 |
.. ====================================================================== |
| 1872 |
|
| 1873 |
|
| 1874 |
.. _module-hashlib: |
| 1875 |
|
| 1876 |
The hashlib package |
| 1877 |
------------------- |
| 1878 |
|
| 1879 |
A new :mod:`hashlib` module, written by Gregory P. Smith, has been added to |
| 1880 |
replace the :mod:`!md5` and :mod:`!sha` modules. :mod:`hashlib` adds support for |
| 1881 |
additional secure hashes (SHA-224, SHA-256, SHA-384, and SHA-512). When |
| 1882 |
available, the module uses OpenSSL for fast platform optimized implementations |
| 1883 |
of algorithms. |
| 1884 |
|
| 1885 |
The old :mod:`!md5` and :mod:`!sha` modules still exist as wrappers around hashlib |
| 1886 |
to preserve backwards compatibility. The new module's interface is very close |
| 1887 |
to that of the old modules, but not identical. The most significant difference |
| 1888 |
is that the constructor functions for creating new hashing objects are named |
| 1889 |
differently. :: |
| 1890 |
|
| 1891 |
# Old versions |
| 1892 |
h = md5.md5() |
| 1893 |
h = md5.new() |
| 1894 |
|
| 1895 |
# New version |
| 1896 |
h = hashlib.md5() |
| 1897 |
|
| 1898 |
# Old versions |
| 1899 |
h = sha.sha() |
| 1900 |
h = sha.new() |
| 1901 |
|
| 1902 |
# New version |
| 1903 |
h = hashlib.sha1() |
| 1904 |
|
| 1905 |
# Hash that weren't previously available |
| 1906 |
h = hashlib.sha224() |
| 1907 |
h = hashlib.sha256() |
| 1908 |
h = hashlib.sha384() |
| 1909 |
h = hashlib.sha512() |
| 1910 |
|
| 1911 |
# Alternative form |
| 1912 |
h = hashlib.new('md5') # Provide algorithm as a string |
| 1913 |
|
| 1914 |
Once a hash object has been created, its methods are the same as before: |
| 1915 |
``update(string)`` hashes the specified string into the current digest |
| 1916 |
state, :meth:`digest` and :meth:`hexdigest` return the digest value as a binary |
| 1917 |
string or a string of hex digits, and :meth:`copy` returns a new hashing object |
| 1918 |
with the same digest state. |
| 1919 |
|
| 1920 |
|
| 1921 |
.. seealso:: |
| 1922 |
|
| 1923 |
The documentation for the :mod:`hashlib` module. |
| 1924 |
|
| 1925 |
.. ====================================================================== |
| 1926 |
|
| 1927 |
|
| 1928 |
.. _module-sqlite: |
| 1929 |
|
| 1930 |
The sqlite3 package |
| 1931 |
------------------- |
| 1932 |
|
| 1933 |
The pysqlite module (https://www.pysqlite.org), a wrapper for the SQLite embedded |
| 1934 |
database, has been added to the standard library under the package name |
| 1935 |
:mod:`sqlite3`. |
| 1936 |
|
| 1937 |
SQLite is a C library that provides a lightweight disk-based database that |
| 1938 |
doesn't require a separate server process and allows accessing the database |
| 1939 |
using a nonstandard variant of the SQL query language. Some applications can use |
| 1940 |
SQLite for internal data storage. It's also possible to prototype an |
| 1941 |
application using SQLite and then port the code to a larger database such as |
| 1942 |
PostgreSQL or Oracle. |
| 1943 |
|
| 1944 |
pysqlite was written by Gerhard Häring and provides a SQL interface compliant |
| 1945 |
with the DB-API 2.0 specification described by :pep:`249`. |
| 1946 |
|
| 1947 |
If you're compiling the Python source yourself, note that the source tree |
| 1948 |
doesn't include the SQLite code, only the wrapper module. You'll need to have |
| 1949 |
the SQLite libraries and headers installed before compiling Python, and the |
| 1950 |
build process will compile the module when the necessary headers are available. |
| 1951 |
|
| 1952 |
To use the module, you must first create a :class:`Connection` object that |
| 1953 |
represents the database. Here the data will be stored in the |
| 1954 |
:file:`/tmp/example` file:: |
| 1955 |
|
| 1956 |
conn = sqlite3.connect('/tmp/example') |
| 1957 |
|
| 1958 |
You can also supply the special name ``:memory:`` to create a database in RAM. |
| 1959 |
|
| 1960 |
Once you have a :class:`Connection`, you can create a :class:`Cursor` object |
| 1961 |
and call its :meth:`execute` method to perform SQL commands:: |
| 1962 |
|
| 1963 |
c = conn.cursor() |
| 1964 |
|
| 1965 |
# Create table |
| 1966 |
c.execute('''create table stocks |
| 1967 |
(date text, trans text, symbol text, |
| 1968 |
qty real, price real)''') |
| 1969 |
|
| 1970 |
# Insert a row of data |
| 1971 |
c.execute("""insert into stocks |
| 1972 |
values ('2006-01-05','BUY','RHAT',100,35.14)""") |
| 1973 |
|
| 1974 |
Usually your SQL operations will need to use values from Python variables. You |
| 1975 |
shouldn't assemble your query using Python's string operations because doing so |
| 1976 |
is insecure; it makes your program vulnerable to an SQL injection attack. |
| 1977 |
|
| 1978 |
Instead, use the DB-API's parameter substitution. Put ``?`` as a placeholder |
| 1979 |
wherever you want to use a value, and then provide a tuple of values as the |
| 1980 |
second argument to the cursor's :meth:`execute` method. (Other database modules |
| 1981 |
may use a different placeholder, such as ``%s`` or ``:1``.) For example:: |
| 1982 |
|
| 1983 |
# Never do this -- insecure! |
| 1984 |
symbol = 'IBM' |
| 1985 |
c.execute("... where symbol = '%s'" % symbol) |
| 1986 |
|
| 1987 |
# Do this instead |
| 1988 |
t = (symbol,) |
| 1989 |
c.execute('select * from stocks where symbol=?', t) |
| 1990 |
|
| 1991 |
# Larger example |
| 1992 |
for t in (('2006-03-28', 'BUY', 'IBM', 1000, 45.00), |
| 1993 |
('2006-04-05', 'BUY', 'MSOFT', 1000, 72.00), |
| 1994 |
('2006-04-06', 'SELL', 'IBM', 500, 53.00), |
| 1995 |
): |
| 1996 |
c.execute('insert into stocks values (?,?,?,?,?)', t) |
| 1997 |
|
| 1998 |
To retrieve data after executing a SELECT statement, you can either treat the |
| 1999 |
cursor as an iterator, call the cursor's :meth:`fetchone` method to retrieve a |
| 2000 |
single matching row, or call :meth:`fetchall` to get a list of the matching |
| 2001 |
rows. |
| 2002 |
|
| 2003 |
This example uses the iterator form:: |
| 2004 |
|
| 2005 |
>>> c = conn.cursor() |
| 2006 |
>>> c.execute('select * from stocks order by price') |
| 2007 |
>>> for row in c: |
| 2008 |
... print row |
| 2009 |
... |
| 2010 |
(u'2006-01-05', u'BUY', u'RHAT', 100, 35.140000000000001) |
| 2011 |
(u'2006-03-28', u'BUY', u'IBM', 1000, 45.0) |
| 2012 |
(u'2006-04-06', u'SELL', u'IBM', 500, 53.0) |
| 2013 |
(u'2006-04-05', u'BUY', u'MSOFT', 1000, 72.0) |
| 2014 |
>>> |
| 2015 |
|
| 2016 |
For more information about the SQL dialect supported by SQLite, see |
| 2017 |
https://www.sqlite.org. |
| 2018 |
|
| 2019 |
|
| 2020 |
.. seealso:: |
| 2021 |
|
| 2022 |
https://www.pysqlite.org |
| 2023 |
The pysqlite web page. |
| 2024 |
|
| 2025 |
https://www.sqlite.org |
| 2026 |
The SQLite web page; the documentation describes the syntax and the available |
| 2027 |
data types for the supported SQL dialect. |
| 2028 |
|
| 2029 |
The documentation for the :mod:`sqlite3` module. |
| 2030 |
|
| 2031 |
:pep:`249` - Database API Specification 2.0 |
| 2032 |
PEP written by Marc-André Lemburg. |
| 2033 |
|
| 2034 |
.. ====================================================================== |
| 2035 |
|
| 2036 |
|
| 2037 |
.. _module-wsgiref: |
| 2038 |
|
| 2039 |
The wsgiref package |
| 2040 |
------------------- |
| 2041 |
|
| 2042 |
The Web Server Gateway Interface (WSGI) v1.0 defines a standard interface |
| 2043 |
between web servers and Python web applications and is described in :pep:`333`. |
| 2044 |
The :mod:`wsgiref` package is a reference implementation of the WSGI |
| 2045 |
specification. |
| 2046 |
|
| 2047 |
.. XXX should this be in a :pep:`333` section instead? |
| 2048 |
|
| 2049 |
The package includes a basic HTTP server that will run a WSGI application; this |
| 2050 |
server is useful for debugging but isn't intended for production use. Setting |
| 2051 |
up a server takes only a few lines of code:: |
| 2052 |
|
| 2053 |
from wsgiref import simple_server |
| 2054 |
|
| 2055 |
wsgi_app = ... |
| 2056 |
|
| 2057 |
host = '' |
| 2058 |
port = 8000 |
| 2059 |
httpd = simple_server.make_server(host, port, wsgi_app) |
| 2060 |
httpd.serve_forever() |
| 2061 |
|
| 2062 |
.. XXX discuss structure of WSGI applications? |
| 2063 |
.. XXX provide an example using Django or some other framework? |
| 2064 |
|
| 2065 |
|
| 2066 |
.. seealso:: |
| 2067 |
|
| 2068 |
https://web.archive.org/web/20160331090247/http://wsgi.readthedocs.org/en/latest/ |
| 2069 |
A central web site for WSGI-related resources. |
| 2070 |
|
| 2071 |
:pep:`333` - Python Web Server Gateway Interface v1.0 |
| 2072 |
PEP written by Phillip J. Eby. |
| 2073 |
|
| 2074 |
.. ====================================================================== |
| 2075 |
|
| 2076 |
|
| 2077 |
.. _build-api: |
| 2078 |
|
| 2079 |
Build and C API Changes |
| 2080 |
======================= |
| 2081 |
|
| 2082 |
Changes to Python's build process and to the C API include: |
| 2083 |
|
| 2084 |
* The Python source tree was converted from CVS to Subversion, in a complex |
| 2085 |
migration procedure that was supervised and flawlessly carried out by Martin von |
| 2086 |
Löwis. The procedure was developed as :pep:`347`. |
| 2087 |
|
| 2088 |
* Coverity, a company that markets a source code analysis tool called Prevent, |
| 2089 |
provided the results of their examination of the Python source code. The |
| 2090 |
analysis found about 60 bugs that were quickly fixed. Many of the bugs were |
| 2091 |
refcounting problems, often occurring in error-handling code. See |
| 2092 |
https://scan.coverity.com for the statistics. |
| 2093 |
|
| 2094 |
* The largest change to the C API came from :pep:`353`, which modifies the |
| 2095 |
interpreter to use a :c:type:`Py_ssize_t` type definition instead of |
| 2096 |
:c:expr:`int`. See the earlier section :ref:`pep-353` for a discussion of this |
| 2097 |
change. |
| 2098 |
|
| 2099 |
* The design of the bytecode compiler has changed a great deal, no longer |
| 2100 |
generating bytecode by traversing the parse tree. Instead the parse tree is |
| 2101 |
converted to an abstract syntax tree (or AST), and it is the abstract syntax |
| 2102 |
tree that's traversed to produce the bytecode. |
| 2103 |
|
| 2104 |
It's possible for Python code to obtain AST objects by using the |
| 2105 |
:func:`compile` built-in and specifying ``_ast.PyCF_ONLY_AST`` as the value of |
| 2106 |
the *flags* parameter:: |
| 2107 |
|
| 2108 |
from _ast import PyCF_ONLY_AST |
| 2109 |
ast = compile("""a=0 |
| 2110 |
for i in range(10): |
| 2111 |
a += i |
| 2112 |
""", "<string>", 'exec', PyCF_ONLY_AST) |
| 2113 |
|
| 2114 |
assignment = ast.body[0] |
| 2115 |
for_loop = ast.body[1] |
| 2116 |
|
| 2117 |
No official documentation has been written for the AST code yet, but :pep:`339` |
| 2118 |
discusses the design. To start learning about the code, read the definition of |
| 2119 |
the various AST nodes in :file:`Parser/Python.asdl`. A Python script reads this |
| 2120 |
file and generates a set of C structure definitions in |
| 2121 |
:file:`Include/Python-ast.h`. The :c:func:`PyParser_ASTFromString` and |
| 2122 |
:c:func:`!PyParser_ASTFromFile`, defined in :file:`Include/pythonrun.h`, take |
| 2123 |
Python source as input and return the root of an AST representing the contents. |
| 2124 |
This AST can then be turned into a code object by :c:func:`!PyAST_Compile`. For |
| 2125 |
more information, read the source code, and then ask questions on python-dev. |
| 2126 |
|
| 2127 |
The AST code was developed under Jeremy Hylton's management, and implemented by |
| 2128 |
(in alphabetical order) Brett Cannon, Nick Coghlan, Grant Edwards, John |
| 2129 |
Ehresman, Kurt Kaiser, Neal Norwitz, Tim Peters, Armin Rigo, and Neil |
| 2130 |
Schemenauer, plus the participants in a number of AST sprints at conferences |
| 2131 |
such as PyCon. |
| 2132 |
|
| 2133 |
.. List of names taken from Jeremy's python-dev post at |
| 2134 |
.. https://mail.python.org/pipermail/python-dev/2005-October/057500.html |
| 2135 |
|
| 2136 |
* Evan Jones's patch to obmalloc, first described in a talk at PyCon DC 2005, |
| 2137 |
was applied. Python 2.4 allocated small objects in 256K-sized arenas, but never |
| 2138 |
freed arenas. With this patch, Python will free arenas when they're empty. The |
| 2139 |
net effect is that on some platforms, when you allocate many objects, Python's |
| 2140 |
memory usage may actually drop when you delete them and the memory may be |
| 2141 |
returned to the operating system. (Implemented by Evan Jones, and reworked by |
| 2142 |
Tim Peters.) |
| 2143 |
|
| 2144 |
Note that this change means extension modules must be more careful when |
| 2145 |
allocating memory. Python's API has many different functions for allocating |
| 2146 |
memory that are grouped into families. For example, :c:func:`PyMem_Malloc`, |
| 2147 |
:c:func:`PyMem_Realloc`, and :c:func:`PyMem_Free` are one family that allocates |
| 2148 |
raw memory, while :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`, and |
| 2149 |
:c:func:`PyObject_Free` are another family that's supposed to be used for |
| 2150 |
creating Python objects. |
| 2151 |
|
| 2152 |
Previously these different families all reduced to the platform's |
| 2153 |
:c:func:`malloc` and :c:func:`free` functions. This meant it didn't matter if |
| 2154 |
you got things wrong and allocated memory with the ``PyMem`` function but |
| 2155 |
freed it with the ``PyObject`` function. With 2.5's changes to obmalloc, |
| 2156 |
these families now do different things and mismatches will probably result in a |
| 2157 |
segfault. You should carefully test your C extension modules with Python 2.5. |
| 2158 |
|
| 2159 |
* The built-in set types now have an official C API. Call :c:func:`PySet_New` |
| 2160 |
and :c:func:`PyFrozenSet_New` to create a new set, :c:func:`PySet_Add` and |
| 2161 |
:c:func:`PySet_Discard` to add and remove elements, and :c:func:`PySet_Contains` |
| 2162 |
and :c:func:`PySet_Size` to examine the set's state. (Contributed by Raymond |
| 2163 |
Hettinger.) |
| 2164 |
|
| 2165 |
* C code can now obtain information about the exact revision of the Python |
| 2166 |
interpreter by calling the :c:func:`Py_GetBuildInfo` function that returns a |
| 2167 |
string of build information like this: ``"trunk:45355:45356M, Apr 13 2006, |
| 2168 |
07:42:19"``. (Contributed by Barry Warsaw.) |
| 2169 |
|
| 2170 |
* Two new macros can be used to indicate C functions that are local to the |
| 2171 |
current file so that a faster calling convention can be used. |
| 2172 |
:c:macro:`Py_LOCAL` declares the function as returning a value of the |
| 2173 |
specified *type* and uses a fast-calling qualifier. |
| 2174 |
:c:macro:`Py_LOCAL_INLINE` does the same thing and also requests the |
| 2175 |
function be inlined. If macro :c:macro:`!PY_LOCAL_AGGRESSIVE` is defined before |
| 2176 |
:file:`python.h` is included, a set of more aggressive optimizations are enabled |
| 2177 |
for the module; you should benchmark the results to find out if these |
| 2178 |
optimizations actually make the code faster. (Contributed by Fredrik Lundh at |
| 2179 |
the NeedForSpeed sprint.) |
| 2180 |
|
| 2181 |
* ``PyErr_NewException(name, base, dict)`` can now accept a tuple of base |
| 2182 |
classes as its *base* argument. (Contributed by Georg Brandl.) |
| 2183 |
|
| 2184 |
* The :c:func:`!PyErr_Warn` function for issuing warnings is now deprecated in |
| 2185 |
favour of ``PyErr_WarnEx(category, message, stacklevel)`` which lets you |
| 2186 |
specify the number of stack frames separating this function and the caller. A |
| 2187 |
*stacklevel* of 1 is the function calling :c:func:`PyErr_WarnEx`, 2 is the |
| 2188 |
function above that, and so forth. (Added by Neal Norwitz.) |
| 2189 |
|
| 2190 |
* The CPython interpreter is still written in C, but the code can now be |
| 2191 |
compiled with a C++ compiler without errors. (Implemented by Anthony Baxter, |
| 2192 |
Martin von Löwis, Skip Montanaro.) |
| 2193 |
|
| 2194 |
* The :c:func:`!PyRange_New` function was removed. It was never documented, never |
| 2195 |
used in the core code, and had dangerously lax error checking. In the unlikely |
| 2196 |
case that your extensions were using it, you can replace it by something like |
| 2197 |
the following:: |
| 2198 |
|
| 2199 |
range = PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", |
| 2200 |
start, stop, step); |
| 2201 |
|
| 2202 |
.. ====================================================================== |
| 2203 |
|
| 2204 |
|
| 2205 |
.. _ports: |
| 2206 |
|
| 2207 |
Port-Specific Changes |
| 2208 |
--------------------- |
| 2209 |
|
| 2210 |
* MacOS X (10.3 and higher): dynamic loading of modules now uses the |
| 2211 |
:c:func:`dlopen` function instead of MacOS-specific functions. |
| 2212 |
|
| 2213 |
* MacOS X: an :option:`!--enable-universalsdk` switch was added to the |
| 2214 |
:program:`configure` script that compiles the interpreter as a universal binary |
| 2215 |
able to run on both PowerPC and Intel processors. (Contributed by Ronald |
| 2216 |
Oussoren; :issue:`2573`.) |
| 2217 |
|
| 2218 |
* Windows: :file:`.dll` is no longer supported as a filename extension for |
| 2219 |
extension modules. :file:`.pyd` is now the only filename extension that will be |
| 2220 |
searched for. |
| 2221 |
|
| 2222 |
.. ====================================================================== |
| 2223 |
|
| 2224 |
|
| 2225 |
.. _porting: |
| 2226 |
|
| 2227 |
Porting to Python 2.5 |
| 2228 |
===================== |
| 2229 |
|
| 2230 |
This section lists previously described changes that may require changes to your |
| 2231 |
code: |
| 2232 |
|
| 2233 |
* ASCII is now the default encoding for modules. It's now a syntax error if a |
| 2234 |
module contains string literals with 8-bit characters but doesn't have an |
| 2235 |
encoding declaration. In Python 2.4 this triggered a warning, not a syntax |
| 2236 |
error. |
| 2237 |
|
| 2238 |
* Previously, the :attr:`gi_frame` attribute of a generator was always a frame |
| 2239 |
object. Because of the :pep:`342` changes described in section :ref:`pep-342`, |
| 2240 |
it's now possible for :attr:`gi_frame` to be ``None``. |
| 2241 |
|
| 2242 |
* A new warning, :class:`UnicodeWarning`, is triggered when you attempt to |
| 2243 |
compare a Unicode string and an 8-bit string that can't be converted to Unicode |
| 2244 |
using the default ASCII encoding. Previously such comparisons would raise a |
| 2245 |
:class:`UnicodeDecodeError` exception. |
| 2246 |
|
| 2247 |
* Library: the :mod:`csv` module is now stricter about multi-line quoted fields. |
| 2248 |
If your files contain newlines embedded within fields, the input should be split |
| 2249 |
into lines in a manner which preserves the newline characters. |
| 2250 |
|
| 2251 |
* Library: the :mod:`locale` module's :func:`format` function's would |
| 2252 |
previously accept any string as long as no more than one %char specifier |
| 2253 |
appeared. In Python 2.5, the argument must be exactly one %char specifier with |
| 2254 |
no surrounding text. |
| 2255 |
|
| 2256 |
* Library: The :mod:`pickle` and :mod:`!cPickle` modules no longer accept a |
| 2257 |
return value of ``None`` from the :meth:`~object.__reduce__` method; the method must |
| 2258 |
return a tuple of arguments instead. The modules also no longer accept the |
| 2259 |
deprecated *bin* keyword parameter. |
| 2260 |
|
| 2261 |
* Library: The :mod:`SimpleXMLRPCServer <xmlrpc.server>` and :mod:`DocXMLRPCServer <xmlrpc.server>` classes now |
| 2262 |
have a :attr:`rpc_paths` attribute that constrains XML-RPC operations to a |
| 2263 |
limited set of URL paths; the default is to allow only ``'/'`` and ``'/RPC2'``. |
| 2264 |
Setting :attr:`rpc_paths` to ``None`` or an empty tuple disables this path |
| 2265 |
checking. |
| 2266 |
|
| 2267 |
* C API: Many functions now use :c:type:`Py_ssize_t` instead of :c:expr:`int` to |
| 2268 |
allow processing more data on 64-bit machines. Extension code may need to make |
| 2269 |
the same change to avoid warnings and to support 64-bit machines. See the |
| 2270 |
earlier section :ref:`pep-353` for a discussion of this change. |
| 2271 |
|
| 2272 |
* C API: The obmalloc changes mean that you must be careful to not mix usage |
| 2273 |
of the ``PyMem_*`` and ``PyObject_*`` families of functions. Memory |
| 2274 |
allocated with one family's ``*_Malloc`` must be freed with the |
| 2275 |
corresponding family's ``*_Free`` function. |
| 2276 |
|
| 2277 |
.. ====================================================================== |
| 2278 |
|
| 2279 |
|
| 2280 |
Acknowledgements |
| 2281 |
================ |
| 2282 |
|
| 2283 |
The author would like to thank the following people for offering suggestions, |
| 2284 |
corrections and assistance with various drafts of this article: Georg Brandl, |
| 2285 |
Nick Coghlan, Phillip J. Eby, Lars Gustäbel, Raymond Hettinger, Ralf W. |
| 2286 |
Grosse-Kunstleve, Kent Johnson, Iain Lowe, Martin von Löwis, Fredrik Lundh, Andrew |
| 2287 |
McNamara, Skip Montanaro, Gustavo Niemeyer, Paul Prescod, James Pryor, Mike |
| 2288 |
Rovner, Scott Weikart, Barry Warsaw, Thomas Wouters. |
| 2289 |
|