| 1 |
**************************** |
| 2 |
What's New in Python 2.1 |
| 3 |
**************************** |
| 4 |
|
| 5 |
:Author: A.M. Kuchling |
| 6 |
|
| 7 |
.. |release| replace:: 1.01 |
| 8 |
|
| 9 |
.. $Id: whatsnew21.tex 50964 2006-07-30 03:03:43Z fred.drake $ |
| 10 |
|
| 11 |
|
| 12 |
Introduction |
| 13 |
============ |
| 14 |
|
| 15 |
This article explains the new features in Python 2.1. While there aren't as |
| 16 |
many changes in 2.1 as there were in Python 2.0, there are still some pleasant |
| 17 |
surprises in store. 2.1 is the first release to be steered through the use of |
| 18 |
Python Enhancement Proposals, or PEPs, so most of the sizable changes have |
| 19 |
accompanying PEPs that provide more complete documentation and a design |
| 20 |
rationale for the change. This article doesn't attempt to document the new |
| 21 |
features completely, but simply provides an overview of the new features for |
| 22 |
Python programmers. Refer to the Python 2.1 documentation, or to the specific |
| 23 |
PEP, for more details about any new feature that particularly interests you. |
| 24 |
|
| 25 |
One recent goal of the Python development team has been to accelerate the pace |
| 26 |
of new releases, with a new release coming every 6 to 9 months. 2.1 is the first |
| 27 |
release to come out at this faster pace, with the first alpha appearing in |
| 28 |
January, 3 months after the final version of 2.0 was released. |
| 29 |
|
| 30 |
The final release of Python 2.1 was made on April 17, 2001. |
| 31 |
|
| 32 |
.. ====================================================================== |
| 33 |
|
| 34 |
|
| 35 |
PEP 227: Nested Scopes |
| 36 |
====================== |
| 37 |
|
| 38 |
The largest change in Python 2.1 is to Python's scoping rules. In Python 2.0, |
| 39 |
at any given time there are at most three namespaces used to look up variable |
| 40 |
names: local, module-level, and the built-in namespace. This often surprised |
| 41 |
people because it didn't match their intuitive expectations. For example, a |
| 42 |
nested recursive function definition doesn't work:: |
| 43 |
|
| 44 |
def f(): |
| 45 |
... |
| 46 |
def g(value): |
| 47 |
... |
| 48 |
return g(value-1) + 1 |
| 49 |
... |
| 50 |
|
| 51 |
The function :func:`!g` will always raise a :exc:`NameError` exception, because |
| 52 |
the binding of the name ``g`` isn't in either its local namespace or in the |
| 53 |
module-level namespace. This isn't much of a problem in practice (how often do |
| 54 |
you recursively define interior functions like this?), but this also made using |
| 55 |
the :keyword:`lambda` expression clumsier, and this was a problem in practice. |
| 56 |
In code which uses :keyword:`lambda` you can often find local variables being |
| 57 |
copied by passing them as the default values of arguments. :: |
| 58 |
|
| 59 |
def find(self, name): |
| 60 |
"Return list of any entries equal to 'name'" |
| 61 |
L = filter(lambda x, name=name: x == name, |
| 62 |
self.list_attribute) |
| 63 |
return L |
| 64 |
|
| 65 |
The readability of Python code written in a strongly functional style suffers |
| 66 |
greatly as a result. |
| 67 |
|
| 68 |
The most significant change to Python 2.1 is that static scoping has been added |
| 69 |
to the language to fix this problem. As a first effect, the ``name=name`` |
| 70 |
default argument is now unnecessary in the above example. Put simply, when a |
| 71 |
given variable name is not assigned a value within a function (by an assignment, |
| 72 |
or the :keyword:`def`, :keyword:`class`, or :keyword:`import` statements), |
| 73 |
references to the variable will be looked up in the local namespace of the |
| 74 |
enclosing scope. A more detailed explanation of the rules, and a dissection of |
| 75 |
the implementation, can be found in the PEP. |
| 76 |
|
| 77 |
This change may cause some compatibility problems for code where the same |
| 78 |
variable name is used both at the module level and as a local variable within a |
| 79 |
function that contains further function definitions. This seems rather unlikely |
| 80 |
though, since such code would have been pretty confusing to read in the first |
| 81 |
place. |
| 82 |
|
| 83 |
One side effect of the change is that the ``from module import *`` and |
| 84 |
``exec`` statements have been made illegal inside a function scope under |
| 85 |
certain conditions. The Python reference manual has said all along that ``from |
| 86 |
module import *`` is only legal at the top level of a module, but the CPython |
| 87 |
interpreter has never enforced this before. As part of the implementation of |
| 88 |
nested scopes, the compiler which turns Python source into bytecodes has to |
| 89 |
generate different code to access variables in a containing scope. ``from |
| 90 |
module import *`` and ``exec`` make it impossible for the compiler to |
| 91 |
figure this out, because they add names to the local namespace that are |
| 92 |
unknowable at compile time. Therefore, if a function contains function |
| 93 |
definitions or :keyword:`lambda` expressions with free variables, the compiler |
| 94 |
will flag this by raising a :exc:`SyntaxError` exception. |
| 95 |
|
| 96 |
To make the preceding explanation a bit clearer, here's an example:: |
| 97 |
|
| 98 |
x = 1 |
| 99 |
def f(): |
| 100 |
# The next line is a syntax error |
| 101 |
exec 'x=2' |
| 102 |
def g(): |
| 103 |
return x |
| 104 |
|
| 105 |
Line 4 containing the ``exec`` statement is a syntax error, since |
| 106 |
``exec`` would define a new local variable named ``x`` whose value should |
| 107 |
be accessed by :func:`!g`. |
| 108 |
|
| 109 |
This shouldn't be much of a limitation, since ``exec`` is rarely used in |
| 110 |
most Python code (and when it is used, it's often a sign of a poor design |
| 111 |
anyway). |
| 112 |
|
| 113 |
Compatibility concerns have led to nested scopes being introduced gradually; in |
| 114 |
Python 2.1, they aren't enabled by default, but can be turned on within a module |
| 115 |
by using a future statement as described in :pep:`236`. (See the following section |
| 116 |
for further discussion of :pep:`236`.) In Python 2.2, nested scopes will become |
| 117 |
the default and there will be no way to turn them off, but users will have had |
| 118 |
all of 2.1's lifetime to fix any breakage resulting from their introduction. |
| 119 |
|
| 120 |
|
| 121 |
.. seealso:: |
| 122 |
|
| 123 |
:pep:`227` - Statically Nested Scopes |
| 124 |
Written and implemented by Jeremy Hylton. |
| 125 |
|
| 126 |
.. ====================================================================== |
| 127 |
|
| 128 |
|
| 129 |
PEP 236: __future__ Directives |
| 130 |
============================== |
| 131 |
|
| 132 |
The reaction to nested scopes was widespread concern about the dangers of |
| 133 |
breaking code with the 2.1 release, and it was strong enough to make the |
| 134 |
Pythoneers take a more conservative approach. This approach consists of |
| 135 |
introducing a convention for enabling optional functionality in release N that |
| 136 |
will become compulsory in release N+1. |
| 137 |
|
| 138 |
The syntax uses a ``from...import`` statement using the reserved module name |
| 139 |
:mod:`__future__`. Nested scopes can be enabled by the following statement:: |
| 140 |
|
| 141 |
from __future__ import nested_scopes |
| 142 |
|
| 143 |
While it looks like a normal :keyword:`import` statement, it's not; there are |
| 144 |
strict rules on where such a future statement can be put. They can only be at |
| 145 |
the top of a module, and must precede any Python code or regular |
| 146 |
:keyword:`!import` statements. This is because such statements can affect how |
| 147 |
the Python bytecode compiler parses code and generates bytecode, so they must |
| 148 |
precede any statement that will result in bytecodes being produced. |
| 149 |
|
| 150 |
|
| 151 |
.. seealso:: |
| 152 |
|
| 153 |
:pep:`236` - Back to the :mod:`__future__` |
| 154 |
Written by Tim Peters, and primarily implemented by Jeremy Hylton. |
| 155 |
|
| 156 |
.. ====================================================================== |
| 157 |
|
| 158 |
|
| 159 |
PEP 207: Rich Comparisons |
| 160 |
========================= |
| 161 |
|
| 162 |
In earlier versions, Python's support for implementing comparisons on user-defined |
| 163 |
classes and extension types was quite simple. Classes could implement a |
| 164 |
:meth:`!__cmp__` method that was given two instances of a class, and could only |
| 165 |
return 0 if they were equal or +1 or -1 if they weren't; the method couldn't |
| 166 |
raise an exception or return anything other than a Boolean value. Users of |
| 167 |
Numeric Python often found this model too weak and restrictive, because in the |
| 168 |
number-crunching programs that numeric Python is used for, it would be more |
| 169 |
useful to be able to perform elementwise comparisons of two matrices, returning |
| 170 |
a matrix containing the results of a given comparison for each element. If the |
| 171 |
two matrices are of different sizes, then the compare has to be able to raise an |
| 172 |
exception to signal the error. |
| 173 |
|
| 174 |
In Python 2.1, rich comparisons were added in order to support this need. |
| 175 |
Python classes can now individually overload each of the ``<``, ``<=``, ``>``, |
| 176 |
``>=``, ``==``, and ``!=`` operations. The new magic method names are: |
| 177 |
|
| 178 |
+-----------+------------------------+ |
| 179 |
| Operation | Method name | |
| 180 |
+===========+========================+ |
| 181 |
| ``<`` | :meth:`~object.__lt__` | |
| 182 |
+-----------+------------------------+ |
| 183 |
| ``<=`` | :meth:`~object.__le__` | |
| 184 |
+-----------+------------------------+ |
| 185 |
| ``>`` | :meth:`~object.__gt__` | |
| 186 |
+-----------+------------------------+ |
| 187 |
| ``>=`` | :meth:`~object.__ge__` | |
| 188 |
+-----------+------------------------+ |
| 189 |
| ``==`` | :meth:`~object.__eq__` | |
| 190 |
+-----------+------------------------+ |
| 191 |
| ``!=`` | :meth:`~object.__ne__` | |
| 192 |
+-----------+------------------------+ |
| 193 |
|
| 194 |
(The magic methods are named after the corresponding Fortran operators ``.LT.``. |
| 195 |
``.LE.``, &c. Numeric programmers are almost certainly quite familiar with |
| 196 |
these names and will find them easy to remember.) |
| 197 |
|
| 198 |
Each of these magic methods is of the form ``method(self, other)``, where |
| 199 |
``self`` will be the object on the left-hand side of the operator, while |
| 200 |
``other`` will be the object on the right-hand side. For example, the |
| 201 |
expression ``A < B`` will cause ``A.__lt__(B)`` to be called. |
| 202 |
|
| 203 |
Each of these magic methods can return anything at all: a Boolean, a matrix, a |
| 204 |
list, or any other Python object. Alternatively they can raise an exception if |
| 205 |
the comparison is impossible, inconsistent, or otherwise meaningless. |
| 206 |
|
| 207 |
The built-in ``cmp(A,B)`` function can use the rich comparison machinery, |
| 208 |
and now accepts an optional argument specifying which comparison operation to |
| 209 |
use; this is given as one of the strings ``"<"``, ``"<="``, ``">"``, ``">="``, |
| 210 |
``"=="``, or ``"!="``. If called without the optional third argument, |
| 211 |
:func:`!cmp` will only return -1, 0, or +1 as in previous versions of Python; |
| 212 |
otherwise it will call the appropriate method and can return any Python object. |
| 213 |
|
| 214 |
There are also corresponding changes of interest to C programmers; there's a new |
| 215 |
slot ``tp_richcmp`` in type objects and an API for performing a given rich |
| 216 |
comparison. I won't cover the C API here, but will refer you to :pep:`207`, or to |
| 217 |
2.1's C API documentation, for the full list of related functions. |
| 218 |
|
| 219 |
|
| 220 |
.. seealso:: |
| 221 |
|
| 222 |
:pep:`207` - Rich Comparisons |
| 223 |
Written by Guido van Rossum, heavily based on earlier work by David Ascher, and |
| 224 |
implemented by Guido van Rossum. |
| 225 |
|
| 226 |
.. ====================================================================== |
| 227 |
|
| 228 |
|
| 229 |
PEP 230: Warning Framework |
| 230 |
========================== |
| 231 |
|
| 232 |
Over its 10 years of existence, Python has accumulated a certain number of |
| 233 |
obsolete modules and features along the way. It's difficult to know when a |
| 234 |
feature is safe to remove, since there's no way of knowing how much code uses it |
| 235 |
--- perhaps no programs depend on the feature, or perhaps many do. To enable |
| 236 |
removing old features in a more structured way, a warning framework was added. |
| 237 |
When the Python developers want to get rid of a feature, it will first trigger a |
| 238 |
warning in the next version of Python. The following Python version can then |
| 239 |
drop the feature, and users will have had a full release cycle to remove uses of |
| 240 |
the old feature. |
| 241 |
|
| 242 |
Python 2.1 adds the warning framework to be used in this scheme. It adds a |
| 243 |
:mod:`warnings` module that provide functions to issue warnings, and to filter |
| 244 |
out warnings that you don't want to be displayed. Third-party modules can also |
| 245 |
use this framework to deprecate old features that they no longer wish to |
| 246 |
support. |
| 247 |
|
| 248 |
For example, in Python 2.1 the :mod:`!regex` module is deprecated, so importing |
| 249 |
it causes a warning to be printed:: |
| 250 |
|
| 251 |
>>> import regex |
| 252 |
__main__:1: DeprecationWarning: the regex module |
| 253 |
is deprecated; please use the re module |
| 254 |
>>> |
| 255 |
|
| 256 |
Warnings can be issued by calling the :func:`warnings.warn` function:: |
| 257 |
|
| 258 |
warnings.warn("feature X no longer supported") |
| 259 |
|
| 260 |
The first parameter is the warning message; an additional optional parameters |
| 261 |
can be used to specify a particular warning category. |
| 262 |
|
| 263 |
Filters can be added to disable certain warnings; a regular expression pattern |
| 264 |
can be applied to the message or to the module name in order to suppress a |
| 265 |
warning. For example, you may have a program that uses the :mod:`!regex` module |
| 266 |
and not want to spare the time to convert it to use the :mod:`re` module right |
| 267 |
now. The warning can be suppressed by calling :: |
| 268 |
|
| 269 |
import warnings |
| 270 |
warnings.filterwarnings(action = 'ignore', |
| 271 |
message='.*regex module is deprecated', |
| 272 |
category=DeprecationWarning, |
| 273 |
module = '__main__') |
| 274 |
|
| 275 |
This adds a filter that will apply only to warnings of the class |
| 276 |
:class:`DeprecationWarning` triggered in the :mod:`__main__` module, and applies |
| 277 |
a regular expression to only match the message about the :mod:`!regex` module |
| 278 |
being deprecated, and will cause such warnings to be ignored. Warnings can also |
| 279 |
be printed only once, printed every time the offending code is executed, or |
| 280 |
turned into exceptions that will cause the program to stop (unless the |
| 281 |
exceptions are caught in the usual way, of course). |
| 282 |
|
| 283 |
Functions were also added to Python's C API for issuing warnings; refer to PEP |
| 284 |
230 or to Python's API documentation for the details. |
| 285 |
|
| 286 |
|
| 287 |
.. seealso:: |
| 288 |
|
| 289 |
:pep:`5` - Guidelines for Language Evolution |
| 290 |
Written by Paul Prescod, to specify procedures to be followed when removing old |
| 291 |
features from Python. The policy described in this PEP hasn't been officially |
| 292 |
adopted, but the eventual policy probably won't be too different from Prescod's |
| 293 |
proposal. |
| 294 |
|
| 295 |
:pep:`230` - Warning Framework |
| 296 |
Written and implemented by Guido van Rossum. |
| 297 |
|
| 298 |
.. ====================================================================== |
| 299 |
|
| 300 |
|
| 301 |
PEP 229: New Build System |
| 302 |
========================= |
| 303 |
|
| 304 |
When compiling Python, the user had to go in and edit the :file:`Modules/Setup` |
| 305 |
file in order to enable various additional modules; the default set is |
| 306 |
relatively small and limited to modules that compile on most Unix platforms. |
| 307 |
This means that on Unix platforms with many more features, most notably Linux, |
| 308 |
Python installations often don't contain all useful modules they could. |
| 309 |
|
| 310 |
Python 2.0 added the Distutils, a set of modules for distributing and installing |
| 311 |
extensions. In Python 2.1, the Distutils are used to compile much of the |
| 312 |
standard library of extension modules, autodetecting which ones are supported on |
| 313 |
the current machine. It's hoped that this will make Python installations easier |
| 314 |
and more featureful. |
| 315 |
|
| 316 |
Instead of having to edit the :file:`Modules/Setup` file in order to enable |
| 317 |
modules, a :file:`setup.py` script in the top directory of the Python source |
| 318 |
distribution is run at build time, and attempts to discover which modules can be |
| 319 |
enabled by examining the modules and header files on the system. If a module is |
| 320 |
configured in :file:`Modules/Setup`, the :file:`setup.py` script won't attempt |
| 321 |
to compile that module and will defer to the :file:`Modules/Setup` file's |
| 322 |
contents. This provides a way to specific any strange command-line flags or |
| 323 |
libraries that are required for a specific platform. |
| 324 |
|
| 325 |
In another far-reaching change to the build mechanism, Neil Schemenauer |
| 326 |
restructured things so Python now uses a single makefile that isn't recursive, |
| 327 |
instead of makefiles in the top directory and in each of the :file:`Python/`, |
| 328 |
:file:`Parser/`, :file:`Objects/`, and :file:`Modules/` subdirectories. This |
| 329 |
makes building Python faster and also makes hacking the Makefiles clearer and |
| 330 |
simpler. |
| 331 |
|
| 332 |
|
| 333 |
.. seealso:: |
| 334 |
|
| 335 |
:pep:`229` - Using Distutils to Build Python |
| 336 |
Written and implemented by A.M. Kuchling. |
| 337 |
|
| 338 |
.. ====================================================================== |
| 339 |
|
| 340 |
|
| 341 |
PEP 205: Weak References |
| 342 |
======================== |
| 343 |
|
| 344 |
Weak references, available through the :mod:`weakref` module, are a minor but |
| 345 |
useful new data type in the Python programmer's toolbox. |
| 346 |
|
| 347 |
Storing a reference to an object (say, in a dictionary or a list) has the side |
| 348 |
effect of keeping that object alive forever. There are a few specific cases |
| 349 |
where this behaviour is undesirable, object caches being the most common one, |
| 350 |
and another being circular references in data structures such as trees. |
| 351 |
|
| 352 |
For example, consider a memoizing function that caches the results of another |
| 353 |
function ``f(x)`` by storing the function's argument and its result in a |
| 354 |
dictionary:: |
| 355 |
|
| 356 |
_cache = {} |
| 357 |
def memoize(x): |
| 358 |
if _cache.has_key(x): |
| 359 |
return _cache[x] |
| 360 |
|
| 361 |
retval = f(x) |
| 362 |
|
| 363 |
# Cache the returned object |
| 364 |
_cache[x] = retval |
| 365 |
|
| 366 |
return retval |
| 367 |
|
| 368 |
This version works for simple things such as integers, but it has a side effect; |
| 369 |
the ``_cache`` dictionary holds a reference to the return values, so they'll |
| 370 |
never be deallocated until the Python process exits and cleans up. This isn't |
| 371 |
very noticeable for integers, but if :func:`!f` returns an object, or a data |
| 372 |
structure that takes up a lot of memory, this can be a problem. |
| 373 |
|
| 374 |
Weak references provide a way to implement a cache that won't keep objects alive |
| 375 |
beyond their time. If an object is only accessible through weak references, the |
| 376 |
object will be deallocated and the weak references will now indicate that the |
| 377 |
object it referred to no longer exists. A weak reference to an object *obj* is |
| 378 |
created by calling ``wr = weakref.ref(obj)``. The object being referred to is |
| 379 |
returned by calling the weak reference as if it were a function: ``wr()``. It |
| 380 |
will return the referenced object, or ``None`` if the object no longer exists. |
| 381 |
|
| 382 |
This makes it possible to write a :func:`!memoize` function whose cache doesn't |
| 383 |
keep objects alive, by storing weak references in the cache. :: |
| 384 |
|
| 385 |
_cache = {} |
| 386 |
def memoize(x): |
| 387 |
if _cache.has_key(x): |
| 388 |
obj = _cache[x]() |
| 389 |
# If weak reference object still exists, |
| 390 |
# return it |
| 391 |
if obj is not None: return obj |
| 392 |
|
| 393 |
retval = f(x) |
| 394 |
|
| 395 |
# Cache a weak reference |
| 396 |
_cache[x] = weakref.ref(retval) |
| 397 |
|
| 398 |
return retval |
| 399 |
|
| 400 |
The :mod:`weakref` module also allows creating proxy objects which behave like |
| 401 |
weak references --- an object referenced only by proxy objects is deallocated -- |
| 402 |
but instead of requiring an explicit call to retrieve the object, the proxy |
| 403 |
transparently forwards all operations to the object as long as the object still |
| 404 |
exists. If the object is deallocated, attempting to use a proxy will cause a |
| 405 |
:exc:`!weakref.ReferenceError` exception to be raised. :: |
| 406 |
|
| 407 |
proxy = weakref.proxy(obj) |
| 408 |
proxy.attr # Equivalent to obj.attr |
| 409 |
proxy.meth() # Equivalent to obj.meth() |
| 410 |
del obj |
| 411 |
proxy.attr # raises weakref.ReferenceError |
| 412 |
|
| 413 |
|
| 414 |
.. seealso:: |
| 415 |
|
| 416 |
:pep:`205` - Weak References |
| 417 |
Written and implemented by Fred L. Drake, Jr. |
| 418 |
|
| 419 |
.. ====================================================================== |
| 420 |
|
| 421 |
|
| 422 |
PEP 232: Function Attributes |
| 423 |
============================ |
| 424 |
|
| 425 |
In Python 2.1, functions can now have arbitrary information attached to them. |
| 426 |
People were often using docstrings to hold information about functions and |
| 427 |
methods, because the :attr:`~function.__doc__` attribute was the only way of |
| 428 |
attaching any |
| 429 |
information to a function. For example, in the Zope web application server, |
| 430 |
functions are marked as safe for public access by having a docstring, and in |
| 431 |
John Aycock's SPARK parsing framework, docstrings hold parts of the BNF grammar |
| 432 |
to be parsed. This overloading is unfortunate, since docstrings are really |
| 433 |
intended to hold a function's documentation; for example, it means you can't |
| 434 |
properly document functions intended for private use in Zope. |
| 435 |
|
| 436 |
Arbitrary attributes can now be set and retrieved on functions using the regular |
| 437 |
Python syntax:: |
| 438 |
|
| 439 |
def f(): pass |
| 440 |
|
| 441 |
f.publish = 1 |
| 442 |
f.secure = 1 |
| 443 |
f.grammar = "A ::= B (C D)*" |
| 444 |
|
| 445 |
The dictionary containing attributes can be accessed as the function's |
| 446 |
:attr:`~function.__dict__`. Unlike the :attr:`~type.__dict__` attribute of class instances, in |
| 447 |
functions you can actually assign a new dictionary to :attr:`~function.__dict__`, though |
| 448 |
the new value is restricted to a regular Python dictionary; you *can't* be |
| 449 |
tricky and set it to a :class:`!UserDict` instance, or any other random object |
| 450 |
that behaves like a mapping. |
| 451 |
|
| 452 |
|
| 453 |
.. seealso:: |
| 454 |
|
| 455 |
:pep:`232` - Function Attributes |
| 456 |
Written and implemented by Barry Warsaw. |
| 457 |
|
| 458 |
.. ====================================================================== |
| 459 |
|
| 460 |
|
| 461 |
PEP 235: Importing Modules on Case-Insensitive Platforms |
| 462 |
======================================================== |
| 463 |
|
| 464 |
Some operating systems have filesystems that are case-insensitive, MacOS and |
| 465 |
Windows being the primary examples; on these systems, it's impossible to |
| 466 |
distinguish the filenames ``FILE.PY`` and ``file.py``, even though they do store |
| 467 |
the file's name in its original case (they're case-preserving, too). |
| 468 |
|
| 469 |
In Python 2.1, the :keyword:`import` statement will work to simulate case-sensitivity |
| 470 |
on case-insensitive platforms. Python will now search for the first |
| 471 |
case-sensitive match by default, raising an :exc:`ImportError` if no such file |
| 472 |
is found, so ``import file`` will not import a module named ``FILE.PY``. |
| 473 |
Case-insensitive matching can be requested by setting the :envvar:`PYTHONCASEOK` |
| 474 |
environment variable before starting the Python interpreter. |
| 475 |
|
| 476 |
.. ====================================================================== |
| 477 |
|
| 478 |
|
| 479 |
PEP 217: Interactive Display Hook |
| 480 |
================================= |
| 481 |
|
| 482 |
When using the Python interpreter interactively, the output of commands is |
| 483 |
displayed using the built-in :func:`repr` function. In Python 2.1, the variable |
| 484 |
:func:`sys.displayhook` can be set to a callable object which will be called |
| 485 |
instead of :func:`repr`. For example, you can set it to a special |
| 486 |
pretty-printing function:: |
| 487 |
|
| 488 |
>>> # Create a recursive data structure |
| 489 |
... L = [1,2,3] |
| 490 |
>>> L.append(L) |
| 491 |
>>> L # Show Python's default output |
| 492 |
[1, 2, 3, [...]] |
| 493 |
>>> # Use pprint.pprint() as the display function |
| 494 |
... import sys, pprint |
| 495 |
>>> sys.displayhook = pprint.pprint |
| 496 |
>>> L |
| 497 |
[1, 2, 3, <Recursion on list with id=135143996>] |
| 498 |
>>> |
| 499 |
|
| 500 |
|
| 501 |
.. seealso:: |
| 502 |
|
| 503 |
:pep:`217` - Display Hook for Interactive Use |
| 504 |
Written and implemented by Moshe Zadka. |
| 505 |
|
| 506 |
.. ====================================================================== |
| 507 |
|
| 508 |
|
| 509 |
PEP 208: New Coercion Model |
| 510 |
=========================== |
| 511 |
|
| 512 |
How numeric coercion is done at the C level was significantly modified. This |
| 513 |
will only affect the authors of C extensions to Python, allowing them more |
| 514 |
flexibility in writing extension types that support numeric operations. |
| 515 |
|
| 516 |
Extension types can now set the type flag ``Py_TPFLAGS_CHECKTYPES`` in their |
| 517 |
``PyTypeObject`` structure to indicate that they support the new coercion model. |
| 518 |
In such extension types, the numeric slot functions can no longer assume that |
| 519 |
they'll be passed two arguments of the same type; instead they may be passed two |
| 520 |
arguments of differing types, and can then perform their own internal coercion. |
| 521 |
If the slot function is passed a type it can't handle, it can indicate the |
| 522 |
failure by returning a reference to the ``Py_NotImplemented`` singleton value. |
| 523 |
The numeric functions of the other type will then be tried, and perhaps they can |
| 524 |
handle the operation; if the other type also returns ``Py_NotImplemented``, then |
| 525 |
a :exc:`TypeError` will be raised. Numeric methods written in Python can also |
| 526 |
return ``Py_NotImplemented``, causing the interpreter to act as if the method |
| 527 |
did not exist (perhaps raising a :exc:`TypeError`, perhaps trying another |
| 528 |
object's numeric methods). |
| 529 |
|
| 530 |
|
| 531 |
.. seealso:: |
| 532 |
|
| 533 |
:pep:`208` - Reworking the Coercion Model |
| 534 |
Written and implemented by Neil Schemenauer, heavily based upon earlier work by |
| 535 |
Marc-André Lemburg. Read this to understand the fine points of how numeric |
| 536 |
operations will now be processed at the C level. |
| 537 |
|
| 538 |
.. ====================================================================== |
| 539 |
|
| 540 |
|
| 541 |
PEP 241: Metadata in Python Packages |
| 542 |
==================================== |
| 543 |
|
| 544 |
A common complaint from Python users is that there's no single catalog of all |
| 545 |
the Python modules in existence. T. Middleton's Vaults of Parnassus at |
| 546 |
``www.vex.net/parnassus/`` (retired in February 2009, `available in the |
| 547 |
Internet Archive Wayback Machine |
| 548 |
<https://web.archive.org/web/20090130140102/http://www.vex.net/parnassus/>`_) |
| 549 |
was the largest catalog of Python modules, but |
| 550 |
registering software at the Vaults is optional, and many people did not bother. |
| 551 |
|
| 552 |
As a first small step toward fixing the problem, Python software packaged using |
| 553 |
the Distutils :command:`sdist` command will include a file named |
| 554 |
:file:`PKG-INFO` containing information about the package such as its name, |
| 555 |
version, and author (metadata, in cataloguing terminology). :pep:`241` contains |
| 556 |
the full list of fields that can be present in the :file:`PKG-INFO` file. As |
| 557 |
people began to package their software using Python 2.1, more and more packages |
| 558 |
will include metadata, making it possible to build automated cataloguing systems |
| 559 |
and experiment with them. With the result experience, perhaps it'll be possible |
| 560 |
to design a really good catalog and then build support for it into Python 2.2. |
| 561 |
For example, the Distutils :command:`sdist` and :command:`bdist_\*` commands |
| 562 |
could support an ``upload`` option that would automatically upload your |
| 563 |
package to a catalog server. |
| 564 |
|
| 565 |
You can start creating packages containing :file:`PKG-INFO` even if you're not |
| 566 |
using Python 2.1, since a new release of the Distutils will be made for users of |
| 567 |
earlier Python versions. Version 1.0.2 of the Distutils includes the changes |
| 568 |
described in :pep:`241`, as well as various bugfixes and enhancements. It will be |
| 569 |
available from the Distutils SIG at https://www.python.org/community/sigs/current/distutils-sig/. |
| 570 |
|
| 571 |
|
| 572 |
.. seealso:: |
| 573 |
|
| 574 |
:pep:`241` - Metadata for Python Software Packages |
| 575 |
Written and implemented by A.M. Kuchling. |
| 576 |
|
| 577 |
:pep:`243` - Module Repository Upload Mechanism |
| 578 |
Written by Sean Reifschneider, this draft PEP describes a proposed mechanism for |
| 579 |
uploading Python packages to a central server. |
| 580 |
|
| 581 |
.. ====================================================================== |
| 582 |
|
| 583 |
|
| 584 |
New and Improved Modules |
| 585 |
======================== |
| 586 |
|
| 587 |
* Ka-Ping Yee contributed two new modules: :mod:`!inspect.py`, a module for |
| 588 |
getting information about live Python code, and :mod:`!pydoc.py`, a module for |
| 589 |
interactively converting docstrings to HTML or text. As a bonus, |
| 590 |
:file:`Tools/scripts/pydoc`, which is now automatically installed, uses |
| 591 |
:mod:`!pydoc.py` to display documentation given a Python module, package, or |
| 592 |
class name. For example, ``pydoc xml.dom`` displays the following:: |
| 593 |
|
| 594 |
Python Library Documentation: package xml.dom in xml |
| 595 |
|
| 596 |
NAME |
| 597 |
xml.dom - W3C Document Object Model implementation for Python. |
| 598 |
|
| 599 |
FILE |
| 600 |
/usr/local/lib/python2.1/xml/dom/__init__.pyc |
| 601 |
|
| 602 |
DESCRIPTION |
| 603 |
The Python mapping of the Document Object Model is documented in the |
| 604 |
Python Library Reference in the section on the xml.dom package. |
| 605 |
|
| 606 |
This package contains the following modules: |
| 607 |
... |
| 608 |
|
| 609 |
:file:`pydoc` also includes a Tk-based interactive help browser. :file:`pydoc` |
| 610 |
quickly becomes addictive; try it out! |
| 611 |
|
| 612 |
* Two different modules for unit testing were added to the standard library. |
| 613 |
The :mod:`doctest` module, contributed by Tim Peters, provides a testing |
| 614 |
framework based on running embedded examples in docstrings and comparing the |
| 615 |
results against the expected output. PyUnit, contributed by Steve Purcell, is a |
| 616 |
unit testing framework inspired by JUnit, which was in turn an adaptation of |
| 617 |
Kent Beck's Smalltalk testing framework. See https://pyunit.sourceforge.net/ for |
| 618 |
more information about PyUnit. |
| 619 |
|
| 620 |
* The :mod:`difflib` module contains a class, :class:`~difflib.SequenceMatcher`, which |
| 621 |
compares two sequences and computes the changes required to transform one |
| 622 |
sequence into the other. For example, this module can be used to write a tool |
| 623 |
similar to the Unix :program:`diff` program, and in fact the sample program |
| 624 |
:file:`Tools/scripts/ndiff.py` demonstrates how to write such a script. |
| 625 |
|
| 626 |
* :mod:`curses.panel`, a wrapper for the panel library, part of ncurses and of |
| 627 |
SYSV curses, was contributed by Thomas Gellekum. The panel library provides |
| 628 |
windows with the additional feature of depth. Windows can be moved higher or |
| 629 |
lower in the depth ordering, and the panel library figures out where panels |
| 630 |
overlap and which sections are visible. |
| 631 |
|
| 632 |
* The PyXML package has gone through a few releases since Python 2.0, and Python |
| 633 |
2.1 includes an updated version of the :mod:`xml` package. Some of the |
| 634 |
noteworthy changes include support for Expat 1.2 and later versions, the ability |
| 635 |
for Expat parsers to handle files in any encoding supported by Python, and |
| 636 |
various bugfixes for SAX, DOM, and the :mod:`!minidom` module. |
| 637 |
|
| 638 |
* Ping also contributed another hook for handling uncaught exceptions. |
| 639 |
:func:`sys.excepthook` can be set to a callable object. When an exception isn't |
| 640 |
caught by any :keyword:`try`...\ :keyword:`except` blocks, the exception will be |
| 641 |
passed to :func:`sys.excepthook`, which can then do whatever it likes. At the |
| 642 |
Ninth Python Conference, Ping demonstrated an application for this hook: |
| 643 |
printing an extended traceback that not only lists the stack frames, but also |
| 644 |
lists the function arguments and the local variables for each frame. |
| 645 |
|
| 646 |
* Various functions in the :mod:`time` module, such as :func:`~time.asctime` and |
| 647 |
:func:`~time.localtime`, require a floating-point argument containing the time in |
| 648 |
seconds since the epoch. The most common use of these functions is to work with |
| 649 |
the current time, so the floating-point argument has been made optional; when a |
| 650 |
value isn't provided, the current time will be used. For example, log file |
| 651 |
entries usually need a string containing the current time; in Python 2.1, |
| 652 |
``time.asctime()`` can be used, instead of the lengthier |
| 653 |
``time.asctime(time.localtime(time.time()))`` that was previously required. |
| 654 |
|
| 655 |
This change was proposed and implemented by Thomas Wouters. |
| 656 |
|
| 657 |
* The :mod:`ftplib` module now defaults to retrieving files in passive mode, |
| 658 |
because passive mode is more likely to work from behind a firewall. This |
| 659 |
request came from the Debian bug tracking system, since other Debian packages |
| 660 |
use :mod:`ftplib` to retrieve files and then don't work from behind a firewall. |
| 661 |
It's deemed unlikely that this will cause problems for anyone, because Netscape |
| 662 |
defaults to passive mode and few people complain, but if passive mode is |
| 663 |
unsuitable for your application or network setup, call ``set_pasv(0)`` on |
| 664 |
FTP objects to disable passive mode. |
| 665 |
|
| 666 |
* Support for raw socket access has been added to the :mod:`socket` module, |
| 667 |
contributed by Grant Edwards. |
| 668 |
|
| 669 |
* The :mod:`pstats` module now contains a simple interactive statistics browser |
| 670 |
for displaying timing profiles for Python programs, invoked when the module is |
| 671 |
run as a script. Contributed by Eric S. Raymond. |
| 672 |
|
| 673 |
* A new implementation-dependent function, ``sys._getframe([depth])``, has |
| 674 |
been added to return a given frame object from the current call stack. |
| 675 |
:func:`sys._getframe` returns the frame at the top of the call stack; if the |
| 676 |
optional integer argument *depth* is supplied, the function returns the frame |
| 677 |
that is *depth* calls below the top of the stack. For example, |
| 678 |
``sys._getframe(1)`` returns the caller's frame object. |
| 679 |
|
| 680 |
This function is only present in CPython, not in Jython or the .NET |
| 681 |
implementation. Use it for debugging, and resist the temptation to put it into |
| 682 |
production code. |
| 683 |
|
| 684 |
.. ====================================================================== |
| 685 |
|
| 686 |
|
| 687 |
Other Changes and Fixes |
| 688 |
======================= |
| 689 |
|
| 690 |
There were relatively few smaller changes made in Python 2.1 due to the shorter |
| 691 |
release cycle. A search through the CVS change logs turns up 117 patches |
| 692 |
applied, and 136 bugs fixed; both figures are likely to be underestimates. Some |
| 693 |
of the more notable changes are: |
| 694 |
|
| 695 |
* A specialized object allocator is now optionally available, that should be |
| 696 |
faster than the system :c:func:`malloc` and have less memory overhead. The |
| 697 |
allocator uses C's :c:func:`!malloc` function to get large pools of memory, and |
| 698 |
then fulfills smaller memory requests from these pools. It can be enabled by |
| 699 |
providing the :option:`!--with-pymalloc` option to the :program:`configure` |
| 700 |
script; see :file:`Objects/obmalloc.c` for the implementation details. |
| 701 |
|
| 702 |
Authors of C extension modules should test their code with the object allocator |
| 703 |
enabled, because some incorrect code may break, causing core dumps at runtime. |
| 704 |
There are a bunch of memory allocation functions in Python's C API that have |
| 705 |
previously been just aliases for the C library's :c:func:`malloc` and |
| 706 |
:c:func:`free`, meaning that if you accidentally called mismatched functions, the |
| 707 |
error wouldn't be noticeable. When the object allocator is enabled, these |
| 708 |
functions aren't aliases of :c:func:`!malloc` and :c:func:`!free` any more, and |
| 709 |
calling the wrong function to free memory will get you a core dump. For |
| 710 |
example, if memory was allocated using :c:macro:`PyMem_New`, it has to be freed |
| 711 |
using :c:func:`PyMem_Del`, not :c:func:`!free`. A few modules included with Python |
| 712 |
fell afoul of this and had to be fixed; doubtless there are more third-party |
| 713 |
modules that will have the same problem. |
| 714 |
|
| 715 |
The object allocator was contributed by Vladimir Marangozov. |
| 716 |
|
| 717 |
* The speed of line-oriented file I/O has been improved because people often |
| 718 |
complain about its lack of speed, and because it's often been used as a naïve |
| 719 |
benchmark. The :meth:`readline` method of file objects has therefore been |
| 720 |
rewritten to be much faster. The exact amount of the speedup will vary from |
| 721 |
platform to platform depending on how slow the C library's :c:func:`!getc` was, but |
| 722 |
is around 66%, and potentially much faster on some particular operating systems. |
| 723 |
Tim Peters did much of the benchmarking and coding for this change, motivated by |
| 724 |
a discussion in comp.lang.python. |
| 725 |
|
| 726 |
A new module and method for file objects was also added, contributed by Jeff |
| 727 |
Epler. The new method, :meth:`!xreadlines`, is similar to the existing |
| 728 |
:func:`!xrange` built-in. :func:`!xreadlines` returns an opaque sequence object |
| 729 |
that only supports being iterated over, reading a line on every iteration but |
| 730 |
not reading the entire file into memory as the existing :meth:`!readlines` method |
| 731 |
does. You'd use it like this:: |
| 732 |
|
| 733 |
for line in sys.stdin.xreadlines(): |
| 734 |
# ... do something for each line ... |
| 735 |
... |
| 736 |
|
| 737 |
For a fuller discussion of the line I/O changes, see the python-dev summary for |
| 738 |
January 1--15, 2001 at https://mail.python.org/pipermail/python-dev/2001-January/. |
| 739 |
|
| 740 |
* A new method, :meth:`~dict.popitem`, was added to dictionaries to enable |
| 741 |
destructively iterating through the contents of a dictionary; this can be faster |
| 742 |
for large dictionaries because there's no need to construct a list containing |
| 743 |
all the keys or values. ``D.popitem()`` removes a random ``(key, value)`` pair |
| 744 |
from the dictionary ``D`` and returns it as a 2-tuple. This was implemented |
| 745 |
mostly by Tim Peters and Guido van Rossum, after a suggestion and preliminary |
| 746 |
patch by Moshe Zadka. |
| 747 |
|
| 748 |
* Modules can now control which names are imported when ``from module import *`` |
| 749 |
is used, by defining an ``__all__`` attribute containing a list of names that |
| 750 |
will be imported. One common complaint is that if the module imports other |
| 751 |
modules such as :mod:`sys` or :mod:`string`, ``from module import *`` will add |
| 752 |
them to the importing module's namespace. To fix this, simply list the public |
| 753 |
names in ``__all__``:: |
| 754 |
|
| 755 |
# List public names |
| 756 |
__all__ = ['Database', 'open'] |
| 757 |
|
| 758 |
A stricter version of this patch was first suggested and implemented by Ben |
| 759 |
Wolfson, but after some python-dev discussion, a weaker final version was |
| 760 |
checked in. |
| 761 |
|
| 762 |
* Applying :func:`repr` to strings previously used octal escapes for |
| 763 |
non-printable characters; for example, a newline was ``'\012'``. This was a |
| 764 |
vestigial trace of Python's C ancestry, but today octal is of very little |
| 765 |
practical use. Ka-Ping Yee suggested using hex escapes instead of octal ones, |
| 766 |
and using the ``\n``, ``\t``, ``\r`` escapes for the appropriate characters, |
| 767 |
and implemented this new formatting. |
| 768 |
|
| 769 |
* Syntax errors detected at compile-time can now raise exceptions containing the |
| 770 |
filename and line number of the error, a pleasant side effect of the compiler |
| 771 |
reorganization done by Jeremy Hylton. |
| 772 |
|
| 773 |
* C extensions which import other modules have been changed to use |
| 774 |
:c:func:`PyImport_ImportModule`, which means that they will use any import hooks |
| 775 |
that have been installed. This is also encouraged for third-party extensions |
| 776 |
that need to import some other module from C code. |
| 777 |
|
| 778 |
* The size of the Unicode character database was shrunk by another 340K thanks |
| 779 |
to Fredrik Lundh. |
| 780 |
|
| 781 |
* Some new ports were contributed: MacOS X (by Steven Majewski), Cygwin (by |
| 782 |
Jason Tishler); RISCOS (by Dietmar Schwertberger); Unixware 7 (by Billy G. |
| 783 |
Allie). |
| 784 |
|
| 785 |
And there's the usual list of minor bugfixes, minor memory leaks, docstring |
| 786 |
edits, and other tweaks, too lengthy to be worth itemizing; see the CVS logs for |
| 787 |
the full details if you want them. |
| 788 |
|
| 789 |
.. ====================================================================== |
| 790 |
|
| 791 |
|
| 792 |
Acknowledgements |
| 793 |
================ |
| 794 |
|
| 795 |
The author would like to thank the following people for offering suggestions on |
| 796 |
various drafts of this article: Graeme Cross, David Goodger, Jay Graves, Michael |
| 797 |
Hudson, Marc-André Lemburg, Fredrik Lundh, Neil Schemenauer, Thomas Wouters. |
| 798 |
|