| 1 |
**************************** |
| 2 |
What's New in Python 2.2 |
| 3 |
**************************** |
| 4 |
|
| 5 |
:Author: A.M. Kuchling |
| 6 |
|
| 7 |
.. |release| replace:: 1.02 |
| 8 |
|
| 9 |
.. $Id: whatsnew22.tex 37315 2004-09-10 19:33:00Z akuchling $ |
| 10 |
|
| 11 |
|
| 12 |
Introduction |
| 13 |
============ |
| 14 |
|
| 15 |
This article explains the new features in Python 2.2.2, released on October 14, |
| 16 |
2002. Python 2.2.2 is a bugfix release of Python 2.2, originally released on |
| 17 |
December 21, 2001. |
| 18 |
|
| 19 |
Python 2.2 can be thought of as the "cleanup release". There are some features |
| 20 |
such as generators and iterators that are completely new, but most of the |
| 21 |
changes, significant and far-reaching though they may be, are aimed at cleaning |
| 22 |
up irregularities and dark corners of the language design. |
| 23 |
|
| 24 |
This article doesn't attempt to provide a complete specification of the new |
| 25 |
features, but instead provides a convenient overview. For full details, you |
| 26 |
should refer to the documentation for Python 2.2, such as the `Python Library |
| 27 |
Reference <https://docs.python.org/2.2/lib/lib.html>`_ and the `Python |
| 28 |
Reference Manual <https://docs.python.org/2.2/ref/ref.html>`_. If you want to |
| 29 |
understand the complete implementation and design rationale for a change, refer |
| 30 |
to the PEP for a particular new feature. |
| 31 |
|
| 32 |
|
| 33 |
.. see also, now defunct |
| 34 |
|
| 35 |
http://www.unixreview.com/documents/s=1356/urm0109h/0109h.htm |
| 36 |
"What's So Special About Python 2.2?" is also about the new 2.2 features, and |
| 37 |
was written by Cameron Laird and Kathryn Soraiz. |
| 38 |
|
| 39 |
.. ====================================================================== |
| 40 |
|
| 41 |
|
| 42 |
PEPs 252 and 253: Type and Class Changes |
| 43 |
======================================== |
| 44 |
|
| 45 |
The largest and most far-reaching changes in Python 2.2 are to Python's model of |
| 46 |
objects and classes. The changes should be backward compatible, so it's likely |
| 47 |
that your code will continue to run unchanged, but the changes provide some |
| 48 |
amazing new capabilities. Before beginning this, the longest and most |
| 49 |
complicated section of this article, I'll provide an overview of the changes and |
| 50 |
offer some comments. |
| 51 |
|
| 52 |
A long time ago I wrote a web page listing flaws in Python's design. One of the |
| 53 |
most significant flaws was that it's impossible to subclass Python types |
| 54 |
implemented in C. In particular, it's not possible to subclass built-in types, |
| 55 |
so you can't just subclass, say, lists in order to add a single useful method to |
| 56 |
them. The :mod:`!UserList` module provides a class that supports all of the |
| 57 |
methods of lists and that can be subclassed further, but there's lots of C code |
| 58 |
that expects a regular Python list and won't accept a :class:`~collections.UserList` |
| 59 |
instance. |
| 60 |
|
| 61 |
Python 2.2 fixes this, and in the process adds some exciting new capabilities. |
| 62 |
A brief summary: |
| 63 |
|
| 64 |
* You can subclass built-in types such as lists and even integers, and your |
| 65 |
subclasses should work in every place that requires the original type. |
| 66 |
|
| 67 |
* It's now possible to define static and class methods, in addition to the |
| 68 |
instance methods available in previous versions of Python. |
| 69 |
|
| 70 |
* It's also possible to automatically call methods on accessing or setting an |
| 71 |
instance attribute by using a new mechanism called :dfn:`properties`. Many uses |
| 72 |
of :meth:`~object.__getattr__` can be rewritten to use properties instead, making the |
| 73 |
resulting code simpler and faster. As a small side benefit, attributes can now |
| 74 |
have docstrings, too. |
| 75 |
|
| 76 |
* The list of legal attributes for an instance can be limited to a particular |
| 77 |
set using :dfn:`slots`, making it possible to safeguard against typos and |
| 78 |
perhaps make more optimizations possible in future versions of Python. |
| 79 |
|
| 80 |
Some users have voiced concern about all these changes. Sure, they say, the new |
| 81 |
features are neat and lend themselves to all sorts of tricks that weren't |
| 82 |
possible in previous versions of Python, but they also make the language more |
| 83 |
complicated. Some people have said that they've always recommended Python for |
| 84 |
its simplicity, and feel that its simplicity is being lost. |
| 85 |
|
| 86 |
Personally, I think there's no need to worry. Many of the new features are |
| 87 |
quite esoteric, and you can write a lot of Python code without ever needed to be |
| 88 |
aware of them. Writing a simple class is no more difficult than it ever was, so |
| 89 |
you don't need to bother learning or teaching them unless they're actually |
| 90 |
needed. Some very complicated tasks that were previously only possible from C |
| 91 |
will now be possible in pure Python, and to my mind that's all for the better. |
| 92 |
|
| 93 |
I'm not going to attempt to cover every single corner case and small change that |
| 94 |
were required to make the new features work. Instead this section will paint |
| 95 |
only the broad strokes. See section :ref:`sect-rellinks`, "Related Links", for |
| 96 |
further sources of information about Python 2.2's new object model. |
| 97 |
|
| 98 |
|
| 99 |
Old and New Classes |
| 100 |
------------------- |
| 101 |
|
| 102 |
First, you should know that Python 2.2 really has two kinds of classes: classic |
| 103 |
or old-style classes, and new-style classes. The old-style class model is |
| 104 |
exactly the same as the class model in earlier versions of Python. All the new |
| 105 |
features described in this section apply only to new-style classes. This |
| 106 |
divergence isn't intended to last forever; eventually old-style classes will be |
| 107 |
dropped, possibly in Python 3.0. |
| 108 |
|
| 109 |
So how do you define a new-style class? You do it by subclassing an existing |
| 110 |
new-style class. Most of Python's built-in types, such as integers, lists, |
| 111 |
dictionaries, and even files, are new-style classes now. A new-style class |
| 112 |
named :class:`object`, the base class for all built-in types, has also been |
| 113 |
added so if no built-in type is suitable, you can just subclass |
| 114 |
:class:`object`:: |
| 115 |
|
| 116 |
class C(object): |
| 117 |
def __init__ (self): |
| 118 |
... |
| 119 |
... |
| 120 |
|
| 121 |
This means that :keyword:`class` statements that don't have any base classes are |
| 122 |
always classic classes in Python 2.2. (Actually you can also change this by |
| 123 |
setting a module-level variable named :attr:`!__metaclass__` --- see :pep:`253` |
| 124 |
for the details --- but it's easier to just subclass :class:`object`.) |
| 125 |
|
| 126 |
The type objects for the built-in types are available as built-ins, named using |
| 127 |
a clever trick. Python has always had built-in functions named :func:`int`, |
| 128 |
:func:`float`, and :func:`str`. In 2.2, they aren't functions any more, but |
| 129 |
type objects that behave as factories when called. :: |
| 130 |
|
| 131 |
>>> int |
| 132 |
<type 'int'> |
| 133 |
>>> int('123') |
| 134 |
123 |
| 135 |
|
| 136 |
To make the set of types complete, new type objects such as :func:`dict` and |
| 137 |
:func:`!file` have been added. Here's a more interesting example, adding a |
| 138 |
:meth:`!lock` method to file objects:: |
| 139 |
|
| 140 |
class LockableFile(file): |
| 141 |
def lock (self, operation, length=0, start=0, whence=0): |
| 142 |
import fcntl |
| 143 |
return fcntl.lockf(self.fileno(), operation, |
| 144 |
length, start, whence) |
| 145 |
|
| 146 |
The now-obsolete :mod:`!posixfile` module contained a class that emulated all of |
| 147 |
a file object's methods and also added a :meth:`!lock` method, but this class |
| 148 |
couldn't be passed to internal functions that expected a built-in file, |
| 149 |
something which is possible with our new :class:`!LockableFile`. |
| 150 |
|
| 151 |
|
| 152 |
Descriptors |
| 153 |
----------- |
| 154 |
|
| 155 |
In previous versions of Python, there was no consistent way to discover what |
| 156 |
attributes and methods were supported by an object. There were some informal |
| 157 |
conventions, such as defining :attr:`!__members__` and :attr:`!__methods__` |
| 158 |
attributes that were lists of names, but often the author of an extension type |
| 159 |
or a class wouldn't bother to define them. You could fall back on inspecting |
| 160 |
the :attr:`~object.__dict__` of an object, but when class inheritance or an arbitrary |
| 161 |
:meth:`!__getattr__` hook were in use this could still be inaccurate. |
| 162 |
|
| 163 |
The one big idea underlying the new class model is that an API for describing |
| 164 |
the attributes of an object using :dfn:`descriptors` has been formalized. |
| 165 |
Descriptors specify the value of an attribute, stating whether it's a method or |
| 166 |
a field. With the descriptor API, static methods and class methods become |
| 167 |
possible, as well as more exotic constructs. |
| 168 |
|
| 169 |
Attribute descriptors are objects that live inside class objects, and have a few |
| 170 |
attributes of their own: |
| 171 |
|
| 172 |
* :attr:`~definition.__name__` is the attribute's name. |
| 173 |
|
| 174 |
* :attr:`~definition.__doc__` is the attribute's docstring. |
| 175 |
|
| 176 |
* ``__get__(object)`` is a method that retrieves the attribute value from |
| 177 |
*object*. |
| 178 |
|
| 179 |
* ``__set__(object, value)`` sets the attribute on *object* to *value*. |
| 180 |
|
| 181 |
* ``__delete__(object, value)`` deletes the *value* attribute of *object*. |
| 182 |
|
| 183 |
For example, when you write ``obj.x``, the steps that Python actually performs |
| 184 |
are:: |
| 185 |
|
| 186 |
descriptor = obj.__class__.x |
| 187 |
descriptor.__get__(obj) |
| 188 |
|
| 189 |
For methods, :meth:`descriptor.__get__ <object.__get__>` returns a temporary |
| 190 |
object that's |
| 191 |
callable, and wraps up the instance and the method to be called on it. This is |
| 192 |
also why static methods and class methods are now possible; they have |
| 193 |
descriptors that wrap up just the method, or the method and the class. As a |
| 194 |
brief explanation of these new kinds of methods, static methods aren't passed |
| 195 |
the instance, and therefore resemble regular functions. Class methods are |
| 196 |
passed the class of the object, but not the object itself. Static and class |
| 197 |
methods are defined like this:: |
| 198 |
|
| 199 |
class C(object): |
| 200 |
def f(arg1, arg2): |
| 201 |
... |
| 202 |
f = staticmethod(f) |
| 203 |
|
| 204 |
def g(cls, arg1, arg2): |
| 205 |
... |
| 206 |
g = classmethod(g) |
| 207 |
|
| 208 |
The :func:`staticmethod` function takes the function :func:`!f`, and returns it |
| 209 |
wrapped up in a descriptor so it can be stored in the class object. You might |
| 210 |
expect there to be special syntax for creating such methods (``def static f``, |
| 211 |
``defstatic f()``, or something like that) but no such syntax has been defined |
| 212 |
yet; that's been left for future versions of Python. |
| 213 |
|
| 214 |
More new features, such as slots and properties, are also implemented as new |
| 215 |
kinds of descriptors, and it's not difficult to write a descriptor class that |
| 216 |
does something novel. For example, it would be possible to write a descriptor |
| 217 |
class that made it possible to write Eiffel-style preconditions and |
| 218 |
postconditions for a method. A class that used this feature might be defined |
| 219 |
like this:: |
| 220 |
|
| 221 |
from eiffel import eiffelmethod |
| 222 |
|
| 223 |
class C(object): |
| 224 |
def f(self, arg1, arg2): |
| 225 |
# The actual function |
| 226 |
... |
| 227 |
def pre_f(self): |
| 228 |
# Check preconditions |
| 229 |
... |
| 230 |
def post_f(self): |
| 231 |
# Check postconditions |
| 232 |
... |
| 233 |
|
| 234 |
f = eiffelmethod(f, pre_f, post_f) |
| 235 |
|
| 236 |
Note that a person using the new :func:`!eiffelmethod` doesn't have to understand |
| 237 |
anything about descriptors. This is why I think the new features don't increase |
| 238 |
the basic complexity of the language. There will be a few wizards who need to |
| 239 |
know about it in order to write :func:`!eiffelmethod` or the ZODB or whatever, |
| 240 |
but most users will just write code on top of the resulting libraries and ignore |
| 241 |
the implementation details. |
| 242 |
|
| 243 |
|
| 244 |
Multiple Inheritance: The Diamond Rule |
| 245 |
-------------------------------------- |
| 246 |
|
| 247 |
Multiple inheritance has also been made more useful through changing the rules |
| 248 |
under which names are resolved. Consider this set of classes (diagram taken |
| 249 |
from :pep:`253` by Guido van Rossum):: |
| 250 |
|
| 251 |
class A: |
| 252 |
^ ^ def save(self): ... |
| 253 |
/ \ |
| 254 |
/ \ |
| 255 |
/ \ |
| 256 |
/ \ |
| 257 |
class B class C: |
| 258 |
^ ^ def save(self): ... |
| 259 |
\ / |
| 260 |
\ / |
| 261 |
\ / |
| 262 |
\ / |
| 263 |
class D |
| 264 |
|
| 265 |
The lookup rule for classic classes is simple but not very smart; the base |
| 266 |
classes are searched depth-first, going from left to right. A reference to |
| 267 |
:meth:`!D.save` will search the classes :class:`!D`, :class:`!B`, and then |
| 268 |
:class:`!A`, where :meth:`!save` would be found and returned. :meth:`!C.save` |
| 269 |
would never be found at all. This is bad, because if :class:`!C`'s :meth:`!save` |
| 270 |
method is saving some internal state specific to :class:`!C`, not calling it will |
| 271 |
result in that state never getting saved. |
| 272 |
|
| 273 |
New-style classes follow a different algorithm that's a bit more complicated to |
| 274 |
explain, but does the right thing in this situation. (Note that Python 2.3 |
| 275 |
changes this algorithm to one that produces the same results in most cases, but |
| 276 |
produces more useful results for really complicated inheritance graphs.) |
| 277 |
|
| 278 |
#. List all the base classes, following the classic lookup rule and include a |
| 279 |
class multiple times if it's visited repeatedly. In the above example, the list |
| 280 |
of visited classes is [:class:`!D`, :class:`!B`, :class:`!A`, :class:`!C`, |
| 281 |
:class:`!A`]. |
| 282 |
|
| 283 |
#. Scan the list for duplicated classes. If any are found, remove all but one |
| 284 |
occurrence, leaving the *last* one in the list. In the above example, the list |
| 285 |
becomes [:class:`!D`, :class:`!B`, :class:`!C`, :class:`!A`] after dropping |
| 286 |
duplicates. |
| 287 |
|
| 288 |
Following this rule, referring to :meth:`!D.save` will return :meth:`!C.save`, |
| 289 |
which is the behaviour we're after. This lookup rule is the same as the one |
| 290 |
followed by Common Lisp. A new built-in function, :func:`super`, provides a way |
| 291 |
to get at a class's superclasses without having to reimplement Python's |
| 292 |
algorithm. The most commonly used form will be ``super(class, obj)``, which |
| 293 |
returns a bound superclass object (not the actual class object). This form |
| 294 |
will be used in methods to call a method in the superclass; for example, |
| 295 |
:class:`!D`'s :meth:`!save` method would look like this:: |
| 296 |
|
| 297 |
class D (B,C): |
| 298 |
def save (self): |
| 299 |
# Call superclass .save() |
| 300 |
super(D, self).save() |
| 301 |
# Save D's private information here |
| 302 |
... |
| 303 |
|
| 304 |
:func:`super` can also return unbound superclass objects when called as |
| 305 |
``super(class)`` or ``super(class1, class2)``, but this probably won't |
| 306 |
often be useful. |
| 307 |
|
| 308 |
|
| 309 |
Attribute Access |
| 310 |
---------------- |
| 311 |
|
| 312 |
A fair number of sophisticated Python classes define hooks for attribute access |
| 313 |
using :meth:`~object.__getattr__`; most commonly this is done for convenience, to make |
| 314 |
code more readable by automatically mapping an attribute access such as |
| 315 |
``obj.parent`` into a method call such as ``obj.get_parent``. Python 2.2 adds |
| 316 |
some new ways of controlling attribute access. |
| 317 |
|
| 318 |
First, ``__getattr__(attr_name)`` is still supported by new-style classes, |
| 319 |
and nothing about it has changed. As before, it will be called when an attempt |
| 320 |
is made to access ``obj.foo`` and no attribute named ``foo`` is found in the |
| 321 |
instance's dictionary. |
| 322 |
|
| 323 |
New-style classes also support a new method, |
| 324 |
``__getattribute__(attr_name)``. The difference between the two methods is |
| 325 |
that :meth:`~object.__getattribute__` is *always* called whenever any attribute is |
| 326 |
accessed, while the old :meth:`~object.__getattr__` is only called if ``foo`` isn't |
| 327 |
found in the instance's dictionary. |
| 328 |
|
| 329 |
However, Python 2.2's support for :dfn:`properties` will often be a simpler way |
| 330 |
to trap attribute references. Writing a :meth:`!__getattr__` method is |
| 331 |
complicated because to avoid recursion you can't use regular attribute accesses |
| 332 |
inside them, and instead have to mess around with the contents of |
| 333 |
:attr:`~object.__dict__`. :meth:`~object.__getattr__` methods also end up being called by Python |
| 334 |
when it checks for other methods such as :meth:`~object.__repr__` or :meth:`!__coerce__`, |
| 335 |
and so have to be written with this in mind. Finally, calling a function on |
| 336 |
every attribute access results in a sizable performance loss. |
| 337 |
|
| 338 |
:class:`property` is a new built-in type that packages up three functions that |
| 339 |
get, set, or delete an attribute, and a docstring. For example, if you want to |
| 340 |
define a :attr:`!size` attribute that's computed, but also settable, you could |
| 341 |
write:: |
| 342 |
|
| 343 |
class C(object): |
| 344 |
def get_size (self): |
| 345 |
result = ... computation ... |
| 346 |
return result |
| 347 |
def set_size (self, size): |
| 348 |
... compute something based on the size |
| 349 |
and set internal state appropriately ... |
| 350 |
|
| 351 |
# Define a property. The 'delete this attribute' |
| 352 |
# method is defined as None, so the attribute |
| 353 |
# can't be deleted. |
| 354 |
size = property(get_size, set_size, |
| 355 |
None, |
| 356 |
"Storage size of this instance") |
| 357 |
|
| 358 |
That is certainly clearer and easier to write than a pair of |
| 359 |
:meth:`!__getattr__`/:meth:`!__setattr__` methods that check for the :attr:`!size` |
| 360 |
attribute and handle it specially while retrieving all other attributes from the |
| 361 |
instance's :attr:`~object.__dict__`. Accesses to :attr:`!size` are also the only ones |
| 362 |
which have to perform the work of calling a function, so references to other |
| 363 |
attributes run at their usual speed. |
| 364 |
|
| 365 |
Finally, it's possible to constrain the list of attributes that can be |
| 366 |
referenced on an object using the new :attr:`~object.__slots__` class attribute. Python |
| 367 |
objects are usually very dynamic; at any time it's possible to define a new |
| 368 |
attribute on an instance by just doing ``obj.new_attr=1``. A new-style class |
| 369 |
can define a class attribute named :attr:`~object.__slots__` to limit the legal |
| 370 |
attributes to a particular set of names. An example will make this clear:: |
| 371 |
|
| 372 |
>>> class C(object): |
| 373 |
... __slots__ = ('template', 'name') |
| 374 |
... |
| 375 |
>>> obj = C() |
| 376 |
>>> print obj.template |
| 377 |
None |
| 378 |
>>> obj.template = 'Test' |
| 379 |
>>> print obj.template |
| 380 |
Test |
| 381 |
>>> obj.newattr = None |
| 382 |
Traceback (most recent call last): |
| 383 |
File "<stdin>", line 1, in ? |
| 384 |
AttributeError: 'C' object has no attribute 'newattr' |
| 385 |
|
| 386 |
Note how you get an :exc:`AttributeError` on the attempt to assign to an |
| 387 |
attribute not listed in :attr:`~object.__slots__`. |
| 388 |
|
| 389 |
|
| 390 |
.. _sect-rellinks: |
| 391 |
|
| 392 |
Related Links |
| 393 |
------------- |
| 394 |
|
| 395 |
This section has just been a quick overview of the new features, giving enough |
| 396 |
of an explanation to start you programming, but many details have been |
| 397 |
simplified or ignored. Where should you go to get a more complete picture? |
| 398 |
|
| 399 |
The :ref:`descriptorhowto` is a lengthy tutorial introduction to |
| 400 |
the descriptor features, written by Guido van Rossum. If my description has |
| 401 |
whetted your appetite, go read this tutorial next, because it goes into much |
| 402 |
more detail about the new features while still remaining quite easy to read. |
| 403 |
|
| 404 |
Next, there are two relevant PEPs, :pep:`252` and :pep:`253`. :pep:`252` is |
| 405 |
titled "Making Types Look More Like Classes", and covers the descriptor API. |
| 406 |
:pep:`253` is titled "Subtyping Built-in Types", and describes the changes to |
| 407 |
type objects that make it possible to subtype built-in objects. :pep:`253` is |
| 408 |
the more complicated PEP of the two, and at a few points the necessary |
| 409 |
explanations of types and meta-types may cause your head to explode. Both PEPs |
| 410 |
were written and implemented by Guido van Rossum, with substantial assistance |
| 411 |
from the rest of the Zope Corp. team. |
| 412 |
|
| 413 |
Finally, there's the ultimate authority: the source code. Most of the machinery |
| 414 |
for the type handling is in :file:`Objects/typeobject.c`, but you should only |
| 415 |
resort to it after all other avenues have been exhausted, including posting a |
| 416 |
question to python-list or python-dev. |
| 417 |
|
| 418 |
.. ====================================================================== |
| 419 |
|
| 420 |
|
| 421 |
PEP 234: Iterators |
| 422 |
================== |
| 423 |
|
| 424 |
Another significant addition to 2.2 is an iteration interface at both the C and |
| 425 |
Python levels. Objects can define how they can be looped over by callers. |
| 426 |
|
| 427 |
In Python versions up to 2.1, the usual way to make ``for item in obj`` work is |
| 428 |
to define a :meth:`~object.__getitem__` method that looks something like this:: |
| 429 |
|
| 430 |
def __getitem__(self, index): |
| 431 |
return <next item> |
| 432 |
|
| 433 |
:meth:`~object.__getitem__` is more properly used to define an indexing operation on an |
| 434 |
object so that you can write ``obj[5]`` to retrieve the sixth element. It's a |
| 435 |
bit misleading when you're using this only to support :keyword:`for` loops. |
| 436 |
Consider some file-like object that wants to be looped over; the *index* |
| 437 |
parameter is essentially meaningless, as the class probably assumes that a |
| 438 |
series of :meth:`~object.__getitem__` calls will be made with *index* incrementing by |
| 439 |
one each time. In other words, the presence of the :meth:`~object.__getitem__` method |
| 440 |
doesn't mean that using ``file[5]`` to randomly access the sixth element will |
| 441 |
work, though it really should. |
| 442 |
|
| 443 |
In Python 2.2, iteration can be implemented separately, and :meth:`~object.__getitem__` |
| 444 |
methods can be limited to classes that really do support random access. The |
| 445 |
basic idea of iterators is simple. A new built-in function, ``iter(obj)`` |
| 446 |
or ``iter(C, sentinel)``, is used to get an iterator. ``iter(obj)`` returns |
| 447 |
an iterator for the object *obj*, while ``iter(C, sentinel)`` returns an |
| 448 |
iterator that will invoke the callable object *C* until it returns *sentinel* to |
| 449 |
signal that the iterator is done. |
| 450 |
|
| 451 |
Python classes can define an :meth:`!__iter__` method, which should create and |
| 452 |
return a new iterator for the object; if the object is its own iterator, this |
| 453 |
method can just return ``self``. In particular, iterators will usually be their |
| 454 |
own iterators. Extension types implemented in C can implement a :c:member:`~PyTypeObject.tp_iter` |
| 455 |
function in order to return an iterator, and extension types that want to behave |
| 456 |
as iterators can define a :c:member:`~PyTypeObject.tp_iternext` function. |
| 457 |
|
| 458 |
So, after all this, what do iterators actually do? They have one required |
| 459 |
method, :meth:`next`, which takes no arguments and returns the next value. When |
| 460 |
there are no more values to be returned, calling :meth:`next` should raise the |
| 461 |
:exc:`StopIteration` exception. :: |
| 462 |
|
| 463 |
>>> L = [1,2,3] |
| 464 |
>>> i = iter(L) |
| 465 |
>>> print i |
| 466 |
<iterator object at 0x8116870> |
| 467 |
>>> i.next() |
| 468 |
1 |
| 469 |
>>> i.next() |
| 470 |
2 |
| 471 |
>>> i.next() |
| 472 |
3 |
| 473 |
>>> i.next() |
| 474 |
Traceback (most recent call last): |
| 475 |
File "<stdin>", line 1, in ? |
| 476 |
StopIteration |
| 477 |
>>> |
| 478 |
|
| 479 |
In 2.2, Python's :keyword:`for` statement no longer expects a sequence; it |
| 480 |
expects something for which :func:`iter` will return an iterator. For backward |
| 481 |
compatibility and convenience, an iterator is automatically constructed for |
| 482 |
sequences that don't implement :meth:`!__iter__` or a :c:member:`~PyTypeObject.tp_iter` slot, so |
| 483 |
``for i in [1,2,3]`` will still work. Wherever the Python interpreter loops |
| 484 |
over a sequence, it's been changed to use the iterator protocol. This means you |
| 485 |
can do things like this:: |
| 486 |
|
| 487 |
>>> L = [1,2,3] |
| 488 |
>>> i = iter(L) |
| 489 |
>>> a,b,c = i |
| 490 |
>>> a,b,c |
| 491 |
(1, 2, 3) |
| 492 |
|
| 493 |
Iterator support has been added to some of Python's basic types. Calling |
| 494 |
:func:`iter` on a dictionary will return an iterator which loops over its keys:: |
| 495 |
|
| 496 |
>>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6, |
| 497 |
... 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12} |
| 498 |
>>> for key in m: print key, m[key] |
| 499 |
... |
| 500 |
Mar 3 |
| 501 |
Feb 2 |
| 502 |
Aug 8 |
| 503 |
Sep 9 |
| 504 |
May 5 |
| 505 |
Jun 6 |
| 506 |
Jul 7 |
| 507 |
Jan 1 |
| 508 |
Apr 4 |
| 509 |
Nov 11 |
| 510 |
Dec 12 |
| 511 |
Oct 10 |
| 512 |
|
| 513 |
That's just the default behaviour. If you want to iterate over keys, values, or |
| 514 |
key/value pairs, you can explicitly call the :meth:`!iterkeys`, |
| 515 |
:meth:`!itervalues`, or :meth:`!iteritems` methods to get an appropriate iterator. |
| 516 |
In a minor related change, the :keyword:`in` operator now works on dictionaries, |
| 517 |
so ``key in dict`` is now equivalent to ``dict.has_key(key)``. |
| 518 |
|
| 519 |
Files also provide an iterator, which calls the :meth:`readline` method until |
| 520 |
there are no more lines in the file. This means you can now read each line of a |
| 521 |
file using code like this:: |
| 522 |
|
| 523 |
for line in file: |
| 524 |
# do something for each line |
| 525 |
... |
| 526 |
|
| 527 |
Note that you can only go forward in an iterator; there's no way to get the |
| 528 |
previous element, reset the iterator, or make a copy of it. An iterator object |
| 529 |
could provide such additional capabilities, but the iterator protocol only |
| 530 |
requires a :meth:`next` method. |
| 531 |
|
| 532 |
|
| 533 |
.. seealso:: |
| 534 |
|
| 535 |
:pep:`234` - Iterators |
| 536 |
Written by Ka-Ping Yee and GvR; implemented by the Python Labs crew, mostly by |
| 537 |
GvR and Tim Peters. |
| 538 |
|
| 539 |
.. ====================================================================== |
| 540 |
|
| 541 |
|
| 542 |
PEP 255: Simple Generators |
| 543 |
========================== |
| 544 |
|
| 545 |
Generators are another new feature, one that interacts with the introduction of |
| 546 |
iterators. |
| 547 |
|
| 548 |
You're doubtless familiar with how function calls work in Python or C. When you |
| 549 |
call a function, it gets a private namespace where its local variables are |
| 550 |
created. When the function reaches a :keyword:`return` statement, the local |
| 551 |
variables are destroyed and the resulting value is returned to the caller. A |
| 552 |
later call to the same function will get a fresh new set of local variables. |
| 553 |
But, what if the local variables weren't thrown away on exiting a function? |
| 554 |
What if you could later resume the function where it left off? This is what |
| 555 |
generators provide; they can be thought of as resumable functions. |
| 556 |
|
| 557 |
Here's the simplest example of a generator function:: |
| 558 |
|
| 559 |
def generate_ints(N): |
| 560 |
for i in range(N): |
| 561 |
yield i |
| 562 |
|
| 563 |
A new keyword, :keyword:`yield`, was introduced for generators. Any function |
| 564 |
containing a :keyword:`!yield` statement is a generator function; this is |
| 565 |
detected by Python's bytecode compiler which compiles the function specially as |
| 566 |
a result. Because a new keyword was introduced, generators must be explicitly |
| 567 |
enabled in a module by including a ``from __future__ import generators`` |
| 568 |
statement near the top of the module's source code. In Python 2.3 this |
| 569 |
statement will become unnecessary. |
| 570 |
|
| 571 |
When you call a generator function, it doesn't return a single value; instead it |
| 572 |
returns a generator object that supports the iterator protocol. On executing |
| 573 |
the :keyword:`yield` statement, the generator outputs the value of ``i``, |
| 574 |
similar to a :keyword:`return` statement. The big difference between |
| 575 |
:keyword:`!yield` and a :keyword:`!return` statement is that on reaching a |
| 576 |
:keyword:`!yield` the generator's state of execution is suspended and local |
| 577 |
variables are preserved. On the next call to the generator's ``next()`` method, |
| 578 |
the function will resume executing immediately after the :keyword:`!yield` |
| 579 |
statement. (For complicated reasons, the :keyword:`!yield` statement isn't |
| 580 |
allowed inside the :keyword:`!try` block of a |
| 581 |
:keyword:`try`...\ :keyword:`finally` statement; read :pep:`255` for a full |
| 582 |
explanation of the interaction between :keyword:`!yield` and exceptions.) |
| 583 |
|
| 584 |
Here's a sample usage of the :func:`!generate_ints` generator:: |
| 585 |
|
| 586 |
>>> gen = generate_ints(3) |
| 587 |
>>> gen |
| 588 |
<generator object at 0x8117f90> |
| 589 |
>>> gen.next() |
| 590 |
0 |
| 591 |
>>> gen.next() |
| 592 |
1 |
| 593 |
>>> gen.next() |
| 594 |
2 |
| 595 |
>>> gen.next() |
| 596 |
Traceback (most recent call last): |
| 597 |
File "<stdin>", line 1, in ? |
| 598 |
File "<stdin>", line 2, in generate_ints |
| 599 |
StopIteration |
| 600 |
|
| 601 |
You could equally write ``for i in generate_ints(5)``, or ``a,b,c = |
| 602 |
generate_ints(3)``. |
| 603 |
|
| 604 |
Inside a generator function, the :keyword:`return` statement can only be used |
| 605 |
without a value, and signals the end of the procession of values; afterwards the |
| 606 |
generator cannot return any further values. :keyword:`!return` with a value, such |
| 607 |
as ``return 5``, is a syntax error inside a generator function. The end of the |
| 608 |
generator's results can also be indicated by raising :exc:`StopIteration` |
| 609 |
manually, or by just letting the flow of execution fall off the bottom of the |
| 610 |
function. |
| 611 |
|
| 612 |
You could achieve the effect of generators manually by writing your own class |
| 613 |
and storing all the local variables of the generator as instance variables. For |
| 614 |
example, returning a list of integers could be done by setting ``self.count`` to |
| 615 |
0, and having the :meth:`next` method increment ``self.count`` and return it. |
| 616 |
However, for a moderately complicated generator, writing a corresponding class |
| 617 |
would be much messier. :file:`Lib/test/test_generators.py` contains a number of |
| 618 |
more interesting examples. The simplest one implements an in-order traversal of |
| 619 |
a tree using generators recursively. :: |
| 620 |
|
| 621 |
# A recursive generator that generates Tree leaves in in-order. |
| 622 |
def inorder(t): |
| 623 |
if t: |
| 624 |
for x in inorder(t.left): |
| 625 |
yield x |
| 626 |
yield t.label |
| 627 |
for x in inorder(t.right): |
| 628 |
yield x |
| 629 |
|
| 630 |
Two other examples in :file:`Lib/test/test_generators.py` produce solutions for |
| 631 |
the N-Queens problem (placing $N$ queens on an $NxN$ chess board so that no |
| 632 |
queen threatens another) and the Knight's Tour (a route that takes a knight to |
| 633 |
every square of an $NxN$ chessboard without visiting any square twice). |
| 634 |
|
| 635 |
The idea of generators comes from other programming languages, especially Icon |
| 636 |
(https://www2.cs.arizona.edu/icon/), where the idea of generators is central. In |
| 637 |
Icon, every expression and function call behaves like a generator. One example |
| 638 |
from "An Overview of the Icon Programming Language" at |
| 639 |
https://www2.cs.arizona.edu/icon/docs/ipd266.htm gives an idea of what this looks |
| 640 |
like:: |
| 641 |
|
| 642 |
sentence := "Store it in the neighboring harbor" |
| 643 |
if (i := find("or", sentence)) > 5 then write(i) |
| 644 |
|
| 645 |
In Icon the :func:`!find` function returns the indexes at which the substring |
| 646 |
"or" is found: 3, 23, 33. In the :keyword:`if` statement, ``i`` is first |
| 647 |
assigned a value of 3, but 3 is less than 5, so the comparison fails, and Icon |
| 648 |
retries it with the second value of 23. 23 is greater than 5, so the comparison |
| 649 |
now succeeds, and the code prints the value 23 to the screen. |
| 650 |
|
| 651 |
Python doesn't go nearly as far as Icon in adopting generators as a central |
| 652 |
concept. Generators are considered a new part of the core Python language, but |
| 653 |
learning or using them isn't compulsory; if they don't solve any problems that |
| 654 |
you have, feel free to ignore them. One novel feature of Python's interface as |
| 655 |
compared to Icon's is that a generator's state is represented as a concrete |
| 656 |
object (the iterator) that can be passed around to other functions or stored in |
| 657 |
a data structure. |
| 658 |
|
| 659 |
|
| 660 |
.. seealso:: |
| 661 |
|
| 662 |
:pep:`255` - Simple Generators |
| 663 |
Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland. Implemented mostly |
| 664 |
by Neil Schemenauer and Tim Peters, with other fixes from the Python Labs crew. |
| 665 |
|
| 666 |
.. ====================================================================== |
| 667 |
|
| 668 |
|
| 669 |
PEP 237: Unifying Long Integers and Integers |
| 670 |
============================================ |
| 671 |
|
| 672 |
In recent versions, the distinction between regular integers, which are 32-bit |
| 673 |
values on most machines, and long integers, which can be of arbitrary size, was |
| 674 |
becoming an annoyance. For example, on platforms that support files larger than |
| 675 |
``2**32`` bytes, the :meth:`!tell` method of file objects has to return a long |
| 676 |
integer. However, there were various bits of Python that expected plain integers |
| 677 |
and would raise an error if a long integer was provided instead. For example, |
| 678 |
in Python 1.5, only regular integers could be used as a slice index, and |
| 679 |
``'abc'[1L:]`` would raise a :exc:`TypeError` exception with the message 'slice |
| 680 |
index must be int'. |
| 681 |
|
| 682 |
Python 2.2 will shift values from short to long integers as required. The 'L' |
| 683 |
suffix is no longer needed to indicate a long integer literal, as now the |
| 684 |
compiler will choose the appropriate type. (Using the 'L' suffix will be |
| 685 |
discouraged in future 2.x versions of Python, triggering a warning in Python |
| 686 |
2.4, and probably dropped in Python 3.0.) Many operations that used to raise an |
| 687 |
:exc:`OverflowError` will now return a long integer as their result. For |
| 688 |
example:: |
| 689 |
|
| 690 |
>>> 1234567890123 |
| 691 |
1234567890123L |
| 692 |
>>> 2 ** 64 |
| 693 |
18446744073709551616L |
| 694 |
|
| 695 |
In most cases, integers and long integers will now be treated identically. You |
| 696 |
can still distinguish them with the :func:`type` built-in function, but that's |
| 697 |
rarely needed. |
| 698 |
|
| 699 |
|
| 700 |
.. seealso:: |
| 701 |
|
| 702 |
:pep:`237` - Unifying Long Integers and Integers |
| 703 |
Written by Moshe Zadka and Guido van Rossum. Implemented mostly by Guido van |
| 704 |
Rossum. |
| 705 |
|
| 706 |
.. ====================================================================== |
| 707 |
|
| 708 |
|
| 709 |
PEP 238: Changing the Division Operator |
| 710 |
======================================= |
| 711 |
|
| 712 |
The most controversial change in Python 2.2 heralds the start of an effort to |
| 713 |
fix an old design flaw that's been in Python from the beginning. Currently |
| 714 |
Python's division operator, ``/``, behaves like C's division operator when |
| 715 |
presented with two integer arguments: it returns an integer result that's |
| 716 |
truncated down when there would be a fractional part. For example, ``3/2`` is |
| 717 |
1, not 1.5, and ``(-1)/2`` is -1, not -0.5. This means that the results of |
| 718 |
division can vary unexpectedly depending on the type of the two operands and |
| 719 |
because Python is dynamically typed, it can be difficult to determine the |
| 720 |
possible types of the operands. |
| 721 |
|
| 722 |
(The controversy is over whether this is *really* a design flaw, and whether |
| 723 |
it's worth breaking existing code to fix this. It's caused endless discussions |
| 724 |
on python-dev, and in July 2001 erupted into a storm of acidly sarcastic |
| 725 |
postings on :newsgroup:`comp.lang.python`. I won't argue for either side here |
| 726 |
and will stick to describing what's implemented in 2.2. Read :pep:`238` for a |
| 727 |
summary of arguments and counter-arguments.) |
| 728 |
|
| 729 |
Because this change might break code, it's being introduced very gradually. |
| 730 |
Python 2.2 begins the transition, but the switch won't be complete until Python |
| 731 |
3.0. |
| 732 |
|
| 733 |
First, I'll borrow some terminology from :pep:`238`. "True division" is the |
| 734 |
division that most non-programmers are familiar with: 3/2 is 1.5, 1/4 is 0.25, |
| 735 |
and so forth. "Floor division" is what Python's ``/`` operator currently does |
| 736 |
when given integer operands; the result is the floor of the value returned by |
| 737 |
true division. "Classic division" is the current mixed behaviour of ``/``; it |
| 738 |
returns the result of floor division when the operands are integers, and returns |
| 739 |
the result of true division when one of the operands is a floating-point number. |
| 740 |
|
| 741 |
Here are the changes 2.2 introduces: |
| 742 |
|
| 743 |
* A new operator, ``//``, is the floor division operator. (Yes, we know it looks |
| 744 |
like C++'s comment symbol.) ``//`` *always* performs floor division no matter |
| 745 |
what the types of its operands are, so ``1 // 2`` is 0 and ``1.0 // 2.0`` is |
| 746 |
also 0.0. |
| 747 |
|
| 748 |
``//`` is always available in Python 2.2; you don't need to enable it using a |
| 749 |
``__future__`` statement. |
| 750 |
|
| 751 |
* By including a ``from __future__ import division`` in a module, the ``/`` |
| 752 |
operator will be changed to return the result of true division, so ``1/2`` is |
| 753 |
0.5. Without the ``__future__`` statement, ``/`` still means classic division. |
| 754 |
The default meaning of ``/`` will not change until Python 3.0. |
| 755 |
|
| 756 |
* Classes can define methods called :meth:`~object.__truediv__` and :meth:`~object.__floordiv__` |
| 757 |
to overload the two division operators. At the C level, there are also slots in |
| 758 |
the :c:type:`PyNumberMethods` structure so extension types can define the two |
| 759 |
operators. |
| 760 |
|
| 761 |
* Python 2.2 supports some command-line arguments for testing whether code will |
| 762 |
work with the changed division semantics. Running python with :option:`!-Q |
| 763 |
warn` will cause a warning to be issued whenever division is applied to two |
| 764 |
integers. You can use this to find code that's affected by the change and fix |
| 765 |
it. By default, Python 2.2 will simply perform classic division without a |
| 766 |
warning; the warning will be turned on by default in Python 2.3. |
| 767 |
|
| 768 |
|
| 769 |
.. seealso:: |
| 770 |
|
| 771 |
:pep:`238` - Changing the Division Operator |
| 772 |
Written by Moshe Zadka and Guido van Rossum. Implemented by Guido van Rossum.. |
| 773 |
|
| 774 |
.. ====================================================================== |
| 775 |
|
| 776 |
|
| 777 |
Unicode Changes |
| 778 |
=============== |
| 779 |
|
| 780 |
Python's Unicode support has been enhanced a bit in 2.2. Unicode strings are |
| 781 |
usually stored as UCS-2, as 16-bit unsigned integers. Python 2.2 can also be |
| 782 |
compiled to use UCS-4, 32-bit unsigned integers, as its internal encoding by |
| 783 |
supplying :option:`!--enable-unicode=ucs4` to the configure script. (It's also |
| 784 |
possible to specify :option:`!--disable-unicode` to completely disable Unicode |
| 785 |
support.) |
| 786 |
|
| 787 |
When built to use UCS-4 (a "wide Python"), the interpreter can natively handle |
| 788 |
Unicode characters from U+000000 to U+110000, so the range of legal values for |
| 789 |
the :func:`!unichr` function is expanded accordingly. Using an interpreter |
| 790 |
compiled to use UCS-2 (a "narrow Python"), values greater than 65535 will still |
| 791 |
cause :func:`!unichr` to raise a :exc:`ValueError` exception. This is all |
| 792 |
described in :pep:`261`, "Support for 'wide' Unicode characters"; consult it for |
| 793 |
further details. |
| 794 |
|
| 795 |
Another change is simpler to explain. Since their introduction, Unicode strings |
| 796 |
have supported an :meth:`!encode` method to convert the string to a selected |
| 797 |
encoding such as UTF-8 or Latin-1. A symmetric ``decode([*encoding*])`` |
| 798 |
method has been added to 8-bit strings (though not to Unicode strings) in 2.2. |
| 799 |
:meth:`!decode` assumes that the string is in the specified encoding and decodes |
| 800 |
it, returning whatever is returned by the codec. |
| 801 |
|
| 802 |
Using this new feature, codecs have been added for tasks not directly related to |
| 803 |
Unicode. For example, codecs have been added for uu-encoding, MIME's base64 |
| 804 |
encoding, and compression with the :mod:`zlib` module:: |
| 805 |
|
| 806 |
>>> s = """Here is a lengthy piece of redundant, overly verbose, |
| 807 |
... and repetitive text. |
| 808 |
... """ |
| 809 |
>>> data = s.encode('zlib') |
| 810 |
>>> data |
| 811 |
'x\x9c\r\xc9\xc1\r\x80 \x10\x04\xc0?Ul...' |
| 812 |
>>> data.decode('zlib') |
| 813 |
'Here is a lengthy piece of redundant, overly verbose,\nand repetitive text.\n' |
| 814 |
>>> print s.encode('uu') |
| 815 |
begin 666 <data> |
| 816 |
M2&5R92!I<R!A(&QE;F=T:'D@<&EE8V4@;V8@<F5D=6YD86YT+"!O=F5R;'D@ |
| 817 |
>=F5R8F]S92P*86YD(')E<&5T:71I=F4@=&5X="X* |
| 818 |
|
| 819 |
end |
| 820 |
>>> "sheesh".encode('rot-13') |
| 821 |
'furrfu' |
| 822 |
|
| 823 |
To convert a class instance to Unicode, a :meth:`!__unicode__` method can be |
| 824 |
defined by a class, analogous to :meth:`!__str__`. |
| 825 |
|
| 826 |
:meth:`!encode`, :meth:`!decode`, and :meth:`!__unicode__` were implemented by |
| 827 |
Marc-André Lemburg. The changes to support using UCS-4 internally were |
| 828 |
implemented by Fredrik Lundh and Martin von Löwis. |
| 829 |
|
| 830 |
|
| 831 |
.. seealso:: |
| 832 |
|
| 833 |
:pep:`261` - Support for 'wide' Unicode characters |
| 834 |
Written by Paul Prescod. |
| 835 |
|
| 836 |
.. ====================================================================== |
| 837 |
|
| 838 |
|
| 839 |
PEP 227: Nested Scopes |
| 840 |
====================== |
| 841 |
|
| 842 |
In Python 2.1, statically nested scopes were added as an optional feature, to be |
| 843 |
enabled by a ``from __future__ import nested_scopes`` directive. In 2.2 nested |
| 844 |
scopes no longer need to be specially enabled, and are now always present. The |
| 845 |
rest of this section is a copy of the description of nested scopes from my |
| 846 |
"What's New in Python 2.1" document; if you read it when 2.1 came out, you can |
| 847 |
skip the rest of this section. |
| 848 |
|
| 849 |
The largest change introduced in Python 2.1, and made complete in 2.2, is to |
| 850 |
Python's scoping rules. In Python 2.0, at any given time there are at most |
| 851 |
three namespaces used to look up variable names: local, module-level, and the |
| 852 |
built-in namespace. This often surprised people because it didn't match their |
| 853 |
intuitive expectations. For example, a nested recursive function definition |
| 854 |
doesn't work:: |
| 855 |
|
| 856 |
def f(): |
| 857 |
... |
| 858 |
def g(value): |
| 859 |
... |
| 860 |
return g(value-1) + 1 |
| 861 |
... |
| 862 |
|
| 863 |
The function :func:`!g` will always raise a :exc:`NameError` exception, because |
| 864 |
the binding of the name ``g`` isn't in either its local namespace or in the |
| 865 |
module-level namespace. This isn't much of a problem in practice (how often do |
| 866 |
you recursively define interior functions like this?), but this also made using |
| 867 |
the :keyword:`lambda` expression clumsier, and this was a problem in practice. |
| 868 |
In code which uses :keyword:`!lambda` you can often find local variables being |
| 869 |
copied by passing them as the default values of arguments. :: |
| 870 |
|
| 871 |
def find(self, name): |
| 872 |
"Return list of any entries equal to 'name'" |
| 873 |
L = filter(lambda x, name=name: x == name, |
| 874 |
self.list_attribute) |
| 875 |
return L |
| 876 |
|
| 877 |
The readability of Python code written in a strongly functional style suffers |
| 878 |
greatly as a result. |
| 879 |
|
| 880 |
The most significant change to Python 2.2 is that static scoping has been added |
| 881 |
to the language to fix this problem. As a first effect, the ``name=name`` |
| 882 |
default argument is now unnecessary in the above example. Put simply, when a |
| 883 |
given variable name is not assigned a value within a function (by an assignment, |
| 884 |
or the :keyword:`def`, :keyword:`class`, or :keyword:`import` statements), |
| 885 |
references to the variable will be looked up in the local namespace of the |
| 886 |
enclosing scope. A more detailed explanation of the rules, and a dissection of |
| 887 |
the implementation, can be found in the PEP. |
| 888 |
|
| 889 |
This change may cause some compatibility problems for code where the same |
| 890 |
variable name is used both at the module level and as a local variable within a |
| 891 |
function that contains further function definitions. This seems rather unlikely |
| 892 |
though, since such code would have been pretty confusing to read in the first |
| 893 |
place. |
| 894 |
|
| 895 |
One side effect of the change is that the ``from module import *`` and |
| 896 |
``exec`` statements have been made illegal inside a function scope under |
| 897 |
certain conditions. The Python reference manual has said all along that ``from |
| 898 |
module import *`` is only legal at the top level of a module, but the CPython |
| 899 |
interpreter has never enforced this before. As part of the implementation of |
| 900 |
nested scopes, the compiler which turns Python source into bytecodes has to |
| 901 |
generate different code to access variables in a containing scope. ``from |
| 902 |
module import *`` and ``exec`` make it impossible for the compiler to |
| 903 |
figure this out, because they add names to the local namespace that are |
| 904 |
unknowable at compile time. Therefore, if a function contains function |
| 905 |
definitions or :keyword:`lambda` expressions with free variables, the compiler |
| 906 |
will flag this by raising a :exc:`SyntaxError` exception. |
| 907 |
|
| 908 |
To make the preceding explanation a bit clearer, here's an example:: |
| 909 |
|
| 910 |
x = 1 |
| 911 |
def f(): |
| 912 |
# The next line is a syntax error |
| 913 |
exec 'x=2' |
| 914 |
def g(): |
| 915 |
return x |
| 916 |
|
| 917 |
Line 4 containing the ``exec`` statement is a syntax error, since |
| 918 |
``exec`` would define a new local variable named ``x`` whose value should |
| 919 |
be accessed by :func:`!g`. |
| 920 |
|
| 921 |
This shouldn't be much of a limitation, since ``exec`` is rarely used in |
| 922 |
most Python code (and when it is used, it's often a sign of a poor design |
| 923 |
anyway). |
| 924 |
|
| 925 |
|
| 926 |
.. seealso:: |
| 927 |
|
| 928 |
:pep:`227` - Statically Nested Scopes |
| 929 |
Written and implemented by Jeremy Hylton. |
| 930 |
|
| 931 |
.. ====================================================================== |
| 932 |
|
| 933 |
|
| 934 |
New and Improved Modules |
| 935 |
======================== |
| 936 |
|
| 937 |
* The :mod:`xmlrpclib <xmlrpc.client>` module was contributed to the standard library by Fredrik |
| 938 |
Lundh, providing support for writing XML-RPC clients. XML-RPC is a simple |
| 939 |
remote procedure call protocol built on top of HTTP and XML. For example, the |
| 940 |
following snippet retrieves a list of RSS channels from the O'Reilly Network, |
| 941 |
and then lists the recent headlines for one channel:: |
| 942 |
|
| 943 |
import xmlrpclib |
| 944 |
s = xmlrpclib.Server( |
| 945 |
'http://www.oreillynet.com/meerkat/xml-rpc/server.php') |
| 946 |
channels = s.meerkat.getChannels() |
| 947 |
# channels is a list of dictionaries, like this: |
| 948 |
# [{'id': 4, 'title': 'Freshmeat Daily News'} |
| 949 |
# {'id': 190, 'title': '32Bits Online'}, |
| 950 |
# {'id': 4549, 'title': '3DGamers'}, ... ] |
| 951 |
|
| 952 |
# Get the items for one channel |
| 953 |
items = s.meerkat.getItems( {'channel': 4} ) |
| 954 |
|
| 955 |
# 'items' is another list of dictionaries, like this: |
| 956 |
# [{'link': 'http://freshmeat.net/releases/52719/', |
| 957 |
# 'description': 'A utility which converts HTML to XSL FO.', |
| 958 |
# 'title': 'html2fo 0.3 (Default)'}, ... ] |
| 959 |
|
| 960 |
The :mod:`SimpleXMLRPCServer <xmlrpc.server>` module makes it easy to create straightforward |
| 961 |
XML-RPC servers. See http://xmlrpc.scripting.com/ for more information about XML-RPC. |
| 962 |
|
| 963 |
* The new :mod:`hmac` module implements the HMAC algorithm described by |
| 964 |
:rfc:`2104`. (Contributed by Gerhard Häring.) |
| 965 |
|
| 966 |
* Several functions that originally returned lengthy tuples now return |
| 967 |
pseudo-sequences that still behave like tuples but also have mnemonic attributes such |
| 968 |
as :attr:`!memberst_mtime` or :attr:`~time.struct_time.tm_year`. The enhanced functions include |
| 969 |
:func:`~os.stat`, :func:`~os.fstat`, :func:`~os.statvfs`, and :func:`~os.fstatvfs` in the |
| 970 |
:mod:`os` module, and :func:`~time.localtime`, :func:`~time.gmtime`, and :func:`~time.strptime` in |
| 971 |
the :mod:`time` module. |
| 972 |
|
| 973 |
For example, to obtain a file's size using the old tuples, you'd end up writing |
| 974 |
something like ``file_size = os.stat(filename)[stat.ST_SIZE]``, but now this can |
| 975 |
be written more clearly as ``file_size = os.stat(filename).st_size``. |
| 976 |
|
| 977 |
The original patch for this feature was contributed by Nick Mathewson. |
| 978 |
|
| 979 |
* The Python profiler has been extensively reworked and various errors in its |
| 980 |
output have been corrected. (Contributed by Fred L. Drake, Jr. and Tim Peters.) |
| 981 |
|
| 982 |
* The :mod:`socket` module can be compiled to support IPv6; specify the |
| 983 |
:option:`!--enable-ipv6` option to Python's configure script. (Contributed by |
| 984 |
Jun-ichiro "itojun" Hagino.) |
| 985 |
|
| 986 |
* Two new format characters were added to the :mod:`struct` module for 64-bit |
| 987 |
integers on platforms that support the C :c:expr:`long long` type. ``q`` is for |
| 988 |
a signed 64-bit integer, and ``Q`` is for an unsigned one. The value is |
| 989 |
returned in Python's long integer type. (Contributed by Tim Peters.) |
| 990 |
|
| 991 |
* In the interpreter's interactive mode, there's a new built-in function |
| 992 |
:func:`help` that uses the :mod:`pydoc` module introduced in Python 2.1 to |
| 993 |
provide interactive help. ``help(object)`` displays any available help text |
| 994 |
about *object*. :func:`help` with no argument puts you in an online help |
| 995 |
utility, where you can enter the names of functions, classes, or modules to read |
| 996 |
their help text. (Contributed by Guido van Rossum, using Ka-Ping Yee's |
| 997 |
:mod:`pydoc` module.) |
| 998 |
|
| 999 |
* Various bugfixes and performance improvements have been made to the SRE engine |
| 1000 |
underlying the :mod:`re` module. For example, the :func:`re.sub` and |
| 1001 |
:func:`re.split` functions have been rewritten in C. Another contributed patch |
| 1002 |
speeds up certain Unicode character ranges by a factor of two, and a new |
| 1003 |
:meth:`~re.finditer` method that returns an iterator over all the non-overlapping |
| 1004 |
matches in a given string. (SRE is maintained by Fredrik Lundh. The |
| 1005 |
BIGCHARSET patch was contributed by Martin von Löwis.) |
| 1006 |
|
| 1007 |
* The :mod:`smtplib` module now supports :rfc:`2487`, "Secure SMTP over TLS", so |
| 1008 |
it's now possible to encrypt the SMTP traffic between a Python program and the |
| 1009 |
mail transport agent being handed a message. :mod:`smtplib` also supports SMTP |
| 1010 |
authentication. (Contributed by Gerhard Häring.) |
| 1011 |
|
| 1012 |
* The :mod:`imaplib` module, maintained by Piers Lauder, has support for several |
| 1013 |
new extensions: the NAMESPACE extension defined in :rfc:`2342`, SORT, GETACL and |
| 1014 |
SETACL. (Contributed by Anthony Baxter and Michel Pelletier.) |
| 1015 |
|
| 1016 |
* The :mod:`!rfc822` module's parsing of email addresses is now compliant with |
| 1017 |
:rfc:`2822`, an update to :rfc:`822`. (The module's name is *not* going to be |
| 1018 |
changed to ``rfc2822``.) A new package, :mod:`email`, has also been added for |
| 1019 |
parsing and generating e-mail messages. (Contributed by Barry Warsaw, and |
| 1020 |
arising out of his work on Mailman.) |
| 1021 |
|
| 1022 |
* The :mod:`difflib` module now contains a new :class:`!Differ` class for |
| 1023 |
producing human-readable lists of changes (a "delta") between two sequences of |
| 1024 |
lines of text. There are also two generator functions, :func:`!ndiff` and |
| 1025 |
:func:`!restore`, which respectively return a delta from two sequences, or one of |
| 1026 |
the original sequences from a delta. (Grunt work contributed by David Goodger, |
| 1027 |
from ndiff.py code by Tim Peters who then did the generatorization.) |
| 1028 |
|
| 1029 |
* New constants :const:`!ascii_letters`, :const:`!ascii_lowercase`, and |
| 1030 |
:const:`!ascii_uppercase` were added to the :mod:`string` module. There were |
| 1031 |
several modules in the standard library that used :const:`!string.letters` to |
| 1032 |
mean the ranges A-Za-z, but that assumption is incorrect when locales are in |
| 1033 |
use, because :const:`!string.letters` varies depending on the set of legal |
| 1034 |
characters defined by the current locale. The buggy modules have all been fixed |
| 1035 |
to use :const:`!ascii_letters` instead. (Reported by an unknown person; fixed by |
| 1036 |
Fred L. Drake, Jr.) |
| 1037 |
|
| 1038 |
* The :mod:`mimetypes` module now makes it easier to use alternative MIME-type |
| 1039 |
databases by the addition of a :class:`~mimetypes.MimeTypes` class, which takes a list of |
| 1040 |
filenames to be parsed. (Contributed by Fred L. Drake, Jr.) |
| 1041 |
|
| 1042 |
* A :class:`~threading.Timer` class was added to the :mod:`threading` module that allows |
| 1043 |
scheduling an activity to happen at some future time. (Contributed by Itamar |
| 1044 |
Shtull-Trauring.) |
| 1045 |
|
| 1046 |
.. ====================================================================== |
| 1047 |
|
| 1048 |
|
| 1049 |
Interpreter Changes and Fixes |
| 1050 |
============================= |
| 1051 |
|
| 1052 |
Some of the changes only affect people who deal with the Python interpreter at |
| 1053 |
the C level because they're writing Python extension modules, embedding the |
| 1054 |
interpreter, or just hacking on the interpreter itself. If you only write Python |
| 1055 |
code, none of the changes described here will affect you very much. |
| 1056 |
|
| 1057 |
* Profiling and tracing functions can now be implemented in C, which can operate |
| 1058 |
at much higher speeds than Python-based functions and should reduce the overhead |
| 1059 |
of profiling and tracing. This will be of interest to authors of development |
| 1060 |
environments for Python. Two new C functions were added to Python's API, |
| 1061 |
:c:func:`PyEval_SetProfile` and :c:func:`PyEval_SetTrace`. The existing |
| 1062 |
:func:`sys.setprofile` and :func:`sys.settrace` functions still exist, and have |
| 1063 |
simply been changed to use the new C-level interface. (Contributed by Fred L. |
| 1064 |
Drake, Jr.) |
| 1065 |
|
| 1066 |
* Another low-level API, primarily of interest to implementers of Python |
| 1067 |
debuggers and development tools, was added. :c:func:`PyInterpreterState_Head` and |
| 1068 |
:c:func:`PyInterpreterState_Next` let a caller walk through all the existing |
| 1069 |
interpreter objects; :c:func:`PyInterpreterState_ThreadHead` and |
| 1070 |
:c:func:`PyThreadState_Next` allow looping over all the thread states for a given |
| 1071 |
interpreter. (Contributed by David Beazley.) |
| 1072 |
|
| 1073 |
* The C-level interface to the garbage collector has been changed to make it |
| 1074 |
easier to write extension types that support garbage collection and to debug |
| 1075 |
misuses of the functions. Various functions have slightly different semantics, |
| 1076 |
so a bunch of functions had to be renamed. Extensions that use the old API will |
| 1077 |
still compile but will *not* participate in garbage collection, so updating them |
| 1078 |
for 2.2 should be considered fairly high priority. |
| 1079 |
|
| 1080 |
To upgrade an extension module to the new API, perform the following steps: |
| 1081 |
|
| 1082 |
* Rename :c:macro:`!Py_TPFLAGS_GC` to :c:macro:`Py_TPFLAGS_HAVE_GC`. |
| 1083 |
|
| 1084 |
* Use :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar` to allocate |
| 1085 |
objects, and :c:func:`PyObject_GC_Del` to deallocate them. |
| 1086 |
|
| 1087 |
* Rename :c:func:`!PyObject_GC_Init` to :c:func:`PyObject_GC_Track` and |
| 1088 |
:c:func:`!PyObject_GC_Fini` to :c:func:`PyObject_GC_UnTrack`. |
| 1089 |
|
| 1090 |
* Remove :c:macro:`!PyGC_HEAD_SIZE` from object size calculations. |
| 1091 |
|
| 1092 |
* Remove calls to :c:func:`!PyObject_AS_GC` and :c:func:`!PyObject_FROM_GC`. |
| 1093 |
|
| 1094 |
* A new ``et`` format sequence was added to :c:func:`PyArg_ParseTuple`; ``et`` |
| 1095 |
takes both a parameter and an encoding name, and converts the parameter to the |
| 1096 |
given encoding if the parameter turns out to be a Unicode string, or leaves it |
| 1097 |
alone if it's an 8-bit string, assuming it to already be in the desired |
| 1098 |
encoding. This differs from the ``es`` format character, which assumes that |
| 1099 |
8-bit strings are in Python's default ASCII encoding and converts them to the |
| 1100 |
specified new encoding. (Contributed by M.-A. Lemburg, and used for the MBCS |
| 1101 |
support on Windows described in the following section.) |
| 1102 |
|
| 1103 |
* A different argument parsing function, :c:func:`PyArg_UnpackTuple`, has been |
| 1104 |
added that's simpler and presumably faster. Instead of specifying a format |
| 1105 |
string, the caller simply gives the minimum and maximum number of arguments |
| 1106 |
expected, and a set of pointers to :c:expr:`PyObject*` variables that will be |
| 1107 |
filled in with argument values. |
| 1108 |
|
| 1109 |
* Two new flags :c:macro:`METH_NOARGS` and :c:macro:`METH_O` are available in method |
| 1110 |
definition tables to simplify implementation of methods with no arguments or a |
| 1111 |
single untyped argument. Calling such methods is more efficient than calling a |
| 1112 |
corresponding method that uses :c:macro:`METH_VARARGS`. Also, the old |
| 1113 |
:c:macro:`!METH_OLDARGS` style of writing C methods is now officially deprecated. |
| 1114 |
|
| 1115 |
* Two new wrapper functions, :c:func:`PyOS_snprintf` and :c:func:`PyOS_vsnprintf` |
| 1116 |
were added to provide cross-platform implementations for the relatively new |
| 1117 |
:c:func:`snprintf` and :c:func:`vsnprintf` C lib APIs. In contrast to the standard |
| 1118 |
:c:func:`sprintf` and :c:func:`!vsprintf` functions, the Python versions check the |
| 1119 |
bounds of the buffer used to protect against buffer overruns. (Contributed by |
| 1120 |
M.-A. Lemburg.) |
| 1121 |
|
| 1122 |
* The :c:func:`_PyTuple_Resize` function has lost an unused parameter, so now it |
| 1123 |
takes 2 parameters instead of 3. The third argument was never used, and can |
| 1124 |
simply be discarded when porting code from earlier versions to Python 2.2. |
| 1125 |
|
| 1126 |
.. ====================================================================== |
| 1127 |
|
| 1128 |
|
| 1129 |
Other Changes and Fixes |
| 1130 |
======================= |
| 1131 |
|
| 1132 |
As usual there were a bunch of other improvements and bugfixes scattered |
| 1133 |
throughout the source tree. A search through the CVS change logs finds there |
| 1134 |
were 527 patches applied and 683 bugs fixed between Python 2.1 and 2.2; 2.2.1 |
| 1135 |
applied 139 patches and fixed 143 bugs; 2.2.2 applied 106 patches and fixed 82 |
| 1136 |
bugs. These figures are likely to be underestimates. |
| 1137 |
|
| 1138 |
Some of the more notable changes are: |
| 1139 |
|
| 1140 |
* The code for the MacOS port for Python, maintained by Jack Jansen, is now kept |
| 1141 |
in the main Python CVS tree, and many changes have been made to support MacOS X. |
| 1142 |
|
| 1143 |
The most significant change is the ability to build Python as a framework, |
| 1144 |
enabled by supplying the :option:`!--enable-framework` option to the configure |
| 1145 |
script when compiling Python. According to Jack Jansen, "This installs a |
| 1146 |
self-contained Python installation plus the OS X framework "glue" into |
| 1147 |
:file:`/Library/Frameworks/Python.framework` (or another location of choice). |
| 1148 |
For now there is little immediate added benefit to this (actually, there is the |
| 1149 |
disadvantage that you have to change your PATH to be able to find Python), but |
| 1150 |
it is the basis for creating a full-blown Python application, porting the |
| 1151 |
MacPython IDE, possibly using Python as a standard OSA scripting language and |
| 1152 |
much more." |
| 1153 |
|
| 1154 |
Most of the MacPython toolbox modules, which interface to MacOS APIs such as |
| 1155 |
windowing, QuickTime, scripting, etc. have been ported to OS X, but they've been |
| 1156 |
left commented out in :file:`setup.py`. People who want to experiment with |
| 1157 |
these modules can uncomment them manually. |
| 1158 |
|
| 1159 |
.. Jack's original comments: |
| 1160 |
The main change is the possibility to build Python as a |
| 1161 |
framework. This installs a self-contained Python installation plus the |
| 1162 |
OSX framework "glue" into /Library/Frameworks/Python.framework (or |
| 1163 |
another location of choice). For now there is little immediate added |
| 1164 |
benefit to this (actually, there is the disadvantage that you have to |
| 1165 |
change your PATH to be able to find Python), but it is the basis for |
| 1166 |
creating a fullblown Python application, porting the MacPython IDE, |
| 1167 |
possibly using Python as a standard OSA scripting language and much |
| 1168 |
more. You enable this with "configure --enable-framework". |
| 1169 |
The other change is that most MacPython toolbox modules, which |
| 1170 |
interface to all the MacOS APIs such as windowing, quicktime, |
| 1171 |
scripting, etc. have been ported. Again, most of these are not of |
| 1172 |
immediate use, as they need a full application to be really useful, so |
| 1173 |
they have been commented out in setup.py. People wanting to experiment |
| 1174 |
can uncomment them. Gestalt and Internet Config modules are enabled by |
| 1175 |
default. |
| 1176 |
|
| 1177 |
* Keyword arguments passed to built-in functions that don't take them now cause a |
| 1178 |
:exc:`TypeError` exception to be raised, with the message "*function* takes no |
| 1179 |
keyword arguments". |
| 1180 |
|
| 1181 |
* Weak references, added in Python 2.1 as an extension module, are now part of |
| 1182 |
the core because they're used in the implementation of new-style classes. The |
| 1183 |
:exc:`ReferenceError` exception has therefore moved from the :mod:`weakref` |
| 1184 |
module to become a built-in exception. |
| 1185 |
|
| 1186 |
* A new script, :file:`Tools/scripts/cleanfuture.py` by Tim Peters, |
| 1187 |
automatically removes obsolete ``__future__`` statements from Python source |
| 1188 |
code. |
| 1189 |
|
| 1190 |
* An additional *flags* argument has been added to the built-in function |
| 1191 |
:func:`compile`, so the behaviour of ``__future__`` statements can now be |
| 1192 |
correctly observed in simulated shells, such as those presented by IDLE and |
| 1193 |
other development environments. This is described in :pep:`264`. (Contributed |
| 1194 |
by Michael Hudson.) |
| 1195 |
|
| 1196 |
* The new license introduced with Python 1.6 wasn't GPL-compatible. This is |
| 1197 |
fixed by some minor textual changes to the 2.2 license, so it's now legal to |
| 1198 |
embed Python inside a GPLed program again. Note that Python itself is not |
| 1199 |
GPLed, but instead is under a license that's essentially equivalent to the BSD |
| 1200 |
license, same as it always was. The license changes were also applied to the |
| 1201 |
Python 2.0.1 and 2.1.1 releases. |
| 1202 |
|
| 1203 |
* When presented with a Unicode filename on Windows, Python will now convert it |
| 1204 |
to an MBCS encoded string, as used by the Microsoft file APIs. As MBCS is |
| 1205 |
explicitly used by the file APIs, Python's choice of ASCII as the default |
| 1206 |
encoding turns out to be an annoyance. On Unix, the locale's character set is |
| 1207 |
used if ``locale.nl_langinfo(CODESET)`` is available. (Windows support was |
| 1208 |
contributed by Mark Hammond with assistance from Marc-André Lemburg. Unix |
| 1209 |
support was added by Martin von Löwis.) |
| 1210 |
|
| 1211 |
* Large file support is now enabled on Windows. (Contributed by Tim Peters.) |
| 1212 |
|
| 1213 |
* The :file:`Tools/scripts/ftpmirror.py` script now parses a :file:`.netrc` |
| 1214 |
file, if you have one. (Contributed by Mike Romberg.) |
| 1215 |
|
| 1216 |
* Some features of the object returned by the :func:`!xrange` function are now |
| 1217 |
deprecated, and trigger warnings when they're accessed; they'll disappear in |
| 1218 |
Python 2.3. :class:`!xrange` objects tried to pretend they were full sequence |
| 1219 |
types by supporting slicing, sequence multiplication, and the :keyword:`in` |
| 1220 |
operator, but these features were rarely used and therefore buggy. The |
| 1221 |
:meth:`!tolist` method and the :attr:`!start`, :attr:`!stop`, and :attr:`!step` |
| 1222 |
attributes are also being deprecated. At the C level, the fourth argument to |
| 1223 |
the :c:func:`!PyRange_New` function, ``repeat``, has also been deprecated. |
| 1224 |
|
| 1225 |
* There were a bunch of patches to the dictionary implementation, mostly to fix |
| 1226 |
potential core dumps if a dictionary contains objects that sneakily changed |
| 1227 |
their hash value, or mutated the dictionary they were contained in. For a while |
| 1228 |
python-dev fell into a gentle rhythm of Michael Hudson finding a case that |
| 1229 |
dumped core, Tim Peters fixing the bug, Michael finding another case, and round |
| 1230 |
and round it went. |
| 1231 |
|
| 1232 |
* On Windows, Python can now be compiled with Borland C thanks to a number of |
| 1233 |
patches contributed by Stephen Hansen, though the result isn't fully functional |
| 1234 |
yet. (But this *is* progress...) |
| 1235 |
|
| 1236 |
* Another Windows enhancement: Wise Solutions generously offered PythonLabs use |
| 1237 |
of their InstallerMaster 8.1 system. Earlier PythonLabs Windows installers used |
| 1238 |
Wise 5.0a, which was beginning to show its age. (Packaged up by Tim Peters.) |
| 1239 |
|
| 1240 |
* Files ending in ``.pyw`` can now be imported on Windows. ``.pyw`` is a |
| 1241 |
Windows-only thing, used to indicate that a script needs to be run using |
| 1242 |
PYTHONW.EXE instead of PYTHON.EXE in order to prevent a DOS console from popping |
| 1243 |
up to display the output. This patch makes it possible to import such scripts, |
| 1244 |
in case they're also usable as modules. (Implemented by David Bolen.) |
| 1245 |
|
| 1246 |
* On platforms where Python uses the C :c:func:`dlopen` function to load |
| 1247 |
extension modules, it's now possible to set the flags used by :c:func:`dlopen` |
| 1248 |
using the :func:`sys.getdlopenflags` and :func:`sys.setdlopenflags` functions. |
| 1249 |
(Contributed by Bram Stolk.) |
| 1250 |
|
| 1251 |
* The :func:`pow` built-in function no longer supports 3 arguments when |
| 1252 |
floating-point numbers are supplied. ``pow(x, y, z)`` returns ``(x**y) % z``, |
| 1253 |
but this is never useful for floating-point numbers, and the final result varies |
| 1254 |
unpredictably depending on the platform. A call such as ``pow(2.0, 8.0, 7.0)`` |
| 1255 |
will now raise a :exc:`TypeError` exception. |
| 1256 |
|
| 1257 |
.. ====================================================================== |
| 1258 |
|
| 1259 |
|
| 1260 |
Acknowledgements |
| 1261 |
================ |
| 1262 |
|
| 1263 |
The author would like to thank the following people for offering suggestions, |
| 1264 |
corrections and assistance with various drafts of this article: Fred Bremmer, |
| 1265 |
Keith Briggs, Andrew Dalke, Fred L. Drake, Jr., Carel Fellinger, David Goodger, |
| 1266 |
Mark Hammond, Stephen Hansen, Michael Hudson, Jack Jansen, Marc-André Lemburg, |
| 1267 |
Martin von Löwis, Fredrik Lundh, Michael McLay, Nick Mathewson, Paul Moore, |
| 1268 |
Gustavo Niemeyer, Don O'Donnell, Joonas Paalasma, Tim Peters, Jens Quade, Tom |
| 1269 |
Reinhardt, Neil Schemenauer, Guido van Rossum, Greg Ward, Edward Welbourne. |
| 1270 |
|