| 1 |
<?xml version="1.0" encoding="utf-8" ?> |
| 2 |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| 3 |
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| 4 |
<head> |
| 5 |
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> |
| 6 |
<meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" /> |
| 7 |
<title>Changelog for processing</title> |
| 8 |
<link rel="stylesheet" href="html4css1.css" type="text/css" /> |
| 9 |
</head> |
| 10 |
<body> |
| 11 |
<div class="document" id="changelog-for-processing"> |
| 12 |
<h1 class="title">Changelog for processing</h1> |
| 13 |
<div class="section"> |
| 14 |
<h1><a id="changes-in-0-52" name="changes-in-0-52">Changes in 0.52</a></h1> |
| 15 |
<ul> |
| 16 |
<li><p class="first">On versions 0.50 and 0.51 Mac OSX <tt class="docutils literal"><span class="pre">Lock.release()</span></tt> would fail with |
| 17 |
<tt class="docutils literal"><span class="pre">OSError(errno.ENOSYS,</span> <span class="pre">"[Errno</span> <span class="pre">78]</span> <span class="pre">Function</span> <span class="pre">not</span> <span class="pre">implemented")</span></tt>. |
| 18 |
This appears to be because on Mac OSX <tt class="docutils literal"><span class="pre">sem_getvalue()</span></tt> has not been |
| 19 |
implemented.</p> |
| 20 |
<p>Now <tt class="docutils literal"><span class="pre">sem_getvalue()</span></tt> is no longer needed. Unfortunately, however, |
| 21 |
on Mac OSX <tt class="docutils literal"><span class="pre">BoundedSemaphore()</span></tt> will not raise <tt class="docutils literal"><span class="pre">ValueError</span></tt> if it |
| 22 |
exceeds its initial value.</p> |
| 23 |
</li> |
| 24 |
<li><p class="first">Some changes to the code for the reduction/rebuilding of connection |
| 25 |
and socket objects so that things work the same on Windows and Unix. |
| 26 |
This should fix a couple of bugs.</p> |
| 27 |
</li> |
| 28 |
<li><p class="first">The code has been changed to consistently use "camelCase" for |
| 29 |
methods and (non-factory) functions. In the few cases where this |
| 30 |
has meant a change to the documented API, the old name has been |
| 31 |
retained as an alias.</p> |
| 32 |
</li> |
| 33 |
</ul> |
| 34 |
</div> |
| 35 |
<div class="section"> |
| 36 |
<h1><a id="changes-in-0-51" name="changes-in-0-51">Changes in 0.51</a></h1> |
| 37 |
<ul> |
| 38 |
<li><p class="first">In 0.50 <tt class="docutils literal"><span class="pre">processing.Value()</span></tt> and <tt class="docutils literal"><span class="pre">processing.sharedctypes.Value()</span></tt> |
| 39 |
were related but had different signatures, which was rather |
| 40 |
confusing.</p> |
| 41 |
<p>Now <tt class="docutils literal"><span class="pre">processing.sharedctypes.Value()</span></tt> has been renamed |
| 42 |
<tt class="docutils literal"><span class="pre">processing.sharedctypes.RawValue()</span></tt> and |
| 43 |
<tt class="docutils literal"><span class="pre">processing.sharedctypes.Value()</span></tt> is the same as <tt class="docutils literal"><span class="pre">processing.Value()</span></tt>.</p> |
| 44 |
</li> |
| 45 |
<li><p class="first">In version 0.50 <tt class="docutils literal"><span class="pre">sendfd()</span></tt> and <tt class="docutils literal"><span class="pre">recvfd()</span></tt> apparently did not work on |
| 46 |
64bit Linux. This has been fixed by reverting to using the CMSG_* |
| 47 |
macros as was done in 0.40.</p> |
| 48 |
<p>However, this means that systems without all the necessary CMSG_* |
| 49 |
macros (such as Solaris 8) will have to disable compilation of |
| 50 |
<tt class="docutils literal"><span class="pre">sendfd()</span></tt> and <tt class="docutils literal"><span class="pre">recvfd()</span></tt> by setting <tt class="docutils literal"><span class="pre">macros['HAVE_FD_TRANSFRER']</span> <span class="pre">=</span> <span class="pre">0</span></tt> |
| 51 |
in <tt class="docutils literal"><span class="pre">setup.py</span></tt>.</p> |
| 52 |
</li> |
| 53 |
<li><p class="first">Fixed an authentication error when using a "remote" manager created |
| 54 |
using <tt class="docutils literal"><span class="pre">BaseManager.from_address()</span></tt>.</p> |
| 55 |
</li> |
| 56 |
<li><p class="first">Fixed a couple of bugs which only affected Python 2.4.</p> |
| 57 |
</li> |
| 58 |
</ul> |
| 59 |
</div> |
| 60 |
<div class="section"> |
| 61 |
<h1><a id="changes-in-0-50" name="changes-in-0-50">Changes in 0.50</a></h1> |
| 62 |
<ul> |
| 63 |
<li><p class="first"><tt class="docutils literal"><span class="pre">ctypes</span></tt> is now a prerequisite if you want to use shared memory -- |
| 64 |
with Python 2.4 you will need to install it separately.</p> |
| 65 |
</li> |
| 66 |
<li><p class="first"><tt class="docutils literal"><span class="pre">LocalManager()</span></tt> has been removed.</p> |
| 67 |
</li> |
| 68 |
<li><p class="first">Added <tt class="docutils literal"><span class="pre">processing.Value()</span></tt> and <tt class="docutils literal"><span class="pre">processing.Array()</span></tt> |
| 69 |
which are similar to <tt class="docutils literal"><span class="pre">LocalManager.SharedValue()</span></tt> and |
| 70 |
<tt class="docutils literal"><span class="pre">LocalManager.SharedArray()</span></tt>.</p> |
| 71 |
</li> |
| 72 |
<li><p class="first">In the <tt class="docutils literal"><span class="pre">sharedctypes</span></tt> module <tt class="docutils literal"><span class="pre">new_value()</span></tt> and <tt class="docutils literal"><span class="pre">new_array()</span></tt> have |
| 73 |
been renamed <tt class="docutils literal"><span class="pre">Value()</span></tt> and <tt class="docutils literal"><span class="pre">Array()</span></tt>.</p> |
| 74 |
</li> |
| 75 |
<li><p class="first"><tt class="docutils literal"><span class="pre">Process.stop()</span></tt>, <tt class="docutils literal"><span class="pre">Process.getStoppable()</span></tt> and |
| 76 |
<tt class="docutils literal"><span class="pre">Process.setStoppable()</span></tt> have been removed. Use |
| 77 |
<tt class="docutils literal"><span class="pre">Process.terminate()</span></tt> instead.</p> |
| 78 |
</li> |
| 79 |
<li><p class="first"><tt class="docutils literal"><span class="pre">procesing.Lock</span></tt> now matches <tt class="docutils literal"><span class="pre">threading.Lock</span></tt> behaviour more |
| 80 |
closely: now a thread can release a lock it does not own, and now |
| 81 |
when a thread tries acquiring a lock it already owns a deadlock |
| 82 |
results instead of an exception.</p> |
| 83 |
</li> |
| 84 |
<li><p class="first">On Windows when the main thread is blocking on a method of <tt class="docutils literal"><span class="pre">Lock</span></tt>, |
| 85 |
<tt class="docutils literal"><span class="pre">RLock</span></tt>, <tt class="docutils literal"><span class="pre">Semaphore</span></tt>, <tt class="docutils literal"><span class="pre">BoundedSemaphore</span></tt>, <tt class="docutils literal"><span class="pre">Condition</span></tt> it will no |
| 86 |
longer ignore Ctrl-C. (The same was already true on Unix.)</p> |
| 87 |
<p>This differs from the behaviour of the equivalent objects in |
| 88 |
<tt class="docutils literal"><span class="pre">threading</span></tt> which will completely ignore Ctrl-C.</p> |
| 89 |
</li> |
| 90 |
<li><p class="first">The <tt class="docutils literal"><span class="pre">test</span></tt> sub-package has been replaced by lots of unit tests in a |
| 91 |
<tt class="docutils literal"><span class="pre">tests</span></tt> sub-package. Some of the old test files have been moved |
| 92 |
over to a new <tt class="docutils literal"><span class="pre">examples</span></tt> sub-package.</p> |
| 93 |
</li> |
| 94 |
<li><p class="first">On Windows it is now possible for a non-console python program |
| 95 |
(i.e. one using <tt class="docutils literal"><span class="pre">pythonw.exe</span></tt> instead of <tt class="docutils literal"><span class="pre">python.exe</span></tt>) to use |
| 96 |
<tt class="docutils literal"><span class="pre">processing</span></tt>.</p> |
| 97 |
<p>Previously an exception was raised when <tt class="docutils literal"><span class="pre">subprocess.py</span></tt> tried to |
| 98 |
duplicate stdin, stdout, stderr.</p> |
| 99 |
</li> |
| 100 |
<li><p class="first">Proxy objects should now be thread safe -- they now use thread local |
| 101 |
storage.</p> |
| 102 |
</li> |
| 103 |
<li><p class="first">Trying to transfer shared resources such as locks, queues etc |
| 104 |
between processes over a pipe or queue will now raise <tt class="docutils literal"><span class="pre">RuntimeError</span></tt> |
| 105 |
with a message saying that the object should only be shared between |
| 106 |
processes using inheritance.</p> |
| 107 |
<p>Previously, this worked unreliably on Windows but would fail with an |
| 108 |
unexplained <tt class="docutils literal"><span class="pre">AssertionError</span></tt> on Unix.</p> |
| 109 |
</li> |
| 110 |
<li><p class="first">The names of some of the macros used for compiling the extension |
| 111 |
have changed. See <tt class="docutils literal"><span class="pre">INSTALL.txt</span></tt> and <tt class="docutils literal"><span class="pre">setup.py</span></tt>.</p> |
| 112 |
</li> |
| 113 |
<li><p class="first">A few changes which (hopefully) make compilation possible on Solaris.</p> |
| 114 |
</li> |
| 115 |
<li><p class="first">Lots of refactoring of the code.</p> |
| 116 |
</li> |
| 117 |
<li><p class="first">Fixed reference leaks so that unit tests pass with "regrtest -R::" |
| 118 |
(at least on Linux).</p> |
| 119 |
</li> |
| 120 |
</ul> |
| 121 |
</div> |
| 122 |
<div class="section"> |
| 123 |
<h1><a id="changes-in-0-40" name="changes-in-0-40">Changes in 0.40</a></h1> |
| 124 |
<ul> |
| 125 |
<li><p class="first">Removed <tt class="docutils literal"><span class="pre">SimpleQueue</span></tt> and <tt class="docutils literal"><span class="pre">PosixQueue</span></tt> types. Just use <tt class="docutils literal"><span class="pre">Queue</span></tt> instead.</p> |
| 126 |
</li> |
| 127 |
<li><p class="first">Previously if you forgot to use the</p> |
| 128 |
<pre class="literal-block"> |
| 129 |
if __name__ == '__main__': |
| 130 |
freezeSupport() |
| 131 |
... |
| 132 |
</pre> |
| 133 |
<p>idiom on Windows then processes could be created recursively |
| 134 |
bringing the computer to its knees. Now <tt class="docutils literal"><span class="pre">RuntimeError</span></tt> will be |
| 135 |
raised instead.</p> |
| 136 |
</li> |
| 137 |
<li><p class="first">Some refactoring of the code.</p> |
| 138 |
</li> |
| 139 |
<li><p class="first">A Unix specific bug meant that a child process might fail to start a |
| 140 |
feeder thread for a queue if its parent process had already started |
| 141 |
its own feeder thread. Fixed.</p> |
| 142 |
</li> |
| 143 |
</ul> |
| 144 |
</div> |
| 145 |
<div class="section"> |
| 146 |
<h1><a id="changes-in-0-39" name="changes-in-0-39">Changes in 0.39</a></h1> |
| 147 |
<ul> |
| 148 |
<li><p class="first">One can now create one-way pipes by doing |
| 149 |
<tt class="docutils literal"><span class="pre">reader,</span> <span class="pre">writer</span> <span class="pre">=</span> <span class="pre">Pipe(duplex=False)</span></tt>.</p> |
| 150 |
</li> |
| 151 |
<li><p class="first">Rewrote code for managing shared memory maps.</p> |
| 152 |
</li> |
| 153 |
<li><p class="first">Added a <tt class="docutils literal"><span class="pre">sharedctypes</span></tt> module for creating <tt class="docutils literal"><span class="pre">ctypes</span></tt> objects allocated |
| 154 |
from shared memory. On Python 2.4 this requires the installation of |
| 155 |
<tt class="docutils literal"><span class="pre">ctypes</span></tt>.</p> |
| 156 |
<p><tt class="docutils literal"><span class="pre">ctypes</span></tt> objects are not protected by any locks so you will need to |
| 157 |
synchronize access to them (such as by using a lock). However they |
| 158 |
can be much faster to access than equivalent objects allocated using |
| 159 |
a <tt class="docutils literal"><span class="pre">LocalManager</span></tt>.</p> |
| 160 |
</li> |
| 161 |
<li><p class="first">Rearranged documentation.</p> |
| 162 |
</li> |
| 163 |
<li><p class="first">Previously the C extension caused a segfault on 64 bit machines with |
| 164 |
Python 2.5 because it used <tt class="docutils literal"><span class="pre">int</span></tt> instead of <tt class="docutils literal"><span class="pre">Py_ssize_t</span></tt> in certain |
| 165 |
places. This is now fixed. Thanks to Alexy Khrabrov for the report.</p> |
| 166 |
</li> |
| 167 |
<li><p class="first">A fix for <tt class="docutils literal"><span class="pre">Pool.terminate()</span></tt>.</p> |
| 168 |
</li> |
| 169 |
<li><p class="first">A fix for cleanup behaviour of <tt class="docutils literal"><span class="pre">Queue</span></tt>.</p> |
| 170 |
</li> |
| 171 |
</ul> |
| 172 |
</div> |
| 173 |
<div class="section"> |
| 174 |
<h1><a id="changes-in-0-38" name="changes-in-0-38">Changes in 0.38</a></h1> |
| 175 |
<ul> |
| 176 |
<li><p class="first">Have revamped the queue types. Now the queue types are |
| 177 |
<tt class="docutils literal"><span class="pre">Queue</span></tt>, <tt class="docutils literal"><span class="pre">SimpleQueue</span></tt> and (on systems which support it) |
| 178 |
<tt class="docutils literal"><span class="pre">PosixQueue</span></tt>.</p> |
| 179 |
<p>Now <tt class="docutils literal"><span class="pre">Queue</span></tt> should behave just like Python's normal <tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> |
| 180 |
class except that <tt class="docutils literal"><span class="pre">qsize()</span></tt>, <tt class="docutils literal"><span class="pre">task_done()</span></tt> and <tt class="docutils literal"><span class="pre">join()</span></tt> are not |
| 181 |
implemented. In particular, if no maximum size was specified when |
| 182 |
the queue was created then <tt class="docutils literal"><span class="pre">put()</span></tt> will always succeed without |
| 183 |
blocking.</p> |
| 184 |
<p>A <tt class="docutils literal"><span class="pre">SimpleQueue</span></tt> instance is really just a pipe protected by a couple |
| 185 |
of locks. It has <tt class="docutils literal"><span class="pre">get()</span></tt>, <tt class="docutils literal"><span class="pre">put()</span></tt> and <tt class="docutils literal"><span class="pre">empty()</span></tt> methods but does |
| 186 |
not not support timeouts or non-blocking.</p> |
| 187 |
<p><tt class="docutils literal"><span class="pre">BufferedPipeQueue()</span></tt> and <tt class="docutils literal"><span class="pre">PipeQueue()</span></tt> remain as deprecated |
| 188 |
aliases of <tt class="docutils literal"><span class="pre">Queue()</span></tt> but <tt class="docutils literal"><span class="pre">BufferedPosixQueue()</span></tt> has been removed. |
| 189 |
(Not sure if we really need to keep <tt class="docutils literal"><span class="pre">PosixQueue()</span></tt>...)</p> |
| 190 |
</li> |
| 191 |
<li><p class="first">Previously the <tt class="docutils literal"><span class="pre">Pool.shutdown()</span></tt> method was a little dodgy -- it |
| 192 |
could block indefinitely if <tt class="docutils literal"><span class="pre">map()</span></tt> or <tt class="docutils literal"><span class="pre">imap*()</span></tt> were used and did |
| 193 |
not try to terminate workers while they were doing a task.</p> |
| 194 |
<p>Now there are three new methods <tt class="docutils literal"><span class="pre">close()</span></tt>, <tt class="docutils literal"><span class="pre">terminate()</span></tt> and |
| 195 |
<tt class="docutils literal"><span class="pre">join()</span></tt> -- <tt class="docutils literal"><span class="pre">shutdown()</span></tt> is retained as a deprecated alias of |
| 196 |
<tt class="docutils literal"><span class="pre">terminate()</span></tt>. Thanks to Gerald John M. Manipon for feature |
| 197 |
request/suggested patch to <tt class="docutils literal"><span class="pre">shutdown()</span></tt>.</p> |
| 198 |
</li> |
| 199 |
<li><p class="first"><tt class="docutils literal"><span class="pre">Pool.imap()</span></tt> and <tt class="docutils literal"><span class="pre">Pool.imap_unordered()</span></tt> has gained a <tt class="docutils literal"><span class="pre">chunksize</span></tt> |
| 200 |
argument which allows the iterable to be submitted to the pool in |
| 201 |
chunks. Choosing <tt class="docutils literal"><span class="pre">chunksize</span></tt> appropriately makes <tt class="docutils literal"><span class="pre">Pool.imap()</span></tt> |
| 202 |
almost as fast as <tt class="docutils literal"><span class="pre">Pool.map()</span></tt> even for long iterables and cheap |
| 203 |
functions.</p> |
| 204 |
</li> |
| 205 |
<li><p class="first">Previously on Windows when the cleanup code for a <tt class="docutils literal"><span class="pre">LocalManager</span></tt> |
| 206 |
attempts to unlink the name of the file which backs the shared |
| 207 |
memory map an exception is raised if a child process still exists |
| 208 |
which has a handle open for that mmap. This is likely to happen if |
| 209 |
a daemon process inherits a <tt class="docutils literal"><span class="pre">LocalManager</span></tt> instance.</p> |
| 210 |
<p>Now the parent process will remember the filename and attempt to |
| 211 |
unlink the file name again once all the child processes have been |
| 212 |
joined or terminated. Reported by Paul Rudin.</p> |
| 213 |
</li> |
| 214 |
<li><p class="first"><tt class="docutils literal"><span class="pre">types.MethodType</span></tt> is registered with <tt class="docutils literal"><span class="pre">copy_reg</span></tt> so now instance |
| 215 |
methods and class methods should be picklable. (Unfortunately there is |
| 216 |
no obvious way of supporting the pickling of staticmethods since |
| 217 |
they are not marked with the class in which they were defined.)</p> |
| 218 |
<p>This means that on Windows it is now possible to use an instance |
| 219 |
method or class method as the target callable of a Process object.</p> |
| 220 |
</li> |
| 221 |
<li><p class="first">On Windows <tt class="docutils literal"><span class="pre">reduction.fromfd()</span></tt> now returns true instances of |
| 222 |
<tt class="docutils literal"><span class="pre">_socket.socket</span></tt>, so there is no more need for the |
| 223 |
<tt class="docutils literal"><span class="pre">_processing.falsesocket</span></tt> type.</p> |
| 224 |
</li> |
| 225 |
</ul> |
| 226 |
</div> |
| 227 |
<div class="section"> |
| 228 |
<h1><a id="changes-in-0-37" name="changes-in-0-37">Changes in 0.37</a></h1> |
| 229 |
<ul class="simple"> |
| 230 |
<li>Updated metadata and documentation because the project is now hosted |
| 231 |
at <tt class="docutils literal"><span class="pre">developer.berlios.de/projects/pyprocessing</span></tt>.</li> |
| 232 |
<li>The <tt class="docutils literal"><span class="pre">Pool.join()</span></tt> method has been removed. <tt class="docutils literal"><span class="pre">Pool.shutdown()</span></tt> will |
| 233 |
now join the worker processes automatically.</li> |
| 234 |
<li>A pool object no longer participates in a reference cycle so |
| 235 |
<tt class="docutils literal"><span class="pre">Pool.shutdown()</span></tt> should get called as soon as its reference count |
| 236 |
falls to zero.</li> |
| 237 |
<li>On Windows if <tt class="docutils literal"><span class="pre">enableLogging()</span></tt> was used at module scope then the |
| 238 |
logger used by a child process would often get two copies of the |
| 239 |
same handler. To fix this, now specifiying a handler type in |
| 240 |
<tt class="docutils literal"><span class="pre">enableLogging()</span></tt> will cause any previous handlers used by the |
| 241 |
logger to be discarded.</li> |
| 242 |
</ul> |
| 243 |
</div> |
| 244 |
<div class="section"> |
| 245 |
<h1><a id="changes-in-0-36" name="changes-in-0-36">Changes in 0.36</a></h1> |
| 246 |
<ul> |
| 247 |
<li><p class="first">In recent versions on Unix the finalizers in a manager process were |
| 248 |
never given a chance to run before <tt class="docutils literal"><span class="pre">os._exit()</span></tt> was called, so old |
| 249 |
unlinked AF_UNIX sockets could accumulate in '/tmp'. Fixed.</p> |
| 250 |
</li> |
| 251 |
<li><p class="first">The shutting down of managers has been cleaned up.</p> |
| 252 |
</li> |
| 253 |
<li><p class="first">In previous versions on Windows trying to acquire a lock owned by a |
| 254 |
different thread of the current process would raise an exception. |
| 255 |
Fixed.</p> |
| 256 |
</li> |
| 257 |
<li><p class="first">In previous versions on Windows trying to use an event object for |
| 258 |
synchronization between two threads of the same process was likely |
| 259 |
to raise an exception. (This was caused by the bug described |
| 260 |
above.) Fixed.</p> |
| 261 |
</li> |
| 262 |
<li><p class="first">Previously the arguments to <tt class="docutils literal"><span class="pre">processing.Semaphore()</span></tt> and |
| 263 |
<tt class="docutils literal"><span class="pre">processing.BoundedSemaphore()</span></tt> did not have any defaults. The |
| 264 |
defaults should be 1 to match <tt class="docutils literal"><span class="pre">threading</span></tt>. Fixed.</p> |
| 265 |
</li> |
| 266 |
<li><p class="first">It should now be possible for a Windows Service created by using |
| 267 |
<tt class="docutils literal"><span class="pre">pywin32</span></tt> to spawn processes using the <tt class="docutils literal"><span class="pre">processing</span></tt> package.</p> |
| 268 |
<p>Note that <tt class="docutils literal"><span class="pre">pywin32</span></tt> apparently has a bug meaning that <tt class="docutils literal"><span class="pre">Py_Finalize()</span></tt> |
| 269 |
is never called when the service exits so functions registered with |
| 270 |
<tt class="docutils literal"><span class="pre">atexit</span></tt> never get a chance to run. Therefore it is advisable to |
| 271 |
explicitly call <tt class="docutils literal"><span class="pre">sys.exitfunc()</span></tt> or <tt class="docutils literal"><span class="pre">atexit._run_exitfuncs()</span></tt> at the |
| 272 |
end of <tt class="docutils literal"><span class="pre">ServiceFramework.DoSvcRun()</span></tt>. Otherwise child processes are |
| 273 |
liable to survive the service when it is stopped. Thanks to Charlie |
| 274 |
Hull for the report.</p> |
| 275 |
</li> |
| 276 |
<li><p class="first">Added <tt class="docutils literal"><span class="pre">getLogger()</span></tt> and <tt class="docutils literal"><span class="pre">enableLogging()</span></tt> to support logging.</p> |
| 277 |
</li> |
| 278 |
</ul> |
| 279 |
</div> |
| 280 |
<div class="section"> |
| 281 |
<h1><a id="changes-in-0-35" name="changes-in-0-35">Changes in 0.35</a></h1> |
| 282 |
<ul> |
| 283 |
<li><p class="first">By default processes are no longer be stoppable using the <tt class="docutils literal"><span class="pre">stop()</span></tt> |
| 284 |
method: one must call <tt class="docutils literal"><span class="pre">setStoppable(True)</span></tt> before <tt class="docutils literal"><span class="pre">start()</span></tt> in order |
| 285 |
to use the <tt class="docutils literal"><span class="pre">stop()</span></tt> method. (Note that <tt class="docutils literal"><span class="pre">terminate()</span></tt> will work |
| 286 |
regardless of whether the process is marked as being "stoppable".)</p> |
| 287 |
<p>The reason for this is that on Windows getting <tt class="docutils literal"><span class="pre">stop()</span></tt> to work |
| 288 |
involves starting a new console for the child process and installing |
| 289 |
a signal handler for the <tt class="docutils literal"><span class="pre">SIGBREAK</span></tt> signal. This unfortunately |
| 290 |
means that Ctrl-Break cannot not be used to kill all processes of |
| 291 |
the program.</p> |
| 292 |
</li> |
| 293 |
<li><p class="first">Added <tt class="docutils literal"><span class="pre">setStoppable()</span></tt> and <tt class="docutils literal"><span class="pre">getStoppable()</span></tt> methods -- see above.</p> |
| 294 |
</li> |
| 295 |
<li><p class="first">Added <tt class="docutils literal"><span class="pre">BufferedQueue</span></tt>/<tt class="docutils literal"><span class="pre">BufferedPipeQueue</span></tt>/<tt class="docutils literal"><span class="pre">BufferedPosixQueue</span></tt>. |
| 296 |
Putting an object on a buffered queue will always succeed without |
| 297 |
blocking (just like with <tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> if no maximum size is |
| 298 |
specified). This makes them potentially safer than the normal queue |
| 299 |
types provided by <tt class="docutils literal"><span class="pre">processing</span></tt> which have finite capacity and may |
| 300 |
cause deadlocks if they fill.</p> |
| 301 |
<p><tt class="docutils literal"><span class="pre">test/test_worker.py</span></tt> has been updated to use <tt class="docutils literal"><span class="pre">BufferedQueue</span></tt> for |
| 302 |
the task queue instead of explicitly spawning a thread to feed tasks |
| 303 |
to the queue without risking a deadlock.</p> |
| 304 |
</li> |
| 305 |
<li><p class="first">Now when the NO_SEM_TIMED macro is set polling will be used to get |
| 306 |
around the lack of <tt class="docutils literal"><span class="pre">sem_timedwait()</span></tt>. This means that |
| 307 |
<tt class="docutils literal"><span class="pre">Condition.wait()</span></tt> and <tt class="docutils literal"><span class="pre">Queue.get()</span></tt> should now work with timeouts |
| 308 |
on Mac OS X.</p> |
| 309 |
</li> |
| 310 |
<li><p class="first">Added a <tt class="docutils literal"><span class="pre">callback</span></tt> argument to <tt class="docutils literal"><span class="pre">Pool.apply_async()</span></tt>.</p> |
| 311 |
</li> |
| 312 |
<li><p class="first">Added <tt class="docutils literal"><span class="pre">test/test_httpserverpool.py</span></tt> which runs a pool of http |
| 313 |
servers which share a single listening socket.</p> |
| 314 |
</li> |
| 315 |
<li><p class="first">Previously on Windows the process object was passed to the child |
| 316 |
process on the commandline (after pickling and hex encoding it). |
| 317 |
This caused errors when the pickled string was too large. Now if |
| 318 |
the pickled string is large then it will be passed to the child |
| 319 |
over a pipe or socket.</p> |
| 320 |
</li> |
| 321 |
<li><p class="first">Fixed bug in the iterator returned by <tt class="docutils literal"><span class="pre">Pool.imap()</span></tt>.</p> |
| 322 |
</li> |
| 323 |
<li><p class="first">Fixed bug in <tt class="docutils literal"><span class="pre">Condition.__repr__()</span></tt>.</p> |
| 324 |
</li> |
| 325 |
<li><p class="first">Fixed a handle/file descriptor leak when sockets or connections are |
| 326 |
unpickled.</p> |
| 327 |
</li> |
| 328 |
</ul> |
| 329 |
</div> |
| 330 |
<div class="section"> |
| 331 |
<h1><a id="changes-in-0-34" name="changes-in-0-34">Changes in 0.34</a></h1> |
| 332 |
<ul> |
| 333 |
<li><p class="first">Although version 0.33 the C extension would compile on Mac OSX |
| 334 |
trying to import it failed with "undefined symbol: _sem_timedwait". |
| 335 |
Unfortunately the <tt class="docutils literal"><span class="pre">ImportError</span></tt> exception was silently swallowed.</p> |
| 336 |
<p>This is now fixed by using the <tt class="docutils literal"><span class="pre">NO_SEM_TIMED</span></tt> macro. Unfortunately |
| 337 |
this means that some methods like <tt class="docutils literal"><span class="pre">Condition.wait()</span></tt> and |
| 338 |
<tt class="docutils literal"><span class="pre">Queue.get()</span></tt> will not work with timeouts on Mac OS X. If you |
| 339 |
really need to be able to use timeouts then you can always use the |
| 340 |
equivalent objects created with a manager. Thanks to Doug Hellmann |
| 341 |
for report and testing.</p> |
| 342 |
</li> |
| 343 |
<li><p class="first">Added a <tt class="docutils literal"><span class="pre">terminate()</span></tt> method to process objects which is more |
| 344 |
forceful than <tt class="docutils literal"><span class="pre">stop()</span></tt>.</p> |
| 345 |
</li> |
| 346 |
<li><p class="first">Fixed bug in the cleanup function registered with <tt class="docutils literal"><span class="pre">atexit</span></tt> which on |
| 347 |
Windows could cause a process which is shutting down to deadlock |
| 348 |
waiting for a manager to exit. Thanks to Dominique Wahli for report |
| 349 |
and testing.</p> |
| 350 |
</li> |
| 351 |
<li><p class="first">Added <tt class="docutils literal"><span class="pre">test/test_workers.py</span></tt> which gives an example of how to create |
| 352 |
a collection of worker processes which execute tasks from one queue |
| 353 |
and return results on another.</p> |
| 354 |
</li> |
| 355 |
<li><p class="first">Added <tt class="docutils literal"><span class="pre">processing.Pool()</span></tt> which returns a process pool object. This |
| 356 |
allows one to execute functions asynchronously. It also has a |
| 357 |
parallel implementation of the <tt class="docutils literal"><span class="pre">map()</span></tt> builtin. This is still |
| 358 |
<em>experimental</em> and undocumented --- see <tt class="docutils literal"><span class="pre">test/test_pool.py</span></tt> for |
| 359 |
example usage.</p> |
| 360 |
</li> |
| 361 |
</ul> |
| 362 |
</div> |
| 363 |
<div class="section"> |
| 364 |
<h1><a id="changes-in-0-33" name="changes-in-0-33">Changes in 0.33</a></h1> |
| 365 |
<ul> |
| 366 |
<li><p class="first">Added a <tt class="docutils literal"><span class="pre">recvbytes_into()</span></tt> method for receiving byte data into |
| 367 |
objects with the writable buffer interface. Also renamed the |
| 368 |
<tt class="docutils literal"><span class="pre">_recv_string()</span></tt> and <tt class="docutils literal"><span class="pre">_send_string()</span></tt> methods of connection objects |
| 369 |
to <tt class="docutils literal"><span class="pre">recvbytes()</span></tt> and <tt class="docutils literal"><span class="pre">sendbytes()</span></tt>.</p> |
| 370 |
</li> |
| 371 |
<li><p class="first">Some optimizations for the transferring of large blocks of data |
| 372 |
using connection objects.</p> |
| 373 |
</li> |
| 374 |
<li><p class="first">On Unix <tt class="docutils literal"><span class="pre">os.sysconf()</span></tt> is now used by default to determine whether |
| 375 |
to compile in support for posix semaphores or posix message queues.</p> |
| 376 |
<p>By using the <tt class="docutils literal"><span class="pre">NO_SEM_TIMED</span></tt> and <tt class="docutils literal"><span class="pre">NO_MQ_TIMED</span></tt> macros (see |
| 377 |
<tt class="docutils literal"><span class="pre">INSTALL.txt</span></tt>) it should now also be possible to compile in |
| 378 |
(partial) semaphore or queue support on Unix systems which lack the |
| 379 |
timeout functions <tt class="docutils literal"><span class="pre">sem_timedwait()</span></tt> or <tt class="docutils literal"><span class="pre">mq_timedreceive()</span></tt> and |
| 380 |
<tt class="docutils literal"><span class="pre">mq_timesend()</span></tt>.</p> |
| 381 |
</li> |
| 382 |
<li><p class="first"><tt class="docutils literal"><span class="pre">gettimeofday()</span></tt> is now used instead of <tt class="docutils literal"><span class="pre">clock_gettime()</span></tt> making |
| 383 |
compilation of the C extension (hopefully) possible on Mac OSX. No |
| 384 |
modificaton of <tt class="docutils literal"><span class="pre">setup.py</span></tt> should be necessary. Thanks to Michele |
| 385 |
Bertoldi for report and proposed patch.</p> |
| 386 |
</li> |
| 387 |
<li><p class="first"><tt class="docutils literal"><span class="pre">cpuCount()</span></tt> function added which returns the number of CPUs |
| 388 |
in the system.</p> |
| 389 |
</li> |
| 390 |
<li><p class="first">Bugfixes to <tt class="docutils literal"><span class="pre">PosixQueue</span></tt> class.</p> |
| 391 |
</li> |
| 392 |
</ul> |
| 393 |
</div> |
| 394 |
<div class="section"> |
| 395 |
<h1><a id="changes-in-0-32" name="changes-in-0-32">Changes in 0.32</a></h1> |
| 396 |
<ul class="simple"> |
| 397 |
<li>Refactored and simplified <tt class="docutils literal"><span class="pre">_nonforking</span></tt> module -- info about |
| 398 |
<tt class="docutils literal"><span class="pre">sys.modules</span></tt> of parent process is no longer passed on to child |
| 399 |
process. Also <tt class="docutils literal"><span class="pre">pkgutil</span></tt> is no longer used.</li> |
| 400 |
<li>Allocated space from an mmap used by <tt class="docutils literal"><span class="pre">LocalManager</span></tt> will now be |
| 401 |
recycled.</li> |
| 402 |
<li>Better tests for <tt class="docutils literal"><span class="pre">LocalManager</span></tt>.</li> |
| 403 |
<li>Fixed bug in <tt class="docutils literal"><span class="pre">managers.py</span></tt> concerning refcounting of shared objects. |
| 404 |
Bug affects the case where the callable used to create a shared |
| 405 |
object does not return a unique object each time it is called. |
| 406 |
Thanks to Alexey Akimov for the report.</li> |
| 407 |
<li>Added a <tt class="docutils literal"><span class="pre">freezeSupport()</span></tt> function. Calling this at the appropriate |
| 408 |
point in the main module is necessary when freezing a multiprocess |
| 409 |
program to produce a Windows executable. (Has been tested with |
| 410 |
<tt class="docutils literal"><span class="pre">py2exe</span></tt>, <tt class="docutils literal"><span class="pre">PyInstaller</span></tt> and <tt class="docutils literal"><span class="pre">cx_Freeze</span></tt>.)</li> |
| 411 |
</ul> |
| 412 |
</div> |
| 413 |
<div class="section"> |
| 414 |
<h1><a id="changes-in-0-31" name="changes-in-0-31">Changes in 0.31</a></h1> |
| 415 |
<ul class="simple"> |
| 416 |
<li>Fixed one line bug in <tt class="docutils literal"><span class="pre">localmanager.py</span></tt> which caused shared memory maps |
| 417 |
not to be resized properly.</li> |
| 418 |
<li>Added tests for shared values/structs/arrays to <tt class="docutils literal"><span class="pre">test/test_processing</span></tt>.</li> |
| 419 |
</ul> |
| 420 |
</div> |
| 421 |
<div class="section"> |
| 422 |
<h1><a id="changes-in-0-30" name="changes-in-0-30">Changes in 0.30</a></h1> |
| 423 |
<ul> |
| 424 |
<li><p class="first">Process objects now support the complete API of thread objects.</p> |
| 425 |
<p>In particular <tt class="docutils literal"><span class="pre">isAlive()</span></tt>, <tt class="docutils literal"><span class="pre">isDaemon()</span></tt>, <tt class="docutils literal"><span class="pre">setDaemon()</span></tt> have been |
| 426 |
added and <tt class="docutils literal"><span class="pre">join()</span></tt> now supports the <tt class="docutils literal"><span class="pre">timeout</span></tt> paramater.</p> |
| 427 |
<p>There are also new methods <tt class="docutils literal"><span class="pre">stop()</span></tt>, <tt class="docutils literal"><span class="pre">getPid()</span></tt> and <tt class="docutils literal"><span class="pre">getExitCode()</span></tt>.</p> |
| 428 |
</li> |
| 429 |
<li><p class="first">Implemented synchronization primitives based on the Windows mutexes |
| 430 |
and semaphores and posix named semaphores.</p> |
| 431 |
</li> |
| 432 |
<li><p class="first">Added support for sharing simple objects between processes by using |
| 433 |
a shared memory map and the <tt class="docutils literal"><span class="pre">struct</span></tt> or <tt class="docutils literal"><span class="pre">array</span></tt> modules.</p> |
| 434 |
</li> |
| 435 |
<li><p class="first">An <tt class="docutils literal"><span class="pre">activeChildren()</span></tt> function has been added to <tt class="docutils literal"><span class="pre">processing</span></tt> which |
| 436 |
returns a list of the child processes which are still alive.</p> |
| 437 |
</li> |
| 438 |
<li><p class="first">A <tt class="docutils literal"><span class="pre">Pipe()</span></tt> function has been added which returns a pair of |
| 439 |
connection objects representing the ends of a duplex connection over |
| 440 |
which picklable objects can be sent.</p> |
| 441 |
</li> |
| 442 |
<li><p class="first">socket objects etc are now picklable and can be transferred between |
| 443 |
processes. (Requires compilation of the <tt class="docutils literal"><span class="pre">_processing</span></tt> extension.)</p> |
| 444 |
</li> |
| 445 |
<li><p class="first">Subclasses of <tt class="docutils literal"><span class="pre">managers.BaseManager</span></tt> no longer automatically spawn a |
| 446 |
child process when an instance is created: the <tt class="docutils literal"><span class="pre">start()</span></tt> method must be |
| 447 |
called explicitly.</p> |
| 448 |
</li> |
| 449 |
<li><p class="first">On Windows child processes are now spawned using <tt class="docutils literal"><span class="pre">subprocess</span></tt>.</p> |
| 450 |
</li> |
| 451 |
<li><p class="first">On Windows the Python 2.5 version of <tt class="docutils literal"><span class="pre">pkgutil</span></tt> is now used for |
| 452 |
loading modules by the <tt class="docutils literal"><span class="pre">_nonforking</span></tt> module. On Python 2.4 this |
| 453 |
version of <tt class="docutils literal"><span class="pre">pkgutil</span></tt> (which uses the standard Python licence) is |
| 454 |
included in <tt class="docutils literal"><span class="pre">processing.compat</span></tt>.</p> |
| 455 |
</li> |
| 456 |
<li><p class="first">The arguments to the functions in <tt class="docutils literal"><span class="pre">processing.connection</span></tt> have |
| 457 |
changed slightly.</p> |
| 458 |
</li> |
| 459 |
<li><p class="first">Connection objects now have a <tt class="docutils literal"><span class="pre">poll()</span></tt> method which tests whether |
| 460 |
there is any data available for reading.</p> |
| 461 |
</li> |
| 462 |
<li><p class="first">The <tt class="docutils literal"><span class="pre">test/py2exedemo</span></tt> folder shows how to get <tt class="docutils literal"><span class="pre">py2exe</span></tt> to create a |
| 463 |
Windows executable from a program using the <tt class="docutils literal"><span class="pre">processing</span></tt> package.</p> |
| 464 |
</li> |
| 465 |
<li><p class="first">More tests.</p> |
| 466 |
</li> |
| 467 |
<li><p class="first">Bugfixes.</p> |
| 468 |
</li> |
| 469 |
<li><p class="first">Rearrangement of various stuff.</p> |
| 470 |
</li> |
| 471 |
</ul> |
| 472 |
</div> |
| 473 |
<div class="section"> |
| 474 |
<h1><a id="changes-in-0-21" name="changes-in-0-21">Changes in 0.21</a></h1> |
| 475 |
<ul class="simple"> |
| 476 |
<li>By default a proxy is now only able to access those methods of its |
| 477 |
referent which have been explicitly exposed.</li> |
| 478 |
<li>The <tt class="docutils literal"><span class="pre">connection</span></tt> sub-package now supports digest authentication.</li> |
| 479 |
<li>Process objects are now given randomly generated 'inheritable' |
| 480 |
authentication keys.</li> |
| 481 |
<li>A manager process will now only accept connections from processes |
| 482 |
using the same authentication key.</li> |
| 483 |
<li>Previously <tt class="docutils literal"><span class="pre">get_module()</span></tt> from <tt class="docutils literal"><span class="pre">_nonforking.py</span></tt> was seriously messed |
| 484 |
up (though it generally worked). It is a lot saner now.</li> |
| 485 |
<li>Python 2.4 or higher is now required.</li> |
| 486 |
</ul> |
| 487 |
</div> |
| 488 |
<div class="section"> |
| 489 |
<h1><a id="changes-in-0-20" name="changes-in-0-20">Changes in 0.20</a></h1> |
| 490 |
<ul class="simple"> |
| 491 |
<li>The <tt class="docutils literal"><span class="pre">doc</span></tt> folder contains HTML documentation.</li> |
| 492 |
<li><tt class="docutils literal"><span class="pre">test</span></tt> is now a subpackage. Running <tt class="docutils literal"><span class="pre">processing.test.main()</span></tt> |
| 493 |
will run test scripts using both processes and threads.</li> |
| 494 |
<li><tt class="docutils literal"><span class="pre">nonforking.py</span></tt> has been renamed <tt class="docutils literal"><span class="pre">_nonforking.py</span></tt>. |
| 495 |
<tt class="docutils literal"><span class="pre">manager.py</span></tt> has been renamed <tt class="docutils literal"><span class="pre">manager.py</span></tt>. |
| 496 |
<tt class="docutils literal"><span class="pre">connection.py</span></tt> has become a sub-package <tt class="docutils literal"><span class="pre">connection</span></tt></li> |
| 497 |
<li><tt class="docutils literal"><span class="pre">Listener</span></tt> and <tt class="docutils literal"><span class="pre">Client</span></tt> have been removed from |
| 498 |
<tt class="docutils literal"><span class="pre">processing</span></tt>, but still exist in <tt class="docutils literal"><span class="pre">processing.connection</span></tt>.</li> |
| 499 |
<li>The package is now <em>probably</em> compatible with versions of Python |
| 500 |
earlier than 2.4.</li> |
| 501 |
<li><tt class="docutils literal"><span class="pre">set</span></tt> is no longer a type supported by the default manager type.</li> |
| 502 |
<li>Many more changes.</li> |
| 503 |
</ul> |
| 504 |
</div> |
| 505 |
<div class="section"> |
| 506 |
<h1><a id="changes-in-0-12" name="changes-in-0-12">Changes in 0.12</a></h1> |
| 507 |
<ul class="simple"> |
| 508 |
<li>Fixed bug where the arguments to <tt class="docutils literal"><span class="pre">processing.Manager()</span></tt> were passed on |
| 509 |
to <tt class="docutils literal"><span class="pre">processing.manager.DefaultManager()</span></tt> in the wrong order.</li> |
| 510 |
<li><tt class="docutils literal"><span class="pre">processing.dummy</span></tt> is now a subpackage of <tt class="docutils literal"><span class="pre">processing</span></tt> |
| 511 |
instead of a module.</li> |
| 512 |
<li>Rearranged package so that the <tt class="docutils literal"><span class="pre">test</span></tt> folder, <tt class="docutils literal"><span class="pre">README.txt</span></tt> and |
| 513 |
<tt class="docutils literal"><span class="pre">CHANGES.txt</span></tt> are copied when the package is installed.</li> |
| 514 |
</ul> |
| 515 |
</div> |
| 516 |
<div class="section"> |
| 517 |
<h1><a id="changes-in-0-11" name="changes-in-0-11">Changes in 0.11</a></h1> |
| 518 |
<ul class="simple"> |
| 519 |
<li>Fixed bug on windows when the full path of <tt class="docutils literal"><span class="pre">nonforking.py</span></tt> contains a |
| 520 |
space.</li> |
| 521 |
<li>On unix there is no longer a need to make the arguments to the |
| 522 |
constructor of <tt class="docutils literal"><span class="pre">Process</span></tt> be picklable or for and instance of a |
| 523 |
subclass of <tt class="docutils literal"><span class="pre">Process</span></tt> to be picklable when you call the start method.</li> |
| 524 |
<li>On unix proxies which a child process inherits from its parent can |
| 525 |
be used by the child without any problem, so there is no longer a |
| 526 |
need to pass them as arguments to <tt class="docutils literal"><span class="pre">Process</span></tt>. (This will never be |
| 527 |
possible on windows.)</li> |
| 528 |
</ul> |
| 529 |
</div> |
| 530 |
</div> |
| 531 |
</body> |
| 532 |
</html> |