| 1 |
<?xml version="1.0" encoding="utf-8" ?> |
| 2 |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| 3 |
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| 4 |
<head> |
| 5 |
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> |
| 6 |
<meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" /> |
| 7 |
<title>Manager objects</title> |
| 8 |
<link rel="stylesheet" href="html4css1.css" type="text/css" /> |
| 9 |
</head> |
| 10 |
<body> |
| 11 |
<div class="header"> |
| 12 |
<a class="reference" href="connection-objects.html">Prev</a> <a class="reference" href="processing-ref.html">Up</a> <a class="reference" href="proxy-objects.html">Next</a> |
| 13 |
<hr class="header"/> |
| 14 |
</div> |
| 15 |
<div class="document" id="manager-objects"> |
| 16 |
<h1 class="title">Manager objects</h1> |
| 17 |
<p>A manager object controls a server process which manages <em>shared |
| 18 |
objects</em>. Other processes can access the shared objects by using |
| 19 |
proxies.</p> |
| 20 |
<p>Manager processes will be shutdown as soon as they are garbage |
| 21 |
collected or their parent process exits. The manager classes are |
| 22 |
defined in the <tt class="docutils literal"><span class="pre">processing.managers</span></tt> module.</p> |
| 23 |
<div class="section"> |
| 24 |
<h1><a id="basemanager" name="basemanager">BaseManager</a></h1> |
| 25 |
<p><tt class="docutils literal"><span class="pre">BaseManager</span></tt> is the base class for all manager classes which use a |
| 26 |
server process. It does not possess any methods which create shared |
| 27 |
objects.</p> |
| 28 |
<p>The public methods of <tt class="docutils literal"><span class="pre">BaseManager</span></tt> are the following:</p> |
| 29 |
<blockquote> |
| 30 |
<dl class="docutils"> |
| 31 |
<dt><tt class="docutils literal"><span class="pre">__init__(self,</span> <span class="pre">address=None,</span> <span class="pre">authkey=None)</span></tt></dt> |
| 32 |
<dd><p class="first">Creates a manager object.</p> |
| 33 |
<p>Once created one should call <tt class="docutils literal"><span class="pre">start()</span></tt> or <tt class="docutils literal"><span class="pre">serveForever()</span></tt> to |
| 34 |
ensure that the manager object refers to a started manager |
| 35 |
process.</p> |
| 36 |
<p>The arguments to the constructor are as follows:</p> |
| 37 |
<dl class="last docutils"> |
| 38 |
<dt><tt class="docutils literal"><span class="pre">address</span></tt></dt> |
| 39 |
<dd><p class="first">The address on which the manager process listens for |
| 40 |
new connections. If <tt class="docutils literal"><span class="pre">address</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> then an arbitrary |
| 41 |
one is chosen.</p> |
| 42 |
<p class="last">See <a class="reference" href="connection-ref.html#listener-objects">Listener objects</a>.</p> |
| 43 |
</dd> |
| 44 |
<dt><tt class="docutils literal"><span class="pre">authkey</span></tt></dt> |
| 45 |
<dd><p class="first">The authentication key which will be used to check the |
| 46 |
validity of incoming connections to the server process.</p> |
| 47 |
<p>If <tt class="docutils literal"><span class="pre">authkey</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> then <tt class="docutils literal"><span class="pre">currentProcess().getAuthKey()</span></tt>. |
| 48 |
Otherwise <tt class="docutils literal"><span class="pre">authkey</span></tt> is used and it must be a string.</p> |
| 49 |
<p class="last">See <a class="reference" href="connection-ref.html#authentication-keys">Authentication keys</a>.</p> |
| 50 |
</dd> |
| 51 |
</dl> |
| 52 |
</dd> |
| 53 |
<dt><tt class="docutils literal"><span class="pre">start()</span></tt></dt> |
| 54 |
<dd>Spawn or fork a subprocess to start the manager.</dd> |
| 55 |
<dt><tt class="docutils literal"><span class="pre">serveForever()</span></tt></dt> |
| 56 |
<dd>Start the manager in the current process. See <a class="reference" href="#using-a-remote-manager">Using a remote |
| 57 |
manager</a>.</dd> |
| 58 |
<dt><tt class="docutils literal"><span class="pre">fromAddress(address,</span> <span class="pre">authkey)</span></tt></dt> |
| 59 |
<dd>A class method which returns a manager object referring to a |
| 60 |
pre-existing server process which is using the given address and |
| 61 |
authentication key. See <a class="reference" href="#using-a-remote-manager">Using a remote manager</a>.</dd> |
| 62 |
<dt><tt class="docutils literal"><span class="pre">shutdown()</span></tt></dt> |
| 63 |
<dd><p class="first">Stop the process used by the manager. This is only available |
| 64 |
if <tt class="docutils literal"><span class="pre">start()</span></tt> has been used to start the server process.</p> |
| 65 |
<p class="last">This can be called multiple times.</p> |
| 66 |
</dd> |
| 67 |
</dl> |
| 68 |
</blockquote> |
| 69 |
<p><tt class="docutils literal"><span class="pre">BaseManager</span></tt> instances also have one read-only property:</p> |
| 70 |
<blockquote> |
| 71 |
<dl class="docutils"> |
| 72 |
<dt><tt class="docutils literal"><span class="pre">address</span></tt></dt> |
| 73 |
<dd>The address used by the manager.</dd> |
| 74 |
</dl> |
| 75 |
</blockquote> |
| 76 |
<p>The creation of managers which support arbitrary types is discussed |
| 77 |
below in <a class="reference" href="#customized-managers">Customized managers</a>.</p> |
| 78 |
</div> |
| 79 |
<div class="section"> |
| 80 |
<h1><a id="syncmanager" name="syncmanager">SyncManager</a></h1> |
| 81 |
<p><tt class="docutils literal"><span class="pre">SyncManager</span></tt> is a subclass of <tt class="docutils literal"><span class="pre">BaseManager</span></tt> which can be used for |
| 82 |
the synchronization of processes. Objects of this type are returned |
| 83 |
by <tt class="docutils literal"><span class="pre">processing.Manager()</span></tt>.</p> |
| 84 |
<p>It also supports creation of shared lists and dictionaries. The |
| 85 |
instance methods defined by <tt class="docutils literal"><span class="pre">SyncManager</span></tt> are</p> |
| 86 |
<blockquote> |
| 87 |
<dl class="docutils"> |
| 88 |
<dt><tt class="docutils literal"><span class="pre">BoundedSemaphore(value=1)</span></tt></dt> |
| 89 |
<dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.BoundedSemaphore</span></tt> object and |
| 90 |
returns a proxy for it.</dd> |
| 91 |
<dt><tt class="docutils literal"><span class="pre">Condition(lock=None)</span></tt></dt> |
| 92 |
<dd><p class="first">Creates a shared <tt class="docutils literal"><span class="pre">threading.Condition</span></tt> object and returns a |
| 93 |
proxy for it.</p> |
| 94 |
<p class="last">If <tt class="docutils literal"><span class="pre">lock</span></tt> is supplied then it should be a proxy for a |
| 95 |
<tt class="docutils literal"><span class="pre">threading.Lock</span></tt> or <tt class="docutils literal"><span class="pre">threading.RLock</span></tt> object.</p> |
| 96 |
</dd> |
| 97 |
<dt><tt class="docutils literal"><span class="pre">Event()</span></tt></dt> |
| 98 |
<dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.Event</span></tt> object and returns a proxy |
| 99 |
for it.</dd> |
| 100 |
<dt><tt class="docutils literal"><span class="pre">Lock()</span></tt></dt> |
| 101 |
<dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.Lock</span></tt> object and returns a proxy |
| 102 |
for it.</dd> |
| 103 |
<dt><tt class="docutils literal"><span class="pre">Namespace()</span></tt></dt> |
| 104 |
<dd><p class="first">Creates a shared <tt class="docutils literal"><span class="pre">Namespace</span></tt> object and returns a |
| 105 |
proxy for it.</p> |
| 106 |
<p class="last">See <a class="reference" href="#namespace-objects">Namespace objects</a>.</p> |
| 107 |
</dd> |
| 108 |
<dt><tt class="docutils literal"><span class="pre">Queue(maxsize=0)</span></tt></dt> |
| 109 |
<dd>Creates a shared <tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> object and returns a proxy for |
| 110 |
it.</dd> |
| 111 |
<dt><tt class="docutils literal"><span class="pre">RLock()</span></tt></dt> |
| 112 |
<dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.RLock</span></tt> object and returns a proxy |
| 113 |
for it.</dd> |
| 114 |
<dt><tt class="docutils literal"><span class="pre">Semaphore(value=1)</span></tt></dt> |
| 115 |
<dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.Semaphore</span></tt> object and returns a |
| 116 |
proxy for it.</dd> |
| 117 |
<dt><tt class="docutils literal"><span class="pre">Array(typecode,</span> <span class="pre">sequence)</span></tt></dt> |
| 118 |
<dd>Create an array and returns a proxy for |
| 119 |
it. (<tt class="docutils literal"><span class="pre">format</span></tt> is ignored.)</dd> |
| 120 |
<dt><tt class="docutils literal"><span class="pre">Value(typecode,</span> <span class="pre">value)</span></tt></dt> |
| 121 |
<dd>Create an object with a writable <tt class="docutils literal"><span class="pre">value</span></tt> attribute and returns |
| 122 |
a proxy for it.</dd> |
| 123 |
<dt><tt class="docutils literal"><span class="pre">dict()</span></tt>, <tt class="docutils literal"><span class="pre">dict(mapping)</span></tt>, <tt class="docutils literal"><span class="pre">dict(sequence)</span></tt></dt> |
| 124 |
<dd>Creates a shared <tt class="docutils literal"><span class="pre">dict</span></tt> object and returns a proxy for it.</dd> |
| 125 |
<dt><tt class="docutils literal"><span class="pre">list()</span></tt>, <tt class="docutils literal"><span class="pre">list(sequence)</span></tt></dt> |
| 126 |
<dd>Creates a shared <tt class="docutils literal"><span class="pre">list</span></tt> object and returns a proxy for it.</dd> |
| 127 |
</dl> |
| 128 |
</blockquote> |
| 129 |
<div class="section"> |
| 130 |
<h2><a id="namespace-objects" name="namespace-objects">Namespace objects</a></h2> |
| 131 |
<p>A namespace object has no public methods but does have writable |
| 132 |
attributes. Its representation shows the values of its attributes.</p> |
| 133 |
<p>However, when using a proxy for a namespace object, an attribute |
| 134 |
beginning with <tt class="docutils literal"><span class="pre">'_'</span></tt> will be an attribute of the proxy and not an |
| 135 |
attribute of the referent:</p> |
| 136 |
<pre class="literal-block"> |
| 137 |
>>> manager = processing.Manager() |
| 138 |
>>> Global = manager.Namespace() |
| 139 |
>>> Global.x = 10 |
| 140 |
>>> Global.y = 'hello' |
| 141 |
>>> Global._z = 12.3 # this is an attribute of the proxy |
| 142 |
>>> print Global |
| 143 |
Namespace(x=10, y='hello') |
| 144 |
</pre> |
| 145 |
</div> |
| 146 |
</div> |
| 147 |
<div class="section"> |
| 148 |
<h1><a id="customized-managers" name="customized-managers">Customized managers</a></h1> |
| 149 |
<p>To create one's own manager one creates a subclass of <tt class="docutils literal"><span class="pre">BaseManager</span></tt>.</p> |
| 150 |
<p>To create a method of the subclass which will create new shared |
| 151 |
objects one uses the following function:</p> |
| 152 |
<blockquote> |
| 153 |
<dl class="docutils"> |
| 154 |
<dt><tt class="docutils literal"><span class="pre">CreatorMethod(callable=None,</span> <span class="pre">proxytype=None,</span> <span class="pre">exposed=None,</span> <span class="pre">typeid=None)</span></tt></dt> |
| 155 |
<dd><p class="first">Returns a function with signature <tt class="docutils literal"><span class="pre">func(self,</span> <span class="pre">*args,</span> <span class="pre">**kwds)</span></tt> |
| 156 |
which will create a shared object using the manager <tt class="docutils literal"><span class="pre">self</span></tt> |
| 157 |
and return a proxy for it.</p> |
| 158 |
<p>The shared objects will be created by evaluating |
| 159 |
<tt class="docutils literal"><span class="pre">callable(*args,</span> <span class="pre">**kwds)</span></tt> in the manager process.</p> |
| 160 |
<p>The arguments are:</p> |
| 161 |
<dl class="last docutils"> |
| 162 |
<dt><tt class="docutils literal"><span class="pre">callable</span></tt></dt> |
| 163 |
<dd>The callable used to create a shared object. If the |
| 164 |
manager will connect to a remote manager then this is ignored.</dd> |
| 165 |
<dt><tt class="docutils literal"><span class="pre">proxytype</span></tt></dt> |
| 166 |
<dd><p class="first">The type of proxy which will be used for object returned |
| 167 |
by <tt class="docutils literal"><span class="pre">callable</span></tt>.</p> |
| 168 |
<p class="last">If <tt class="docutils literal"><span class="pre">proxytype</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> then each time an object is |
| 169 |
returned by <tt class="docutils literal"><span class="pre">callable</span></tt> either a new proxy type is created |
| 170 |
or a cached one is reused. The methods of the shared |
| 171 |
object which will be exposed via the proxy will then be |
| 172 |
determined by the <tt class="docutils literal"><span class="pre">exposed</span></tt> argument, see below.</p> |
| 173 |
</dd> |
| 174 |
<dt><tt class="docutils literal"><span class="pre">exposed</span></tt></dt> |
| 175 |
<dd><p class="first">Given a shared object returned by <tt class="docutils literal"><span class="pre">callable</span></tt>, the |
| 176 |
<tt class="docutils literal"><span class="pre">exposed</span></tt> argument is the list of those method names which |
| 177 |
should be exposed via <a class="reference" href="proxy-objects.html#methods-of-baseproxy"><tt class="docutils literal"><span class="pre">BaseProxy._callMethod()</span></tt></a>. <a class="footnote-reference" href="#id3" id="id1" name="id1">[1]</a> <a class="footnote-reference" href="#id4" id="id2" name="id2">[2]</a></p> |
| 178 |
<p>If <tt class="docutils literal"><span class="pre">exposed</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> and <tt class="docutils literal"><span class="pre">callable.__exposed__</span></tt> exists then |
| 179 |
<tt class="docutils literal"><span class="pre">callable.__exposed__</span></tt> is used instead.</p> |
| 180 |
<p>If <tt class="docutils literal"><span class="pre">exposed</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> and <tt class="docutils literal"><span class="pre">callable.__exposed__</span></tt> does not |
| 181 |
exist then all methods of the shared object which do not |
| 182 |
start with <tt class="docutils literal"><span class="pre">'_'</span></tt> will be exposed.</p> |
| 183 |
<p class="last">An attempt to use <tt class="docutils literal"><span class="pre">BaseProxy._callMethod()</span></tt> with a method name which is |
| 184 |
not exposed will raise an exception.</p> |
| 185 |
</dd> |
| 186 |
<dt><tt class="docutils literal"><span class="pre">typeid</span></tt></dt> |
| 187 |
<dd>If <tt class="docutils literal"><span class="pre">typeid</span></tt> is a string then it is used as an identifier |
| 188 |
for the callable. Otherwise, <tt class="docutils literal"><span class="pre">typeid</span></tt> must be <tt class="docutils literal"><span class="pre">None</span></tt> and |
| 189 |
a string prefixed by <tt class="docutils literal"><span class="pre">callable.__name__</span></tt> is used as the |
| 190 |
identifier.</dd> |
| 191 |
</dl> |
| 192 |
</dd> |
| 193 |
</dl> |
| 194 |
</blockquote> |
| 195 |
<table class="docutils footnote" frame="void" id="id3" rules="none"> |
| 196 |
<colgroup><col class="label" /><col /></colgroup> |
| 197 |
<tbody valign="top"> |
| 198 |
<tr><td class="label"><a class="fn-backref" href="#id1" name="id3">[1]</a></td><td>A method here means any attribute which has a <tt class="docutils literal"><span class="pre">__call__</span></tt> |
| 199 |
attribute.</td></tr> |
| 200 |
</tbody> |
| 201 |
</table> |
| 202 |
<table class="docutils footnote" frame="void" id="id4" rules="none"> |
| 203 |
<colgroup><col class="label" /><col /></colgroup> |
| 204 |
<tbody valign="top"> |
| 205 |
<tr><td class="label"><a class="fn-backref" href="#id2" name="id4">[2]</a></td><td><p class="first">The method names <tt class="docutils literal"><span class="pre">__repr__</span></tt>, <tt class="docutils literal"><span class="pre">__str__</span></tt>, and <tt class="docutils literal"><span class="pre">__cmp__</span></tt> of a |
| 206 |
shared object are always exposed by the manager. However, instead |
| 207 |
of invoking the <tt class="docutils literal"><span class="pre">__repr__()</span></tt>, <tt class="docutils literal"><span class="pre">__str__()</span></tt>, <tt class="docutils literal"><span class="pre">__cmp__()</span></tt> instance |
| 208 |
methods (none of which are guaranteed to exist) they invoke the |
| 209 |
builtin functions <tt class="docutils literal"><span class="pre">repr()</span></tt>, <tt class="docutils literal"><span class="pre">str()</span></tt> and <tt class="docutils literal"><span class="pre">cmp()</span></tt>.</p> |
| 210 |
<p class="last">Note that one should generally avoid exposing rich comparison |
| 211 |
methods like <tt class="docutils literal"><span class="pre">__eq__()</span></tt>, <tt class="docutils literal"><span class="pre">__ne__()</span></tt>, <tt class="docutils literal"><span class="pre">__le__()</span></tt>. To make the proxy |
| 212 |
type support comparison by value one can just expose <tt class="docutils literal"><span class="pre">__cmp__()</span></tt> |
| 213 |
instead (even if the referent does not have such a method).</p> |
| 214 |
</td></tr> |
| 215 |
</tbody> |
| 216 |
</table> |
| 217 |
<div class="section"> |
| 218 |
<h2><a id="example" name="example">Example</a></h2> |
| 219 |
<pre class="literal-block"> |
| 220 |
from processing.managers import BaseManager, CreatorMethod |
| 221 |
|
| 222 |
class FooClass(object): |
| 223 |
def bar(self): |
| 224 |
print 'BAR' |
| 225 |
def baz(self): |
| 226 |
print 'BAZ' |
| 227 |
|
| 228 |
class NewManager(BaseManager): |
| 229 |
Foo = CreatorMethod(FooClass) |
| 230 |
|
| 231 |
if __name__ == '__main__': |
| 232 |
manager = NewManager() |
| 233 |
manager.start() |
| 234 |
foo = manager.Foo() |
| 235 |
foo.bar() # prints 'BAR' |
| 236 |
foo.baz() # prints 'BAZ' |
| 237 |
manager.shutdown() |
| 238 |
</pre> |
| 239 |
<p>See <a class="reference" href="../examples/ex_newtype.py">ex_newtype.py</a> for more examples.</p> |
| 240 |
</div> |
| 241 |
</div> |
| 242 |
<div class="section"> |
| 243 |
<h1><a id="using-a-remote-manager" name="using-a-remote-manager">Using a remote manager</a></h1> |
| 244 |
<p>It is possible to run a manager server on one machine and have clients |
| 245 |
use it from other machines (assuming that the firewalls involved allow |
| 246 |
it).</p> |
| 247 |
<p>Running the following commands creates a server for a shared queue which |
| 248 |
remote clients can use:</p> |
| 249 |
<pre class="literal-block"> |
| 250 |
>>> from processing.managers import BaseManager, CreatorMethod |
| 251 |
>>> import Queue |
| 252 |
>>> queue = Queue.Queue() |
| 253 |
>>> class QueueManager(BaseManager): |
| 254 |
... get_proxy = CreatorMethod(callable=lambda:queue, typeid='get_proxy') |
| 255 |
... |
| 256 |
>>> m = QueueManager(address=('foo.bar.org', 50000), authkey='none') |
| 257 |
>>> m.serveForever() |
| 258 |
</pre> |
| 259 |
<p>One client can access the server as follows:</p> |
| 260 |
<pre class="literal-block"> |
| 261 |
>>> from processing.managers import BaseManager, CreatorMethod |
| 262 |
>>> class QueueManager(BaseManager): |
| 263 |
... get_proxy = CreatorMethod(typeid='get_proxy') |
| 264 |
... |
| 265 |
>>> m = QueueManager.fromAddress(address=('foo.bar.org', 50000), authkey='none') |
| 266 |
>>> queue = m.get_proxy() |
| 267 |
>>> queue.put('hello') |
| 268 |
</pre> |
| 269 |
<p>Another client can also use it:</p> |
| 270 |
<pre class="literal-block"> |
| 271 |
>>> from processing.managers import BaseManager, CreatorMethod |
| 272 |
>>> class QueueManager(BaseManager): |
| 273 |
... get_proxy = CreatorMethod(typeid='get_proxy') |
| 274 |
... |
| 275 |
>>> m = QueueManager.fromAddress(address=('foo.bar.org', 50000), authkey='none') |
| 276 |
>>> queue = m.get_proxy() |
| 277 |
>>> queue.get() |
| 278 |
'hello' |
| 279 |
</pre> |
| 280 |
</div> |
| 281 |
</div> |
| 282 |
<div class="footer"> |
| 283 |
<hr class="footer" /> |
| 284 |
<a class="reference" href="connection-objects.html">Prev</a> <a class="reference" href="processing-ref.html">Up</a> <a class="reference" href="proxy-objects.html">Next</a> |
| 285 |
</div> |
| 286 |
</body> |
| 287 |
</html> |