Debian Codegraph @

pypy3.9/doc/manager-objects.html

raw
1 <?xml version="1.0" encoding="utf-8" ?>
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
4 <head>
5 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
6 <meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" />
7 <title>Manager objects</title>
8 <link rel="stylesheet" href="html4css1.css" type="text/css" />
9 </head>
10 <body>
11 <div class="header">
12 <a class="reference" href="connection-objects.html">Prev</a> &nbsp; &nbsp; &nbsp; &nbsp; <a class="reference" href="processing-ref.html">Up</a> &nbsp; &nbsp; &nbsp; &nbsp; <a class="reference" href="proxy-objects.html">Next</a>
13 <hr class="header"/>
14 </div>
15 <div class="document" id="manager-objects">
16 <h1 class="title">Manager objects</h1>
17 <p>A manager object controls a server process which manages <em>shared
18 objects</em>. Other processes can access the shared objects by using
19 proxies.</p>
20 <p>Manager processes will be shutdown as soon as they are garbage
21 collected or their parent process exits. The manager classes are
22 defined in the <tt class="docutils literal"><span class="pre">processing.managers</span></tt> module.</p>
23 <div class="section">
24 <h1><a id="basemanager" name="basemanager">BaseManager</a></h1>
25 <p><tt class="docutils literal"><span class="pre">BaseManager</span></tt> is the base class for all manager classes which use a
26 server process. It does not possess any methods which create shared
27 objects.</p>
28 <p>The public methods of <tt class="docutils literal"><span class="pre">BaseManager</span></tt> are the following:</p>
29 <blockquote>
30 <dl class="docutils">
31 <dt><tt class="docutils literal"><span class="pre">__init__(self,</span> <span class="pre">address=None,</span> <span class="pre">authkey=None)</span></tt></dt>
32 <dd><p class="first">Creates a manager object.</p>
33 <p>Once created one should call <tt class="docutils literal"><span class="pre">start()</span></tt> or <tt class="docutils literal"><span class="pre">serveForever()</span></tt> to
34 ensure that the manager object refers to a started manager
35 process.</p>
36 <p>The arguments to the constructor are as follows:</p>
37 <dl class="last docutils">
38 <dt><tt class="docutils literal"><span class="pre">address</span></tt></dt>
39 <dd><p class="first">The address on which the manager process listens for
40 new connections. If <tt class="docutils literal"><span class="pre">address</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> then an arbitrary
41 one is chosen.</p>
42 <p class="last">See <a class="reference" href="connection-ref.html#listener-objects">Listener objects</a>.</p>
43 </dd>
44 <dt><tt class="docutils literal"><span class="pre">authkey</span></tt></dt>
45 <dd><p class="first">The authentication key which will be used to check the
46 validity of incoming connections to the server process.</p>
47 <p>If <tt class="docutils literal"><span class="pre">authkey</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> then <tt class="docutils literal"><span class="pre">currentProcess().getAuthKey()</span></tt>.
48 Otherwise <tt class="docutils literal"><span class="pre">authkey</span></tt> is used and it must be a string.</p>
49 <p class="last">See <a class="reference" href="connection-ref.html#authentication-keys">Authentication keys</a>.</p>
50 </dd>
51 </dl>
52 </dd>
53 <dt><tt class="docutils literal"><span class="pre">start()</span></tt></dt>
54 <dd>Spawn or fork a subprocess to start the manager.</dd>
55 <dt><tt class="docutils literal"><span class="pre">serveForever()</span></tt></dt>
56 <dd>Start the manager in the current process. See <a class="reference" href="#using-a-remote-manager">Using a remote
57 manager</a>.</dd>
58 <dt><tt class="docutils literal"><span class="pre">fromAddress(address,</span> <span class="pre">authkey)</span></tt></dt>
59 <dd>A class method which returns a manager object referring to a
60 pre-existing server process which is using the given address and
61 authentication key. See <a class="reference" href="#using-a-remote-manager">Using a remote manager</a>.</dd>
62 <dt><tt class="docutils literal"><span class="pre">shutdown()</span></tt></dt>
63 <dd><p class="first">Stop the process used by the manager. This is only available
64 if <tt class="docutils literal"><span class="pre">start()</span></tt> has been used to start the server process.</p>
65 <p class="last">This can be called multiple times.</p>
66 </dd>
67 </dl>
68 </blockquote>
69 <p><tt class="docutils literal"><span class="pre">BaseManager</span></tt> instances also have one read-only property:</p>
70 <blockquote>
71 <dl class="docutils">
72 <dt><tt class="docutils literal"><span class="pre">address</span></tt></dt>
73 <dd>The address used by the manager.</dd>
74 </dl>
75 </blockquote>
76 <p>The creation of managers which support arbitrary types is discussed
77 below in <a class="reference" href="#customized-managers">Customized managers</a>.</p>
78 </div>
79 <div class="section">
80 <h1><a id="syncmanager" name="syncmanager">SyncManager</a></h1>
81 <p><tt class="docutils literal"><span class="pre">SyncManager</span></tt> is a subclass of <tt class="docutils literal"><span class="pre">BaseManager</span></tt> which can be used for
82 the synchronization of processes. Objects of this type are returned
83 by <tt class="docutils literal"><span class="pre">processing.Manager()</span></tt>.</p>
84 <p>It also supports creation of shared lists and dictionaries. The
85 instance methods defined by <tt class="docutils literal"><span class="pre">SyncManager</span></tt> are</p>
86 <blockquote>
87 <dl class="docutils">
88 <dt><tt class="docutils literal"><span class="pre">BoundedSemaphore(value=1)</span></tt></dt>
89 <dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.BoundedSemaphore</span></tt> object and
90 returns a proxy for it.</dd>
91 <dt><tt class="docutils literal"><span class="pre">Condition(lock=None)</span></tt></dt>
92 <dd><p class="first">Creates a shared <tt class="docutils literal"><span class="pre">threading.Condition</span></tt> object and returns a
93 proxy for it.</p>
94 <p class="last">If <tt class="docutils literal"><span class="pre">lock</span></tt> is supplied then it should be a proxy for a
95 <tt class="docutils literal"><span class="pre">threading.Lock</span></tt> or <tt class="docutils literal"><span class="pre">threading.RLock</span></tt> object.</p>
96 </dd>
97 <dt><tt class="docutils literal"><span class="pre">Event()</span></tt></dt>
98 <dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.Event</span></tt> object and returns a proxy
99 for it.</dd>
100 <dt><tt class="docutils literal"><span class="pre">Lock()</span></tt></dt>
101 <dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.Lock</span></tt> object and returns a proxy
102 for it.</dd>
103 <dt><tt class="docutils literal"><span class="pre">Namespace()</span></tt></dt>
104 <dd><p class="first">Creates a shared <tt class="docutils literal"><span class="pre">Namespace</span></tt> object and returns a
105 proxy for it.</p>
106 <p class="last">See <a class="reference" href="#namespace-objects">Namespace objects</a>.</p>
107 </dd>
108 <dt><tt class="docutils literal"><span class="pre">Queue(maxsize=0)</span></tt></dt>
109 <dd>Creates a shared <tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> object and returns a proxy for
110 it.</dd>
111 <dt><tt class="docutils literal"><span class="pre">RLock()</span></tt></dt>
112 <dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.RLock</span></tt> object and returns a proxy
113 for it.</dd>
114 <dt><tt class="docutils literal"><span class="pre">Semaphore(value=1)</span></tt></dt>
115 <dd>Creates a shared <tt class="docutils literal"><span class="pre">threading.Semaphore</span></tt> object and returns a
116 proxy for it.</dd>
117 <dt><tt class="docutils literal"><span class="pre">Array(typecode,</span> <span class="pre">sequence)</span></tt></dt>
118 <dd>Create an array and returns a proxy for
119 it. (<tt class="docutils literal"><span class="pre">format</span></tt> is ignored.)</dd>
120 <dt><tt class="docutils literal"><span class="pre">Value(typecode,</span> <span class="pre">value)</span></tt></dt>
121 <dd>Create an object with a writable <tt class="docutils literal"><span class="pre">value</span></tt> attribute and returns
122 a proxy for it.</dd>
123 <dt><tt class="docutils literal"><span class="pre">dict()</span></tt>, <tt class="docutils literal"><span class="pre">dict(mapping)</span></tt>, <tt class="docutils literal"><span class="pre">dict(sequence)</span></tt></dt>
124 <dd>Creates a shared <tt class="docutils literal"><span class="pre">dict</span></tt> object and returns a proxy for it.</dd>
125 <dt><tt class="docutils literal"><span class="pre">list()</span></tt>, <tt class="docutils literal"><span class="pre">list(sequence)</span></tt></dt>
126 <dd>Creates a shared <tt class="docutils literal"><span class="pre">list</span></tt> object and returns a proxy for it.</dd>
127 </dl>
128 </blockquote>
129 <div class="section">
130 <h2><a id="namespace-objects" name="namespace-objects">Namespace objects</a></h2>
131 <p>A namespace object has no public methods but does have writable
132 attributes. Its representation shows the values of its attributes.</p>
133 <p>However, when using a proxy for a namespace object, an attribute
134 beginning with <tt class="docutils literal"><span class="pre">'_'</span></tt> will be an attribute of the proxy and not an
135 attribute of the referent:</p>
136 <pre class="literal-block">
137 &gt;&gt;&gt; manager = processing.Manager()
138 &gt;&gt;&gt; Global = manager.Namespace()
139 &gt;&gt;&gt; Global.x = 10
140 &gt;&gt;&gt; Global.y = 'hello'
141 &gt;&gt;&gt; Global._z = 12.3 # this is an attribute of the proxy
142 &gt;&gt;&gt; print Global
143 Namespace(x=10, y='hello')
144 </pre>
145 </div>
146 </div>
147 <div class="section">
148 <h1><a id="customized-managers" name="customized-managers">Customized managers</a></h1>
149 <p>To create one's own manager one creates a subclass of <tt class="docutils literal"><span class="pre">BaseManager</span></tt>.</p>
150 <p>To create a method of the subclass which will create new shared
151 objects one uses the following function:</p>
152 <blockquote>
153 <dl class="docutils">
154 <dt><tt class="docutils literal"><span class="pre">CreatorMethod(callable=None,</span> <span class="pre">proxytype=None,</span> <span class="pre">exposed=None,</span> <span class="pre">typeid=None)</span></tt></dt>
155 <dd><p class="first">Returns a function with signature <tt class="docutils literal"><span class="pre">func(self,</span> <span class="pre">*args,</span> <span class="pre">**kwds)</span></tt>
156 which will create a shared object using the manager <tt class="docutils literal"><span class="pre">self</span></tt>
157 and return a proxy for it.</p>
158 <p>The shared objects will be created by evaluating
159 <tt class="docutils literal"><span class="pre">callable(*args,</span> <span class="pre">**kwds)</span></tt> in the manager process.</p>
160 <p>The arguments are:</p>
161 <dl class="last docutils">
162 <dt><tt class="docutils literal"><span class="pre">callable</span></tt></dt>
163 <dd>The callable used to create a shared object. If the
164 manager will connect to a remote manager then this is ignored.</dd>
165 <dt><tt class="docutils literal"><span class="pre">proxytype</span></tt></dt>
166 <dd><p class="first">The type of proxy which will be used for object returned
167 by <tt class="docutils literal"><span class="pre">callable</span></tt>.</p>
168 <p class="last">If <tt class="docutils literal"><span class="pre">proxytype</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> then each time an object is
169 returned by <tt class="docutils literal"><span class="pre">callable</span></tt> either a new proxy type is created
170 or a cached one is reused. The methods of the shared
171 object which will be exposed via the proxy will then be
172 determined by the <tt class="docutils literal"><span class="pre">exposed</span></tt> argument, see below.</p>
173 </dd>
174 <dt><tt class="docutils literal"><span class="pre">exposed</span></tt></dt>
175 <dd><p class="first">Given a shared object returned by <tt class="docutils literal"><span class="pre">callable</span></tt>, the
176 <tt class="docutils literal"><span class="pre">exposed</span></tt> argument is the list of those method names which
177 should be exposed via <a class="reference" href="proxy-objects.html#methods-of-baseproxy"><tt class="docutils literal"><span class="pre">BaseProxy._callMethod()</span></tt></a>. <a class="footnote-reference" href="#id3" id="id1" name="id1">[1]</a> <a class="footnote-reference" href="#id4" id="id2" name="id2">[2]</a></p>
178 <p>If <tt class="docutils literal"><span class="pre">exposed</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> and <tt class="docutils literal"><span class="pre">callable.__exposed__</span></tt> exists then
179 <tt class="docutils literal"><span class="pre">callable.__exposed__</span></tt> is used instead.</p>
180 <p>If <tt class="docutils literal"><span class="pre">exposed</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> and <tt class="docutils literal"><span class="pre">callable.__exposed__</span></tt> does not
181 exist then all methods of the shared object which do not
182 start with <tt class="docutils literal"><span class="pre">'_'</span></tt> will be exposed.</p>
183 <p class="last">An attempt to use <tt class="docutils literal"><span class="pre">BaseProxy._callMethod()</span></tt> with a method name which is
184 not exposed will raise an exception.</p>
185 </dd>
186 <dt><tt class="docutils literal"><span class="pre">typeid</span></tt></dt>
187 <dd>If <tt class="docutils literal"><span class="pre">typeid</span></tt> is a string then it is used as an identifier
188 for the callable. Otherwise, <tt class="docutils literal"><span class="pre">typeid</span></tt> must be <tt class="docutils literal"><span class="pre">None</span></tt> and
189 a string prefixed by <tt class="docutils literal"><span class="pre">callable.__name__</span></tt> is used as the
190 identifier.</dd>
191 </dl>
192 </dd>
193 </dl>
194 </blockquote>
195 <table class="docutils footnote" frame="void" id="id3" rules="none">
196 <colgroup><col class="label" /><col /></colgroup>
197 <tbody valign="top">
198 <tr><td class="label"><a class="fn-backref" href="#id1" name="id3">[1]</a></td><td>A method here means any attribute which has a <tt class="docutils literal"><span class="pre">__call__</span></tt>
199 attribute.</td></tr>
200 </tbody>
201 </table>
202 <table class="docutils footnote" frame="void" id="id4" rules="none">
203 <colgroup><col class="label" /><col /></colgroup>
204 <tbody valign="top">
205 <tr><td class="label"><a class="fn-backref" href="#id2" name="id4">[2]</a></td><td><p class="first">The method names <tt class="docutils literal"><span class="pre">__repr__</span></tt>, <tt class="docutils literal"><span class="pre">__str__</span></tt>, and <tt class="docutils literal"><span class="pre">__cmp__</span></tt> of a
206 shared object are always exposed by the manager. However, instead
207 of invoking the <tt class="docutils literal"><span class="pre">__repr__()</span></tt>, <tt class="docutils literal"><span class="pre">__str__()</span></tt>, <tt class="docutils literal"><span class="pre">__cmp__()</span></tt> instance
208 methods (none of which are guaranteed to exist) they invoke the
209 builtin functions <tt class="docutils literal"><span class="pre">repr()</span></tt>, <tt class="docutils literal"><span class="pre">str()</span></tt> and <tt class="docutils literal"><span class="pre">cmp()</span></tt>.</p>
210 <p class="last">Note that one should generally avoid exposing rich comparison
211 methods like <tt class="docutils literal"><span class="pre">__eq__()</span></tt>, <tt class="docutils literal"><span class="pre">__ne__()</span></tt>, <tt class="docutils literal"><span class="pre">__le__()</span></tt>. To make the proxy
212 type support comparison by value one can just expose <tt class="docutils literal"><span class="pre">__cmp__()</span></tt>
213 instead (even if the referent does not have such a method).</p>
214 </td></tr>
215 </tbody>
216 </table>
217 <div class="section">
218 <h2><a id="example" name="example">Example</a></h2>
219 <pre class="literal-block">
220 from processing.managers import BaseManager, CreatorMethod
221
222 class FooClass(object):
223 def bar(self):
224 print 'BAR'
225 def baz(self):
226 print 'BAZ'
227
228 class NewManager(BaseManager):
229 Foo = CreatorMethod(FooClass)
230
231 if __name__ == '__main__':
232 manager = NewManager()
233 manager.start()
234 foo = manager.Foo()
235 foo.bar() # prints 'BAR'
236 foo.baz() # prints 'BAZ'
237 manager.shutdown()
238 </pre>
239 <p>See <a class="reference" href="../examples/ex_newtype.py">ex_newtype.py</a> for more examples.</p>
240 </div>
241 </div>
242 <div class="section">
243 <h1><a id="using-a-remote-manager" name="using-a-remote-manager">Using a remote manager</a></h1>
244 <p>It is possible to run a manager server on one machine and have clients
245 use it from other machines (assuming that the firewalls involved allow
246 it).</p>
247 <p>Running the following commands creates a server for a shared queue which
248 remote clients can use:</p>
249 <pre class="literal-block">
250 &gt;&gt;&gt; from processing.managers import BaseManager, CreatorMethod
251 &gt;&gt;&gt; import Queue
252 &gt;&gt;&gt; queue = Queue.Queue()
253 &gt;&gt;&gt; class QueueManager(BaseManager):
254 ... get_proxy = CreatorMethod(callable=lambda:queue, typeid='get_proxy')
255 ...
256 &gt;&gt;&gt; m = QueueManager(address=('foo.bar.org', 50000), authkey='none')
257 &gt;&gt;&gt; m.serveForever()
258 </pre>
259 <p>One client can access the server as follows:</p>
260 <pre class="literal-block">
261 &gt;&gt;&gt; from processing.managers import BaseManager, CreatorMethod
262 &gt;&gt;&gt; class QueueManager(BaseManager):
263 ... get_proxy = CreatorMethod(typeid='get_proxy')
264 ...
265 &gt;&gt;&gt; m = QueueManager.fromAddress(address=('foo.bar.org', 50000), authkey='none')
266 &gt;&gt;&gt; queue = m.get_proxy()
267 &gt;&gt;&gt; queue.put('hello')
268 </pre>
269 <p>Another client can also use it:</p>
270 <pre class="literal-block">
271 &gt;&gt;&gt; from processing.managers import BaseManager, CreatorMethod
272 &gt;&gt;&gt; class QueueManager(BaseManager):
273 ... get_proxy = CreatorMethod(typeid='get_proxy')
274 ...
275 &gt;&gt;&gt; m = QueueManager.fromAddress(address=('foo.bar.org', 50000), authkey='none')
276 &gt;&gt;&gt; queue = m.get_proxy()
277 &gt;&gt;&gt; queue.get()
278 'hello'
279 </pre>
280 </div>
281 </div>
282 <div class="footer">
283 <hr class="footer" />
284 <a class="reference" href="connection-objects.html">Prev</a> &nbsp; &nbsp; &nbsp; &nbsp; <a class="reference" href="processing-ref.html">Up</a> &nbsp; &nbsp; &nbsp; &nbsp; <a class="reference" href="proxy-objects.html">Next</a>
285 </div>
286 </body>
287 </html>