| 1 |
<?xml version="1.0" encoding="utf-8" ?> |
| 2 |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| 3 |
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| 4 |
<head> |
| 5 |
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> |
| 6 |
<meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" /> |
| 7 |
<title>Queue objects</title> |
| 8 |
<link rel="stylesheet" href="html4css1.css" type="text/css" /> |
| 9 |
</head> |
| 10 |
<body> |
| 11 |
<div class="header"> |
| 12 |
<a class="reference" href="process-objects.html">Prev</a> <a class="reference" href="processing-ref.html">Up</a> <a class="reference" href="connection-objects.html">Next</a> |
| 13 |
<hr class="header"/> |
| 14 |
</div> |
| 15 |
<div class="document" id="queue-objects"> |
| 16 |
<h1 class="title">Queue objects</h1> |
| 17 |
<p>The queue type provided by <tt class="docutils literal"><span class="pre">processing</span></tt> is a multi-producer, |
| 18 |
multi-consumer FIFO queue modelled on the <tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> class in the |
| 19 |
standard library.</p> |
| 20 |
<dl class="docutils"> |
| 21 |
<dt><tt class="docutils literal"><span class="pre">Queue(maxsize=0)</span></tt></dt> |
| 22 |
<dd><p class="first">Returns a process shared queue implemented using a pipe and a few |
| 23 |
locks/semaphores. When a process first puts an item on the queue |
| 24 |
a feeder thread is started which transfers objects from a |
| 25 |
buffer into the pipe.</p> |
| 26 |
<p><tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> implements all the methods of <tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> except for |
| 27 |
<tt class="docutils literal"><span class="pre">qsize()</span></tt>, <tt class="docutils literal"><span class="pre">task_done()</span></tt> and <tt class="docutils literal"><span class="pre">join()</span></tt>.</p> |
| 28 |
<blockquote> |
| 29 |
<dl class="docutils"> |
| 30 |
<dt><tt class="docutils literal"><span class="pre">empty()</span></tt></dt> |
| 31 |
<dd>Return <tt class="docutils literal"><span class="pre">True</span></tt> if the queue is empty, <tt class="docutils literal"><span class="pre">False</span></tt> |
| 32 |
otherwise. Because of multithreading/multiprocessing |
| 33 |
semantics, this is not reliable.</dd> |
| 34 |
<dt><tt class="docutils literal"><span class="pre">full()</span></tt></dt> |
| 35 |
<dd>Return <tt class="docutils literal"><span class="pre">True</span></tt> if the queue is full, <tt class="docutils literal"><span class="pre">False</span></tt> otherwise. Because |
| 36 |
of multithreading/multiprocessing semantics, this is not |
| 37 |
reliable.</dd> |
| 38 |
<dt><tt class="docutils literal"><span class="pre">put(item,</span> <span class="pre">block=True,</span> <span class="pre">timeout=None)</span></tt></dt> |
| 39 |
<dd>Put item into the queue. If optional args <tt class="docutils literal"><span class="pre">block</span></tt> is true |
| 40 |
and <tt class="docutils literal"><span class="pre">timeout</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> (the default), block if necessary |
| 41 |
until a free slot is available. If <tt class="docutils literal"><span class="pre">timeout</span></tt> is a positive |
| 42 |
number, it blocks at most <tt class="docutils literal"><span class="pre">timeout</span></tt> seconds and raises the |
| 43 |
<tt class="docutils literal"><span class="pre">Full</span></tt> exception if no free slot was available within that |
| 44 |
time. Otherwise (<tt class="docutils literal"><span class="pre">block</span></tt> is false), put an item on the queue |
| 45 |
if a free slot is immediately available, else raise the |
| 46 |
<tt class="docutils literal"><span class="pre">Full</span></tt> exception (<tt class="docutils literal"><span class="pre">timeout</span></tt> is ignored in that case).</dd> |
| 47 |
<dt><tt class="docutils literal"><span class="pre">put_nowait(item)</span></tt>, <tt class="docutils literal"><span class="pre">putNoWait(item)</span></tt></dt> |
| 48 |
<dd>Equivalent to <tt class="docutils literal"><span class="pre">put(item,</span> <span class="pre">False)</span></tt>.</dd> |
| 49 |
<dt><tt class="docutils literal"><span class="pre">get(block=True,</span> <span class="pre">timeout=None)</span></tt></dt> |
| 50 |
<dd>Remove and return an item from the queue. If optional args |
| 51 |
<tt class="docutils literal"><span class="pre">block</span></tt> is true and <tt class="docutils literal"><span class="pre">timeout</span></tt> is <tt class="docutils literal"><span class="pre">None</span></tt> (the default), |
| 52 |
block if necessary until an item is available. If |
| 53 |
<tt class="docutils literal"><span class="pre">timeout</span></tt> is a positive number, it blocks at most |
| 54 |
<tt class="docutils literal"><span class="pre">timeout</span></tt> seconds and raises the <tt class="docutils literal"><span class="pre">Empty</span></tt> exception if no |
| 55 |
item was available within that time. Otherwise (block is |
| 56 |
false), return an item if one is immediately available, |
| 57 |
else raise the <tt class="docutils literal"><span class="pre">Empty</span></tt> exception (<tt class="docutils literal"><span class="pre">timeout</span></tt> is ignored in |
| 58 |
that case).</dd> |
| 59 |
<dt><tt class="docutils literal"><span class="pre">get_nowait()</span></tt>, <tt class="docutils literal"><span class="pre">getNoWait()</span></tt></dt> |
| 60 |
<dd>Equivalent to <tt class="docutils literal"><span class="pre">get(False)</span></tt>.</dd> |
| 61 |
</dl> |
| 62 |
</blockquote> |
| 63 |
<p><tt class="docutils literal"><span class="pre">processing.Queue</span></tt> has a few additional methods not found in |
| 64 |
<tt class="docutils literal"><span class="pre">Queue.Queue</span></tt> which are usually unnecessary:</p> |
| 65 |
<blockquote class="last"> |
| 66 |
<dl class="docutils"> |
| 67 |
<dt><tt class="docutils literal"><span class="pre">putMany(iterable)</span></tt></dt> |
| 68 |
<dd>If the queue has infinite size then this adds all |
| 69 |
items in the iterable to the queue's buffer. So |
| 70 |
<tt class="docutils literal"><span class="pre">q.putMany(X)</span></tt> is a faster alternative to <tt class="docutils literal"><span class="pre">for</span> <span class="pre">x</span> <span class="pre">in</span> <span class="pre">X:</span> |
| 71 |
<span class="pre">q.put(x)</span></tt>. Raises an error if the queue has finite |
| 72 |
size.</dd> |
| 73 |
<dt><tt class="docutils literal"><span class="pre">close()</span></tt></dt> |
| 74 |
<dd>Indicates that no more data will be put on this queue by |
| 75 |
the current process. The background thread will quit once |
| 76 |
it has flushed all buffered data to the pipe. This is |
| 77 |
called automatically when the queue is garbage collected.</dd> |
| 78 |
<dt><tt class="docutils literal"><span class="pre">joinThread()</span></tt></dt> |
| 79 |
<dd><p class="first">This joins the background thread and can only be used |
| 80 |
after <tt class="docutils literal"><span class="pre">close()</span></tt> has been called. This blocks until |
| 81 |
the background thread exits, ensuring that all data in |
| 82 |
the buffer has been flushed to the pipe.</p> |
| 83 |
<p class="last">By default if a process is not the creator of the |
| 84 |
queue then on exit it will attempt to join the queue's |
| 85 |
background thread. The process can call |
| 86 |
<tt class="docutils literal"><span class="pre">cancelJoin()</span></tt> to prevent this behaviour.</p> |
| 87 |
</dd> |
| 88 |
<dt><tt class="docutils literal"><span class="pre">cancelJoin()</span></tt></dt> |
| 89 |
<dd>Prevents the background thread from being joined |
| 90 |
automatically when the process exits. Unnecessary if |
| 91 |
the current process created the queue.</dd> |
| 92 |
</dl> |
| 93 |
</blockquote> |
| 94 |
</dd> |
| 95 |
</dl> |
| 96 |
<div class="admonition-empty-and-full admonition"> |
| 97 |
<p class="first admonition-title"><tt class="docutils literal"><span class="pre">Empty</span></tt> and <tt class="docutils literal"><span class="pre">Full</span></tt></p> |
| 98 |
<p class="last"><tt class="docutils literal"><span class="pre">processing</span></tt> uses the usual <tt class="docutils literal"><span class="pre">Queue.Empty</span></tt> and <tt class="docutils literal"><span class="pre">Queue.Full</span></tt> |
| 99 |
exceptions to signal a timeout. They are not available in the |
| 100 |
<tt class="docutils literal"><span class="pre">processing</span></tt> namespace so you need to import them from <tt class="docutils literal"><span class="pre">Queue</span></tt>.</p> |
| 101 |
</div> |
| 102 |
<div class="warning"> |
| 103 |
<p class="first admonition-title">Warning</p> |
| 104 |
<p class="last">If a process is killed using the <tt class="docutils literal"><span class="pre">terminate()</span></tt> method or |
| 105 |
<tt class="docutils literal"><span class="pre">os.kill()</span></tt> while it is trying to use a <tt class="docutils literal"><span class="pre">Queue</span></tt> then the data in |
| 106 |
the queue is likely to become corrupted. This may cause any other |
| 107 |
processes to get an exception when it tries to use the queue later |
| 108 |
on.</p> |
| 109 |
</div> |
| 110 |
<div class="warning"> |
| 111 |
<p class="first admonition-title">Warning</p> |
| 112 |
<p>As mentioned above, if a child process has put items on a queue |
| 113 |
(and it has not used <tt class="docutils literal"><span class="pre">cancelJoin()</span></tt>) then that process will not |
| 114 |
terminate until all buffered items have been flushed to the pipe.</p> |
| 115 |
<p>This means that if you try joining that process you may get a |
| 116 |
deadlock unless you are sure that all items which have been put on |
| 117 |
the queue have been consumed. Similarly, if the child process is |
| 118 |
non-daemonic then the parent process may hang on exit when it tries |
| 119 |
to join all it non-daemonic children.</p> |
| 120 |
<p class="last">Note that a queue created using a manager does not have this issue. |
| 121 |
See <a class="reference" href="programming-guidelines.html">Programming Guidelines</a>.</p> |
| 122 |
</div> |
| 123 |
</div> |
| 124 |
<div class="footer"> |
| 125 |
<hr class="footer" /> |
| 126 |
<a class="reference" href="process-objects.html">Prev</a> <a class="reference" href="processing-ref.html">Up</a> <a class="reference" href="connection-objects.html">Next</a> |
| 127 |
</div> |
| 128 |
</body> |
| 129 |
</html> |