| 1 |
Summary: |
| 2 |
|
| 3 |
1. Don't use threads. |
| 4 |
2. If you still want to use threads, only communicate with the UI |
| 5 |
thread by injecting events into its main loop. |
| 6 |
3. Handle slots you receive from the main thread via safe_slot. |
| 7 |
4. Watch out for what happens when the cache is closed underneath |
| 8 |
you; use the appropriate signals to avoid disaster. |
| 9 |
5. Use job_background_queue to implement your background thread if |
| 10 |
at all possible. |
| 11 |
|
| 12 |
The basic threading philosophy followed in aptitude can be |
| 13 |
summarized thus: "if you aren't sure it's safe, do it in the main |
| 14 |
thread; if it's going to take a long time, run it in the background |
| 15 |
and only communicate with the main thread via event injection". |
| 16 |
aptitude does *not* perform any concurrent computation in the normal |
| 17 |
sense; it uses threads only to keep the UI responsive while a |
| 18 |
background computation or I/O operation is going on. |
| 19 |
|
| 20 |
The general pattern is this: |
| 21 |
|
| 22 |
+-- UI thread -------+-------------------------+--- handle event --- |
| 23 |
\ / post_event |
| 24 |
+---- background job ------------------------+---------------------- |
| 25 |
|
| 26 |
The UI thread is assumed to be running an event-handling loop where |
| 27 |
it waits for a new event to appear; when an event appears, it invokes |
| 28 |
a callback contained in the event. Even the command-line UI can do |
| 29 |
this, although when it needs to interact with a background thread, it |
| 30 |
usually just uses a "dummy" main loop. |
| 31 |
|
| 32 |
Note that the background thread continues running. In general, |
| 33 |
background threads have their own input queue (containing "work to be |
| 34 |
done") and are automatically started and stopped as work appears or |
| 35 |
disappears. The class job_queue_thread provides this behavior and |
| 36 |
should be used for new code if possible. This architecture should |
| 37 |
eventually evolve into something like Java's system of thread pools |
| 38 |
and executors, but it's not there yet. |
| 39 |
|
| 40 |
Background threads are given a safe mechanism for passing slot |
| 41 |
objects to the main thread, in the form of a callback function. |
| 42 |
|
| 43 |
void post_thunk(const sigc::slot<void> &); |
| 44 |
|
| 45 |
You can assume that the slot object will be wrapped in a safe_slot. |
| 46 |
It's probably not necesary, but I've had enough bad experiences with |
| 47 |
slots and threads (see below) that I prefer handling them at arm's |
| 48 |
length. |
| 49 |
|
| 50 |
When the main thread has time, the slot will be invoked, then |
| 51 |
destroyed. Normally, the slot will in turn invoke some callback slot |
| 52 |
provided by the main thread. |
| 53 |
|
| 54 |
Using this idiom allows the main thread code to be insulated from |
| 55 |
the details of the threading model. As far as it's concerned, it |
| 56 |
invokes a frontend routine, passing a callback, and some time later |
| 57 |
receives an event indicating that the background process completed. |
| 58 |
|
| 59 |
*** THIS IS FAR SUPERIOR TO THE THREADING MODELS APTITUDE WAS |
| 60 |
PREVIOUSLY USING AND SHOULD BE USED FOR ALL NEW CODE UNLESS YOU HAVE A |
| 61 |
VERY, VERY, VERY GOOD REASON ***. You can probably find some vestiges |
| 62 |
of the old threading model in various places (especially the |
| 63 |
resolver). Don't imitate them. |
| 64 |
|
| 65 |
|
| 66 |
|
| 67 |
Even with this mechanism, you have to watch out for a few gotchas. |
| 68 |
A particularly nasty one involves object lifetimes. If you want to |
| 69 |
invoke a method on an object that was created in a background thread, |
| 70 |
but you want to invoke it in the main thread, it's tempting to do |
| 71 |
this: |
| 72 |
|
| 73 |
post_thunk(sigc::mem_fun(*obj, &obj_type::method)); |
| 74 |
|
| 75 |
The problem is that unless obj is a bare pointer and obj_type::method |
| 76 |
ends with "delete this", one of two things will happen: |
| 77 |
|
| 78 |
1) No-one is managing obj's lifetime, and it will leak. |
| 79 |
|
| 80 |
2) The background thread holds strong references to obj, and the |
| 81 |
thunk might try to invoke a method on a destroyed obj (or, |
| 82 |
worse, encounter sigc++ type-safety issues when the implicit |
| 83 |
weak reference is being invalidated). |
| 84 |
|
| 85 |
The safest way to pass objects between threads in this way is to |
| 86 |
manage the object's lifetime with shared_ptr, and use a *keepalive |
| 87 |
slot*: |
| 88 |
|
| 89 |
post_thunk(keepalive(sigc::mem_fun(*obj, &obj_type::method), obj)); |
| 90 |
|
| 91 |
This will take a temporary shared reference to obj, ensuring that it |
| 92 |
is not deleted until the thunk fires. Note that this idiom cannot |
| 93 |
create cycles, since the only new reference to obj is stored in the |
| 94 |
posted thunk, which will eventually be destroyed by the main loop. |
| 95 |
|
| 96 |
An alternative is to use bare pointers and manage the object |
| 97 |
lifetime by hand. This is error-prone and I recommend against it. |
| 98 |
|
| 99 |
|
| 100 |
|
| 101 |
Very little global or shared state should be accessed by background |
| 102 |
threads. The one large exception is the currently loaded apt cache; |
| 103 |
any background thread that needs to perform computations on packages |
| 104 |
will obviously need access to the cache (even if it was not a global |
| 105 |
variable, it would still be widely shared). |
| 106 |
|
| 107 |
The apt module provides sigc signals that are invoked in the main |
| 108 |
thread prior to destroying the cache object and after a new one is |
| 109 |
created. If you create a background thread that accesses the cache, |
| 110 |
you should hook into these signals, stopping your thread on the first |
| 111 |
one and resuming it on the second one. The job_queue_thread class |
| 112 |
provides methods that you can hook into to achieve this effect. |
| 113 |
|
| 114 |
|
| 115 |
|
| 116 |
The actual threading constructs used are the pthread wrappers in |
| 117 |
cwidget/generic/threads.h (and also cwidget/generic/event_queue.h). |
| 118 |
|
| 119 |
|
| 120 |
WARNINGS ABOUT THINGS THAT ARE DANGEROUS: |
| 121 |
|
| 122 |
Things that you might thank are threadsafe but aren't include: |
| 123 |
|
| 124 |
* sigc++ objects. Not only do you have to watch out for manual |
| 125 |
additions and deletions to connection lists during invocation, you |
| 126 |
also have to watch out for automatic invalidation of slots at any |
| 127 |
time. In particular, COPYING SIGC++ SLOTS IS NOT THREADSAFE. If |
| 128 |
you write a background thread that accepts a slot from the main |
| 129 |
thread (for instance, as a callback), you should wrap that slot in |
| 130 |
a safe_slot to ensure that it is only copied in the main thread. |
| 131 |
Read safe_slot's header for more details. |
| 132 |
|
| 133 |
* Smart pointers other than boost::shared_ptr. Most smart pointers |
| 134 |
that aptitude uses are NOT threadsafe. This means that *EVEN |
| 135 |
READ-ONLY ACCESS* from another thread will cause horrible ghastly |
| 136 |
problems that you don't even want to think about. At the moment |
| 137 |
it's almost never necessary to pass these between threads, so it's |
| 138 |
not a big deal; the exception is the problem resolver's solution |
| 139 |
objects (and the shared trees contained inside them), which are |
| 140 |
dealt with by doing a deep copy of the object. (see |
| 141 |
resolver_manager::do_get_solution) |
| 142 |
|
| 143 |
The reason this is the case is basically that the pthread |
| 144 |
abstraction doesn't give you a fast lock for low-contention |
| 145 |
situations -- adding locking around the reference counts of set |
| 146 |
tree nodes made the overall resolver take 50% longer to run in |
| 147 |
single-threaded mode! I'm not eager to add nonportable threading |
| 148 |
constructs, so I decided to see whether it was possible to just be |
| 149 |
very careful about handling reference-counted objects. |
| 150 |
|
| 151 |
Some existing background threads: |
| 152 |
|
| 153 |
* The cwidget library creates threads to handle keyboard input, |
| 154 |
certain asynchronous signals, and timed events. You generally |
| 155 |
don't need to worry about these. |
| 156 |
|
| 157 |
* Downloads are performed by a background thread. This predates |
| 158 |
job_queue_thread and doesn't use it. Only the actual download is |
| 159 |
handled in the background thread -- the setup of the download and |
| 160 |
any actions taken once it completes are handled by the main |
| 161 |
thread. The gatekeeper for downloads is in download_thread.cc; it |
| 162 |
provides the basic thread harness, as well as a convenience class |
| 163 |
that forwards the various download messages to a foreground |
| 164 |
progress meter. (these are basically inter-thread RPC calls, and |
| 165 |
they block the download thread until the progress meter's method |
| 166 |
call returns a value) |
| 167 |
|
| 168 |
* The problem resolver runs in a background thread. This thread |
| 169 |
always exists, even when there is no resolver (in which case it |
| 170 |
will just sleep); the foreground thread can post jobs to it, and |
| 171 |
will also stop the resolver whenever its state needs to be |
| 172 |
modified (for instance, if the rejected set is changed). The |
| 173 |
interface for this is in src/generic/resolver_manager.cc. |
| 174 |
|
| 175 |
* From the GTK+ interface, changelog parsing and checking for |
| 176 |
changelogs in the download cache both happen in a background |
| 177 |
thread, using job_queue_thread. |