Debian Codegraph @

README.THREADS

raw
1 Summary:
2
3 1. Don't use threads.
4 2. If you still want to use threads, only communicate with the UI
5 thread by injecting events into its main loop.
6 3. Handle slots you receive from the main thread via safe_slot.
7 4. Watch out for what happens when the cache is closed underneath
8 you; use the appropriate signals to avoid disaster.
9 5. Use job_background_queue to implement your background thread if
10 at all possible.
11
12 The basic threading philosophy followed in aptitude can be
13 summarized thus: "if you aren't sure it's safe, do it in the main
14 thread; if it's going to take a long time, run it in the background
15 and only communicate with the main thread via event injection".
16 aptitude does *not* perform any concurrent computation in the normal
17 sense; it uses threads only to keep the UI responsive while a
18 background computation or I/O operation is going on.
19
20 The general pattern is this:
21
22 +-- UI thread -------+-------------------------+--- handle event ---
23 \ / post_event
24 +---- background job ------------------------+----------------------
25
26 The UI thread is assumed to be running an event-handling loop where
27 it waits for a new event to appear; when an event appears, it invokes
28 a callback contained in the event. Even the command-line UI can do
29 this, although when it needs to interact with a background thread, it
30 usually just uses a "dummy" main loop.
31
32 Note that the background thread continues running. In general,
33 background threads have their own input queue (containing "work to be
34 done") and are automatically started and stopped as work appears or
35 disappears. The class job_queue_thread provides this behavior and
36 should be used for new code if possible. This architecture should
37 eventually evolve into something like Java's system of thread pools
38 and executors, but it's not there yet.
39
40 Background threads are given a safe mechanism for passing slot
41 objects to the main thread, in the form of a callback function.
42
43 void post_thunk(const sigc::slot<void> &);
44
45 You can assume that the slot object will be wrapped in a safe_slot.
46 It's probably not necesary, but I've had enough bad experiences with
47 slots and threads (see below) that I prefer handling them at arm's
48 length.
49
50 When the main thread has time, the slot will be invoked, then
51 destroyed. Normally, the slot will in turn invoke some callback slot
52 provided by the main thread.
53
54 Using this idiom allows the main thread code to be insulated from
55 the details of the threading model. As far as it's concerned, it
56 invokes a frontend routine, passing a callback, and some time later
57 receives an event indicating that the background process completed.
58
59 *** THIS IS FAR SUPERIOR TO THE THREADING MODELS APTITUDE WAS
60 PREVIOUSLY USING AND SHOULD BE USED FOR ALL NEW CODE UNLESS YOU HAVE A
61 VERY, VERY, VERY GOOD REASON ***. You can probably find some vestiges
62 of the old threading model in various places (especially the
63 resolver). Don't imitate them.
64
65
66
67 Even with this mechanism, you have to watch out for a few gotchas.
68 A particularly nasty one involves object lifetimes. If you want to
69 invoke a method on an object that was created in a background thread,
70 but you want to invoke it in the main thread, it's tempting to do
71 this:
72
73 post_thunk(sigc::mem_fun(*obj, &obj_type::method));
74
75 The problem is that unless obj is a bare pointer and obj_type::method
76 ends with "delete this", one of two things will happen:
77
78 1) No-one is managing obj's lifetime, and it will leak.
79
80 2) The background thread holds strong references to obj, and the
81 thunk might try to invoke a method on a destroyed obj (or,
82 worse, encounter sigc++ type-safety issues when the implicit
83 weak reference is being invalidated).
84
85 The safest way to pass objects between threads in this way is to
86 manage the object's lifetime with shared_ptr, and use a *keepalive
87 slot*:
88
89 post_thunk(keepalive(sigc::mem_fun(*obj, &obj_type::method), obj));
90
91 This will take a temporary shared reference to obj, ensuring that it
92 is not deleted until the thunk fires. Note that this idiom cannot
93 create cycles, since the only new reference to obj is stored in the
94 posted thunk, which will eventually be destroyed by the main loop.
95
96 An alternative is to use bare pointers and manage the object
97 lifetime by hand. This is error-prone and I recommend against it.
98
99
100
101 Very little global or shared state should be accessed by background
102 threads. The one large exception is the currently loaded apt cache;
103 any background thread that needs to perform computations on packages
104 will obviously need access to the cache (even if it was not a global
105 variable, it would still be widely shared).
106
107 The apt module provides sigc signals that are invoked in the main
108 thread prior to destroying the cache object and after a new one is
109 created. If you create a background thread that accesses the cache,
110 you should hook into these signals, stopping your thread on the first
111 one and resuming it on the second one. The job_queue_thread class
112 provides methods that you can hook into to achieve this effect.
113
114
115
116 The actual threading constructs used are the pthread wrappers in
117 cwidget/generic/threads.h (and also cwidget/generic/event_queue.h).
118
119
120 WARNINGS ABOUT THINGS THAT ARE DANGEROUS:
121
122 Things that you might thank are threadsafe but aren't include:
123
124 * sigc++ objects. Not only do you have to watch out for manual
125 additions and deletions to connection lists during invocation, you
126 also have to watch out for automatic invalidation of slots at any
127 time. In particular, COPYING SIGC++ SLOTS IS NOT THREADSAFE. If
128 you write a background thread that accepts a slot from the main
129 thread (for instance, as a callback), you should wrap that slot in
130 a safe_slot to ensure that it is only copied in the main thread.
131 Read safe_slot's header for more details.
132
133 * Smart pointers other than boost::shared_ptr. Most smart pointers
134 that aptitude uses are NOT threadsafe. This means that *EVEN
135 READ-ONLY ACCESS* from another thread will cause horrible ghastly
136 problems that you don't even want to think about. At the moment
137 it's almost never necessary to pass these between threads, so it's
138 not a big deal; the exception is the problem resolver's solution
139 objects (and the shared trees contained inside them), which are
140 dealt with by doing a deep copy of the object. (see
141 resolver_manager::do_get_solution)
142
143 The reason this is the case is basically that the pthread
144 abstraction doesn't give you a fast lock for low-contention
145 situations -- adding locking around the reference counts of set
146 tree nodes made the overall resolver take 50% longer to run in
147 single-threaded mode! I'm not eager to add nonportable threading
148 constructs, so I decided to see whether it was possible to just be
149 very careful about handling reference-counted objects.
150
151 Some existing background threads:
152
153 * The cwidget library creates threads to handle keyboard input,
154 certain asynchronous signals, and timed events. You generally
155 don't need to worry about these.
156
157 * Downloads are performed by a background thread. This predates
158 job_queue_thread and doesn't use it. Only the actual download is
159 handled in the background thread -- the setup of the download and
160 any actions taken once it completes are handled by the main
161 thread. The gatekeeper for downloads is in download_thread.cc; it
162 provides the basic thread harness, as well as a convenience class
163 that forwards the various download messages to a foreground
164 progress meter. (these are basically inter-thread RPC calls, and
165 they block the download thread until the progress meter's method
166 call returns a value)
167
168 * The problem resolver runs in a background thread. This thread
169 always exists, even when there is no resolver (in which case it
170 will just sleep); the foreground thread can post jobs to it, and
171 will also stop the resolver whenever its state needs to be
172 modified (for instance, if the rejected set is changed). The
173 interface for this is in src/generic/resolver_manager.cc.
174
175 * From the GTK+ interface, changelog parsing and checking for
176 changelogs in the download cache both happen in a background
177 thread, using job_queue_thread.