Debian Codegraph @

TODO

raw
1 ***** Future Features / Interface Changes *****
2
3 optimizations:
4 - support alternate, faster locking mechanisms (filesystem and/or SysV shmem)
5 - allow for shared memory as transport medium
6 - current alternative is to use BerkeleyDB, e.g. forks::BerkeleyDB
7 - implement more methods using custom filters (to reduce Storable usage)
8
9 Make thread objects shared in nature, such that their properties may be read
10 across all threads. Will make _handle more reliable, and allow for thread
11 state params to be persisted.
12
13 Add a method to give the user run-time privilege control over Unix socket
14 file descriptors (for additional security control).
15
16 Possibly implement TTY wrapper control for Perl debugger, such that user has simpler
17 API functions to call to bind a thread to a TTY. Idea is to integrate debugger
18 TTY control functionality as thread create arguments, to allow for unique tty
19 subroutines to be defined per thread, or unique TTY values to be "bound" to specific
20 threads.
21
22 Consider interfacing with graphical Perl debugger intefaces, to simplify
23 multi-threaded debugging; managing multiple TTYs, although supported in different
24 terminals (1 per thread), can be conceptually difficult. Considering interface
25 to Devel::PDB. Consider support for Eclipse. Open to suggestions, comments, patches.
26
27
28 ***** Issues to Address *****
29
30 Need to load internally cached, shared REF values from server, not from first use
31 to insure that values are correctly loaded, to insure they correctly evaluate when
32 used in is_shared(). (TODO temp patch seen in t/forks09.t)
33
34 Circular references and ref(), Scalar::Util::reftype(), attribute::reftype() don't
35 work well with perltie shared variables. Likely a limitation of how we're using
36 variables in a remote process. May be able to improve on this by overloading
37 all these functions with a forks::shared aware function that contacts remote process
38 when variable is shared to determine what has circular recursion and what doesn't.
39
40 END { } blocks should be skipped unless they were defined in a thread: eval "END {...}"
41 or module use/require). One idea is to somehow sneak POSIX::_exit(), as long as we
42 are careful to flush buffered sockets before calling it. NOTE: this behavior likely
43 should ONLY occur when in native ithreads mode; I consider END {...} block execution
44 a good thing for all threads that were aware of it otherwise. (Note: better programming
45 practice would be to use DESTROY method whenever possible, but END is important
46 for some modules.
47
48 Environment variables should be shared among all threads (ithreads behavior).
49
50 Current working directory should be shared among all threads (ithreads behavior). Note
51 that this may be considered bug in current ithreads implementation.
52
53 Should implement to-the-letter behavior: when parent thread calls ->kill and target
54 doesn't have a signal handler registered, caller should croak. Current behavior is
55 target emits a warning, exits, and kill caller quietly continues.
56
57 Memory is currently not reclaimed for shared variables that are no longer
58 referenced by any thread. Possible solutions: implement PL_destroyhook; or use
59 STORE for threads::shared objects, and DESTROY for threads, to track shared var _refcnt.
60 DESTROY will be called on shared objects in an exiting thread, even if still
61 referenced in other threads. If can track global _refcnt, could determine refcnt
62 state from server and destroy only when 0.
63
64 Determine if it is possible to recover all memory from detached threads. Joined
65 threads reclaim all shared memory (very stable memory usage); detached threads
66 currently use a small amount of memory (~200Kb/100K threads) even after they
67 complete.
68
69 exit() in a thread may allow other threads that were waiting to join that thread
70 to continue before they are terminated (race condition). Join should not return.
71
72 Add more defensive logic in _join() to prevent hang on join (for processes
73 that have already terminated). May wish to check client socket when join for
74 that thread is requested: if socket exception, then assume client terminated;
75 otherwise, wait for client (regularly checking client socket connection to
76 determine if it has prematurely closed).
77
78 Perl 5.6 doesn't seem to respond to thread global exit in this case:
79 use forks; threads->new(sub { exit($desired_exit_val);} )->join(); sleep 10; sleep 10;
80
81 ***** Possible Issues to Address *****
82
83 In rare cases, if the main thread dies, the server process doesn't appear to
84 shut down correctly. Sometimes, it appears to use high CPU usage (is stuck in a loop
85 somewhere). It is likely that a socket disconnect is getting a read or write
86 while loop stuck in a loop, and that additional error checking (or non-error
87 occurrence but no change to the amount of data read/written) might help the
88 state of things. Reproducing this one is pretty hard--I can't create the
89 exact conditions yet to manually trigger it. Problem last seen in forks 0.20.
90 Note that this issue may have been squashed in forks 0.21, as most of the socket
91 handling was rewritten in that release.
92
93 Need to look into why some signals very rarely are not handled by threads
94 (i.e. SIGTERM should cause thread to exit--but thread appears to have ignored
95 signal. Is this a side effect of Perl's safe signal handling mechanism?).
96 The thread does appear to actually get the signal, but sometimes it doesn't
97 seem to completely exit. Problem last seen in forks 0.23, but not seen since
98 forks 0.25, so may be resolved.
99
100
101 ***** Miscellaneous Items *****
102
103 Consider implementing virtual signals that map to one signal handler, thus allowing
104 use of special signals like 'STOP', 'KILL', etc. for improved portability with
105 scripts that might use these signal names already. e.g. ABRT might be used across
106 all threads and a virtual signal mask check could be tracked by the shared server.
107
108 Update documentation to discuss fault-tolerance (individual thread can't take down the thread group, even with memory faults like segfault). Very helpful for high-availability systems.
109
110 Consider checking _check_pl_signal_unsafe_flag for any thread signaling behavior,
111 and warn user about (or prevent from using) signals (i.e. deadlock detect, $t->kill).
112
113 Allow manual disabling of forks.pm signal management (for embedding with other
114 pragmas or interfaces that implement their own signals management, such as mod_perl
115 via Apache::forks).
116
117 Need a way to keep deserialized shared variable references consistent between
118 lookups, e.g.
119 my $shared :shared = &share({});
120 $$shared{'foo'} = 'bar';
121 my $str1 = "$shared";
122 my $str2 = "$shared";
123 ok($str1 eq $str2, 'stringify');
124 This is a side-effect of Storable, which reconstitutes a new tied hash each time
125 FETCH returns a shared variable.
126 This might be solved with overload.pm, although documentation indicates that it
127 may not work with Storable since we're masking the master package in a different
128 file (Storable uses XS load_module, which tries to load real threads/shared.pm even
129 though %INC is populated). (I wonder if it's possible to fake out load_module?)
130 Thus, maybe we can implement a thread-local reference cache for all shared
131 variables (dynamically populated as they are FETCHed), which returns the
132 thread-local copy instead of the server copy. (Same idea as forks::BerkeleyDB,
133 but caching the server copy instead.) Benefit to this method is that ref()
134 and refaddr() will return the same results, although won't return the same value
135 as is_shared() unless we change this to use the same logic.
136 UPDATE: we need to get away from the caching mechanism, or somehow have logic
137 that resolves REF types (before or in) the shared server such that it returns
138 a shared variable to the code. The code itself should be able to know the following:
139 1. when a REF is a shared variable
140 2. when a REF has changed in the shared process, we need to make sure all child threads
141 that have a reference to it are correctly updated.
142
143 Enable custom concurrent lock interface to support optimized locking mechanisms;
144 specifically, SysV semaphores, filesystem (+alarm/ALRM), or other possibilities.
145
146 Add additional tests to rigorously test tied handle (for shared var) behavior.
147 Test delete with single, hash slice, and array slice elements.
148
149 Cache signal names from Config.pm for higher performance (avoid tied access)?
150
151 Begin adding some sort of CLONE forks.pm compatibility framework to insure that
152 modules that implement non-forks.pm CLONE methods are supported. Currently,
153 DBI is the only known module that may need a CLONE patch, and this is only to
154 suppress annoying warning for resources that are closed (or is this something
155 that actually needs to be fixed in DBI?)
156
157 Signals sent to main thread can propagate to all threads. We need to
158 prevent this from happening, if possible, to replicate threads.pm behavior.
159 This may be due to signal being sent to a process group (so it may not be
160 blockable).
161
162 Add recursive lock tests with cond_* (to test that recursive locks are
163 correctly acquired and released).
164
165 Add tests to validate that storing CODE refs in shared variables works.
166
167 Add documentation explaining that forks installs a SIGCHLD handler by default,
168 and that system calls in your user code may be interrupted by this handler
169 at any time (threads exiting), depending on the system call and if your kernel
170 auto-continues the system call after handling the signal.
171
172 NOTE: There still appears to be an issue with the last value of $! not being
173 cleared and EINTR still being returned on the next accept() vall. Setting $!
174 to undef before calling the system call appears to clear the issue. This is
175 system behavior outside of forks module that needs more research to resolve.
176
177 A few suggestions if this is an issue:
178
179 1. Check if $! == POSIX::EINTR() if your system call returns undef. If so,
180 manually re-enter into (call again) the system call, like:
181
182 while((my $conn = $server->accept) || $! == EINTR) {
183 next if $! == EINTR;
184 ...
185 }
186
187 This is the most portable solution.
188
189 2. Localize $SIG{CHLD} = 'IGNORE' when performing your interruptible
190 (non-reentrant) system call. For example:
191
192 my $conn;
193 my $loop = 1;
194 while($loop) {
195 {
196 local $SIG{CHLD} = 'IGNORE';
197 $conn = $server->accept;
198 next if $! == EINTR;
199 last unless $conn;
200 }
201 ...
202 }
203
204 This is a more verbose but acceptable solution.
205
206
207 3. Set $SIG{CHLD} = 'IGNORE' after loading the forks module.
208
209 This is the simplest solution as it applies globally to all system calls, but be
210 warned that this may cause the exit value of your script to be ignored
211 (returns -1) depending on your kernel behavior. You can always check your kernel
212 behavior by executing the following at a shell command line:
213
214 perl -e '$SIG{CHLD} = "IGNORE"; exit(42);'; echo "Expected 42. Got $?"