| 1 |
|
| 2 |
Revision history for Perl module MCE. |
| 3 |
|
| 4 |
1.902 Tue Sep 09 15:00:00 EST 2025 |
| 5 |
|
| 6 |
* Add support for Iterator:: classes. |
| 7 |
|
| 8 |
1.901 Thu Jan 02 17:30:00 EST 2025 |
| 9 |
|
| 10 |
* Add MCE::Core package for future development. |
| 11 |
|
| 12 |
1.900 Mon Sep 09 20:30:00 EST 2024 |
| 13 |
|
| 14 |
* Improve MCE::Child exiting when signaled. |
| 15 |
|
| 16 |
1.899 Fri Sep 06 00:15:00 EST 2024 |
| 17 |
|
| 18 |
* Fix for MCE::Child and MCE::Channel signal anomaly #24. |
| 19 |
Thank you, @exodist. |
| 20 |
|
| 21 |
1.898 Wed Aug 21 15:30:00 EST 2024 |
| 22 |
|
| 23 |
* Fix for MCE::Child #22, Can't call method "len" on an undefined value |
| 24 |
during global destruction. Thanks, @exodist. |
| 25 |
|
| 26 |
1.897 Wed Jun 19 22:00:00 EST 2024 |
| 27 |
|
| 28 |
* In scalar context, the init function in MCE Child and models Flow, Grep, |
| 29 |
Loop, Map, Step, and Stream returns a guard to call finish automatically |
| 30 |
upon leaving the { scope i.e. omitting finish }. |
| 31 |
|
| 32 |
* Add out_iter_callback to MCE::Candy. |
| 33 |
|
| 34 |
1.896 Tue Jun 11 16:00:00 EST 2024 |
| 35 |
|
| 36 |
* Weaken internal core MCE reference to reap workers automatically |
| 37 |
upon leaving the { scope i.e. omitting shutdown }. Note: No change |
| 38 |
to MCE models Flow, Grep, Loop, Map, Step, and Stream. Call finish |
| 39 |
explicitly to reap workers. |
| 40 |
|
| 41 |
This resolves the case when omitting calling either one of $mce->run(), |
| 42 |
$mce->run(1), or $mce->shutdown() inside a scope, causing workers to |
| 43 |
linger around until completion of the script. |
| 44 |
|
| 45 |
1.895 Sun Jun 09 01:00:00 EST 2024 |
| 46 |
|
| 47 |
* Revert back to calling CORE::rand() to set the internal seed. |
| 48 |
MCE and MCE::Child cannot assume the srand or setter function used |
| 49 |
by the application for predictability. |
| 50 |
1. https://perlmonks.org/?node_id=11159834 |
| 51 |
2. https://perlmonks.org/?node_id=11159827 |
| 52 |
|
| 53 |
* Add class methods MCE->seed and MCE::Child->seed to retrieve the seed. |
| 54 |
|
| 55 |
1.894 Sun Jun 09 15:30:00 EST 2024 |
| 56 |
|
| 57 |
* Improve support for PDL. |
| 58 |
|
| 59 |
1.893 Sat Jun 08 08:30:00 EST 2024 |
| 60 |
|
| 61 |
* Preserve functionality for older Perl, non-threads. |
| 62 |
|
| 63 |
1.892 Sat Jun 08 08:00:00 EST 2024 |
| 64 |
|
| 65 |
* Remove check if spinning threads i.e. use_threads. |
| 66 |
Predictable output matches non-threads for CORE, |
| 67 |
Math::Prime::Util, and Math::Random::MT::Auto. |
| 68 |
https://perlmonks.org/?node_id=11159834 |
| 69 |
|
| 70 |
1.891 Fri Jun 06 04:00:00 EST 2024 |
| 71 |
|
| 72 |
* Apply workaround for PDL::srand in MCE and MCE::Child. Thank you, PerlMonks. |
| 73 |
https://www.perlmonks.org/?node_id=11159773 |
| 74 |
* Add PDL::srand (v2.062 ~ v2.089) and PDL::srandom (v2.089_01+). |
| 75 |
* Call CORE::srand inside child processes, only. |
| 76 |
|
| 77 |
1.890 Fri May 24 19:00:00 EST 2024 |
| 78 |
|
| 79 |
* Improve reaping completed workers in MCE::Child. |
| 80 |
* Fix the _sprintf function, failing multiple arguments. |
| 81 |
|
| 82 |
1.889 Wed Sep 13 18:00:00 EST 2023 |
| 83 |
|
| 84 |
* Add Android support. Thank you, Dimitrios Kechagias. |
| 85 |
* Revert defer signal-handling in MCE::Channel (send2 method). |
| 86 |
* Improve mutex synchronize (aka enter) with guard capability. |
| 87 |
Thank you, José Joaquín Atria. |
| 88 |
* Fix mutex re-entrant lock on the Windows platform. |
| 89 |
* Add mutex guard_lock method. |
| 90 |
|
| 91 |
1.888 Wed Jun 21 17:00:00 EST 2023 |
| 92 |
|
| 93 |
* Fix typos caught by lintian. Thank you, Étienne Mollier. |
| 94 |
|
| 95 |
1.887 Fri Jun 09 08:00:00 EST 2023 |
| 96 |
|
| 97 |
* Fix typo in Queue dequeue_timed documentation. |
| 98 |
Thank you, Łukasz Strzelecki. |
| 99 |
|
| 100 |
1.886 Tue Jun 06 12:00:00 EST 2023 |
| 101 |
|
| 102 |
* Added dequeue_timed method to MCE::Queue. |
| 103 |
* Fixed taint mode in MCE->printf and _sprintf. |
| 104 |
* Improved reliability on the Windows platform. |
| 105 |
|
| 106 |
1.885 Tue May 30 20:00:00 EST 2023 |
| 107 |
|
| 108 |
* Improved reliability on the Windows platform. |
| 109 |
|
| 110 |
1.884 Thu Jan 05 10:00:00 EST 2023 |
| 111 |
|
| 112 |
* Disabled non-blocking dequeue_nb and recv_nb tests on the Windows platform. |
| 113 |
Reason: Author cannot reproduce failing tests reported by CPAN Tester aero. |
| 114 |
Copied nb tests to xt folder: nonblocking_channel.t and nonblocking_queue.t |
| 115 |
|
| 116 |
1.883 Tue Jan 03 20:00:00 EST 2023 |
| 117 |
|
| 118 |
* Fix typo in MCE::Channel::SimpleFast documentation. |
| 119 |
* Improve 05_mce_child.t test. |
| 120 |
|
| 121 |
1.882 Fri Dec 02 21:00:00 EST 2022 |
| 122 |
|
| 123 |
* Added ABRT to the list of signals to trap in MCE::Signal. |
| 124 |
* Added a guard to MCE::Core::Worker for checking if exited prematurely. |
| 125 |
* Added init_relay and use_threads import options to MCE and MCE Models. |
| 126 |
* Separated input mutexes from the rest of IPC for lesser latency. |
| 127 |
|
| 128 |
* Auto-detect if init_relay is defined and set chunk_size to 1 in |
| 129 |
MCE::Grep, MCE::Map, and MCE::Stream. |
| 130 |
* Update the import function in MCE models, detecting if the caller |
| 131 |
is another MCE module, to not export model functions. |
| 132 |
|
| 133 |
* Update the error status if MCE::Child died due to receiving a signal. |
| 134 |
* Improved reaping in MCE::Child, before creating a new child. |
| 135 |
* Improved the timeout handler in MCE::Child and MCE::Mutex::Channel. |
| 136 |
* Fixed private functions _quit and _trap not setting the return value. |
| 137 |
|
| 138 |
1.881 Thu Oct 13 23:45:00 EST 2022 |
| 139 |
|
| 140 |
* Improved the private _parse_chunk_size function. For better |
| 141 |
utilization of CPU cores in MCE::Grep, MCE::Map, and MCE::Stream, |
| 142 |
processing small input sizes. |
| 143 |
|
| 144 |
Previously, chunk_size => 'auto' equals 2 minimally. |
| 145 |
Starting with MCE v1.881, 'auto' equals 1 minimally. |
| 146 |
|
| 147 |
1.880 Mon Oct 10 04:00:00 EST 2022 |
| 148 |
|
| 149 |
* Improved reliability on the Windows platform. |
| 150 |
* Improved MCE::Mutex::Channel::timedwait on the Windows platform. |
| 151 |
* Improved MCE::Mutex::Channel performance on UNIX platforms. |
| 152 |
* Resolved edge case in MCE::Child reaching deadlock. |
| 153 |
|
| 154 |
1.879 Tue May 24 05:00:00 EST 2022 |
| 155 |
|
| 156 |
* Replace http with https in documentation and meta files. |
| 157 |
* Call PDL::set_autopthread_targ(1); disables PDL auto-threading. |
| 158 |
|
| 159 |
1.878 Sun Feb 20 06:45:00 EST 2022 |
| 160 |
|
| 161 |
* Fix for the fast channel implementations. |
| 162 |
Thank you, twata for the test report. |
| 163 |
|
| 164 |
1.877 Sun Feb 20 02:30:00 EST 2022 |
| 165 |
|
| 166 |
* Improved suppressing the PDL CLONE warning. Piddles should not be |
| 167 |
naively copied into new threads. |
| 168 |
|
| 169 |
* Added fast channel implementations optimized for non-Unicode strings. |
| 170 |
The main difference is that these lack freeze-thaw serialization. |
| 171 |
MCE::Channel::MutexFast, MCE::Channel::SimpleFast, and |
| 172 |
MCE::Channel::ThreadsFast. |
| 173 |
|
| 174 |
1.876 Thu Dec 02 18:00:00 EST 2021 |
| 175 |
|
| 176 |
* Allow percentage above 100% for max_workers in MCE. |
| 177 |
|
| 178 |
* MCE::Child update. |
| 179 |
Improved _ordhash. |
| 180 |
Renamed JOINED to REAPED in code for better clarity. |
| 181 |
Specify a percentage for max_workers. |
| 182 |
Added t/05_mce_child_max_workers.t |
| 183 |
|
| 184 |
1.875 Tue Nov 16 04:00:00 EST 2021 |
| 185 |
|
| 186 |
* Specify a percentage for max_workers. |
| 187 |
Thank you, kcott@PerlMonks (Ken) for the idea. |
| 188 |
https://www.perlmonks.org/?node_id=11134439 |
| 189 |
|
| 190 |
* Added t/03_max_workers.t |
| 191 |
|
| 192 |
1.874 Tue Aug 18 16:00:00 EST 2020 |
| 193 |
|
| 194 |
* Improved MCE->yield when used together with MCE::Relay. |
| 195 |
|
| 196 |
1.873 Sat Aug 01 16:00:00 EST 2020 |
| 197 |
|
| 198 |
* Removed unused variable in MCE::Mutex::Channel. |
| 199 |
|
| 200 |
* Fixed typo in prior change log. |
| 201 |
Replaced "completed" with "feature complete". |
| 202 |
|
| 203 |
1.872 Sun Jun 14 22:30:00 EST 2020 |
| 204 |
|
| 205 |
* Added open to required dependencies. |
| 206 |
* Set default encodings on standard filehandles in tests using UTF-8. |
| 207 |
|
| 208 |
* Bumped minimal Perl version to 5.8.1. |
| 209 |
* Bumped MCE version to 1.872 to align with MCE::Shared. |
| 210 |
|
| 211 |
* The MCE project is feature complete. |
| 212 |
|
| 213 |
1.868 Sun May 10 22:00:00 EST 2020 |
| 214 |
|
| 215 |
* Completed threads-like detach capability in MCE::Child. |
| 216 |
|
| 217 |
* Resolved MCE::Channel failing when calling dequeue multiple times |
| 218 |
on an ended channel. |
| 219 |
|
| 220 |
* MCE->say, MCE->print, and MCE->printf now return 1. |
| 221 |
|
| 222 |
1.867 Sun May 03 18:00:00 EST 2020 |
| 223 |
|
| 224 |
* Bug fix for UTF-8 issues during inter-process communication. |
| 225 |
This update required undoing optimizations specific to scalar args. |
| 226 |
Essentially, IPC involves serialization for everything going forward. |
| 227 |
Install Sereal::Encoder and Sereal::Decoder for better performance |
| 228 |
in Perl 5.8.8+. |
| 229 |
|
| 230 |
* MCE options flush_stdout, flush_stderr, and flush_file now default to |
| 231 |
enabled for the MCE->print, MCE->printf, and MCE->say output routines. |
| 232 |
|
| 233 |
* Improved MCE::Child with threads-like detach capability. See POD. |
| 234 |
|
| 235 |
* Improved IPC in MCE::Queue with permanent fast-like dequeue including |
| 236 |
dequeue_nb. Going forward, the fast and barrier options are silently |
| 237 |
ignored if specified (i.e. no-op). |
| 238 |
|
| 239 |
* Improved IPC performance on Linux. |
| 240 |
|
| 241 |
1.866 Sat Feb 08 21:00:00 EST 2020 |
| 242 |
|
| 243 |
* Bug fix for restart_worker, race condition introduced in 1.863. |
| 244 |
Thank you, Oliver Gorwits for reporting the issue. |
| 245 |
|
| 246 |
1.865 Wed Dec 25 18:00:00 EST 2019 |
| 247 |
|
| 248 |
* Bug fix for two-way IPC stalling on Windows in MCE::Channel::Threads. |
| 249 |
See https://www.perlmonks.org/?node_id=11110612 |
| 250 |
|
| 251 |
* Remove the check for MSWin32 in MCE::Channel::Mutex. MCE::Channel since |
| 252 |
the 1st release silently defaults to MCE::Channel::Threads on Windows. |
| 253 |
|
| 254 |
* Small tweak to MCE::Signal. |
| 255 |
|
| 256 |
1.864 Wed Dec 04 13:00:00 EST 2019 |
| 257 |
|
| 258 |
* Bug fix in MCE::Signal - Shared manager not exiting, introduced in 1.863. |
| 259 |
|
| 260 |
* Use monotonic clock if available in MCE->yield and MCE::Child->yield. |
| 261 |
See https://www.perlmonks.org/?node_id=11109673 |
| 262 |
|
| 263 |
1.863 Sun Nov 26 20:00:00 EST 2019 |
| 264 |
|
| 265 |
* On Cygwin, silently use Mutex in MCE::Channel when Threads is specified |
| 266 |
for better performance. |
| 267 |
|
| 268 |
* New defer capability in MCE::Signal. This applies to MCE::Shared 1.863. |
| 269 |
See POD section labled "DEFER SIGNAL" in MCE::Signal. |
| 270 |
|
| 271 |
* Reverted $child->exit back to sending the SIGQUIT signal in MCE::Child |
| 272 |
now that MCE::Shared::Server 1.863 defers signal during IPC. |
| 273 |
|
| 274 |
* Improved reliability for spawning MCE and MCE::Child inside threads |
| 275 |
including nested parallelization, made possible using a global lock |
| 276 |
$MCE::_GMUTEX. |
| 277 |
|
| 278 |
* Updated signal handling in mce-examples/framebuffer on GitHub. |
| 279 |
|
| 280 |
1.862 Wed Sep 18 22:00:00 EST 2019 |
| 281 |
|
| 282 |
* Hotfix for 1.861. |
| 283 |
|
| 284 |
1.861 Wed Sep 18 08:30:00 EST 2019 |
| 285 |
|
| 286 |
* Hotfix for 1.849-1.860. The edge cases regarding signal handling have |
| 287 |
been finally resolved for MCE::Child. Thank you, Richard Kelsch for the |
| 288 |
use case involving ctrl-c. See mce-examples/framebuffer on GitHub. |
| 289 |
|
| 290 |
1.860 Sun Sep 15 20:00:00 EST 2019 |
| 291 |
|
| 292 |
* Signal-handling update release. |
| 293 |
|
| 294 |
* Localized input and output record separators in MCE::Channel. |
| 295 |
* IPC safety in MCE::Child during SIGINT and SIGTERM. |
| 296 |
|
| 297 |
* Method $child->exit in MCE::Child now sends the SIGINT signal |
| 298 |
for extra reliability with MCE::Shared (previously SIGQUIT). |
| 299 |
|
| 300 |
1.850 Mon Sep 09 12:30:00 EST 2019 |
| 301 |
|
| 302 |
* Bumped version to coincide with the stable MCE::Shared 1.850 release. |
| 303 |
|
| 304 |
1.849 Sun Sep 08 23:30:00 EST 2019 |
| 305 |
|
| 306 |
* Fixed edge case in MCE::Child when reaping inside a signal handler. |
| 307 |
* Added list_pids class method to MCE::Child. |
| 308 |
|
| 309 |
1.848 Tue Sep 03 23:30:00 EST 2019 |
| 310 |
|
| 311 |
* Improved IO::All::{ File, Pipe, STDIO } output via MCE->print($io, ...), |
| 312 |
printf, and say. This resolves a bug introduced in 1.845 when using |
| 313 |
App::Cmd::Tester to capture output. |
| 314 |
|
| 315 |
1.847 Mon Sep 02 23:30:00 EST 2019 |
| 316 |
|
| 317 |
* Obsolete RedHat MCE-1.840-Sereal-deps.patch file. This patch file |
| 318 |
is no longer needed and finally resolved with this release. |
| 319 |
|
| 320 |
* PDL random numbers now unique between threads. Thank you, PerlMonks (vr). |
| 321 |
See https://www.perlmonks.org/?node_id=1214439. |
| 322 |
|
| 323 |
* Replaced "PF_UNIX" with "AF_UNIX" in MCE::Util. |
| 324 |
|
| 325 |
1.846 Mon Aug 26 21:30:00 EST 2019 |
| 326 |
|
| 327 |
* Fixed code tags in documentation. Thank you, Mohammad S Anwar. |
| 328 |
|
| 329 |
1.845 Sun Aug 25 22:00:00 EST 2019 |
| 330 |
|
| 331 |
* Croak if is_joinable, is_running, or join is called by a non-managed |
| 332 |
process in MCE::Child. Added LIMITATION section to the documentation. |
| 333 |
|
| 334 |
* Improved is_joinable, is_running, list_joinable, and list_running |
| 335 |
in MCE::Child. Thank you, Philippe Baumgart. |
| 336 |
|
| 337 |
* Added example (Consumer requests item) to MCE::Channel documentation. |
| 338 |
|
| 339 |
* Support the task_end option regardless if user_tasks is specified. |
| 340 |
|
| 341 |
* Support IO::All::{ File, Pipe, STDIO } for input data including |
| 342 |
output via MCE->print($io, ...), printf, and say. |
| 343 |
|
| 344 |
* Support gather => MCE::Candy::out_iter_fh($io) using MCE::Candy. |
| 345 |
|
| 346 |
1.844 Wed Aug 14 21:30:00 EST 2019 |
| 347 |
|
| 348 |
* Resolved MCE stalling when specifying max_retries with init_relay. |
| 349 |
Ditto for loop_timeout with init_relay on UNIX platforms. |
| 350 |
Thank you, Chris Denley. |
| 351 |
|
| 352 |
* Enhanced loop_timeout to handle workers dieing uncontrollably from |
| 353 |
any user_tasks (i.e. task_id >= 0). Previously, only task_id == 0. |
| 354 |
|
| 355 |
* Improved IPC on the Windows platform for edge case when a worker is |
| 356 |
awaiting input while the manager process is restarting a worker. |
| 357 |
|
| 358 |
* MCE, MCE::Child workers exit immediately upon receiving a SIGSEGV signal. |
| 359 |
This safeguards IPC from stalling inside the manager process. |
| 360 |
|
| 361 |
* Enhanced the _wait_one private function in MCE::Child. |
| 362 |
|
| 363 |
* Removed Prima from the list for auto-enabling the posix_exit option. |
| 364 |
Prima (since 1.52) is parallel safe during global cleanup. |
| 365 |
|
| 366 |
* Reached 100% Pod coverage. |
| 367 |
|
| 368 |
1.843 Tue Jul 23 22:30:00 EST 2019 |
| 369 |
|
| 370 |
* Updated results in MCE::Child (Parallel::ForkManager-like demonstration). |
| 371 |
|
| 372 |
* Completed missing interrupt signal-safety for the non-blocking methods |
| 373 |
in MCE::Channel::Mutex and MCE::Channel::Threads. |
| 374 |
|
| 375 |
1.842 Sun Jul 21 19:00:00 EST 2019 |
| 376 |
|
| 377 |
* Fixed race condition abnormalities in MCE::Child. |
| 378 |
* Added Parallel::ForkManager-like demonstration to MCE::Child. |
| 379 |
|
| 380 |
1.841 Sun Jul 07 23:30:00 EST 2019 |
| 381 |
|
| 382 |
* Disabled t/04_channel_threads testing on Unix platforms for Perl less than |
| 383 |
5.10.1. Basically, the MCE::Channel::Threads implementation is not supported |
| 384 |
on older Perls unless the OS vendor applied upstream patches (i.e. works on |
| 385 |
RedHat/CentOS 5.x running Perl 5.8.x). |
| 386 |
|
| 387 |
* Added LIMITATIONS section to MCE::Channel::Threads. |
| 388 |
|
| 389 |
1.840 Sun Jul 07 05:00:00 EST 2019 |
| 390 |
|
| 391 |
* Update MCE::Channel POD documentation. |
| 392 |
|
| 393 |
1.839 Sun Jul 07 04:30:00 EST 2019 |
| 394 |
|
| 395 |
[NEW FEATURES] |
| 396 |
|
| 397 |
* Added MCE::Mutex::Channel2 providing two locks using a single channel. |
| 398 |
The secondary lock is accessible by calling methods with the '2' suffix. |
| 399 |
E.g. primary mutex ->lock, ->unlock; secondary mutex ->lock2, ->unlock2 |
| 400 |
|
| 401 |
* Added MCE::Channel providing queue-like and two-way communication |
| 402 |
supporting threads and processes. |
| 403 |
|
| 404 |
* Added MCE::Child and compatibility with Perl 5.8. MCE::Child is based on |
| 405 |
MCE::Hobo, but using MCE::Channel for data retrieval without involving |
| 406 |
a shared-manager process. |
| 407 |
|
| 408 |
* Added MCE::Channel examples { channel1.pl and channel2.pl } using |
| 409 |
threads and MCE::Child respectively. |
| 410 |
https://github.com/marioroy/mce-examples/tree/master/chameneos |
| 411 |
|
| 412 |
[ENHANCEMENTS] |
| 413 |
|
| 414 |
* IPC update; removed unnecessary overhead including private methods |
| 415 |
_sysseek and _syswrite from MCE::Util (no longer needed). |
| 416 |
|
| 417 |
* Improved MCE->do, now callable by workers and the manager process. |
| 418 |
|
| 419 |
* Updated MCE::{ Flow, Grep, Loop, Map, Step, and Stream } documentation |
| 420 |
on passing an array reference versus a list for deeply input data. |
| 421 |
|
| 422 |
* Updated and re-organized the top-level MCE documentation, particularly |
| 423 |
improved clarity for the 'MCE Models' section. |
| 424 |
|
| 425 |
* Removed MANIFEST.SKIP. |
| 426 |
|
| 427 |
1.838 Wed Jan 23 08:30:00 EST 2019 |
| 428 |
|
| 429 |
* IPC update, raising reliability across multiple platforms. |
| 430 |
* Improved hack for the Windows platform for nested MCE sessions. |
| 431 |
* Added _sysread, _sysseek, _syswrite, and _nonblocking to MCE::Util. |
| 432 |
* Added barrier option to MCE::Queue allowing one to disable. |
| 433 |
|
| 434 |
1.837 Sat Aug 25 13:00:00 EST 2018 |
| 435 |
|
| 436 |
* Seeds the Math::Random::MT::Auto generator automatically when present |
| 437 |
for non-threads, similarly to Math::Random and Math::Prime::Util, to |
| 438 |
avoid child processes sharing the same seed value as the parent and |
| 439 |
each other. The new seed is computed using the current seed. |
| 440 |
|
| 441 |
1.836 Mon Jun 25 08:30:00 EST 2018 |
| 442 |
|
| 443 |
* Moved validation code from MCE::Util to MCE::Core::Validation. |
| 444 |
* Applied small optimizations. |
| 445 |
|
| 446 |
1.835 Tue Mar 13 15:00:00 EST 2018 |
| 447 |
|
| 448 |
* Added gather and relay demonstrations to MCE::Relay. |
| 449 |
* Load IO::Handle for extra stability, preventing workers loading uniquely. |
| 450 |
|
| 451 |
* Load Net::HTTP and Net::HTTPS before spawning if present LWP::UserAgent. |
| 452 |
See https://www.perlmonks.org/?node_id=1199760 |
| 453 |
and https://www.perlmonks.org/?node_id=1199891. |
| 454 |
|
| 455 |
1.834 Tue Jan 23 08:00:00 EST 2018 |
| 456 |
|
| 457 |
* Improved Queue await and dequeue performance on the Windows platform. |
| 458 |
|
| 459 |
* Added chameneos-redux parallel demonstrations on GitHub: |
| 460 |
https://github.com/marioroy/mce-examples/tree/master/chameneos |
| 461 |
|
| 462 |
1.833 Thu Dec 28 16:00:00 EST 2017 |
| 463 |
|
| 464 |
* Fixed bug with sequence (#10), broken in 1.832. Thank you, @djerius. |
| 465 |
|
| 466 |
1.832 Tue Nov 21 15:30:00 EST 2017 |
| 467 |
|
| 468 |
* Added LWP::UserAgent to list for enabling posix_exit. |
| 469 |
* Improved number-sequence generation for big integers. |
| 470 |
* Improved wantarray support in MCE::Mutex synchronize. |
| 471 |
* Removed limit check on chunk_size option. |
| 472 |
|
| 473 |
1.831 Sun Oct 08 20:30:00 EST 2017 |
| 474 |
|
| 475 |
* Added STFL (Terminal UI) to list for enabling posix_exit. |
| 476 |
See https://www.perlmonks.org/?node_id=1200923. |
| 477 |
|
| 478 |
* Math::Prime::Util random numbers now unique between MCE workers. |
| 479 |
See https://www.perlmonks.org/?node_id=1200960. |
| 480 |
|
| 481 |
1.830 Tue Sep 12 17:00:00 EST 2017 |
| 482 |
|
| 483 |
[BUG FIXES] |
| 484 |
|
| 485 |
* Fixed MCE and MCE::Relay stalling when setting the input record separator. |
| 486 |
See https://www.perlmonks.org/?node_id=1196701. Thank you, JediWombat. |
| 487 |
|
| 488 |
* Fixed bug with dequeue_nb in MCE::Queue (#8). Thank you, @bokutin. |
| 489 |
* Fixed signal handler (#9). Thank you, @chrisdenley. |
| 490 |
|
| 491 |
[ENHANCEMENTS] |
| 492 |
|
| 493 |
* Added Coro and Win32::GUI to list for enabling posix_exit. |
| 494 |
* Added support for Haiku to get_ncpu in MCE::Util. |
| 495 |
* Allow gathering to a shared array in MCE::Candy. |
| 496 |
|
| 497 |
* Improved CPU count on the AIX platform in MCE::Util. |
| 498 |
* Improved signal handling, including nested parallel-sessions. |
| 499 |
* Improved running MCE::Hobo inside MCE workers. |
| 500 |
* Improved running MCE with PDL. |
| 501 |
|
| 502 |
* Refactored logic for MCE->do, bi-directional callback feature. |
| 503 |
* Preserve lexical type for numbers during IPC: MCE->do and MCE::Queue. |
| 504 |
|
| 505 |
* No longer loads threads on the Windows platform in MCE::Signal. |
| 506 |
This enables MCE::Hobo 1.827 to spin faster, including lesser |
| 507 |
memory consumption. Threads isn't required to run MCE::Hobo. |
| 508 |
|
| 509 |
* Removed extra white-space from POD documentation. |
| 510 |
* Validated MCE on SmartOS. |
| 511 |
|
| 512 |
1.829 Wed May 03 03:00:00 EST 2017 |
| 513 |
|
| 514 |
* Reduced memory consumption. |
| 515 |
|
| 516 |
1.828 Fri Apr 28 16:00:00 EST 2017 |
| 517 |
|
| 518 |
[BUG FIXES] |
| 519 |
|
| 520 |
* Do not enable barrier mode for Queue on the Windows platform. |
| 521 |
* Fixed MCE::Mutex::Flock, tmp_file missing script name in path. |
| 522 |
|
| 523 |
[ENHANCEMENTS] |
| 524 |
|
| 525 |
* Added Curses and Prima to list for enabling the posix_exit option. |
| 526 |
* Allow a hash as input_data: Core API, MCE::{ Flow, Loop, Step }. |
| 527 |
* Improved API documentation on MCE models with more synopsis. |
| 528 |
* Enhanced IPC and signal handling. Reduced memory consumption. |
| 529 |
* Make tmp_dir on demand in MCE::Signal. Ditto for sess_dir in MCE. |
| 530 |
* Load Fcntl, File::Path, Symbol on demand. |
| 531 |
|
| 532 |
1.827 Wed Apr 05 01:30:00 EST 2017 |
| 533 |
|
| 534 |
* Do not enable barrier mode for Queue if constructed inside a thread |
| 535 |
or by MCE models (e.g. Step, Stream). Ditto for fast => 1 option. |
| 536 |
|
| 537 |
* Updated MCE to not croak when running Perl in taint mode via perl -T. |
| 538 |
Failing -T was MCE::Core::Input::{ Generator, Sequence }, MCE::Signal, |
| 539 |
and MCE::Util. |
| 540 |
|
| 541 |
* Added Denis Fateyev, Felipe Gasper, and Paul Howarth to Credits. |
| 542 |
|
| 543 |
1.826 Sun Apr 02 23:00:00 EST 2017 |
| 544 |
|
| 545 |
* Is now safe running MCE with the Wx GUI toolkit (wxWidgets). |
| 546 |
|
| 547 |
1.825 Sun Apr 02 07:00:00 EST 2017 |
| 548 |
|
| 549 |
* Updated MCE::Queue. The following provides a comparison for the |
| 550 |
enhancements made regarding IPC during 1.822 through 1.825, in order |
| 551 |
to run on machines having "many" cores. I ran with 12, 96, and 192 |
| 552 |
workers on an 8 core box. |
| 553 |
|
| 554 |
A. MCE queue, dequeue 100k items. |
| 555 |
|
| 556 |
my $Q = MCE::Queue->new(); |
| 557 |
|
| 558 |
$Q->enqueue( 1 .. 100000 ); |
| 559 |
$Q->end(); MCE 1.608 $Q->enqueue((undef) x 12, 96, or 192); |
| 560 |
|
| 561 |
MCE->new( |
| 562 |
max_workers => 12, 96, or 192, |
| 563 |
user_func => sub { |
| 564 |
while ( defined ( my $item = $Q->dequeue ) ) { |
| 565 |
; |
| 566 |
} |
| 567 |
} |
| 568 |
)->run(); |
| 569 |
|
| 570 |
MCE 1.608: 12 ~ 1.799 secs 96 ~ 8.702 secs 192 ~ 18.083 secs |
| 571 |
MCE 1.821: ~ 1.450 secs ~ 5.231 secs ~ 8.102 secs |
| 572 |
MCE 1.825: ~ 0.976 secs ~ 1.067 secs ~ 1.509 secs |
| 573 |
|
| 574 |
B. Input file containing 250k lines ( 300 MiB ). |
| 575 |
|
| 576 |
MCE->new( |
| 577 |
input_data => "/dev/shm/file_250k.txt", |
| 578 |
max_workers => 12, 96, or 192, |
| 579 |
chunk_size => 1, |
| 580 |
use_slurpio => 1, |
| 581 |
user_func => sub { } |
| 582 |
)->run(); |
| 583 |
|
| 584 |
MCE 1.608: 12 ~ 3.605 secs 96 ~ 8.074 secs 192 ~ 8.465 secs |
| 585 |
MCE 1.821: ~ 3.613 secs ~ 8.058 secs ~ 8.607 secs |
| 586 |
MCE 1.825: ~ 3.567 secs ~ 2.601 secs ~ 3.199 secs |
| 587 |
|
| 588 |
C. Sequence of numbers from 1 to 200k. |
| 589 |
|
| 590 |
MCE->new( |
| 591 |
sequence => [ 1, 200000 ], |
| 592 |
max_workers => 12, 96, or 192, |
| 593 |
chunk_size => 1, |
| 594 |
user_func => sub { } |
| 595 |
)->run(); |
| 596 |
|
| 597 |
MCE 1.608: 12 ~ 1.236 secs 96 ~ 2.922 secs 192 ~ 3.113 secs |
| 598 |
MCE 1.821: ~ 1.227 secs ~ 2.915 secs ~ 3.106 secs |
| 599 |
MCE 1.825: ~ 1.250 secs ~ 1.203 secs ~ 1.581 secs |
| 600 |
|
| 601 |
* Results were captured on a fast 8 core system running CentOS Linux 7. |
| 602 |
The thing to take from this is that running many workers "no longer" |
| 603 |
results in up to 5.6x penalty regarding IPC. |
| 604 |
|
| 605 |
1.824 Sat Apr 01 01:00:00 EST 2017 |
| 606 |
|
| 607 |
* Completed validation for running MCE on a box having 100+ cores. |
| 608 |
|
| 609 |
* Tuned the number of data-channels for IPC. Set upper limit in |
| 610 |
MCE::Core::Input::{ Handle and Sequence } to not impact the |
| 611 |
OS kernel. The result is better performance, yet graceful. |
| 612 |
|
| 613 |
1.823 Fri Mar 31 19:30:00 EST 2017 |
| 614 |
|
| 615 |
* Calibrated the number of data-channels for IPC. |
| 616 |
|
| 617 |
1.822 Fri Mar 31 11:00:00 EST 2017 |
| 618 |
|
| 619 |
* Check for EINTR during sysread and syswrite. |
| 620 |
* Improved reliability when running nested MCE sessions. |
| 621 |
* Updated MCE::Mutex with Channel and Fcntl implementations. |
| 622 |
|
| 623 |
* Completed validation for using MCE with 200+ cores. |
| 624 |
|
| 625 |
1.821 Sun Mar 19 04:00:00 EST 2017 |
| 626 |
|
| 627 |
* Improved reliability when running MCE with threads. |
| 628 |
|
| 629 |
* Added parallel Net::Pcap and Ping demonstrations on GitHub: |
| 630 |
https://github.com/marioroy/mce-examples/tree/master/network |
| 631 |
|
| 632 |
* Optimized 'dequeue' method in MCE::Queue. |
| 633 |
* Optimized 'synchronize' method in MCE::Mutex. |
| 634 |
|
| 635 |
1.820 Thu Mar 09 02:00:00 EST 2017 |
| 636 |
|
| 637 |
* Improved reliability when running MCE inside an eval block. |
| 638 |
|
| 639 |
1.819 Fri Mar 03 23:00:00 EST 2017 |
| 640 |
|
| 641 |
* Fixed issue with localizing AUTOFLUSH variable before autoflush handles. |
| 642 |
Thank you, Charles Hendry for raising the issue. |
| 643 |
|
| 644 |
1.818 Wed Mar 01 22:00:00 EST 2017 |
| 645 |
|
| 646 |
* Updated bin/mce_grep for determining chunk level. Ditto for chunk size. |
| 647 |
Fixed an issue for not seeing STDERR output with '--chunk-level=file'. |
| 648 |
Added support for zgrep, zegrep, and zfgrep. Thank you, Jeff Rouse. |
| 649 |
|
| 650 |
* Changed Sereal to Sereal::Decoder and Sereal::Encoder in recommends |
| 651 |
section inside Makefile and META files. |
| 652 |
|
| 653 |
* Refactored MCE::Queue. Merged local and manager code base into one. |
| 654 |
Removed t/04_norm_que_local.t and t/04_prio_que_local.t. |
| 655 |
|
| 656 |
* Added 'end' method to MCE::Queue. |
| 657 |
Updated documentation on dequeue and pending. |
| 658 |
|
| 659 |
1.817 Sat Feb 25 02:00:00 EST 2017 |
| 660 |
|
| 661 |
* Improved bin/mce_grep. When -r is specified and zero paths are given, |
| 662 |
start recursively in the current directory versus await data from STDIN. |
| 663 |
Set chunk-level accordingly to list mode. |
| 664 |
|
| 665 |
1.816 Fri Feb 24 19:00:00 EST 2017 |
| 666 |
|
| 667 |
* Revised the description on max_retries in MCE::Core.pod. |
| 668 |
|
| 669 |
1.815 Fri Feb 24 01:00:00 EST 2017 |
| 670 |
|
| 671 |
* Fixed divide-by-zero error in MCE->yield. |
| 672 |
|
| 673 |
* Refactored code for the interval option by moving the code to the manager |
| 674 |
process. This allows the manager process to accomodate the next available |
| 675 |
worker and ready to run. Previoulsy, a worker taking a long time resulted |
| 676 |
in empty time slots. Thank you, Philippe Baumgart for your patience. |
| 677 |
|
| 678 |
* Revised the description on posix_exit in MCE::Core.pod. |
| 679 |
|
| 680 |
1.814 Mon Feb 20 05:30:00 EST 2017 |
| 681 |
|
| 682 |
* Enhanced the progress option for use with MCE->process. |
| 683 |
Updated demonstrations in MCE::Core.pod. |
| 684 |
|
| 685 |
1.813 Thu Feb 16 02:30:00 EST 2017 |
| 686 |
|
| 687 |
* Last minute request by Philippe Baumgart (reminder and long overdue). |
| 688 |
Added progress option, a code block for receiving info on progress made. |
| 689 |
See MCE::Core.pod for demonstrations accommodating all input data types. |
| 690 |
|
| 691 |
1.812 Tue Feb 14 17:00:00 EST 2017 |
| 692 |
|
| 693 |
* Bumped minimum requirement for Sereal to 3.015 when available. |
| 694 |
Added check ensuring matching version for Encoder and Decoder. |
| 695 |
|
| 696 |
1.811 Mon Feb 13 23:30:00 EST 2017 |
| 697 |
|
| 698 |
* Fixed bug in MCE::Queue (dequeue_nb) when queue has zero items. |
| 699 |
* Applied small optimization in MCE::Core::Input::Sequence and Generator. |
| 700 |
|
| 701 |
* Added cross-platform template to MCE::Examples for making an executable. |
| 702 |
* Removed signal handling for XCPU and XFSZ from MCE::Signal. |
| 703 |
|
| 704 |
* Imply posix_exit => 1 if Gearman::XS or Gearman::Util is present during |
| 705 |
MCE construction. |
| 706 |
|
| 707 |
* Added MCE + Gearman demonstrations (xs and non-xs) on GitHub: |
| 708 |
https://github.com/marioroy/mce-examples/tree/master/gearman_xs |
| 709 |
https://github.com/marioroy/mce-examples/tree/master/gearman |
| 710 |
|
| 711 |
* Changed kilobytes and megabytes to kibiBytes (KiB) and mebiBytes (MiB) |
| 712 |
respectively inside the documentation. |
| 713 |
|
| 714 |
1.810 Fri Dec 09 23:30:00 EST 2016 |
| 715 |
|
| 716 |
* Updated check for IO handle allowed. This allows $gather_fh = *STDOUT{IO}, |
| 717 |
construction in Perl <= 5.10.1. Thank you, Qiang Wang. |
| 718 |
|
| 719 |
1.809 Wed Nov 23 16:00:00 EST 2016 |
| 720 |
|
| 721 |
* Bug fixes for running MCE inside threads. |
| 722 |
* Random numbers are unique between workers. |
| 723 |
|
| 724 |
1.808 Sat Nov 05 02:00:00 EST 2016 |
| 725 |
|
| 726 |
* Workers persist unless shutdown explicity while running alongside |
| 727 |
the Mojolicious framework. |
| 728 |
|
| 729 |
1.807 Tue Nov 01 16:00:00 EST 2016 |
| 730 |
|
| 731 |
* Enhanced relay capabilities. Added Mandelbrot example to MCE::Example. |
| 732 |
Added extra demonstrations to MCE::Relay. Also, added test script. |
| 733 |
|
| 734 |
* Tweaked manager-loop delay for special cases -- applies to MSWin32 only. |
| 735 |
|
| 736 |
1.806 Tue Oct 11 21:30:00 EST 2016 |
| 737 |
|
| 738 |
* Fixed two typos. Thank you, Florian Schlichting. |
| 739 |
* Support input_data with nested arrays in MCE models. |
| 740 |
|
| 741 |
1.805 Thu Sep 01 16:00:00 EST 2016 |
| 742 |
|
| 743 |
* Fixed bug in MCE::Queue (#4). Thank you, Mary Ehlers. |
| 744 |
|
| 745 |
* Improved support for running MCE with Tk. Added Tk demonstrations to |
| 746 |
MCE::Examples. Thank you, Götz Meyer. |
| 747 |
|
| 748 |
1.804 Thu Jul 28 23:00:00 EST 2016 |
| 749 |
|
| 750 |
* Removed the sleep statement in MCE->restart_worker. |
| 751 |
|
| 752 |
* Added FCGI::ProcManager demonstrations to MCE::Examples. |
| 753 |
* Automatically set posix_exit to 1 whenever (F)CGI.pm is present. |
| 754 |
* Thank you Kai Wasserbäch (TheRealCuran) for the cool test case. |
| 755 |
https://github.com/marioroy/mce-perl/issues/1 |
| 756 |
|
| 757 |
1.803 Sun Jul 10 23:30:00 EST 2016 |
| 758 |
|
| 759 |
* Re-enabled Sereal 3.008+ for Perl < v5.12.0, if available. |
| 760 |
* Optimized dequeue methods in MCE::Queue. |
| 761 |
|
| 762 |
1.802 Mon Jul 04 03:30:00 EST 2016 |
| 763 |
|
| 764 |
* Default to Storable for serialization in Perl less than v5.12.0. |
| 765 |
Sereal 3.008+, if available, is loaded automatically in Perl v5.12+. |
| 766 |
|
| 767 |
1.801 Sun Jul 03 00:30:00 EST 2016 |
| 768 |
|
| 769 |
[BUG FIXES] |
| 770 |
|
| 771 |
* Fixed race condition in Queue->await. |
| 772 |
|
| 773 |
[ENHANCEMENTS] |
| 774 |
|
| 775 |
* MCE 1.801 is stable on all supported platforms. |
| 776 |
|
| 777 |
* Completed work supporting cyclical include of MCE Core / models. |
| 778 |
* Updated MCE to support Perl included with Git Bash. |
| 779 |
|
| 780 |
* Renamed temp dir from 'mce' to 'Perl-MCE' under user's %TEMP% |
| 781 |
location on Windows. E.g. Native Perl, Cygwin, Git Bash. |
| 782 |
|
| 783 |
1.800 Sat Jun 18 16:30:00 EST 2016 |
| 784 |
|
| 785 |
[BUG FIXES] |
| 786 |
|
| 787 |
* Fixed dequeue (count) in MCE::Queue for standalone mode. |
| 788 |
|
| 789 |
[ENHANCEMENTS] |
| 790 |
|
| 791 |
* On Windows, improved stablity and feature parity with UNIX. |
| 792 |
* Use Sereal 3.008+ automatically if available on the box. |
| 793 |
|
| 794 |
[NEW FEATURES] |
| 795 |
|
| 796 |
* Added support for cyclical include of MCE Core, MCE models, and |
| 797 |
MCE Queue by scoping the configuration to the local package. |
| 798 |
This resolves (RT#107384), bug reported by Kai Wasserbäch. |
| 799 |
|
| 800 |
1.708 Sat May 28 14:00:00 EST 2016 |
| 801 |
|
| 802 |
[BUG FIXES] |
| 803 |
|
| 804 |
* Improved import routine in MCE models and MCE::Subs. This resolves an |
| 805 |
issue where functions are not exported; e.g. mce_flow, mce_flow_s. |
| 806 |
|
| 807 |
[ENHANCEMENTS] |
| 808 |
|
| 809 |
* Added support for IO::TieCombine handles. This enables MCE->print and |
| 810 |
MCE->sendto to work reliably with App::Cmd and App::Cmd::Tester. |
| 811 |
See Testing and Capturing Output in MCE::Examples. |
| 812 |
|
| 813 |
1.707 Wed May 25 16:00:00 EST 2016 |
| 814 |
|
| 815 |
[BUG FIXES] |
| 816 |
|
| 817 |
* Fixed logic when workers exit. Improved reliability on Windows. |
| 818 |
|
| 819 |
[ENHANCEMENTS] |
| 820 |
|
| 821 |
* Applied MCE-1.700-provides.patch from RedHat. Thank you, Paul Howarth. |
| 822 |
|
| 823 |
* Added META.json to the distribution. |
| 824 |
|
| 825 |
1.706 Fri Apr 22 21:30:00 EST 2016 |
| 826 |
|
| 827 |
[ENHANCEMENTS] |
| 828 |
|
| 829 |
* Time::HiRes sleep resolution is 15 milliseconds on Windows and Cygwin. |
| 830 |
Adjusted timeout values accordingly. Thank you, Daniel Dragan. |
| 831 |
|
| 832 |
* Reinstated the hack for faster IO when use_slurpio => 1 is specified. |
| 833 |
Tuned chunk_size => 'auto'. |
| 834 |
|
| 835 |
1.705 Thu Apr 14 10:00:00 EST 2016 |
| 836 |
|
| 837 |
* Bumped version for Test::More to 0.88. Thank you, Paul Howarth. |
| 838 |
|
| 839 |
1.704 Thu Apr 14 05:00:00 EST 2016 |
| 840 |
|
| 841 |
[BUG FIXES] |
| 842 |
|
| 843 |
* Fixed restart on the Windows platform, bug introduced in 1.700. |
| 844 |
* Reached *stable* on all major platforms for MCE 1.7x. |
| 845 |
|
| 846 |
[ENHANCEMENTS] |
| 847 |
|
| 848 |
* Enabled auto-destroy for MCE objects. |
| 849 |
* Enabled freeze callbacks for Sereal. |
| 850 |
* Switched bug tracking to GitHub. |
| 851 |
* Tweaked test scripts. |
| 852 |
|
| 853 |
1.703 Sat Mar 19 23:00:00 EST 2016 |
| 854 |
|
| 855 |
* Completed IPC optimizations for 1.7. |
| 856 |
|
| 857 |
1.702 Tue Mar 15 17:00:00 EST 2016 |
| 858 |
|
| 859 |
* Bumped version. |
| 860 |
|
| 861 |
1.701 Tue Mar 15 12:00:00 EST 2016 |
| 862 |
|
| 863 |
[ENHANCEMENTS] |
| 864 |
|
| 865 |
* Some folks have expressed a wish for running MCE 1.7 with Perl v5.8. |
| 866 |
To restore support for Perl v5.8, removed MCE::Shared and MCE::Hobo |
| 867 |
from the MCE 1.701 distribution. |
| 868 |
|
| 869 |
* MCE::Shared will be released after MCE 1.700 is deleted from CPAN. |
| 870 |
Thank you for your patience during this transition. |
| 871 |
|
| 872 |
1.700 Tue Mar 08 15:30:00 EST 2016 |
| 873 |
|
| 874 |
[BUG FIXES] |
| 875 |
|
| 876 |
* Fixed race condition on Windows for non-threaded workers. |
| 877 |
|
| 878 |
* Updated MCE models to not fail when running inside an eval statement. |
| 879 |
This addresses (RT#105557) and (RT#105559). Thank you, Benjamin McKeown. |
| 880 |
|
| 881 |
* Added new MCE option loop_timeout to prevent the MCE Manager process |
| 882 |
from hanging perpetually. The manager process wrongly assumes a worker |
| 883 |
is still running when the worker died in an uncontrollable manner. |
| 884 |
This resolves (RT#111780). Thank you, Benjamin McKeown. |
| 885 |
|
| 886 |
[ENHANCEMENTS] |
| 887 |
|
| 888 |
* Perl 5.10.1 or later is required to run MCE 1.7. Perl < 5.10.1 lacks |
| 889 |
'overloading.pm'. |
| 890 |
|
| 891 |
* Added code in MCE::Grep's documentation for parsing huge files. |
| 892 |
* Added support for running MCE with Perl under MobaXterm on Windows. |
| 893 |
|
| 894 |
* MCE/examples and MCE/images are no longer included with the distribution. |
| 895 |
These are maintained separately at https://github.com/marioroy/mce-examples |
| 896 |
and https://github.com/marioroy/mce-assets respectively. |
| 897 |
|
| 898 |
* MCE performs channel locking via a pipe or socket depending on platform. |
| 899 |
Previously, locking was through file locking using flock. This resolves |
| 900 |
the slow locking performance on Cygwin. |
| 901 |
|
| 902 |
* Optimized signal handling including improved support on Windows. |
| 903 |
|
| 904 |
* Reduced overhead during spawning and job submissions on Windows and Cygwin. |
| 905 |
This enables IPC to complete up to 20x faster, thus benefiting Monte-Carlo |
| 906 |
simulations; e.g. calling ->run(0) or ->process(...) repeatedly. |
| 907 |
|
| 908 |
* The MCE::Flow and MCE::Step models can take an anonymous array for |
| 909 |
specifying use_threads uniquely for sub-tasks. |
| 910 |
|
| 911 |
[NEW FEATURES] |
| 912 |
|
| 913 |
* Added MCE::Hobo for running code asynchronously. This provides async/join |
| 914 |
functionality for processes similarly to async/join in threads. It includes |
| 915 |
->is_joinable, ->is_running, ->join, ->kill, ->list, ->waitall, ->waitone, |
| 916 |
and other methods not metioned here. |
| 917 |
|
| 918 |
* Added MCE::Shared for sharing objects/data between threads/processes. |
| 919 |
* Added MCE::Shared::{ Array, Handle, Hash, Ordhash, and Scalar }. |
| 920 |
* Added MCE::Shared::{ Condvar, Minidb, Queue, and Sequence }. |
| 921 |
* Added MCE::Shared::{ Server }. |
| 922 |
|
| 923 |
* Added methods ->await, ->enq, and ->enqp to MCE::Step. |
| 924 |
* Added method ->await to MCE::Queue. |
| 925 |
|
| 926 |
* Added option max_retries => N for retrying a failed chunk from a worker |
| 927 |
dying while processing input data or sequence of numbers. |
| 928 |
|
| 929 |
* Added option posix_exit => 1 to avoid END and destructor processing. |
| 930 |
This is necessary for running with Tk and child processes or with |
| 931 |
use_threads => 0. |
| 932 |
|
| 933 |
* Seeds the Math::Random generator automatically when present for non-threads |
| 934 |
to avoid child processes sharing the same seed value as the parent and |
| 935 |
each other. The new seed is computed using the current seed. Thus, okay |
| 936 |
to set the seed at the application level for predictable results. |
| 937 |
|
| 938 |
1.608 Fri Apr 10 03:00:00 EST 2015 |
| 939 |
|
| 940 |
* Correction for $prog_name (-e) to (perl) in MCE::Signal. |
| 941 |
|
| 942 |
1.607 Fri Apr 10 01:00:00 EST 2015 |
| 943 |
|
| 944 |
[BUG FIXES] |
| 945 |
|
| 946 |
* Updated t/01_load_signal_arg.t to address false-positive from bug fix |
| 947 |
in 1.606. This was missed in the 1.606 release. Thank you Nigel Horne. |
| 948 |
|
| 949 |
[ENHANCEMENTS] |
| 950 |
|
| 951 |
* Replaced (-e) with (perl) for the $prog_name value in MCE::Signal; |
| 952 |
e.g. from running perl -e 'command'. |
| 953 |
|
| 954 |
1.606 Wed Apr 08 18:00:00 EST 2015 |
| 955 |
|
| 956 |
[BUG FIXES] |
| 957 |
|
| 958 |
* Added -d and -w tests to ensure $ENV{TEMP} exists and writeable in |
| 959 |
MCE::Signal. Otherwise /tmp is used as usual. |
| 960 |
|
| 961 |
[ENHANCEMENTS] |
| 962 |
|
| 963 |
* Determine running state in MCE->exit. Call stop_and_exit if not |
| 964 |
already running to not hang the script. |
| 965 |
|
| 966 |
1.605 Mon Apr 06 00:30:00 EST 2015 |
| 967 |
|
| 968 |
[BUG FIXES] |
| 969 |
|
| 970 |
* Improved fix for the die handler in MCE::Signal and MCE::Core::Worker. |
| 971 |
* Improved support for threads in MCE::Signal's stop_and_exit function. |
| 972 |
|
| 973 |
1.604 Sat Mar 21 21:00:00 EST 2015 |
| 974 |
|
| 975 |
[BUG FIXES] |
| 976 |
|
| 977 |
* All bugs found during testing of the upcoming 1.7 release have been |
| 978 |
backported to the 1.6 branch. |
| 979 |
|
| 980 |
[NEW FEATURES] |
| 981 |
|
| 982 |
* Added out_iter_array and out_iter_fh to MCE::Candy. These preserve output |
| 983 |
order and cover the two general use cases. |
| 984 |
|
| 985 |
1.603 Tue Mar 17 21:00:00 EST 2015 |
| 986 |
|
| 987 |
[ENHANCEMENTS] |
| 988 |
|
| 989 |
* A safer solution by Dmitry Karasik for the die handler in MCE::Signal |
| 990 |
and MCE::Core::Worker. |
| 991 |
|
| 992 |
* Moved ->forchunk, ->foreach, and ->forseq sugar methods to MCE::Candy. |
| 993 |
Stubs exist in MCE. Thus, no breakage to existing apps. |
| 994 |
|
| 995 |
* Removed the link to MCE::Shared on the main page. I decided to backport |
| 996 |
all the fixes into 1.6. The MCE::Shared link was missed and requires |
| 997 |
the upcoming 1.7 release. |
| 998 |
|
| 999 |
1.602 Mon Mar 16 20:00:00 EST 2015 |
| 1000 |
|
| 1001 |
[BUG FIXES] |
| 1002 |
|
| 1003 |
* Updated die handler in MCE::Core::Worker and MCE::Signal (RT#102802) |
| 1004 |
Bug reported by Dmitry Karasik. |
| 1005 |
|
| 1006 |
[ENHANCEMENTS] |
| 1007 |
|
| 1008 |
* MCE child processes call CORE::exit during exiting; previously CORE::kill. |
| 1009 |
* Improved IPC stability on Windows and Cygwin. |
| 1010 |
|
| 1011 |
* Multiple calls to mutex->lock by the same process or thread is now safe. |
| 1012 |
The mutex will remain locked until mutex->unlock is called. |
| 1013 |
|
| 1014 |
1.601 Sun Mar 15 21:00:00 EST 2015 |
| 1015 |
|
| 1016 |
This release is attributed to the many use cases sent by George Bouras. |
| 1017 |
|
| 1018 |
[BUG FIXES] |
| 1019 |
|
| 1020 |
* Updated Makefile.PL allowing installation of bin/mce_grep (RT#102040). |
| 1021 |
Running (MCE_INSTALL_TOOLS=1 perl Makefile.PL) installs bin/mce_grep. |
| 1022 |
|
| 1023 |
* Ensure MCE instances spawned by workers have shutdown before leaving. |
| 1024 |
* An exiting forked non-MCE process will not cause the worker to exit. |
| 1025 |
|
| 1026 |
* Fixed sockets not closing immediately in Cygwin and Windows. |
| 1027 |
* Fixed a pod error in MCE::Mutex. |
| 1028 |
|
| 1029 |
* Fixed a rare condition where socket handles for MCE::Mutex and |
| 1030 |
MCE::Queue were closing pre-maturely when using threads. |
| 1031 |
|
| 1032 |
* Fixed an undefined variable inside croak statements in MCE models' |
| 1033 |
import subroutine. |
| 1034 |
|
| 1035 |
* Fixed automatic shutdown due to loading threads and specifying |
| 1036 |
use_threads => 0 with workers persisting before exiting the script. |
| 1037 |
|
| 1038 |
[ENHANCEMENTS] |
| 1039 |
|
| 1040 |
* Moved relay methods, introduced in 1.600, from MCE to MCE::Relay. |
| 1041 |
The MCE init_relay option loads and enables MCE::Relay automatically. |
| 1042 |
|
| 1043 |
* Captured metrics from Linux (previously OS X) for forchunk.pl, |
| 1044 |
foreach.pl, and forseq.pl. Updated Examples.pod. |
| 1045 |
|
| 1046 |
* Default to 'auto' for max_workers in bin/mce_grep, examples/egrep.pl, |
| 1047 |
forchunk.pl, foreach.pl, forseq.pl, iterator.pl, and wc.pl. |
| 1048 |
|
| 1049 |
* Moved _create_socket_pair from MCE to MCE::Util as _make_socket_pair. |
| 1050 |
Added _destroy_sockets. Updated MCE, Mutex, and Queue to use MCE::Util. |
| 1051 |
|
| 1052 |
* Added a CARP_NOT line to MCE models. |
| 1053 |
* Added support for ->next and ->last from any sub-tasks in MCE::Step. |
| 1054 |
* Reverted a small change applied in 1.600 to ->do and ->gather. |
| 1055 |
* The 'synchronize' method in Mutex is wantarray aware. |
| 1056 |
* Updated POD header lines for method names; from =item to =head2. |
| 1057 |
* Workers now set STDERR and STDOUT to flush automatically. |
| 1058 |
|
| 1059 |
[NEW FEATURES] |
| 1060 |
|
| 1061 |
* Added ->pid method to MCE. |
| 1062 |
|
| 1063 |
1.600 Sat Jan 31 20:00:00 EST 2015 |
| 1064 |
|
| 1065 |
[BUG FIXES] |
| 1066 |
|
| 1067 |
* Die handlers (in MCE::Signal and MCE::Core::Worker) are finally 100%. |
| 1068 |
Furthermore, eval { die ... } statements behave correctly while running |
| 1069 |
MCE itself inside an eval block. Thus, MCE on iPerl on top of iPython |
| 1070 |
is possible via Devel::IPerl. |
| 1071 |
|
| 1072 |
* Fixed queues stalling from running (MCE::Queue fast => 1) on Linux. |
| 1073 |
|
| 1074 |
* MCE models now set $MCE::FREEZE, $MCE::THAW, and $MCE::TMP_DIR when |
| 1075 |
overriding freeze, thaw, and tmp_dir respectively at load time. |
| 1076 |
|
| 1077 |
* Pressing CTRL-D now ends STDIN the first time. |
| 1078 |
|
| 1079 |
[ENHANCEMENTS] |
| 1080 |
|
| 1081 |
* Added seven names to CREDITS; David Farrell, Demian Riccardi, |
| 1082 |
Hisham Eldai, Joel Berger, Oleksandr Kharchenko, Wei Shen, and |
| 1083 |
Zakariyya Mughal. |
| 1084 |
|
| 1085 |
* Refactored ->print, ->printf, and ->say. Optimized ->print some more. |
| 1086 |
* Shorten $_queue to $_Q in Queue. ID is always sent first during IPC. |
| 1087 |
* The init method in MCE models can now take an array of options. |
| 1088 |
|
| 1089 |
* Optimized memory consumption in Handle.pm, Iterator.pm, and Request.pm. |
| 1090 |
* Optimized memory consumption for ->sendto, ->do, and ->gather. |
| 1091 |
* Optimized memory consumption for overall IPC in general. |
| 1092 |
* Refreshed the MCE->new method. |
| 1093 |
|
| 1094 |
[NEW FEATURES] |
| 1095 |
|
| 1096 |
* Added a new module; MCE::Mutex providing simple semaphore. |
| 1097 |
|
| 1098 |
* Added ->relay_recv, ->relay, and ->relay_final methods to the Core API. |
| 1099 |
Refreshed cat.pl and findnull.pl examples to relay the number of lines |
| 1100 |
read. Workers output exclusively and orderly to STDOUT in cat.pl. |
| 1101 |
|
| 1102 |
* Added several examples; biofasta (folder), mutex.pl, and relay.pl. |
| 1103 |
The FASTA examples process by records "\n>", not by lines. |
| 1104 |
|
| 1105 |
* MCE applies additional logic when RS begins with a newline character; |
| 1106 |
e.g. RS => "\n>". It trims away characters after the newline and |
| 1107 |
prepends them to the next record. This happens automatically when not |
| 1108 |
slurping. Otherwise, the logic is applied to the first and last records |
| 1109 |
only. This is illustrated in the Core API documentation. |
| 1110 |
|
| 1111 |
* Updated the Core API documentation (RS, added relay methods). |
| 1112 |
|
| 1113 |
1.522 Thu Dec 25 16:00:00 EST 2014 |
| 1114 |
|
| 1115 |
[BUG FIXES] |
| 1116 |
|
| 1117 |
* Applied fix to MCE->shutdown so that MCE models do not err when |
| 1118 |
receiving signal to terminate. |
| 1119 |
|
| 1120 |
* Optimization for MCE->print, MCE->printf, MCE->say and MCE->sendto. |
| 1121 |
This was calling fileno unnecessarily. MCE->print('STDERR', ...) |
| 1122 |
is not supported. This works; MCE->print(\*STDERR, ...) |
| 1123 |
|
| 1124 |
[ENHANCEMENTS] |
| 1125 |
|
| 1126 |
* Inserts the actual lib-path at the head of @INC in example files. |
| 1127 |
* Massive documentation updates throughout the entire distribution. |
| 1128 |
* Renamed barrier_sync.pl to sync.pl; scaling_pings.pl to ping.pl. |
| 1129 |
* Updated requires in META.yml, Makefile.PL and perl-MCE.spec. |
| 1130 |
* Updated comment in MCE::Queue test scripts. |
| 1131 |
* Updated the README file. |
| 1132 |
|
| 1133 |
[NEW FEATURES] |
| 1134 |
|
| 1135 |
* Added sampledb examples demonstrating DBI and SQLite with MCE. |
| 1136 |
* Added step_demo.pl to examples. |
| 1137 |
|
| 1138 |
1.521 Thu Dec 11 16:00:00 EST 2014 |
| 1139 |
|
| 1140 |
[BUG FIXES] |
| 1141 |
|
| 1142 |
* Fixed broken MCE::Queue ->insert and ->peek methods. FIFO and LIFO |
| 1143 |
are fully supported with this release. |
| 1144 |
|
| 1145 |
* Support running in taint mode. |
| 1146 |
|
| 1147 |
[ENHANCEMENTS] |
| 1148 |
|
| 1149 |
* Added support for negative index in MCE::Queue ->insert and ->peek. |
| 1150 |
Updated the documentation. |
| 1151 |
|
| 1152 |
* CBOR::XS serialization is mentioned in documentation along with |
| 1153 |
JSON::XS and Sereal. |
| 1154 |
|
| 1155 |
* Completed code refactoring for the 1.5 branch. |
| 1156 |
* Optimized argument parsing in import routines. |
| 1157 |
* Removed the MCE spawn_delay option from test scripts. |
| 1158 |
|
| 1159 |
[NEW FEATURES] |
| 1160 |
|
| 1161 |
* Added 15 new test scripts for testing user_args, MCE::Queue |
| 1162 |
and MCE models. |
| 1163 |
|
| 1164 |
* An upper-limit of 8 is set when specifying max_workers => 'auto'. |
| 1165 |
Several folks have requested this. More info at MCE::Util::get_ncpu |
| 1166 |
for increasing or decreasing the limit. |
| 1167 |
|
| 1168 |
1.520 Wed Nov 05 03:00:00 EST 2014 |
| 1169 |
|
| 1170 |
[ENHANCEMENTS] |
| 1171 |
|
| 1172 |
* MCE::Step and MCE::Stream can take the 'fast' option when including |
| 1173 |
the module. The 'fast' option was introduced in 1.518. |
| 1174 |
|
| 1175 |
* Removed the type declaration for self (feedback from Sri). |
| 1176 |
* Removed -s from files_flow.pl (was left behind). |
| 1177 |
* Added support for GNU Hurd OS in get_ncpu. |
| 1178 |
|
| 1179 |
1.519 Mon Oct 27 19:00:00 EST 2014 |
| 1180 |
|
| 1181 |
[ENHANCEMENTS] |
| 1182 |
|
| 1183 |
* Unset the need for channel locking if only worker riding the channel. |
| 1184 |
There are 8 data channels in MCE. Basically, a worker will obtain a |
| 1185 |
lock only when sharing the data channel with another worker. |
| 1186 |
|
| 1187 |
* Updated files_flow.pl, files_mce.pl and files_thr.pl to allow for |
| 1188 |
many workers for the first task. Updated the synopsis in MCE::Queue. |
| 1189 |
Synced example listing in Examples.pod with the examples folder. |
| 1190 |
|
| 1191 |
* Remove period from summary line. |
| 1192 |
|
| 1193 |
1.518 Mon Oct 27 10:00:00 EST 2014 |
| 1194 |
|
| 1195 |
[BUG FIXES] |
| 1196 |
|
| 1197 |
* Corrected MCE::Queue's synopsis due to missing List::MoreUtils line. |
| 1198 |
Changed the synopsis to use 1 worker for the 'dir' task in the event |
| 1199 |
one were to copy/patse the code and use threads. The glob() function |
| 1200 |
is not thread-safe in Perl 5.16.x; fixed in 5.18.2, okay in 5.8 - 14. |
| 1201 |
|
| 1202 |
* Use portable syntax for setpgrp in MCE::Signal. This closes issue 1 |
| 1203 |
at https://code.google.com/p/many-core-engine-perl. |
| 1204 |
|
| 1205 |
[NEW FEATURES] |
| 1206 |
|
| 1207 |
* New 'fast' option for MCE::Queue. The 'fast' option speeds up ->dequeue |
| 1208 |
ops and not enabled by default. It is beneficial for queues not needing |
| 1209 |
->clear or ->dequeue_nb and not altering the optional count value |
| 1210 |
while running; e.g. ->dequeue( [ $count ] ). |
| 1211 |
|
| 1212 |
* Added three examples: files_flow.pl, files_mce.pl, and files_thr.pl. |
| 1213 |
* Benchmarked on several OSes and appended results to MCE::Queue synopsis. |
| 1214 |
|
| 1215 |
1.517 Thu Oct 23 10:00:00 EST 2014 |
| 1216 |
|
| 1217 |
[BUG FIXES] |
| 1218 |
|
| 1219 |
* Correction applied to MCE::Util::get_ncpu for Tru64 UNIX. This method |
| 1220 |
will emit a warning (not croak) whenever the OS is unknown. |
| 1221 |
|
| 1222 |
* Changed ${^CHILD_ERROR_NATIVE} to $? in examples/pipe2.pl. This was |
| 1223 |
missed in the previous release. |
| 1224 |
|
| 1225 |
[ENHANCEMENTS] |
| 1226 |
|
| 1227 |
* Added support for DragonFly BSD, SCO OpenServer 5, SCO OpenServer 6 and |
| 1228 |
SCO UnixWare 7 to MCE::Util::get_ncpu. |
| 1229 |
|
| 1230 |
* Also, validated MCE on FreeBSD, NetBSD, OpenBSD, PC-BSD and JabirOS. |
| 1231 |
No further changes to MCE::Util::get_ncpu. |
| 1232 |
|
| 1233 |
1.516 Fri Oct 03 02:00:00 EST 2014 |
| 1234 |
|
| 1235 |
[BUG FIXES] |
| 1236 |
|
| 1237 |
* Updated IPC for better stability across multiple environments. The fix |
| 1238 |
addresses an issue on Windows where sockets fail to respond after a |
| 1239 |
period of inactivity; i.e. 4 minutes. Added George Bouras to CREDITS |
| 1240 |
for reporting the issue. |
| 1241 |
|
| 1242 |
* Tip for folks developing on Windows: |
| 1243 |
Open an explorer window and go to C:\Users\<user>\AppData\Local\Temp. |
| 1244 |
Right-click on the 'mce' folder and create a shortcut on the desktop. |
| 1245 |
Although MCE removes its temp files automatically, it is possible for |
| 1246 |
files to remain from a failing app. Go inside the 'mce' folder and |
| 1247 |
press ctrl-a to select all files. Then press shift-delete; |
| 1248 |
(macbook[pro] folks: fn-shift-delete). |
| 1249 |
|
| 1250 |
[ENHANCEMENTS] |
| 1251 |
|
| 1252 |
* Corrections to documentation. |
| 1253 |
|
| 1254 |
1.515 Thu Jul 24 23:30:00 EST 2014 |
| 1255 |
|
| 1256 |
[BUG FIXES] |
| 1257 |
|
| 1258 |
* bin/mce_grep: ${^CHILD_ERROR_NATIVE} is not defined in Perl 5.8.x. |
| 1259 |
Changed to $?. |
| 1260 |
|
| 1261 |
[ENHANCEMENTS] |
| 1262 |
|
| 1263 |
* Tweaked bin/mce_grep. Compute chunk_level => 'auto' to use 'file' when |
| 1264 |
reading STDIN. Set chunk_size to 8M when not specified (4M previously). |
| 1265 |
|
| 1266 |
* Added the following names to CREDITS. |
| 1267 |
|
| 1268 |
Stephan Kulow ; for making the OpenSUSE MCE package |
| 1269 |
Henry Lu ; for listening while I chatted away about MCE |
| 1270 |
Jillian Rowe ; for reporting IO::File failing as input_data |
| 1271 |
Sylvia Roy ; for driving while I worked in the passenger seat |
| 1272 |
Tom Sasser ; for reporting bin/mce_grep failing with Perl 5.8.x |
| 1273 |
Florian Schlichting ; for making the Debian MCE package |
| 1274 |
|
| 1275 |
[NEW FEATURES] |
| 1276 |
|
| 1277 |
* Added support for IO::File handles as valid input_data including |
| 1278 |
IO:Uncompress:Gunzip. |
| 1279 |
|
| 1280 |
Modified SYNTAX section for INPUT_DATA in MCE::Core.pod. |
| 1281 |
|
| 1282 |
input_data => $fh, ## new IO::File "file", "r" |
| 1283 |
input_data => $fh, ## new IO::Uncompress::Gunzip "file.gz" |
| 1284 |
|
| 1285 |
1.514 Thu Jun 05 09:00:00 EST 2014 |
| 1286 |
|
| 1287 |
[BUG FIXES] |
| 1288 |
|
| 1289 |
* Fixed typo in MCE::Step POD (RT#95250) (Florian Schlichting). |
| 1290 |
* Updated MCE::Util's get_ncpu function for AIX (Dana Jacobsen). |
| 1291 |
* Do not send a KILL signal after receiving a SIGPIPE. |
| 1292 |
* Fixed issue with Makefile.PL on defining the minimum Perl version. |
| 1293 |
|
| 1294 |
* Use Scalar::Util (looks_like_number) for validation logic. The regex |
| 1295 |
previously were insufficient for large numbers containing scientific |
| 1296 |
notation. This impacted the sequence option in MCE. |
| 1297 |
|
| 1298 |
[ENHANCEMENTS] |
| 1299 |
|
| 1300 |
* The mce-sandbox demo has been released on GitHub demonstrating |
| 1301 |
Perl + MCE + Inline::C. The theme is Prime Numbers. This journey |
| 1302 |
which began 2 years ago has been completed. There is also the |
| 1303 |
mce-sort exercise demonstrating Perl + MCE + External C. |
| 1304 |
|
| 1305 |
https://github.com/marioroy/mce-sandbox (is thread-safe) |
| 1306 |
https://github.com/marioroy/mce-sort (not thread-safe) |
| 1307 |
|
| 1308 |
* A hard decision had to made for MCE, particularly Perl under Windows |
| 1309 |
excluding Cygwin. MCE will now load the 'threads' module automatically |
| 1310 |
for Windows only. Folks may specify use_threads => 0 if threads is |
| 1311 |
not desired. |
| 1312 |
|
| 1313 |
The reason for this is from seeing Math::Prime::Util crashing once |
| 1314 |
workers exit. The same is true without MCE and forking a child process. |
| 1315 |
Threads does not exhibit this behavior. It is a hard problem to solve. |
| 1316 |
Why not default to threads for Windows since forking is emulated. |
| 1317 |
|
| 1318 |
1.513 Sat Apr 19 20:30:00 EST 2014 |
| 1319 |
|
| 1320 |
[BUG FIXES] |
| 1321 |
|
| 1322 |
* Added fix for bug RT#94869 -- crash when restarting workers with 9+ |
| 1323 |
workers. Updated the perldoc for restart_worker in MCE::Core.pod. |
| 1324 |
|
| 1325 |
[ENHANCEMENTS] |
| 1326 |
|
| 1327 |
* Replaced $self with $mce in MCE::/Core.pod to be consistent with |
| 1328 |
examples described in MCE::Examples.pod. |
| 1329 |
|
| 1330 |
1.512 Fri Apr 18 21:00:00 EST 2014 |
| 1331 |
|
| 1332 |
[BUG FIXES] |
| 1333 |
|
| 1334 |
* Signal-handling update for MCE::Signal. Passing the -setpgrp option is not |
| 1335 |
necessary, even with Daemon::Control. Piping data into and out is better |
| 1336 |
supported with this release (\*STDIN). This resolves bug RT#94706. |
| 1337 |
|
| 1338 |
cat infile | mce_script | head |
| 1339 |
mce_script < infile | head |
| 1340 |
|
| 1341 |
Added Shawn Halpenny to the CREDITS file. |
| 1342 |
|
| 1343 |
* The utf8.pl example now runs under the BSD 9.0 environment. This was |
| 1344 |
failing due to $^H{charnames} is not defined error. Removed the constant |
| 1345 |
from the list of unicode characters inside the script. |
| 1346 |
|
| 1347 |
[ENHANCEMENTS] |
| 1348 |
|
| 1349 |
* Added examples/pipe1.pl and pipe2.pl. These process STDIN or FILE in |
| 1350 |
parallel. Processing is via Perl for pipe1.pl, whereas an external |
| 1351 |
command for pipe2.pl. |
| 1352 |
|
| 1353 |
1.511 Fri Apr 04 22:30:00 EST 2014 |
| 1354 |
|
| 1355 |
[BUG FIXES] |
| 1356 |
|
| 1357 |
* Added "use bytes;" in several files to have length() return physical |
| 1358 |
bytes, not logical characters. MCE is now UTF-8 safe when passing scalar |
| 1359 |
data between workers and the manager process. |
| 1360 |
|
| 1361 |
Added Marcus Smith to the CREDITS file for reporting this bug. |
| 1362 |
|
| 1363 |
[ENHANCEMENTS] |
| 1364 |
|
| 1365 |
* Added examples/utf8.pl |
| 1366 |
|
| 1367 |
1.510 Thu Mar 27 10:00:00 EST 2014 |
| 1368 |
|
| 1369 |
[ENHANCEMENTS] |
| 1370 |
|
| 1371 |
* The user_begin and user_end functions now receive 3 arguments. |
| 1372 |
my ($mce, $task_id, $task_name) = @_; |
| 1373 |
|
| 1374 |
* Pass the chunk_size value when calling the iterator function. Added a |
| 1375 |
DBI example under "SYNTAX for INPUT_DATA" in MCE::Core.pod. |
| 1376 |
|
| 1377 |
* Store the last scalar reference to not have workers re-spawn |
| 1378 |
unnecessarily when input_data => \$same_scalar_ref. |
| 1379 |
|
| 1380 |
[NEW FEATURES] |
| 1381 |
|
| 1382 |
* New parallel_io option to further enhance slurp IO when specifying |
| 1383 |
use_slurpio => 1 and chunk_size is greater than 8192. Try with |
| 1384 |
chunk_size => '300k' or chunk_size => '2m'. |
| 1385 |
|
| 1386 |
The parallel_io option is beneficial when reading from fast storage. |
| 1387 |
However, possibly not recommended if running MCE on many compute nodes |
| 1388 |
and having workers read various input_data from shared storage. |
| 1389 |
|
| 1390 |
Enable parallel_io only if it makes sence and without impacting the |
| 1391 |
environment such as nfsd. Because use_slurpio => 1, parallel_io => 1, |
| 1392 |
and chunk_size => '2560k' is truly parallel. |
| 1393 |
|
| 1394 |
1.509 Sat Feb 03 05:30:00 EST 2014 |
| 1395 |
|
| 1396 |
[BUG FIXES] |
| 1397 |
|
| 1398 |
* Fixed an issue with all the models (Flow, Grep, Loop, Map, Step, and |
| 1399 |
Stream) ending immediately on subsequent runs when input_data is |
| 1400 |
specified through the init method. |
| 1401 |
|
| 1402 |
* Things have finally settled down with all the models. |
| 1403 |
|
| 1404 |
[ENHANCEMENTS] |
| 1405 |
|
| 1406 |
* Refactored the fix applied in 1.508 for addressing bug #92627. |
| 1407 |
|
| 1408 |
* MCE, for the most part, has been completed. This release touches up on |
| 1409 |
many examples under the 'examples' directory. |
| 1410 |
|
| 1411 |
* Brought the MCE::Examples Perl documentation up to date. |
| 1412 |
Inlined MCE::Loop snippets under the chunking sections. |
| 1413 |
Added a new section GLOBALLY SCOPED VARIABLES AND MCE MODELS. |
| 1414 |
Added a new section MONTE CARLO SIMULATION. |
| 1415 |
|
| 1416 |
1.508 Sat Feb 01 04:00:00 EST 2014 |
| 1417 |
|
| 1418 |
[BUG FIXES] |
| 1419 |
|
| 1420 |
* Applied fix for bug #92627 submitted by Philip Mabon. MCE scripts may |
| 1421 |
exit with a non zero error code due to leaving workers up thereby |
| 1422 |
having MCE performing the shut down in its END block. |
| 1423 |
|
| 1424 |
For folks running on an older release, the workaround is to shut down |
| 1425 |
workers prior to exiting. |
| 1426 |
|
| 1427 |
1.507 Fri Jan 30 23:00:00 EST 2014 |
| 1428 |
|
| 1429 |
[ENHANCEMENTS] |
| 1430 |
|
| 1431 |
* Added clarity around preserving output order for several models; |
| 1432 |
MCE::Flow, MCE::Loop, and MCE::Step. |
| 1433 |
|
| 1434 |
* Inlined comments in the code for overriding MCE options when using |
| 1435 |
an anonymous hash for the first argument; MCE::Flow, MCE::Step, and |
| 1436 |
MCE::Stream. |
| 1437 |
|
| 1438 |
* Removed unnecessary code due to one not able to pass MCE options other |
| 1439 |
than through the init method for MCE::Loop, MCE::Grep, and MCE::Map. |
| 1440 |
|
| 1441 |
* Added to doc a use-case example searching a large file with mce_grep_f. |
| 1442 |
The emphasis is comparing the memory consumption against the native |
| 1443 |
grep function as well as time to complete. |
| 1444 |
|
| 1445 |
1.506 Fri Jan 30 03:00:00 EST 2014 |
| 1446 |
|
| 1447 |
[BUG FIXES] |
| 1448 |
|
| 1449 |
* Fixed issue with all 5 models croaking for an internal '_file' option. |
| 1450 |
This has been broken since the 1.502 release. Methods impacted were: |
| 1451 |
mce_flow_f, mce_grep_f, mce_loop_f, mce_map_f, and mce_stream_f |
| 1452 |
|
| 1453 |
* Specifying a different value for gather on subsequent runs is now taking |
| 1454 |
effect for when workers persist after running. The models impacted are |
| 1455 |
MCE::Flow and MCE::Loop. |
| 1456 |
|
| 1457 |
* Updated _parse_chunk_size in MCE::Util to compute chunk_size correctly |
| 1458 |
for new edge cases. One may call either mce_grep or mce_grep_f for a |
| 1459 |
GLOB or scalar reference as input data. The same also applies for the |
| 1460 |
other models; mce_flow, mce_loop, mce_map and mce_stream. |
| 1461 |
|
| 1462 |
[ENHANCEMENTS] |
| 1463 |
|
| 1464 |
* Updates to MCE::Grep and MCE::Map. The logic is now aware of wantarray |
| 1465 |
for faster processing when storing to a scalar value. In addition, the |
| 1466 |
use_slurpio option is enabled for efficient IO when processing large |
| 1467 |
files. |
| 1468 |
|
| 1469 |
* The task_end option can now be specified for MCE::Stream, although being |
| 1470 |
used internally. |
| 1471 |
|
| 1472 |
* The interval option can take a decimal number. Previously, this |
| 1473 |
wanted a hash reference with up to 3 key/value pairs. Most often, all |
| 1474 |
one needed was delay and not max_nodes or node_id. |
| 1475 |
|
| 1476 |
interval => { delay => 0.05 } ## Choose either format for specifying |
| 1477 |
interval => 0.05 ## delay in 1.506 and above. MCE will |
| 1478 |
## translate 0.05 to { delay => 0.05 } |
| 1479 |
## automatically. |
| 1480 |
|
| 1481 |
* Slight optimization to get_ncpu (public method) in MCE::Util. |
| 1482 |
|
| 1483 |
* Tweak to _parse_chunk_size (private method) in MCE::Util. |
| 1484 |
|
| 1485 |
[NEW FEATURES] |
| 1486 |
|
| 1487 |
* Added a new model MCE::Step for transparent use of MCE::Queue when passing |
| 1488 |
data among sub-tasks. MCE::Step is basically a spin off from MCE::Flow |
| 1489 |
with a touch of MCE::Stream. |
| 1490 |
|
| 1491 |
This new model is crazy :) |
| 1492 |
|
| 1493 |
1.505 Thu Jan 21 01:30:00 EST 2014 |
| 1494 |
|
| 1495 |
[BUG FIXES] |
| 1496 |
|
| 1497 |
* Delete $self->{input_data} inside the worker immediately during spawning |
| 1498 |
(if input_data is an ARRAY, GLOB, or Iterator reference). |
| 1499 |
|
| 1500 |
* Reverted the logic for the RS (record separator) option from the 1.4x |
| 1501 |
code base. This is now working as expected. |
| 1502 |
|
| 1503 |
* Specifying chunk_size => 'auto' via the 'init' method (all 5 models) |
| 1504 |
causes MCE validation to croak due to 'auto' being carried over to the |
| 1505 |
core API. Note that only the 5 models are allowed 'auto' for chunk_size |
| 1506 |
as of this time. |
| 1507 |
|
| 1508 |
* Update to the MCE->do method. Previously, an undef sent back from the |
| 1509 |
callback function ended up as a blank value "". Both "" and undef are |
| 1510 |
now properly captured and sent back to the worker process. |
| 1511 |
|
| 1512 |
* Added an if statement inside the 'abort' method. |
| 1513 |
* Removed an old unlink _store.db statement (old code left behind). |
| 1514 |
* Removed an unused _next variable inside Request.pm. |
| 1515 |
|
| 1516 |
[ENHANCEMENTS] |
| 1517 |
|
| 1518 |
* Enhanced egrep.pl to handle additional options including recursion. |
| 1519 |
This script now supports many egrep options [ceHhiLlmnqRrsv]. |
| 1520 |
|
| 1521 |
* IO performance for examples/cat.pl was improved. The real focus here is |
| 1522 |
demonstrating output order. |
| 1523 |
|
| 1524 |
* The chunk_size option can take a suffix; K (Kilobytes) or M (Megabytes). |
| 1525 |
|
| 1526 |
* The following examples except --max-workers=NUM --chunk-size=NUM options. |
| 1527 |
cat.pl, egrep.pl, findnull.pl, scaling_pings.pl, and wc.pl |
| 1528 |
|
| 1529 |
[NEW FEATURES] |
| 1530 |
|
| 1531 |
* The input_data option can now receive an iterator reference. Added a |
| 1532 |
new example iterator.pl for demonstraton. In addition, there are |
| 1533 |
several examples listed under a new section "SYNTAX for INPUT_DATA" |
| 1534 |
under MCE::Core.pod. |
| 1535 |
|
| 1536 |
* Added a new demo script; bin/mce_grep. This is a wrapper script for the |
| 1537 |
grep binary. This script supports agrep, grep, egrep, fgrep, & tre-agrep. |
| 1538 |
Simply create a link to mce_grep or make a copy. Both Windows and Cygwin |
| 1539 |
are supported as well. Recursion works for all binaries including agrep |
| 1540 |
(-R, -r options). |
| 1541 |
|
| 1542 |
ln mce_grep mce_agrep |
| 1543 |
ln mce_grep mce_egrep |
| 1544 |
ln mce_grep mce_fgrep |
| 1545 |
ln mce_grep mce_tre-agrep |
| 1546 |
|
| 1547 |
Try with the --lang=C option for faster execution time (-i runs faster). |
| 1548 |
Try mce_agrep or mce_tre-agrep against very large files. The speedup is |
| 1549 |
linear and makes good utilization of all available cores on the box. |
| 1550 |
|
| 1551 |
One may specify the chunking level via the --chunk-level option. For large |
| 1552 |
files, specify 'file' (chunks file). For many small files, use 'list'. |
| 1553 |
|
| 1554 |
I have tested against the following GnuWin32 packages found at this URL. |
| 1555 |
http://gnuwin32.sourceforge.net/packages.html |
| 1556 |
|
| 1557 |
tre-0.7.5-bin.zip ## Contains agrep.exe (also runs under Cygwin) |
| 1558 |
grep-2.5.4-bin.zip ## Contains egrep.exe, fgrep.exe, grep.exe |
| 1559 |
libiconv-1.9.2-1-bin.zip ## These are required for the GnuWin32 binaries. |
| 1560 |
libintl-0.14.4-bin.zip |
| 1561 |
pcre-7.0-bin.zip |
| 1562 |
regex-2.7-bin.zip |
| 1563 |
|
| 1564 |
Btw, bin/mce_grep is optional and therefore not installed by default |
| 1565 |
when running make install for the MCE module. Simply copy mce_grep as |
| 1566 |
mce_agrep.pl for Windows. The '.pl' suffix is optional for other |
| 1567 |
environments. |
| 1568 |
|
| 1569 |
1.504 Tue Oct 29 16:00:00 EST 2013 |
| 1570 |
|
| 1571 |
[BUG FIXES] |
| 1572 |
|
| 1573 |
* MCE::Grep, MCE::Map, and MCE::Stream were failing for mce_grep_s, |
| 1574 |
mce_map_s, and mce_stream_s when specifying chunk_size => 1. |
| 1575 |
|
| 1576 |
[ENHANCEMENTS] |
| 1577 |
|
| 1578 |
* Calibrated 'auto' slightly in MCE::Util::_parse_chunk_size. |
| 1579 |
|
| 1580 |
1.503 Mon Oct 28 17:00:00 EST 2013 |
| 1581 |
|
| 1582 |
[BUG FIXES] |
| 1583 |
|
| 1584 |
* The previous release introduced a bug by removing the line closing |
| 1585 |
STDERR and STDOUT prior to workers exiting. The proper fix intended |
| 1586 |
is to flush (not close) the handles. Closing them was not the thing |
| 1587 |
to do in the first place due to possibly needed by the END block or |
| 1588 |
WARN/DIE handlers initiated from inside the END block. |
| 1589 |
|
| 1590 |
[ENHANCEMENTS] |
| 1591 |
|
| 1592 |
* Updated README under examples/tbray and examples/matmult. |
| 1593 |
* Changed 0.499 to 0.5 inside the yield method. |
| 1594 |
* Small cosmetic changes otherwise. |
| 1595 |
|
| 1596 |
1.502 Mon Oct 21 16:00:00 EST 2013 |
| 1597 |
|
| 1598 |
[BUG FIXES] |
| 1599 |
|
| 1600 |
* The END block for all models will return immediately when called by a |
| 1601 |
worker thread or process. Removed the line closing STDERR and STDOUT |
| 1602 |
prior to workers exiting. It turns out that workers were calling |
| 1603 |
shutdown which is not allowed. This impacted MCE::Flow, MCE::Grep, |
| 1604 |
MCE::Loop, MCE::Map, and MCE::Stream. |
| 1605 |
|
| 1606 |
* An update was applied to DESTROY in MCE::Queue to address an edge |
| 1607 |
case during additional testing across several environments. |
| 1608 |
|
| 1609 |
* Added documentation describing the core methods in MCE. I had moved the |
| 1610 |
missing sections to another file some time back and totally forgotten |
| 1611 |
about them when finalizing on MCE::Core.pod for the 1.5 release. |
| 1612 |
|
| 1613 |
[ENHANCEMENTS] |
| 1614 |
|
| 1615 |
* All models will croak when specifying an invalid MCE option. |
| 1616 |
|
| 1617 |
[NEW FEATURES] |
| 1618 |
|
| 1619 |
* One can specify a hash reference for the gather option. Updated |
| 1620 |
documentation describing the gather option and the gather method. |
| 1621 |
|
| 1622 |
1.501 Wed Oct 16 01:20:00 EST 2013 |
| 1623 |
|
| 1624 |
[BUG FIXES] |
| 1625 |
|
| 1626 |
* Addressed an issue with the worker signal "die" handler (#89538). Eval'd |
| 1627 |
code should not raise an exception causing the app to die. This is working |
| 1628 |
as expected for workers spawned as children. |
| 1629 |
|
| 1630 |
For threads, a die called explicitly inside an eval block will cause the |
| 1631 |
worker to exit (same as previously). However, an eval 'use MissingModule' |
| 1632 |
will no longer cause the thread to die. |
| 1633 |
|
| 1634 |
[ENHANCEMENTS] |
| 1635 |
|
| 1636 |
* Changelog section uses a new format beginning with this release. |
| 1637 |
|
| 1638 |
[NEW FEATURES] |
| 1639 |
|
| 1640 |
* Added examples/flow_model.pl for demonstrating MCE::Flow, MCE::Queue, and |
| 1641 |
MCE->gather. |
| 1642 |
|
| 1643 |
1.500 Thu Oct 10 01:00:00 EST 2013 |
| 1644 |
|
| 1645 |
* MCE 1.5 is backwards compatible with 1.4 and below. |
| 1646 |
|
| 1647 |
* The documentation, previously MCE.pod, moved to MCE::Core.pod with the |
| 1648 |
examples section placed in MCE::Examples.pod. The MCE.pod file serves |
| 1649 |
as an index page for the various documentation. |
| 1650 |
|
| 1651 |
* IPC has been enhanced with 8 data channels. Many operations run 3x when |
| 1652 |
compared with MCE 1.4. |
| 1653 |
|
| 1654 |
* Five models: MCE::Flow, MCE::Grep, MCE::Loop, MCE::Map, and MCE::Stream. |
| 1655 |
|
| 1656 |
* Hybrid queues via MCE::Queue allowing for normal and priority queues. |
| 1657 |
|
| 1658 |
* MCE::Subs for exporting functions prefixed with mce_; e.g. mce_wid. |
| 1659 |
|
| 1660 |
* All public methods can be called directly using the package name |
| 1661 |
and method e.g. MCE->wid, MCE->run. |
| 1662 |
|
| 1663 |
MCE->new( |
| 1664 |
max_workers => 'auto', |
| 1665 |
user_func => sub { |
| 1666 |
my $wid = MCE->wid; |
| 1667 |
MCE->sendto("STDOUT", "Hello from $wid\n"); |
| 1668 |
} |
| 1669 |
); |
| 1670 |
|
| 1671 |
MCE->run; |
| 1672 |
|
| 1673 |
* Localize the input scalar $_ prior to calling user_func. Folks can |
| 1674 |
use $_ for input_data and sequence of numbers. Added section to docs |
| 1675 |
explaining DEFAULT INPUT SCALAR. |
| 1676 |
|
| 1677 |
* New options (bounds_only, gather, interval, task_name). |
| 1678 |
|
| 1679 |
The task_end option can now be specified at the top level. |
| 1680 |
See docs for use case. |
| 1681 |
|
| 1682 |
Input_data can be specified inside the first task instead of having |
| 1683 |
to specify this at the top level. Input_data is ignored when specified |
| 1684 |
for tasks other than the first task. |
| 1685 |
|
| 1686 |
user_tasks => [{ |
| 1687 |
input_data => \@list, |
| 1688 |
... |
| 1689 |
},{ |
| 1690 |
... |
| 1691 |
} |
| 1692 |
|
| 1693 |
* New public methods |
| 1694 |
chunk_id, gather, freeze, thaw, yield, task_name, print, printf, say |
| 1695 |
|
| 1696 |
* New example (interval.pl). |
| 1697 |
|
| 1698 |
* Optimized the egrep.pl and wc.pl examples. These run much faster. |
| 1699 |
Try these out against large log files. Both examples fly. |
| 1700 |
|
| 1701 |
* Barrier synchronization update. Two sockets are utilized instead of |
| 1702 |
2 lock files. This, now works wonderfully under the Cygwin environment. |
| 1703 |
For threading, the removal of 2 lock files increases the number of |
| 1704 |
threads allowed from about 1/3rd previously to under 1/2 of ulimit, |
| 1705 |
e.g. int(ulimit -n / 2 - 20). |
| 1706 |
|
| 1707 |
* Removed the logic for determining MAX_OPEN_FILES and MAX_USER_PROCS. |
| 1708 |
MCE no longer has a constraint on max_workers allowed. |
| 1709 |
|
| 1710 |
* Code re-factor work. Added a private method _validate_runstate called |
| 1711 |
by various methods. Organized the code slightly such as placement of |
| 1712 |
methods. |
| 1713 |
|
| 1714 |
1.415 Mon Jun 17 15:00:00 EST 2013 |
| 1715 |
|
| 1716 |
* Code-refactor in preparation for the upcoming 1.5 release. |
| 1717 |
This completes the 1.4x branch for a very solid and stable release. |
| 1718 |
|
| 1719 |
* IPC optimization. Localize $\ and $/ only as needed. Small tweak to |
| 1720 |
logic when specifying RS (record separator) for input data. |
| 1721 |
|
| 1722 |
Foreach is very communication intensive. Forseq is less so. Chunking |
| 1723 |
was made faster by increasing chunk size from 500 to 2500. I've reached |
| 1724 |
my goal for forseq by reaching 60,000+. A while back, foreach could |
| 1725 |
barely reach 18,000. |
| 1726 |
|
| 1727 |
Before and after results with IPC optimization. |
| 1728 |
|
| 1729 |
(Before) (After) |
| 1730 |
Parallel::Loops 600 600 |
| 1731 |
$mce->foreach 20,000 21,500 (+ 1,500) |
| 1732 |
$mce->forseq 55,000 64,000 (+ 9,000) |
| 1733 |
$mce->forchunk 395,000 450,000 (+ 55,000) |
| 1734 |
|
| 1735 |
* Updated the process method. Allows sequence to be specified as an |
| 1736 |
option. Also allow the hash to be specified as the 1st or 2nd |
| 1737 |
argument. |
| 1738 |
|
| 1739 |
$mce->process( \@input_array, { options } ); |
| 1740 |
$mce->process( { options }, [ 1..1000 ] ); |
| 1741 |
$mce->process( { sequence => [ 20, 40, 2 ] } ); |
| 1742 |
|
| 1743 |
* The forseq method will now honor chunk_size greater than 1. Updated |
| 1744 |
docs with use case. |
| 1745 |
|
| 1746 |
* Added CONST short for EXPORT_CONST, e.g. use MCE CONST => 1; |
| 1747 |
|
| 1748 |
1.414 Sun Jun 16 13:00:00 EST 2013 |
| 1749 |
|
| 1750 |
* Upstream bug fixes. Mainly chunk_size was not honored when specifying |
| 1751 |
user_tasks and sequence/chunk_size for the first task. Bug is with |
| 1752 |
not setting abort_msg correctly. |
| 1753 |
|
| 1754 |
1.413 Mon Jun 10 02:00:00 EST 2013 |
| 1755 |
|
| 1756 |
* Fixed typo: Changed local @_ to local $@ inside _parse_max_workers |
| 1757 |
before evaling. |
| 1758 |
|
| 1759 |
1.412 Sun Jun 09 14:00:00 EST 2013 |
| 1760 |
|
| 1761 |
* Updated the main README file and CHANGES under 1.411 below. |
| 1762 |
|
| 1763 |
* Added support for multiplication and division as well as |
| 1764 |
case insentivity to the private _parse_max_workers method. |
| 1765 |
|
| 1766 |
max_workers => 'auto', ## = MCE::Util::get_ncpu() |
| 1767 |
max_workers => 'Auto-1', ## = MCE::Util::get_ncpu() - 1 |
| 1768 |
max_workers => 'AUTO + 3', ## = MCE::Util::get_ncpu() + 3 |
| 1769 |
max_workers => 'AUTO * 1.5', |
| 1770 |
max_workers => 'auto / 1.333', |
| 1771 |
|
| 1772 |
1.411 Sun Jun 09 02:00:00 EST 2013 |
| 1773 |
|
| 1774 |
* Reverted the change made in MCE-1.410. That degraded slurp IO on |
| 1775 |
large files. This restores slurp IO performance in MCE. |
| 1776 |
|
| 1777 |
* Corrected an if statement towards the end of the shutdown method. |
| 1778 |
The delay was occurring always when it was meant only if MCE was |
| 1779 |
launched from inside a non-main thread. |
| 1780 |
|
| 1781 |
* Addressed a race condition with barrier synchronization under the |
| 1782 |
Cygwin environment. Was not able to reproduce this with other OS's. |
| 1783 |
This takes barrier synchronization work to 100%. Delay statements |
| 1784 |
for Cygwin were removed around barrier synchronization. |
| 1785 |
|
| 1786 |
* Optimized writes to sockets. Arguments passed to print statements |
| 1787 |
are concatenated as one big string. Removed local $/ = $LF when |
| 1788 |
reading from the queue socket. It's not required there since using |
| 1789 |
read and known size. The foreach.pl example (very communication |
| 1790 |
intensive) can now do 20000 in 1 second on my Macbook Pro. Updated |
| 1791 |
benchmark results inside foreach.pl, forchunk.pl, and forseq.pl. |
| 1792 |
|
| 1793 |
* MAX_WORKERS, CHUNK_SIZE, TMP_DIR, FREEZE and THAW can be specified |
| 1794 |
when loading the module. FREEZE and THAW allows one to choose an |
| 1795 |
alternative serialization module if preferred for your project. |
| 1796 |
|
| 1797 |
use Sereal qw(encode_sereal decode_sereal); |
| 1798 |
use MCE FREEZE => \&encode_sereal, THAW => \&decode_sereal; |
| 1799 |
|
| 1800 |
* Passing EXPORT_CONST => 1 will export 3 constants: SELF, CHUNK, CID |
| 1801 |
The "my ($self, $chunk_ref, $chunk_id) = @_" line is not necessary. |
| 1802 |
$_[SELF], $_[CHUNK], $_[CID] refers to $_[0], $_[1], $_[2]. |
| 1803 |
|
| 1804 |
* The MCE::Util module was created. It contains the get_ncpu function. |
| 1805 |
This is largely borrowed from Test::Smoke::Util.pm. MCE supports |
| 1806 |
'auto' when specifying max_workers. Read doc for other use case. |
| 1807 |
|
| 1808 |
max_workers => 'auto'; ## Same as MCE::Util::get_ncpu() |
| 1809 |
|
| 1810 |
* Re-factored code in preparation for the upcoming 1.5 release. The |
| 1811 |
1.5 release will use just 3 socket pairs versus 4. I did not want |
| 1812 |
to include that change in this release. Therefore, I decided to |
| 1813 |
take the upstream bug fixes and apply them to the 1.4 base. |
| 1814 |
|
| 1815 |
Re-factored documentation by making use of the full 78 character |
| 1816 |
width per line. |
| 1817 |
|
| 1818 |
* Updated the main README file under the top level dir. |
| 1819 |
|
| 1820 |
1.410 Tue May 28 23:30:00 EST 2013 |
| 1821 |
|
| 1822 |
* Use threads under MSWin32 for 02* and 03* test scripts. |
| 1823 |
* Removed sysopen/sysseek/sysread to simplify logic due to negligible |
| 1824 |
performance gains over open/seek/read. |
| 1825 |
* Minor updates to documentation. |
| 1826 |
|
| 1827 |
1.409 Sun May 12 22:45:00 EST 2013 |
| 1828 |
|
| 1829 |
* Croak if user_func is not defined and input_data/sequence is specified. |
| 1830 |
* Fix barrier synchronization when running multiple tasks via user_tasks. |
| 1831 |
* Updated Perl documentation for easier reading through cpan/metapan. |
| 1832 |
* Renamed private method _worker_sequence to _worker_sequence_queue. |
| 1833 |
|
| 1834 |
1.408 Tue Mar 19 22:00:00 EST 2013 |
| 1835 |
|
| 1836 |
* Minor tweaks here and there to further increase reliability. |
| 1837 |
* Updated the barrier synchronization logic to not stall. Updated the |
| 1838 |
perl docs on mixing "sync" with "do" or "sendto" methods. |
| 1839 |
* Added new "status" method for the manager process. |
| 1840 |
* Added new arguments for MCE::Signal: -no_kill9, -no_sigmsg |
| 1841 |
|
| 1842 |
1.407 Thu Mar 14 21:00:00 EST 2013 |
| 1843 |
|
| 1844 |
* This marks a tremendous effort in achieving parity across the board |
| 1845 |
from Cygwin to Windows and obviously UNIX. MCE now works beautifully |
| 1846 |
under the Cygwin environment including Perl for Windows. |
| 1847 |
* Up'ed the maximum workers allowed for both Cygwin and Windows to |
| 1848 |
56 (forking) and 80 (threading). |
| 1849 |
* Barrier synchronization requires an extra semaphore file, therefore |
| 1850 |
the maximum workers allowed under the UNIX environment for threading |
| 1851 |
decreased. It shouldn't be a problem as there are many threads already. |
| 1852 |
* Addressed an issue with barrier synchronization under Cygwin. |
| 1853 |
* Addressed an issue with the die handler for the main worker method |
| 1854 |
when threading. |
| 1855 |
* Thank you for hanging in there. It took quite some time to get there. |
| 1856 |
This is the most stable release thus far for MCE across the board. |
| 1857 |
|
| 1858 |
1.406 Tue Mar 12 19:00:00 EST 2013 |
| 1859 |
|
| 1860 |
* Added support for barrier synchronization (via new sync method). |
| 1861 |
Added barrier_sync.pl example. |
| 1862 |
* Addressed rounding errors with the sequence generator. |
| 1863 |
The sequence option now follows a bank-teller queuing model when |
| 1864 |
generating numbers. This applies to task 0 only. Other tasks can |
| 1865 |
still specify sequence where numbers will be distributed equally |
| 1866 |
among workers like before. |
| 1867 |
* Optimized the _worker_request_chunk private method. |
| 1868 |
* A take 2 on the matrix multiplication examples. This is better |
| 1869 |
organized with updated README file to include the script running |
| 1870 |
time in the results. |
| 1871 |
|
| 1872 |
1.405 Mon Mar 04 19:00:00 EST 2013 |
| 1873 |
|
| 1874 |
* Added strassen_pdl_t.pl in the event folks cannot make use of /dev/shm |
| 1875 |
used by the strassen_pdl_s.pl example. |
| 1876 |
* Optimized the 'send' method -- workers process immediately after receiving |
| 1877 |
data. Updated run times in README for the strassen examples. |
| 1878 |
* MCE no longer calls setpgrp by default as of MCE 1.405. There is only |
| 1879 |
one reason to call setpgrp, but many reasons not to. The sole reason |
| 1880 |
was for MCE to run correctly with Daemon::Control. If needed, one can |
| 1881 |
pass the option to MCE::Signal qw(-setpgrp). |
| 1882 |
* Return void in the shutdown method (previously was returning $self). |
| 1883 |
* Tidy code inside sequence generator. |
| 1884 |
|
| 1885 |
1.404 Sun Feb 24 13:00:00 EST 2013 |
| 1886 |
|
| 1887 |
* Added sess_dir method |
| 1888 |
* Completed work with matmult/* examples |
| 1889 |
Added matmult_pdl_q.pl, removed strassen_pdl_h.pl |
| 1890 |
Added strassen_pdl_o/p/q/r/s.pl |
| 1891 |
Added benchmark results from a 32-way box at the end of the readme |
| 1892 |
* Removed lines setting max limit for files/procs |
| 1893 |
|
| 1894 |
1.403 Sun Feb 17 15:00:00 EST 2013 |
| 1895 |
|
| 1896 |
* Wrap sub PDL::CLONE_SKIP into a no warnings 'redefine' block |
| 1897 |
MCE now works with PDL::Parallel::threads without any warnings |
| 1898 |
* Added missing examples/matmult/matmult_pdl_n.pl to MANIFEST |
| 1899 |
* Refactored strassen examples, memory consumption reduced by > than 50% |
| 1900 |
* Added matmult_pdl_o.pl -- uses PDL::Parallel::threads to share matrices |
| 1901 |
* Added matmult_pdl_p.pl -- matrix b is read from shared memory, not mmap |
| 1902 |
* Added strassen_pdl_n.pl -- additional improvements to memory reduction |
| 1903 |
* Added strassen_pdl_h.pl -- shown running with 4 workers (half and half) |
| 1904 |
* Re-ran matrix multiplication examples and updated results in README file |
| 1905 |
* Added -no_setpgrp option to MCE::Signal.pm |
| 1906 |
Ctrl-C does not respond when running /usr/bin/time mce_script.pl |
| 1907 |
* Added undef $buffer in a couple of places within MCE.pm |
| 1908 |
* Added David Mertens and Adam Sjøgren to CREDITS |
| 1909 |
* The 'send' method now checks if sending > total workers after spawning |
| 1910 |
not before |
| 1911 |
|
| 1912 |
1.402 Thu Feb 14 07:30:00 EST 2013 |
| 1913 |
|
| 1914 |
* Updated matrix multiplication examples including README |
| 1915 |
* Added examples/matmult/matmult_pdl_n.pl |
| 1916 |
|
| 1917 |
1.401 Tue Feb 12 19:00:00 EST 2013 |
| 1918 |
|
| 1919 |
* Added sub PDL::CLONE_SKIP { 1 } to MCE.pm. Running PDL + MCE threads no |
| 1920 |
longer crashes during exiting. |
| 1921 |
* Updated matrix multiplication examples. All examples now work under the |
| 1922 |
Windows environment no matter if threading or forking. Unix is stable as |
| 1923 |
well if wanting to use PDL + MCE and use_threads => 1 or 0. |
| 1924 |
* Added benchmark results for 2048x2048, 4096x4096, and 8192x8192 to the |
| 1925 |
README file under examples/matmult/ |
| 1926 |
* Updated documentation |
| 1927 |
|
| 1928 |
1.400 Mon Feb 11 07:00:00 EST 2013 |
| 1929 |
|
| 1930 |
* Slight optimization in the _do_callback method |
| 1931 |
* Added 2 new options: user_args and RS (record separator) |
| 1932 |
* Added new send method for sending data to workers after spawning and |
| 1933 |
prior to running |
| 1934 |
* The sequence option can now take an ARRAY reference |
| 1935 |
* Updated documentation on new features |
| 1936 |
* Added matrix multiplication examples |
| 1937 |
|
| 1938 |
1.306 Sat Jan 05 18:00:00 EST 2013 |
| 1939 |
|
| 1940 |
* Added if statement around setpgrp(0,0). That function is not supported |
| 1941 |
under Windows. |
| 1942 |
* Updated logic for removing any remaining MCE session directories inside |
| 1943 |
MCE::Signal. |
| 1944 |
|
| 1945 |
1.305 Sat Jan 05 16:00:00 EST 2013 |
| 1946 |
|
| 1947 |
* Added check for $^S to the DIE handler inside the _worker_main method |
| 1948 |
* Added setpgrp(0,0) to MCE::Signal's BEGIN block |
| 1949 |
* MCE::Signal points to a _mce_sess_dir hash in the event of a signal, |
| 1950 |
will remove the sess_dir(s) as well. This is needed when tmp_dir is |
| 1951 |
specified during instantiation and pointing to another location than |
| 1952 |
MCE::Signal::tmp_dir. |
| 1953 |
|
| 1954 |
1.304 Wed Jan 02 22:00:00 EST 2013 |
| 1955 |
|
| 1956 |
* Added Oliver Gorwits to CREDITS for identifying 2 issues |
| 1957 |
* Direct die to CORE::die inside handler if executing an eval |
| 1958 |
* Undef $mce_spawned_ref if signal was caught (stop_and_exit) |
| 1959 |
* Changed INIT to sub import in MCE.pm |
| 1960 |
|
| 1961 |
1.303 Tue Jan 01 20:00:00 EST 2013 |
| 1962 |
|
| 1963 |
* Bump version -- MCE.pm VERSION now matches with META.yml |
| 1964 |
* Sorted forchunk, foreach, forseq methods inside MCE.pm |
| 1965 |
* Modified if statement in run method |
| 1966 |
* Task ID is never undef, therefore removed check inside |
| 1967 |
restart_worker and worker_do methods |
| 1968 |
* Added 2 package variables to MCE::Signal |
| 1969 |
$display_die_with_localtime and $display_warn_with_localtime |
| 1970 |
* Completed updates to documentation |
| 1971 |
* Update to forseq.pl and seq_demo.pl examples |
| 1972 |
|
| 1973 |
1.302 Tue Jan 01 07:30:00 EST 2013 |
| 1974 |
|
| 1975 |
* Fixed validation logic for sequence |
| 1976 |
* Updated the sequence generator -- now supports chunking |
| 1977 |
* Updated seq_demo.pl example to demo user_tasks, sequence, and |
| 1978 |
chunk_size applied together |
| 1979 |
* Documentation updates |
| 1980 |
|
| 1981 |
1.301 Mon Dec 31 15:00:00 EST 2012 |
| 1982 |
|
| 1983 |
* Emphasis on documentation -- better flow plus additional clarity |
| 1984 |
* Minor updates to sequence option validation |
| 1985 |
* Minor updates to included examples |
| 1986 |
|
| 1987 |
1.300 Mon Dec 31 06:00:00 EST 2012 |
| 1988 |
|
| 1989 |
* New methods...: chunk_size, restart_worker, task_id, task_wid, tmp_dir |
| 1990 |
* New options...: on_post_exit, on_post_run, sequence |
| 1991 |
* New examples..: forseq.pl, seq_demo.pl |
| 1992 |
* Overhaul to exit method |
| 1993 |
Workers can exit or die without impacting the manager process |
| 1994 |
* Enabled executable bit for test files |
| 1995 |
* Removed localtime output in die and warn handlers |
| 1996 |
* All 3 delay options are consistent whether user_tasks is specified |
| 1997 |
* Removed logic around total_ended count -- replaced with new exit logic |
| 1998 |
* Code refactoring plus documentation updates |
| 1999 |
* Added LICENSE file |
| 2000 |
|
| 2001 |
1.201 Fri Dec 21 00:00:00 EST 2012 |
| 2002 |
|
| 2003 |
* Added MCE.pod -- moved documentation from MCE.pm to pod file |
| 2004 |
* Added missing use strict/warnings to test scripts |
| 2005 |
* Default to 1 for chunk_size and max_workers if not specified |
| 2006 |
* Test::More is not a requirement to run MCE, only for building |
| 2007 |
* Changed the format for the change log file |
| 2008 |
|
| 2009 |
1.200 Thu Dec 20 00:00:00 EST 2012 |
| 2010 |
|
| 2011 |
* Added new user_tasks option |
| 2012 |
* Added space between method name and left-paren for header lines in POD |
| 2013 |
* Remove not-needed BSD::Resource and forks inside BEGIN/INIT blocks |
| 2014 |
|
| 2015 |
1.106 Wed Dec 19 05:00:00 EST 2012 |
| 2016 |
|
| 2017 |
* Added t/pod-coverage.t |
| 2018 |
* Big overhaul of the MCE documentation -- all methods are documented |
| 2019 |
* Croak if method suited for a MCE worker is called by the main MCE process |
| 2020 |
* Croak if method suited for the main MCE process is called by a MCE worker |
| 2021 |
* Updated Makefile.PL to declare the minimum Perl version |
| 2022 |
|
| 2023 |
1.105 Sun Dec 16 23:00:00 EST 2012 |
| 2024 |
|
| 2025 |
* Completed code re-factoring |
| 2026 |
* Added t/pod.t |
| 2027 |
|
| 2028 |
1.104 Sun Nov 25 17:00:00 EST 2012 |
| 2029 |
|
| 2030 |
* Added 1 new example to MCE's Perl documentation |
| 2031 |
* Use module::method name versus constant symbol when calling _croak |
| 2032 |
* Croak if session directory is not writeable inside MCE::spawn |
| 2033 |
* Renamed _mce_id to _mce_sid (meant to be spawn id actually) |
| 2034 |
* Re-calibrated maximum workers allowed |
| 2035 |
|
| 2036 |
1.103 Fri Nov 23 13:00:00 EST 2012 |
| 2037 |
|
| 2038 |
* Added writeable check on /dev/shm |
| 2039 |
* Croak if tmp dir is not writeable inside MCE::Signal::import |
| 2040 |
|
| 2041 |
1.102 Thu Nov 22 13:00:00 EST 2012 |
| 2042 |
|
| 2043 |
* Woohoot !!! MCE now passes with Perl 5.17.x |
| 2044 |
* Added Copying file -- same as in Perl |
| 2045 |
|
| 2046 |
1.101 Wed Nov 21 16:00:00 EST 2012 |
| 2047 |
|
| 2048 |
* Shifted white space to the left for code blocks inside documentation |
| 2049 |
|
| 2050 |
1.100 Wed Nov 21 10:00:00 EST 2012 |
| 2051 |
|
| 2052 |
* Completed optimization and shakeout for MCE's existing API |
| 2053 |
* File handles are cached when calling sendto and appending to a file |
| 2054 |
* The sendto method now supports multiple arguments -- see perldoc |
| 2055 |
* Added new option: flush_file |
| 2056 |
|
| 2057 |
1.008 Sat Nov 17 23:00:00 EST 2012 |
| 2058 |
|
| 2059 |
* Updates to __DIE__ and __WARN__ handling in MCE. These address the |
| 2060 |
unreferenced scalars seen in packaging logs at activestate.com for |
| 2061 |
Perl under Windows: https://code.activestate.com/ppm/MCE/ |
| 2062 |
* Update t/01_load_signal_arg.t -- added check for $ENV{TEMP} |
| 2063 |
This fixes issue seen under Cygwin |
| 2064 |
|
| 2065 |
1.007 Thu Nov 15 21:30:00 EST 2012 |
| 2066 |
|
| 2067 |
* At last, the "Voila" release :) |
| 2068 |
* Small change to __DIE__ and __WARN__ signal handling for spawn method |
| 2069 |
|
| 2070 |
1.006 Thu Nov 15 03:30:00 EST 2012 |
| 2071 |
|
| 2072 |
* Added description section to MCE::Signal's Perl doc |
| 2073 |
* Do not set trap on __DIE__ and __WARN__ inside MCE::Signal |
| 2074 |
* Localized __DIE__ and __WARN__ handlers inside MCE instead |
| 2075 |
* Clarify the use of threads in documentation |
| 2076 |
|
| 2077 |
1.005 Tue Nov 13 06:11:00 EST 2012 |
| 2078 |
|
| 2079 |
* Removed underscore from package globals in MCE::Signal |
| 2080 |
* Optimized _worker_read_handle method in MCE |
| 2081 |
* Updated files under examples/tbray/ |
| 2082 |
|
| 2083 |
1.004 Mon Nov 12 01:50:00 EST 2012 |
| 2084 |
|
| 2085 |
* Updated examples/mce_usage.readme |
| 2086 |
* Updated examples/wide_finder.pl |
| 2087 |
* Added examples/tbray/README |
| 2088 |
* Added examples/tbray/tbray_baseline1.pl |
| 2089 |
* Added examples/tbray/tbray_baseline2.pl |
| 2090 |
* Added examples/tbray/wf_mce1.pl |
| 2091 |
* Added examples/tbray/wf_mce2.pl |
| 2092 |
* Added examples/tbray/wf_mce3.pl (../wide_finder.pl moved here) |
| 2093 |
* Added examples/tbray/wf_mmap.pl |
| 2094 |
|
| 2095 |
1.003 Sat Nov 10 12:55:00 EST 2012 |
| 2096 |
|
| 2097 |
* Updated README |
| 2098 |
* Updated images/06_Shared_Sockets.gif |
| 2099 |
* Updated images/10_Scaling_Pings.gif |
| 2100 |
* Added images/11_SNMP_Collection.gif |
| 2101 |
* Minor updates to MCE::Signal |
| 2102 |
|
| 2103 |
1.002 Thu Nov 08 01:13:10 EST 2012 |
| 2104 |
|
| 2105 |
* Renamed continue method to next |
| 2106 |
|
| 2107 |
1.001 Wed Nov 07 23:58:20 EST 2012 |
| 2108 |
|
| 2109 |
* Added perl-MCE.spec to trunk |
| 2110 |
https://code.google.com/p/many-core-engine-perl/source/browse/trunk/ |
| 2111 |
* Added CREDITS |
| 2112 |
* Added 3 new methods to MCE.pm: continue, last, and exit |
| 2113 |
* Both foreach & forchunk now call run(1, {...}) to auto-shutdown workers |
| 2114 |
|
| 2115 |
1.000 Mon Nov 05 10:00:00 EST 2012 |
| 2116 |
|
| 2117 |
* First release |
| 2118 |
|