Debian Codegraph @

ChangeLog

raw
1 Version 3.10.0 2026-06-11
2
3 * Enforce the stricter `nltk.pathsec` security policy by default
4 * Document the new security model and migration guidance
5 * Harden resource loading against path traversal and SSRF/DNS-rebinding
6 * Harden downloader path handling and block XML entity expansion
7 * Close remaining corpus-reader security edge cases
8 * Replace unsafe `exec()` usage in the utility CLI
9 * Warn on unpickling user-provided pickles
10 * Add HuggingFace datasets integration (`nltk.huggingface`)
11 * Align TnT with Brants (2000) specifications
12 * Fix PorterStemmer irregular-form lowercasing in NLTK mode
13 * Fix TransitionParser sparse index dtype for scikit-learn 1.9
14 * Fix TextCat tie handling
15 * Fix WordNet object comparisons for incompatible types
16 * Cache WordNet max depth lazily for `lch_similarity()`
17 * Fix CCG variable direction, substitution, and type-raising bugs
18 * Fix Jaro similarity for single-character and empty-string cases
19 * Improve CI and release-maintenance workflows
20
21 Thanks to the following contributors to 3.10.0:
22 13rac1, alvations, bowiechen, devesh-2002, ekaf, elias-ba,
23 haosenwang1018, HyperPS, ihitamandal, jancallewaert, jhnwnstd,
24 JuanIMartinezB, Lemm1, LinZiyuu, Mr-Neutr0n, PastelStorm,
25 scruge1, Syzygy2048, ylwango613, yzhaoinuw
26
27 Version 3.9.4 2026-03-24
28
29 * Support Python 3.14
30 * Fix bug in Levenshtein distance when substitution_cost > 2
31 * Fix bug in Treebank detokeniser re quote ordering
32 * Fix bug in Jaro similarity for empty strings
33 * Several security enhancements
34 * Fix GHSA-rf74-v2fm-23pw: unbounded recursion in JSONTaggedDecoder
35 * Implement TextTiling vocabulary introduction method (Hearst 1997)
36 * Fix ALINE feature matrix errors and add comprehensive tests
37 * Support multiple VerbNet versions, fix longid/shortid regex for VerbNet ids
38 * Let downloader fallback to md5 when sha256 is unavailable
39 * Several other minor bugfixes and code cleanups
40
41 Thanks to the following contributors to 3.9.4:
42 Min-Yen Kan, Eric Kafe, Emily Voss, bowiechen, Hrudhai01,
43 jancallewaert, Mr-Neutr0n, pollak.peter89, ylwango613,
44
45 Version 3.9.3 2026-02-21
46
47 * Fix CVE-2025-14009: secure ZIP extraction in nltk.downloader (#3468)
48 * Block path traversal/arbitrary reads in nltk.data for protocol-less refs (#3467)
49 * Block path traversal/abs paths in corpus readers and FS pointers (#3479, #3480)
50 * Validate external StanfordSegmenter JARs using SHA256 (#3477)
51 * Add optional sandbox enforcement for filestring() (#3485)
52 * Maintenance: downloader/zipped models, CI/tooling updates
53
54 Thanks to the following contributors to 3.9.3:
55 Chris Clauss, Eric Kafe, HyperPS, purificant, Shivansh-Game, Christopher Smith
56
57 Version 3.9.2 2025-10-01
58
59 * Update download checksums to use SHA256 in built index
60 * Fix percentage escape in new-style string formatting
61 * replace shortened URLs using goo.gl
62 * Make Wordnet interoperable with various taggers and tagged corpora
63 * Fix saving PerceptronTagger
64 * Document how to reproduce old Wordnet studies
65 * properly initialize Portuguese corpus reader
66 * support for mixed rules conversion into Chomsky Normal Form
67 * only import tkinter if a GUI is needed
68 * issue #2112 with Corenlp
69 * new environment variable NLTK_DOWNLOADER_FORCE_INTERACTIVE_SHELL
70 * Lesk defaults to most frequent sense in case of ties
71
72 Thanks to the following contributors to 3.9.2:
73 Jose Cols, Peter de Blanc, GeneralPoxter, Eric Kafe, William LaCroix, Jason Liu,
74 Samer Masterson, Mike014, purificant, Andrew Ernest Ritz, samertm, Ikram Ul Haq,
75 Christopher Smith, Ryan Mannion
76
77 Version 3.9.1 2024-08-19
78 * Fixed bug that prevented wordnet from loading
79
80 Version 3.9 2024-08-18
81 * Fix security vulnerability CVE-2024-39705 (breaking change)
82 * Replace pickled models (punkt, chunker, taggers) by new pickle-free "_tab" packages
83 * No longer sort Wordnet synsets and relations (sort in calling function when required)
84 * Only strip the last suffix in Wordnet Morphy, thus restricting synsets() results
85 * Add Python 3.12 support
86 * Many other minor fixes
87
88 Thanks to the following contributors to 3.8.2:
89 Tom Aarsen, Cat Lee Ball, Veralara Bernhard, Carlos Brandt, Konstantin Chernyshev, Michael Higgins,
90 Eric Kafe, Vivek Kalyan, David Lukes, Rob Malouf, purificant, Alex Rudnick, Liling Tan, Akihiro Yamazaki.
91
92 Version 3.8.1 2023-01-02
93
94 * Resolve RCE vulnerability in localhost WordNet Browser (#3100)
95 * Remove unused tool scripts (#3099)
96 * Resolve XSS vulnerability in localhost WordNet Browser (#3096)
97 * Add Python 3.11 support (#3090)
98
99 Thanks to the following contributors to 3.8.1:
100 Francis Bond, John Vandenberg, Tom Aarsen
101
102 Version 3.8 2022-12-12
103
104 * Refactor dispersion plot (#3082)
105 * Provide type hints for LazyCorpusLoader variables (#3081)
106 * Throw warning when LanguageModel is initialized with incorrect vocabulary (#3080)
107 * Fix WordNet's all_synsets() function (#3078)
108 * Resolve TreebankWordDetokenizer inconsistency with end-of-string contractions (#3070)
109 * Support both iso639-3 codes and BCP-47 language tags (#3060)
110 * Avoid DeprecationWarning in Regexp tokenizer (#3055)
111 * Fix many doctests, add doctests to CI (#3054, #3050, #3048)
112 * Fix bool field not being read in VerbNet (#3044)
113 * Greatly improve time efficiency of SyllableTokenizer when tokenizing numbers (#3042)
114 * Fix encodings of Polish udhr corpus reader (#3038)
115 * Allow TweetTokenizer to tokenize emoji flag sequences (#3034)
116 * Prevent LazyModule from increasing the size of nltk.__dict__ (#3033)
117 * Fix CoreNLPServer non-default port issue (#3031)
118 * Add "acion" suffix to the Spanish SnowballStemmer (#3030)
119 * Allow loading WordNet without OMW (#3026)
120 * Use input() in nltk.chat.chatbot() for Jupyter support (#3022)
121 * Fix edit_distance_align() in distance.py (#3017)
122 * Tackle performance and accuracy regression of sentence tokenizer since NLTK 3.6.6 (#3014)
123 * Add the Iota operator to semantic logic (#3010)
124 * Resolve critical errors in WordNet app (#3008)
125 * Resolve critical error in CHILDES Corpus (#2998)
126 * Make WordNet information_content() accept adjective satellites (#2995)
127 * Add "strict=True" parameter to CoreNLP (#2993, #3043)
128 * Resolve issue with WordNet's synset_from_sense_key (#2988)
129 * Handle WordNet synsets that were lost in mapping (#2985)
130 * Resolve TypeError in Boxer (#2979)
131 * Add function to retrieve WordNet synonyms (#2978)
132 * Warn about nonexistent OMW offsets instead of raising an error (#2974)
133 * Fix missing ic argument in res, jcn and lin similarity functions of WordNet (#2970)
134 * Add support for the extended OMW (#2946)
135 * Fix LC cutoff policy of text tiling (#2936)
136 * Optimize ConditionalFreqDist.__add__ performance (#2939)
137 * Add Markdown corpus reader (#2902)
138
139 Thanks to the following contributors to 3.8:
140 Alexandre Perez-Lebel, David Lukes, Eric Kafe, Fernando Carranza, Heungson Lee,
141 Hoyeol Kim, James Huang, Jelle Zijlstra, Louis-Justin Tallot, M.K. Pawelkiewicz,
142 Jan Lennartz, Malinda Dilhara, Martin Kondratzky, Rob Malouf, Saud Kadiri,
143 Siddhesh Mhadnak, Stephan Hasler, Steve Smith, Tom Aarsen, Tyler Sheaffer,
144 Yue Zhao, cestwc, elespike, purificant, richardyy1188
145
146 Version 3.7 2022-02-09
147
148 * Improve and update the NLTK team page on nltk.org (#2855, #2941)
149 * Drop support for Python 3.6, support Python 3.10 (#2920)
150
151 Thanks to the following contributors to 3.7:
152 Tom Aarsen
153
154 Version 3.6.7 2021-12-28
155
156 * Resolve IndexError in `sent_tokenize` and `word_tokenize` (#2922)
157
158 Thanks to the following contributors to 3.6.7:
159 Tom Aarsen
160
161 Version 3.6.6 2021-12-21
162
163 * Refactor `gensim.doctest` to work for gensim 4.0.0 and up (#2914)
164 * Add Precision, Recall, F-measure, Confusion Matrix to Taggers (#2862)
165 * Added warnings if .zip files exist without any corresponding .csv files. (#2908)
166 * Fix `FileNotFoundError` when the `download_dir` is a non-existing nested folder (#2910)
167 * Rename omw to omw-1.4 (#2907)
168 * Resolve ReDoS opportunity by fixing incorrectly specified regex (#2906)
169 * Support OMW 1.4 (#2899)
170 * Deprecate Tree get and set node methods (#2900)
171 * Fix broken inaugural test case (#2903)
172 * Use Multilingual Wordnet Data from OMW with newer Wordnet versions (#2889)
173 * Keep NLTKs "tokenize" module working with pathlib (#2896)
174 * Make prettyprinter to be more readable (#2893)
175 * Update links to the nltk book (#2895)
176 * Add `CITATION.cff` to nltk (#2880)
177 * Resolve serious ReDoS in PunktSentenceTokenizer (#2869)
178 * Delete old CI config files (#2881)
179 * Improve Tokenize documentation + add TokenizerI as superclass for TweetTokenizer (#2878)
180 * Fix expected value for BLEU score doctest after changes from #2572
181 * Add multi Bleu functionality and tests (#2793)
182 * Deprecate 'return_str' parameter in NLTKWordTokenizer and TreebankWordTokenizer (#2883)
183 * Allow empty string in CFG's + more (#2888)
184 * Partition `tree.py` module into `tree` package + pickle fix (#2863)
185 * Fix several TreebankWordTokenizer and NLTKWordTokenizer bugs (#2877)
186 * Rewind Wordnet data file after each lookup (#2868)
187 * Correct __init__ call for SyntaxCorpusReader subclasses (#2872)
188 * Documentation fixes (#2873)
189 * Fix levenstein distance for duplicated letters (#2849)
190 * Support alternative Wordnet versions (#2860)
191 * Remove hundreds of formatting warnings for nltk.org (#2859)
192 * Modernize `nltk.org/howto` pages (#2856)
193 * Fix Bleu Score smoothing function from taking log(0) (#2839)
194 * Update third party tools to newer versions and removing MaltParser fixed version (#2832)
195 * Fix TypeError: _pretty() takes 1 positional argument but 2 were given in sem/drt.py (#2854)
196 * Replace `http` with `https` in most URLs (#2852)
197
198 Thanks to the following contributors to 3.6.6:
199 Adam Hawley, BatMrE, Danny Sepler, Eric Kafe, Gavish Poddar, Panagiotis Simakis,
200 RnDevelover, Robby Horvath, Tom Aarsen, Yuta Nakamura, Mohaned Mashaly
201
202 Version 3.6.5 2021-10-11
203
204 * modernised nltk.org website
205 * addressed LGTM.com issues
206 * support ZWJ sequences emoji and skin tone modifer emoji in TweetTokenizer
207 * METEOR evaluation now requires pre-tokenized input
208 * Code linting and type hinting
209 * implement get_refs function for DrtLambdaExpression
210 * Enable automated CoreNLP, Senna, Prover9/Mace4, Megam, MaltParser CI tests
211 * specify minimum regex version that supports regex.Pattern
212 * avoid re.Pattern and regex.Pattern which fail for Python 3.6, 3.7
213
214 Thanks to the following contributors to 3.6.5:
215 Tom Aarsen, Saibo Geng, Mohaned Mashaly, Dimitri Papadopoulos, Danny Sepler,
216 Ahmet Yildirim, RnDevelover, yutanakamura
217
218 Version 3.6.4 2021-10-01
219
220 * deprecate `nltk.usage(obj)` in favor of `help(obj)`
221 * resolve ReDoS vulnerability in Corpus Reader
222 * solidify performance tests
223 * improve phone number recognition in tweet tokenizer
224 * refactored CISTEM stemmer for German
225 * identify NLTK Team as the author
226 * replace travis badge with github actions badge
227 * add SECURITY.md
228
229 Thanks to the following contributors to 3.6.4:
230 Tom Aarsen, Mohaned Mashaly, Dimitri Papadopoulos Orfanos, purificant, Danny Sepler
231
232 Version 3.6.3 2021-09-19
233 * Dropped support for Python 3.5
234 * Run CI tests on Windows, too
235 * Moved from Travis CI to GitHub Actions
236 * Code and comment cleanups
237 * Visualize WordNet relation graphs using Graphviz
238 * Fixed large error in METEOR score
239 * Apply isort, pyupgrade, black, added as pre-commit hooks
240 * Prevent debug_decisions in Punkt from throwing IndexError
241 * Resolved ZeroDivisionError in RIBES with dissimilar sentences
242 * Initialize WordNet IC total counts with smoothing value
243 * Fixed AttributeError for Arabic ARLSTem2 stemmer
244 * Many fixes and improvements to lm language model package
245 * Fix bug in nltk.metrics.aline, C_skip = -10
246 * Improvements to TweetTokenizer
247 * Optional show arg for FreqDist.plot, ConditionalFreqDist.plot
248 * edit_distance now computes Damerau-Levenshtein edit-distance
249
250 Thanks to the following contributors to 3.6.3:
251 Tom Aarsen, Abhijnan Bajpai, Michael Wayne Goodman, Michał Górny, Maarten ter Huurne,
252 Manu Joseph, Eric Kafe, Ilia Kurenkov, Daniel Loney, Rob Malouf, Mohaned Mashaly,
253 purificant, Danny Sepler, Anthony Sottile
254
255 Version 3.6.2 2021-04-20
256 * move test code to nltk/test
257 * clean up some doctests
258 * fix bug in NgramAssocMeasures (order preserving fix)
259 * fixes for compatibility with Pypy 7.3.4
260
261 Thanks to the following contributors to 3.6.2:
262 Ruben Cartuyvels, Rob Malouf, Dalton Pearson, Danny Sepler
263
264 Version 3.6 2021-04-07
265 * add support for Python 3.9
266 * add Tree.fromlist
267 * compute Minimum Spanning Tree of unweighted graph using BFS
268 * fix bug with infinite loop in Wordnet closure and tree
269 * fix bug in calculating BLEU using smoothing method 4
270 * Wordnet synset similarities work for all pos
271 * new Arabic light stemmer (ARLSTem2)
272 * new syllable tokenizer (LegalitySyllableTokenizer)
273 * remove nose in favor of pytest
274 * misc bug fixes, code cleanups, test cleanups, efficiency improvements
275
276 Thanks to the following contributors to 3.6:
277 Tom Aarsen, K Abainia, Akshita Bhagia, Andrew Bird, Thomas Bird,
278 Tom Conroy, Christopher Hench, Andrew Jorgensen, Eric Kafe,
279 Ilia Kurenkov, Yeting Li, Joseph Manu, Marius Mather, Denali Molitor,
280 Jacob Moorman, Philippe Ombredanne, Vassilis Palassopoulos, Ram Rachum,
281 Danny Sepler, Or Sharir, Brad Solomon, Hiroki Teranishi, Constantin Weisser,
282 Pratap Yadav, Louis Yang
283
284 Version 3.5 2020-04-13
285 * add support for Python 3.8
286 * drop support for Python 2
287 * create NLTK's own Tokenizer class distinct from the Treebank reference tokeniser
288 * update Vader sentiment analyser
289 * fix JSON serialization of some PoS taggers
290 * minor improvements in grammar.CFG, Vader, pl196x corpus reader, StringTokenizer
291 * change implementation <= and >= for FreqDist so they are partial orders
292 * make FreqDist iterable
293 * correctly handle Penn Treebank trees with a unlabeled branching top node.
294
295 Thanks to the following contributors to 3.5:
296 Nicolas Darr, Gerhard Kremer, Liling Tan, Christopher Hench, Alexandre Dias, Hervé Nicol,
297 Pierpaolo Pantone, Bonifacio de Oliveira, Maciej Gawinecki, BLKSerene, hoefling, alvations,
298 pyfisch, srhrshr
299
300 Version 3.4.5 2019-08-20
301 * Fixed security bug in downloader: Zip slip vulnerability - for the unlikely
302 situation where a user configures their downloader to use a compromised server
303 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14751)
304
305 Thanks to the following contributors to 3.4.5:
306 Mike Salvatore
307
308 Version 3.4.4 2019-07-04
309 * fix bug in plot function (probability.py)
310 * add improved PanLex Swadesh corpus reader
311
312 Thanks to the following contributors to 3.4.4:
313 Devashish Lal, Liling Tan
314
315 Version 3.4.3 2019-06-07
316
317 * add Text.generate()
318 * add QuadgramAssocMeasures
319 * add SSP to tokenizers
320 * return confidence of best tag from AveragedPerceptron
321 * make plot methods return Axes objects
322 * don't require list arguments to PositiveNaiveBayesClassifier.train
323 * fix Tree classes to work with native Python copy library
324 * fix inconsistency for NomBank
325 * fix random seeding in LanguageModel.generate
326 * fix ConditionalFreqDist mutation on tabulate/plot call
327 * fix broken links in documentation
328 * fix misc Wordnet issues
329 * update installation instructions
330
331 Thanks to the following contributors to 3.4.3:
332 alvations, Bharat123rox, cifkao, drewmiller, free-variation, henchc
333 irisxzhou, nick-ulle, ppartarr, simonepri, yigitsever, zhaoyanpeng
334
335 Version 3.4.1 2019-04-17
336
337 * add chomsky_normal_form for CFGs
338 * add meteor score
339 * add minimum edit/Levenshtein distance based alignment function
340 * allow access to collocation list via text.collocation_list()
341 * support corenlp server options
342 * drop support for Python 3.4
343 * other minor fixes
344
345 Thanks to the following contributors to 3.4.1:
346 Adrian Ellis, Andrew Martin, Ayush Kaushal, BLKSerene, Bharat
347 Raghunathan, Franklin Chen, KMiNT21 Kevin Brown, Liling Tan,
348 Matan Rak, Nat Quayle Nelson, Osman Zubair, Purificant,
349 Uday Krishna, Viresh Gupta
350
351 Version 3.4 2018-11-17
352 * Support Python 3.7
353 * Language Modeling incl Kneser-Ney, Witten-Bell, Good-Turing
354 * Cistem Stemmer for German
355 * Support Russian National Corpus incl POS tag model
356 * Decouple sentiment and twitter packages
357 * Minor extensions for WordNet
358 * K-alpha
359 * Fix warning messages for corenlp
360 * Comprehensive code cleanups
361 * Many other minor fixes
362 * Switch continuous integration from Jenkins to Travis
363
364 Special thanks to Ilia Kurenkov (Language Model package), Liling Tan (Python 3.7, Travis-CI),
365 and purificant (code cleanups). Thanks also to: Afshin Sadeghi, Ales Tamchyna, Alok Debnath,
366 aquatiko, Coykto, Denis Kataev, dnc1994, Fabian Howard, Frankie Robertson, Iaroslav Tymchenko,
367 Jayakrishna Sahit, LBenzahia, Leonie Weißweiler, Linghao Zhang, Rohit Kumar, sahitpj,
368 Tim Gianitsos, vagrant, 53X
369
370 Version 3.3 2018-05-06
371 * Support Python 3.6
372 * New interface to CoreNLP
373 * Support synset retrieval by sense key
374 * Minor fixes to CoNLL Corpus Reader, AlignedSent
375 * Fixed minor inconsistencies in APIs and API documentation
376 * Better conformance to PEP8
377 * Drop moses.py (incompatible license)
378
379 Special thanks to Liling Tan for leading our transition to Python 3.6.
380 Thanks to other contributors listed here: https://github.com/nltk/nltk/blob/develop/AUTHORS.md
381
382 Version 3.2.5 2017-09-24
383
384 * Arabic stemmers (ARLSTem, Snowball)
385 * NIST MT evaluation metric and added NIST international_tokenize
386 * Moses tokenizer
387 * Document Russian tagger
388 * Fix to Stanford segmenter
389 * Improve treebank detokenizer, VerbNet, Vader
390 * Misc code and documentation cleanups
391 * Implement fixes suggested by LGTM
392
393 Thanks to the following contributors to 3.2.5:
394 Ali Abdullah, Lakhdar Benzahia, Henry Elder, Campion Fellin,
395 Tsolak Ghukasyan, Thanh Ha, Jean Helie, Nelson Liu,
396 Nathan Schneider, Chintan Shah, Fábio Silva, Liling Tan,
397 Ziyao Wei, Zicheng Xu, Albert Au Yeung, AbdealiJK,
398 porqupine, sbagan, xprogramer
399
400 Version 3.2.4 2017-05-21
401
402 * remove load-time dependency on Python requests library
403 * add support for Arabic in StanfordSegmenter
404 * fix MosesDetokenizer on irregular quote tokens
405
406 Thanks to the following contributors to 3.2.4:
407 Alex Constantin, Hatem Nassrat, Liling Tan
408
409 Version 3.2.3 2017-05-16
410
411 * new interface to Stanford CoreNLP Web API
412 * improved Lancaster stemmer with customizable rules from Whoosh
413 * improved Treebank tokenizer
414 * improved support for GLEU score
415 * adopt new Abstract base class style
416 * support custom tab files for extending WordNet
417 * make synset_from_pos_and_offset a public method
418 * make non-English WordNet lemma lookups case-insensitive
419 * speed up TnT tagger
420 * speed up FreqDist and ConditionalFreqDist
421 * support additional quotes in TreebankWordTokenizer
422 * clean up Tk's postscript output
423 * drop explicit support for corpora not distributed with NLTK to streamline testing
424 * allow iterator in perceptron tagger training
425 * allow for curly bracket quantifiers in chunk.regexp.CHUNK_TAG_PATTERN
426 * new corpus reader for MWA subset of PPDB
427 * improved testing framework
428
429 Thanks to the following contributors to 3.2.3:
430 Mark Amery, Carl Bolz, Abdelhak Bougouffa, Matt Chaput, Michael Goodman,
431 Jaehoon Hwang, Naoya Kanai, Jackson Lee, Christian Meyer, Dmitrijs Milajevs,
432 Adam Nelson, Pierpaolo Pantone, Liling Tan, Vilhjalmur Thorsteinsson,
433 Arthur Tilley, jmhutch, Yorwba, eromoe and others
434
435 Version 3.2.2 2016-12-31
436 * added Kondrak's Aline algorithm
437 * added ChrF and GLEU MT evaluation metrics
438 * added Russian pos tagger model
439 * added Moses detokenizer
440 * rewrite Porter Stemmer
441 * rewrite FrameNet corpus reader
442 (adds frame parameter to fes(), lus(), exemplars()
443 see https://www.nltk.org/howto/framenet.html)
444 * updated FrameNet Corpus to version 1.7
445 * fixes to stanford_segmenter.py, SentiText, CoNLL Corpus Reader
446 * fixes to BLEU, naivebayes, Krippendorff's alpha, Punkt
447 * fixes to tests for TransitionParser, Senna, edit distance
448 * fixes to Moses Tokenizer and Detokenizer
449 * improved TweetTokenizer
450 * strip trailing whitespace when splitting sentences
451 * handle inverted exclamation mark in ToktokTokenizer
452 * resolved some issues with Python 3.5 support
453 * improvements to testing framework
454 * clean up dependencies
455
456 Thanks to the following contributors to 3.2.2:
457
458 Prasasto Adi, Mark Amery, Geoff Bacon, George Berry, Colin Carroll, Alexis Dimitriadis,
459 Nicholas Fabina, German Ferrero, Tsolak Ghukasyan, Hyuckin David Lim, Naoya Kanai,
460 Greg Kondrak, Igor Korolev, Tim Leslie, Rob Malouf, Heguang Miao, Dmitrijs Milajevs,
461 Adam Nelson, Dennis O'Brien, Qi Liu, Pierpaolo Pantone, Andy Reagan, Mike Recachinas,
462 Nathan Schneider, Jānis Šlapiņš, Richard Snape, Liling Tan, Marcus Uneson,
463 Linghao Zhang, drevicko, SaintNazaire
464
465 Version 3.2.1 2016-04-09
466 * Support for CCG semantics, Stanford segmenter, VADER lexicon
467 * Fixes to BLEU score calculation, CHILDES corpus reader
468 * Other miscellaneous fixes
469
470 Thanks to the following contributors to 3.2.1:
471 Andrew Giel, Casper Lehmann-Strøm, David Madl, Tanin Na Nakorn,
472 Guilherme Nardari, Philippe Ombredanne, Nathan Schneider, Liling Tan,
473 Josiah Wang, venticello
474
475 Version 3.2 2016-03-03
476 * Fixes for Python 3.5
477 * Code cleanups now Python 2.6 is no longer supported
478 * Improvements to documentation
479 * Comprehensive use of os.path for platform-specific path handling
480 * Support for PanLex
481 * Support for third party download locations for NLTK data
482 * Fix bugs in IBM method 3 smoothing and BLEU calculation
483 * Support smoothing for BLEU score and corpus-level BLEU
484 * Support RIBES score
485 * Improvements to TweetTokenizer
486 * Updates for Stanford API
487 * Add mathematical operators to ConditionalFreqDist
488 * Fix bug in sentiwordnet for adjectives
489 * Merged internal implementations of Trie
490
491 Thanks to the following contributors to 3.2:
492 Santiago Castro, Jihun Choi, Graham Christensen, Andrew Drozdov, Long
493 Duong, Kyriakos Georgiou, Michael Wayne Goodman, Clark Grubb, Tah Wei
494 Hoon, David Kamholz, Ewan Klein, Reed Loden, Rob Malouf, Philippe
495 Ombredanne, Josh Owen, Pierpaolo Pantone, Mike Recachinas, Elijah
496 Rippeth, Thomas Stieglmaier, Liling Tan, Philip Tzou, Pratap Vardhan.
497
498 Version 3.1 2015-10-15
499 * Fixes for Python 3.5 (drop support for capturing groups in regexp tokenizer)
500 * Drop support for Python 2.6
501 * Adopt perceptron tagger for new default POS tagger nltk.pos_tag
502 * Stanford Neural Dependency Parser wrapper
503 * Sentiment analysis package incl VADER
504 * Improvements to twitter package
505 * Multi word expression tokenizer
506 * Support for everygram and skipgram
507 * consistent evaluation metric interfaces, putting reference before hypothesis
508 * new nltk.translate module, incorporating the old align module
509 * implement stack decoder
510 * clean up Alignment interface
511 * CorpusReader method to support access to license and citation
512 * Multext East Corpus and MTECorpusReader
513 * include six module to streamline installation on MS Windows
514
515 Thanks to the following contributors to 3.1:
516 Le Tuan Anh, Petra Barancikova, Alexander Böhm, Francis Bond,
517 Long Duong, Anna Garbar, Matthew Honnibal, Tah Wei Hoon, Ewan Klein,
518 Rob Malouf, Dmitrijs Milajevs, Will Monroe, Sergio Oller, Pierpaolo
519 Pantone, Jacob Perkins, Lorenzo Rubio, Thomas Stieglmaier, Liling Tan,
520 Pratap Vardhan
521
522 Version 3.0.5 2015-09-05
523 * rewritten IBM models, and new IBM Model 4 and 5 implementations
524 * new Twitter package
525 * stabilized MaltParser API
526 * improved regex tagger
527 * improved documentation on contributing
528 * minor improvements to documentation and testing
529
530 Thanks to the following contributors to 3.0.5:
531 Álvaro Justen, Dmitrijs Milajevs, Ewan Klein, Heran Lin, Justin Hammar,
532 Liling Tan, Long Duong, Lorenzo Rubio, Pierpaolo Pantone, Tah Wei Hoon
533
534 Version 3.0.4 2015-07-13
535 * minor bug fixes and enhancements
536
537 Thanks to the following contributors to 3.0.4:
538 Nicola Bova, Santiago Castro, Len Remmerswaal, Keith Suderman, kabayan55,
539 pln-fing-udelar (NLP Group, Instituto de Computación, Facultad de Ingeniería, Universidad de la República, Uruguay).
540
541 Version 3.0.3 2015-06-12
542 * bug fixes (Stanford NER, Boxer, Snowball, treebank tokenizer,
543 dependency graph, KneserNey, BLEU)
544 * code clean-ups
545 * default POS tagger permits tagset to be specified
546 * gensim illustration
547 * tgrep implementation
548 * added PanLex Swadesh corpora
549 * visualisation for aligned bitext
550 * support for Google App Engine
551 * POSTagger renamed StanfordPOSTagger, NERTagger renamed StanfordNERTagger
552
553 Thanks to the following contributors to 3.0.3:
554
555 Long Duong, Pedro Fialho, Dan Garrette, Helder, Saimadhav Heblikar,
556 Chris Inskip, David Kamholz, Dmitrijs Milajevs, Smitha Milli,
557 Tom Mortimer-Jones, Avital Pekker, Jonathan Pool, Sam Raker,
558 Will Roberts, Dmitry Sadovnychyi, Nathan Schneider, Anirudh W
559
560 Version 3.0.2 2015-03-13
561 * make pretty-printing method names consistent
562 * improvements to Portuguese stemmer
563 * transition-based dependency parsers
564 * dependency graph visualisation for ipython notebook
565 * interfaces for Senna, BLLIP, python-crfsuite
566 * NKJP corpus reader
567 * code clean ups, minor bug fixes
568
569 Thanks to the following contributors to 3.0.2:
570
571 Long Duong, Saimadhav Heblikar, Helder, Mikhail Korobov, Denis Krusko,
572 Alex Louden, Felipe Madrigal, David McClosky, Dmitrijs Milajevs,
573 Ondrej Platek, Nathan Schneider, Dávid Márk Nemeskey, 0ssifrage, ducki13, kiwipi.
574
575 Version 3.0.1 2015-01-12
576 * fix setup.py for new version of setuptools
577
578 Version 3.0.0 2014-09-07
579 * minor bugfixes
580 * added phrase extraction code by Liling Tan and Fredrik Hedman
581
582 Thanks to the following contributors to 3.0.0:
583 Mark Amery, Ivan Barria, Ingolf Becker, Francis Bond, Lars
584 Buitinck, Cristian Capdevila, Arthur Darcet, Michelle Fullwood,
585 Dan Garrette, Dougal Graham, Dan Garrette, Dougal Graham, Lauri
586 Hallila, Tyler Hartley, Fredrik Hedman, Ofer Helman, Bruce Hill,
587 Marcus Huderle, Nancy Ide, Nick Johnson, Angelos Katharopoulos,
588 Ewan Klein, Mikhail Korobov, Chris Liechti, Peter Ljunglof,
589 Joseph Lynch, Haejoong Lee, Peter Ljunglöf, Dean Malmgren, Rob
590 Malouf, Thorsten Marek, Dmitrijs Milajevs, Shari A’aidil
591 Nasruddin, Lance Nathan, Joel Nothman, Alireza Nourian, Alexander
592 Oleynikov, Ted Pedersen, Jacob Perkins, Will Roberts, Alex
593 Rudnick, Nathan Schneider, Geraldine Sim Wei Ying, Lynn Soe,
594 Liling Tan, Louis Tiao, Marcus Uneson, Yu Usami, Steven Xu, Zhe
595 Wang, Chuck Wooters, lade, isnowfy, onesandzeros, pquentin, wvanlint
596
597 Version 3.0b2 2014-08-21
598 * minor bugfixes and clean-ups
599 * renamed remaining parse_ methods to read_ or load_, cf issue #656
600 * added Paice's method of evaluating stemming algorithms
601
602 Thanks to the following contributors to 3.0.0b2: Lars Buitinck,
603 Cristian Capdevila, Lauri Hallila, Ofer Helman, Dmitrijs Milajevs,
604 lade, Liling Tan, Steven Xu
605
606 Version 3.0.0b1 2014-07-11
607 * Added SentiWordNet corpus and corpus reader
608 * Fixed support for 10-column dependency file format
609 * Changed Tree initialization to use fromstring
610
611 Thanks to the following contributors to 3.0b1: Mark Amery, Ivan
612 Barria, Ingolf Becker, Francis Bond, Lars Buitinck, Arthur Darcet,
613 Michelle Fullwood, Dan Garrette, Dougal Graham, Dan Garrette, Dougal
614 Graham, Tyler Hartley, Ofer Helman, Bruce Hill, Marcus Huderle, Nancy
615 Ide, Nick Johnson, Angelos Katharopoulos, Ewan Klein, Mikhail Korobov,
616 Chris Liechti, Peter Ljunglof, Joseph Lynch, Haejoong Lee, Peter
617 Ljunglöf, Dean Malmgren, Rob Malouf, Thorsten Marek, Dmitrijs
618 Milajevs, Shari A’aidil Nasruddin, Lance Nathan, Joel Nothman, Alireza
619 Nourian, Alexander Oleynikov, Ted Pedersen, Jacob Perkins, Will
620 Roberts, Alex Rudnick, Nathan Schneider, Geraldine Sim Wei Ying, Lynn
621 Soe, Liling Tan, Louis Tiao, Marcus Uneson, Yu Usami, Steven Xu, Zhe
622 Wang, Chuck Wooters, isnowfy, onesandzeros, pquentin, wvanlint
623
624 Version 3.0a4 2014-05-25
625 * IBM Models 1-3, BLEU, Gale-Church aligner
626 * Lesk algorithm for WSD
627 * Open Multilingual WordNet
628 * New implementation of Brill Tagger
629 * Extend BNCCorpusReader to parse the whole BNC
630 * MASC Tagged Corpus and corpus reader
631 * Interface to Stanford Parser
632 * Code speed-ups and clean-ups
633 * API standardisation, including fromstring method for many objects
634 * Improved regression testing setup
635 * Removed PyYAML dependency
636
637 Thanks to the following contributors to 3.0a4:
638 Ivan Barria, Ingolf Becker, Francis Bond, Arthur Darcet, Dan Garrette,
639 Ofer Helman, Dougal Graham, Nancy Ide, Ewan Klein, Mikhail Korobov,
640 Chris Liechti, Peter Ljunglof, Joseph Lynch, Rob Malouf, Thorsten Marek,
641 Dmitrijs Milajevs, Shari A’aidil Nasruddin, Lance Nathan, Joel Nothman,
642 Jacob Perkins, Lynn Soe, Liling Tan, Louis Tiao, Marcus Uneson, Steven Xu,
643 Geraldine Sim Wei Ying
644
645 Version 3.0a3 2013-11-02
646 * support for FrameNet contributed by Chuck Wooters
647 * support for Universal Declaration of Human Rights Corpus (udhr2)
648 * major API changes:
649 - Tree.node -> Tree.label() / Tree.set_label()
650 - Chunk parser: top_node -> root_label; chunk_node -> chunk_label
651 - WordNet properties are now access methods, e.g. Synset.definition -> Synset.definition()
652 - relextract: show_raw_rtuple() -> rtuple(), show_clause() -> clause()
653 * bugfix in texttiling
654 * replaced simplify_tags with support for universal tagset (simplify_tags=True -> tagset='universal')
655 * Punkt default behavior changed to realign sentence boundaries after trailing parenthesis and quotes
656 * deprecated classify.svm (use scikit-learn instead)
657 * various efficiency improvements
658
659 Thanks to the following contributors to 3.0a3:
660 Lars Buitinck, Marcus Huderle, Nick Johnson, Dougal Graham, Ewan Klein,
661 Mikhail Korobov, Haejoong Lee, Peter Ljunglöf, Dean Malmgren, Lance Nathan,
662 Alexander Oleynikov, Nathan Schneider, Chuck Wooters, Yu Usami, Steven Xu,
663 pquentin, wvanlint
664
665 Version 3.0a2 2013-07-12
666 * speed improvements in word_tokenize, GAAClusterer, TnT tagger, Baum Welch, HMM tagger
667 * small improvements in collocation finders, probability, modelling, Porter Stemmer
668 * bugfix in lowest common hypernyn calculation (used in path similarity measures)
669 * code cleanups, docstring cleanups, demo fixes
670
671 Thanks to the following contributors to 3.0a2:
672 Mark Amery, Lars Buitinck, Michelle Fullwood, Dan Garrette, Dougal Graham,
673 Tyler Hartley, Bruce Hill, Angelos Katharopoulos, Mikhail Korobov,
674 Rob Malouf, Joel Nothman, Ted Pedersen, Will Roberts, Alex Rudnick,
675 Steven Xu, isnowfy, onesandzeros
676
677 Version 3.0a1 2013-02-14
678 * reinstated tkinter support (Haejoong Lee)
679
680 Version 3.0a0 2013-01-14
681 * alpha release of first version to support Python 2.6, 2.7, and 3.
682
683 Version 2.0.4 2012-11-07
684 * minor bugfix (removed numpy dependency)
685
686 Version 2.0.3 2012-09-24
687
688 * fixed corpus/reader/util.py to support Python 2.5
689 * make MaltParser safe to use in parallel
690 * fixed bug in inter-annotator agreement
691 * updates to various doctests (nltk/test)
692 * minor bugfixes
693
694 Thanks to the following contributors to 2.0.3:
695 Robin Cooper, Pablo Duboue, Christian Federmann, Dan Garrette, Ewan Klein,
696 Pierre-François Laquerre, Max Leonov, Peter Ljunglöf, Nitin Madnani, Ceri Stagg
697
698 Version 2.0.2 2012-07-05
699
700 * improvements to PropBank, NomBank, and SemCor corpus readers
701 * interface to full Penn Treebank Corpus V3 (corpus.ptb)
702 * made wordnet.lemmas case-insensitive
703 * more flexible padding in model.ngram
704 * minor bugfixes and documentation enhancements
705 * better support for automated testing
706
707 Thanks to the following contributors to 2.0.2:
708 Daniel Blanchard, Mikhail Korobov, Nitin Madnani, Duncan McGreggor,
709 Morten Neergaard, Nathan Schneider, Rico Sennrich.
710
711 Version 2.0.1 2012-05-15
712
713 * moved NLTK to GitHub: https://github.com/nltk
714 * set up integration testing: https://jenkins.shiningpanda.com/nltk/ (Morten Neergaard)
715 * converted documentation to Sphinx format: https://www.nltk.org/api/nltk.html
716 * dozens of minor enhancements and bugfixes: https://github.com/nltk/nltk/commits/
717 * dozens of fixes for conformance with PEP-8
718 * dozens of fixes to ensure operation with Python 2.5
719 * added interface to Lin's Dependency Thesaurus (Dan Blanchard)
720 * added interface to scikit-learn classifiers (Lars Buitinck)
721 * added segmentation evaluation measures (David Doukhan)
722
723 Thanks to the following contributors to 2.0.1 (since 2.0b9, July 2010):
724 Rami Al-Rfou', Yonatan Becker, Steven Bethard, Daniel Blanchard, Lars
725 Buitinck, David Coles, Lucas Cooper, David Doukhan, Dan Garrette,
726 Masato Hagiwara, Michael Hansen, Michael Heilman, Rebecca Ingram,
727 Sudharshan Kaushik, Mikhail Korobov, Peter Ljunglof, Nitin Madnani,
728 Rob Malouf, Tomonori Nagano, Morten Neergaard, David Nemeskey,
729 Joel Nothman, Jacob Perkins, Alessandro Presta, Alex Rudnick,
730 Nathan Schneider, Stefano Lattarini, Peter Stahl, Jason Yoder
731
732 Version 2.0.1 (rc1) 2011-04-11
733
734 NLTK:
735 * added interface to the Stanford POS Tagger
736 * updates to sem.Boxer, sem.drt.DRS
737 * allow unicode strings in grammars
738 * allow non-string features in classifiers
739 * modifications to HunposTagger
740 * issues with DRS printing
741 * fixed bigram collocation finder for window_size > 2
742 * doctest paths no longer presume unix-style pathname separators
743 * fixed issue with NLTK's tokenize module colliding with the Python tokenize module
744 * fixed issue with stemming Unicode strings
745 * changed ViterbiParser.nbest_parse to parse
746 * ChaSen and KNBC Japanese corpus readers
747 * preserve case in concordance display
748 * fixed bug in simplification of Brown tags
749 * a version of IBM Model 1 as described in Koehn 2010
750 * new class AlignedSent for aligned sentence data and evaluation metrics
751 * new nltk.util.set_proxy to allow easy configuration of HTTP proxy
752 * improvements to downloader user interface to catch URL and HTTP errors
753 * added CHILDES corpus reader
754 * created special exception hierarchy for Prover9 errors
755 * significant changes to the underlying code of the boxer interface
756 * path-based wordnet similarity metrics use a fake root node for verbs, following the Perl version
757 * added ability to handle multi-sentence discourses in Boxer
758 * added the 'english' Snowball stemmer
759 * simplifications and corrections of Earley Chart Parser rules
760 * several changes to the feature chart parsers for correct unification
761 * bugfixes: FreqDist.plot, FreqDist.max, NgramModel.entropy, CategorizedCorpusReader, DecisionTreeClassifier
762 * removal of Python >2.4 language features for 2.4 compatibility
763 * removal of deprecated functions and associated warnings
764 * added semantic domains to wordnet corpus reader
765 * changed wordnet similarity functions to include instance hyponyms
766 * updated to use latest version of Boxer
767
768 Data:
769 * JEITA Public Morphologically Tagged Corpus (in ChaSen format)
770 * KNB Annotated corpus of Japanese blog posts
771 * Fixed some minor bugs in alvey.fcfg, and added number of parse trees in alvey_sentences.txt
772 * added more comtrans data
773
774 Documentation:
775 * minor fixes to documentation
776 * NLTK Japanese book (chapter 12) by Masato Hagiwara
777
778 NLTK-Contrib:
779 * Viethen and Dale referring expression algorithms
780
781
782 Version 2.0b9 2010-07-25
783
784 NLTK:
785 * many code and documentation cleanups
786 * Added port of Snowball stemmers
787 * Fixed loading of pickled tokenizers (issue 556)
788 * DecisionTreeClassifier now handles unknown features (issue 570)
789 * Added error messages to LogicParser
790 * Replaced max_models with end_size to prevent Mace from hanging
791 * Added interface to Boxer
792 * Added nltk.corpus.semcor to give access to SemCor 3.0 corpus (issue 530)
793 * Added support for integer- and float-valued features in maxent classifiers
794 * Permit NgramModels to be pickled
795 * Added Sourced Strings (see test/sourcedstring.doctest for details)
796 * Fixed bugs in with Good-Turing and Simple Good-Turing Estimation (issue 26)
797 * Add support for span tokenization, aka standoff annotation of segmentation (incl Punkt)
798 * allow unicode nodes in Tree.productions()
799 * Fixed WordNet's morphy to be consistent with the original implementation,
800 taking the shortest returned form instead of an arbitrary one (issues 427, 487)
801 * Fixed bug in MaxentClassifier
802 * Accepted bugfixes for YCOE corpus reader (issue 435)
803 * Added test to _cumulative_frequencies() to correctly handle the case when no arguments are supplied
804 * Added a TaggerI interface to the HunPos open-source tagger
805 * Return 0, not None, when no count is present for a lemma in WordNet
806 * fixed pretty-printing of unicode leaves
807 * More efficient calculation of the leftcorner relation for left corner parsers
808 * Added two functions for graph calculations: transitive closure and inversion.
809 * FreqDist.pop() and FreqDist.popitems() now invalidate the caches (issue 511)
810
811 Data:
812 * Added SemCor 3.0 corpus (Brown Corpus tagged with WordNet synsets)
813 * Added LanguageID corpus (trigram counts for 451 languages)
814 * Added grammar for a^n b^n c^n
815
816 NLTK-Contrib:
817 * minor updates
818
819 Thanks to the following contributors to 2.0b9:
820
821 Steven Bethard, Francis Bond, Dmitry Chichkov, Liang Dong, Dan Garrette,
822 Simon Greenhill, Bjorn Maeland, Rob Malouf, Joel Nothman, Jacob Perkins,
823 Alberto Planas, Alex Rudnick, Geoffrey Sampson, Kevin Scannell, Richard Sproat
824
825
826 Version 2.0b8 2010-02-05
827
828 NLTK:
829 * fixed copyright and license statements
830 * removed PyYAML, and added dependency to installers and download instructions
831 * updated to LogicParser, DRT (Dan Garrette)
832 * WordNet similarity metrics return None instead of -1 when
833 they fail to find a path (Steve Bethard)
834 * shortest_path_distance uses instance hypernyms (Jordan Boyd-Graber)
835 * clean_html improved (Bjorn Maeland)
836 * batch_parse, batch_interpret and batch_evaluate functions allow
837 grammar or grammar filename as argument
838 * more Portuguese examples (portuguese_en.doctest, examples/pt.py)
839
840 NLTK-Contrib:
841 * Aligner implementations (Christopher Crowner, Torsten Marek)
842 * ScriptTranscriber package (Richard Sproat and Kristy Hollingshead)
843
844 Book:
845 * updates for second printing, correcting errata
846 https://nltk.googlecode.com/svn/trunk/nltk/doc/book/errata.txt
847
848 Data:
849 * added Europarl sample, with 10 docs for each of 11 langs (Nitin Madnani)
850 * added SMULTRON sample corpus (Torsten Marek, Martin Volk)
851
852
853 Version 2.0b7 2009-11-09
854
855 NLTK:
856 * minor bugfixes and enhancements: data loader, inference package, FreqDist, Punkt
857 * added Portuguese example module, similar to nltk.book for English (examples/pt.py)
858 * added all_lemma_names() method to WordNet corpus reader
859 * added update() and __add__() extensions to FreqDist (enhances alignment with Python 3.0 counters)
860 * reimplemented clean_html
861 * added test-suite runner for automatic/manual regression testing
862
863 NLTK-Data:
864 * updated Punkt models for sentence segmentation
865 * added corpus of the works of Machado de Assis (Brazilian Portuguese)
866
867 Book:
868 * Added translation of preface into Portuguese, contributed by Tiago Tresoldi.
869
870 Version 2.0b6 2009-09-20
871
872 NLTK:
873 * minor fixes for Python 2.4 compatibility
874 * added words() method to XML corpus reader
875 * minor bugfixes and code clean-ups
876 * fixed downloader to put data in %APPDATA% on Windows
877
878 Data:
879 * Updated Punkt models
880 * Fixed utf8 encoding issues with UDHR and Stopwords Corpora
881 * Renamed CoNLL "cat" files to "esp" (different language)
882 * Added Alvey NLT feature-based grammar
883 * Added Polish PL196x corpus
884
885 Version 2.0b5 2009-07-19
886
887 NLTK:
888 * minor bugfixes (incl FreqDist, Python eggs)
889 * added reader for Europarl Corpora (contributed by Nitin Madnani)
890 * added reader for IPI PAN Polish Corpus (contributed by Konrad Goluchowski)
891 * fixed data.py so that it doesn't generate a warning for Windows Python 2.6
892
893 NLTK-Contrib:
894 * updated Praat reader (contributed by Margaret Mitchell)
895
896 Version 2.0b4 2009-07-10
897
898 NLTK:
899 * switched to Apache License, Version 2.0
900 * minor bugfixes in semantics and inference packages
901 * support for Python eggs
902 * fixed stale regression tests
903
904 Data:
905 * added NomBank 1.0
906 * uppercased feature names in some grammars
907
908 Version 2.0b3 2009-06-25
909
910 NLTK:
911 * several bugfixes
912 * added nombank corpus reader (Paul Bedaride)
913
914 Version 2.0b2 2009-06-15
915
916 NLTK:
917 * minor bugfixes and optimizations for parsers, updated some doctests
918 * added bottom-up filtered left corner parsers,
919 LeftCornerChartParser and IncrementalLeftCornerChartParser.
920 * fixed dispersion plot bug which prevented empty plots
921
922 Version 2.0b1 2009-06-09
923
924 NLTK:
925 * major refactor of chart parser code and improved API (Peter Ljungl喃)
926 * added new bottom-up left-corner chart parser strategy
927 * misc bugfixes (ChunkScore, chart rules, chatbots, jcn-similarity)
928 * improved efficiency of "import nltk" using lazy module imports
929 * moved CCG package and ISRI Arabic stemmer from NLTK-Contrib into core NLTK
930 * misc code cleanups
931
932 Contrib:
933 * moved out of the main NLTK distribution into a separate distribution
934
935 Book:
936 * Ongoing polishing ahead of print publication
937
938 Version 0.9.9 2009-05-06
939
940 NLTK:
941 * Finalized API for NLTK 2.0 and the book, incl dozens of small fixes
942 * Names of the form nltk.foo.Bar now available as nltk.Bar
943 for significant functionality; in some cases the name was modified
944 (using old names will produce a deprecation warning)
945 * Bugfixes in downloader, WordNet
946 * Expanded functionality in DecisionTree
947 * Bigram collocations extended for discontiguous bigrams
948 * Translation toy nltk.misc.babelfish
949 * New module nltk.help giving access to tagset documentation
950 * Fix imports so that NLTK builds without Tkinter (Bjorn Maeland)
951
952 Data:
953 * new maxent NE chunker model
954 * updated grammar packages for the book
955 * data for new tagsets collection, documenting several tagsets
956 * added lolcat translation to the Genesis collection
957
958 Contrib (work in progress):
959 * Updates to coreference package (Joseph Frazee)
960 * New ISRI Arabic stemmer (Hosam Algasaier)
961 * Updates to Toolbox package (Greg Aumann)
962
963 Book:
964 * Substantial editorial corrections ahead of final submission
965
966 Version 0.9.8 2009-02-18
967
968 NLTK:
969 * New off-the-shelf tokenizer, POS tagger, and named-entity tagger
970 * New metrics package with inter-annotator agreement scores,
971 distance metrics, rank correlation
972 * New collocations package (Joel Nothman)
973 * Many clean-ups to WordNet package (Steven Bethard, Jordan Boyd-Graber)
974 * Moved old pywordnet-based WordNet package to nltk_contrib
975 * WordNet browser (Paul Bone)
976 * New interface to dependency treebank corpora
977 * Moved MinimalSet class into nltk.misc package
978 * Put NLTK applications in new nltk.app package
979 * Many other improvements incl semantics package, toolbox, MaltParser
980 * Misc changes to many API names in preparation for 1.0, old names deprecated
981 * Most classes now available in the top-level namespace
982 * Work on Python egg distribution (Brandon Rhodes)
983 * Removed deprecated code remaining from 0.8.* versions
984 * Fixes for Python 2.4 compatibility
985
986 Data:
987 * Corrected identifiers in Dependency Treebank corpus
988 * Basque and Catalan Dependency Treebanks (CoNLL 2007)
989 * PE08 Parser Evaluation data
990 * New models for POS tagger and named-entity tagger
991
992 Book:
993 * Substantial editorial corrections
994
995 Version 0.9.7 2008-12-19
996
997 NLTK:
998 * fixed problems with accessing zipped corpora
999 * improved design and efficiency of grammars and chart parsers
1000 including new bottom-up combine strategy and a redesigned
1001 Earley strategy (Peter Ljunglof)
1002 * fixed bugs in smoothed probability distributions and added
1003 regression tests (Peter Ljunglof)
1004 * improvements to Punkt (Joel Nothman)
1005 * improvements to text classifiers
1006 * simple word-overlap RTE classifier
1007
1008 Data:
1009 * A new package of large grammars (Peter Ljunglof)
1010 * A small gazetteer corpus and corpus reader
1011 * Organized example grammars into separate packages
1012 * Childrens' stories added to gutenberg package
1013
1014 Contrib (work in progress):
1015 * fixes and demonstration for named-entity feature extractors in nltk_contrib.coref
1016
1017 Book:
1018 * extensive changes throughout, including new chapter 5 on classification
1019 and substantially revised chapter 11 on managing linguistic data
1020
1021 Version 0.9.6 2008-12-07
1022
1023 NLTK:
1024 * new WordNet corpus reader (contributed by Steven Bethard)
1025 * incorporated dependency parsers into NLTK (was NLTK-Contrib) (contributed by Jason Narad)
1026 * moved nltk/cfg.py to nltk/grammar.py and incorporated dependency grammars
1027 * improved efficiency of unification algorithm
1028 * various enhancements to the semantics package
1029 * added plot() and tabulate() methods to FreqDist and ConditionalFreqDist
1030 * FreqDist.keys() and list(FreqDist) provide keys reverse-sorted by value,
1031 to avoid the confusion caused by FreqDist.sorted()
1032 * new downloader module to support interactive data download: nltk.download()
1033 run using "python -m nltk.downloader all"
1034 * fixed WordNet bug that caused min_depth() to sometimes give incorrect result
1035 * added nltk.util.Index as a wrapper around defaultdict(list) plus
1036 a functional-style initializer
1037 * fixed bug in Earley chart parser that caused it to break
1038 * added basic TnT tagger nltk.tag.tnt
1039 * new corpus reader for CoNLL dependency format (contributed by Kepa Sarasola and Iker Manterola)
1040 * misc other bugfixes
1041
1042 Contrib (work in progress):
1043 * TIGERSearch implementation by Torsten Marek
1044 * extensions to hole and glue semantics modules by Dan Garrette
1045 * new coreference package by Joseph Frazee
1046 * MapReduce interface by Xinfan Meng
1047
1048 Data:
1049 * Corpora are stored in compressed format if this will not compromise speed of access
1050 * Swadesh Corpus of comparative wordlists in 23 languages
1051 * Split grammar collection into separate packages
1052 * New Basque and Spanish grammar samples (contributed by Kepa Sarasola and Iker Manterola)
1053 * Brown Corpus sections now have meaningful names (e.g. 'a' is now 'news')
1054 * Fixed bug that forced users to manually unzip the WordNet corpus
1055 * New dependency-parsed version of Treebank corpus sample
1056 * Added movie script "Monty Python and the Holy Grail" to webtext corpus
1057 * Replaced words corpus data with a much larger list of English words
1058 * New URL for list of available NLTK corpora
1059 https://nltk.googlecode.com/svn/trunk/nltk_data/index.xml
1060
1061 Book:
1062 * complete rewrite of first three chapters to make the book accessible
1063 to a wider audience
1064 * new chapter on data-intensive language processing
1065 * extensive reworking of most chapters
1066 * Dropped subsection numbering; moved exercises to end of chapters
1067
1068 Distributions:
1069 * created Portfile to support Mac installation
1070
1071
1072 Version 0.9.5 2008-08-27
1073
1074 NLTK:
1075 * text module with support for concordancing, text generation, plotting
1076 * book module
1077 * Major reworking of the automated theorem proving modules (Dan Garrette)
1078 * draw.dispersion now uses pylab
1079 * draw.concordance GUI tool
1080 * nltk.data supports for reading corpora and other data files from within zipfiles
1081 * trees can be constructed from strings with Tree(s) (cf Tree.parse(s))
1082
1083 Contrib (work in progress):
1084 * many updates to student projects
1085 - nltk_contrib.agreement (Thomas Lippincott)
1086 - nltk_contrib.coref (Joseph Frazee)
1087 - nltk_contrib.depparser (Jason Narad)
1088 - nltk_contrib.fuf (Petro Verkhogliad)
1089 - nltk_contrib.hadoop (Xinfan Meng)
1090 * clean-ups: deleted stale files; moved some packages to misc
1091
1092 Data
1093 * Cleaned up Gutenberg text corpora
1094 * added Moby Dick; removed redundant copy of Blake songs.
1095 * more tagger models
1096 * renamed to nltk_data to facilitate installation
1097 * stored each corpus as a zip file for quicker installation
1098 and access, and to solve a problem with the Propbank
1099 corpus including a file with an illegal name for MSWindows
1100 (con.xml).
1101
1102 Book:
1103 * changed filenames to chNN format
1104 * reworked opening chapters (work in progress)
1105
1106 Distributions:
1107 * fixed problem with mac installer that arose when Python binary
1108 couldn't be found
1109 * removed dependency of NLTK on nltk_data so that NLTK code can be
1110 installed before the data
1111
1112 Version 0.9.4 2008-08-01
1113
1114 NLTK:
1115 - Expanded semantics package for first order logic, linear logic,
1116 glue semantics, DRT, LFG (Dan Garrette)
1117 - new WordSense class in wordnet.synset supporting access to synsets
1118 from sense keys and accessing sense counts (Joel Nothman)
1119 - interface to Mallet's linear chain CRF implementation (nltk.tag.crf)
1120 - misc bugfixes incl Punkt, synsets, maxent
1121 - improved support for chunkers incl flexible chunk corpus reader,
1122 new rule type: ChunkRuleWithContext
1123 - new GUI for pos-tagged concordancing nltk.draw.pos_concordance
1124 - new GUI for developing regexp chunkers nltk.draw.rechunkparser
1125 - added bio_sents() and bio_words() methods to ConllChunkCorpusReader in conll.py
1126 to allow reading (word, tag, chunk_typ) tuples off of CoNLL-2000 corpus. Also
1127 modified ConllChunkCorpusView to support these changes.
1128 - feature structures support values with custom unification methods
1129 - new flag on tagged corpus readers to use simplified tagsets
1130 - new package for ngram language modeling with Katz backoff nltk.model
1131 - added classes for single-parented and multi-parented trees that
1132 automatically maintain parent pointers (nltk.tree.ParentedTree and
1133 nltk.tree.MultiParentedTree)
1134 - new WordNet browser GUI (Jussi Salmela, Paul Bone)
1135 - improved support for lazy sequences
1136 - added generate() method to probability distributions
1137 - more flexible parser for converting bracketed strings to trees
1138 - made fixes to docstrings to improve API documentation
1139
1140 Contrib (work in progress)
1141 - new NLG package, FUF/SURGE (Petro Verkhogliad)
1142 - new dependency parser package (Jason Narad)
1143 - new Coreference package, incl support for
1144 ACE-2, MUC-6 and MUC-7 corpora (Joseph Frazee)
1145 - CCG Parser (Graeme Gange)
1146 - first order resolution theorem prover (Dan Garrette)
1147
1148 Data:
1149 - Nnw NPS Chat Corpus and corpus reader (nltk.corpus.nps_chat)
1150 - ConllCorpusReader can now be used to read CoNLL 2004 and 2005 corpora.
1151 - Implemented HMM-based Treebank POS tagger and phrase chunker for
1152 nltk_contrib.coref in api.py. Pickled versions of these objects are checked
1153 in in data/taggers and data/chunkers.
1154
1155 Book:
1156 - misc corrections in response to feedback from readers
1157
1158 Version 0.9.3 2008-06-03
1159
1160 NLTK:
1161 - modified WordNet similarity code to use pre-built information content files
1162 - new classifier-based tagger, BNC corpus reader
1163 - improved unicode support for corpus readers
1164 - improved interfaces to Weka, Prover9/Mace4
1165 - new support for using MEGAM and SciPy to train maxent classifiers
1166 - rewrite of Punkt sentence segmenter (Joel Nothman)
1167 - bugfixes for WordNet information content module (Jordan Boyd-Graber)
1168 - code clean-ups throughout
1169
1170 Book:
1171 - miscellaneous fixes in response to feedback from readers
1172
1173 Contrib:
1174 - implementation of incremental algorithm for generating
1175 referring expressions (contributed by Margaret Mitchell)
1176 - refactoring WordNet browser (Paul Bone)
1177
1178 Corpora:
1179 - included WordNet information content files
1180
1181 Version 0.9.2 2008-03-04
1182
1183 NLTK:
1184 - new theorem-prover and model-checker module nltk.inference,
1185 including interface to Prover9/Mace4 (Dan Garrette, Ewan Klein)
1186 - bugfix in Reuters corpus reader that causes Python
1187 to complain about too many open files
1188 - VerbNet and PropBank corpus readers
1189
1190 Data:
1191 - VerbNet Corpus version 2.1: hierarchical, verb lexicon linked to WordNet
1192 - PropBank Corpus: predicate-argument structures, as stand-off annotation of Penn Treebank
1193
1194 Contrib:
1195 - New work on WordNet browser, incorporating a client-server model (Jussi Salmela)
1196
1197 Distributions:
1198 - Mac OS 10.5 distribution
1199
1200 Version 0.9.1 2008-01-24
1201
1202 NLTK:
1203 - new interface for text categorization corpora
1204 - new corpus readers: RTE, Movie Reviews, Question Classification, Brown Corpus
1205 - bugfix in ConcatenatedCorpusView that caused iteration to fail if it didn't start from the beginning of the corpus
1206
1207 Data:
1208 - Question classification data, included with permission of Li & Roth
1209 - Reuters 21578 Corpus, ApteMod version, from CPAN
1210 - Movie Reviews corpus (sentiment polarity), included with permission of Lillian Lee
1211 - Corpus for Recognising Textual Entailment (RTE) Challenges 1, 2 and 3
1212 - Brown Corpus (reverted to original file structure: ca01-cr09)
1213 - Penn Treebank corpus sample (simplified implementation, new readers treebank_raw and treebank_chunk)
1214 - Minor redesign of corpus readers, to use filenames instead of "items" to identify parts of a corpus
1215
1216 Contrib:
1217 - theorem_prover: Prover9, tableau, MaltParser, Mace4, glue semantics, docs (Dan Garrette, Ewan Klein)
1218 - drt: improved drawing, conversion to FOL (Dan Garrette)
1219 - gluesemantics: GUI demonstration, abstracted LFG code, documentation (Dan Garrette)
1220 - readability: various text readability scores (Thomas Jakobsen, Thomas Skardal)
1221 - toolbox: code to normalize toolbox databases (Greg Aumann)
1222
1223 Book:
1224 - many improvements in early chapters in response to reader feedback
1225 - updates for revised corpus readers
1226 - moved unicode section to chapter 3
1227 - work on engineering.txt (not included in 0.9.1)
1228
1229 Distributions:
1230 - Fixed installation for Mac OS 10.5 (Joshua Ritterman)
1231 - Generalize doctest_driver to work with doc_contrib
1232
1233 Version 0.9 2007-10-12
1234
1235 NLTK:
1236 - New naming of packages and modules, and more functions imported into
1237 top-level nltk namespace, e.g. nltk.chunk.Regexp -> nltk.RegexpParser,
1238 nltk.tokenize.Line -> nltk.LineTokenizer, nltk.stem.Porter -> nltk.PorterStemmer,
1239 nltk.parse.ShiftReduce -> nltk.ShiftReduceParser
1240 - processing class names changed from verbs to nouns, e.g.
1241 StemI -> StemmerI, ParseI -> ParserI, ChunkParseI -> ChunkParserI, ClassifyI -> ClassifierI
1242 - all tokenizers are now available as subclasses of TokenizeI,
1243 selected tokenizers are also available as functions, e.g. wordpunct_tokenize()
1244 - rewritten ngram tagger code, collapsed lookup tagger with unigram tagger
1245 - improved tagger API, permitting training in the initializer
1246 - new system for deprecating code so that users are notified of name changes.
1247 - support for reading feature cfgs to parallel reading cfgs (parse_featcfg())
1248 - text classifier package, maxent (GIS, IIS), naive Bayes, decision trees, weka support
1249 - more consistent tree printing
1250 - wordnet's morphy stemmer now accessible via stemmer package
1251 - RSLP Portuguese stemmer (originally developed by Viviane Moreira Orengo, reimplemented by Tiago Tresoldi)
1252 - promoted ieer_rels.py to the sem package
1253 - improvements to WordNet package (Jussi Salmela)
1254 - more regression tests, and support for checking coverage of tests
1255 - miscellaneous bugfixes
1256 - remove numpy dependency
1257
1258 Data:
1259 - new corpus reader implementation, refactored syntax corpus readers
1260 - new data package: corpora, grammars, tokenizers, stemmers, samples
1261 - CESS-ESP Spanish Treebank and corpus reader
1262 - CESS-CAT Catalan Treebank and corpus reader
1263 - Alpino Dutch Treebank and corpus reader
1264 - MacMorpho POS-tagged Brazilian Portuguese news text and corpus reader
1265 - trained model for Portuguese sentence segmenter
1266 - Floresta Portuguese Treebank version 7.4 and corpus reader
1267 - TIMIT player audio support
1268
1269 Contrib:
1270 - BioReader (contributed by Carlos Rodriguez)
1271 - TnT tagger (contributed by Sam Huston)
1272 - wordnet browser (contributed by Jussi Salmela, requires wxpython)
1273 - lpath interpreter (contributed by Haejoong Lee)
1274 - timex -- regular expression-based temporal expression tagger
1275
1276 Book:
1277 - polishing of early chapters
1278 - introductions to parts 1, 2, 3
1279 - improvements in book processing software (xrefs, avm & gloss formatting, javascript clipboard)
1280 - updates to book organization, chapter contents
1281 - corrections throughout suggested by readers (acknowledged in preface)
1282 - more consistent use of US spelling throughout
1283 - all examples redone to work with single import statement: "import nltk"
1284 - reordered chapters: 5->7->8->9->11->12->5
1285 * language engineering in part 1 to broaden the appeal
1286 of the earlier part of the book and to talk more about
1287 evaluation and baselines at an earlier stage
1288 * concentrate the partial and full parsing material in part 2,
1289 and remove the specialized feature-grammar material into part 3
1290
1291 Distributions:
1292 - streamlined mac installation (Joshua Ritterman)
1293 - included mac distribution with ISO image
1294
1295 Version 0.8 2007-07-01
1296
1297 Code:
1298 - changed nltk.__init__ imports to explicitly import names from top-level modules
1299 - changed corpus.util to use the 'rb' flag for opening files, to fix problems
1300 reading corpora under MSWindows
1301 - updated stale examples in engineering.txt
1302 - extended feature structure interface to permit chained features, e.g. fs['F','G']
1303 - further misc improvements to test code plus some bugfixes
1304 Tutorials:
1305 - rewritten opening section of tagging chapter
1306 - reorganized some exercises
1307
1308 Version 0.8b2 2007-06-26
1309
1310 Code (major):
1311 - new corpus package, obsoleting old corpora package
1312 - supports caching, slicing, corpus search path
1313 - more flexible API
1314 - global updates so all NLTK modules use new corpus package
1315 - moved nltk/contrib to separate top-level package nltk_contrib
1316 - changed wordpunct tokenizer to use \w instead of a-zA-Z0-9
1317 as this will be more robust for languages other than English,
1318 with implications for many corpus readers that use it
1319 - known bug: certain re-entrant structures in featstruct
1320 - known bug: when the LHS of an edge contains an ApplicationExpression,
1321 variable values in the RHS bindings aren't copied over when the
1322 fundamental rule applies
1323 - known bug: HMM tagger is broken
1324 Tutorials:
1325 - global updates to NLTK and docs
1326 - ongoing polishing
1327 Corpora:
1328 - treebank sample reverted to published multi-file structure
1329 Contrib:
1330 - DRT and Glue Semantics code (nltk_contrib.drt, nltk_contrib.gluesemantics, by Dan Garrette)
1331
1332 Version 0.8b1 2007-06-18
1333
1334 Code (major):
1335 - changed package name to nltk
1336 - import all top-level modules into nltk, reducing need for import statements
1337 - reorganization of sub-package structures to simplify imports
1338 - new featstruct module, unifying old featurelite and featurestructure modules
1339 - FreqDist now inherits from dict, fd.count(sample) becomes fd[sample]
1340 - FreqDist initializer permits: fd = FreqDist(len(token) for token in text)
1341 - made numpy optional
1342 Code (minor):
1343 - changed GrammarFile initializer to accept filename
1344 - consistent tree display format
1345 - fixed loading process for WordNet and TIMIT that prevented code installation if data not installed
1346 - taken more care with unicode types
1347 - incorporated pcfg code into cfg module
1348 - moved cfg, tree, featstruct to top level
1349 - new filebroker module to make handling of example grammar files more transparent
1350 - more corpus readers (webtext, abc)
1351 - added cfg.covers() to check that a grammar covers a sentence
1352 - simple text-based wordnet browser
1353 - known bug: parse/featurechart.py uses incorrect apply() function
1354 Corpora:
1355 - csv data file to document NLTK corpora
1356 Contrib:
1357 - added Glue semantics code (contrib.glue, by Dan Garrette)
1358 - Punkt sentence segmenter port (contrib.punkt, by Willy)
1359 - added LPath interpreter (contrib.lpath, by Haejoong Lee)
1360 - extensive work on classifiers (contrib.classifier*, Sumukh Ghodke)
1361 Tutorials:
1362 - polishing on parts I, II
1363 - more illustrations, data plots, summaries, exercises
1364 - continuing to make prose more accessible to non-linguistic audience
1365 - new default import that all chapters presume: from nltk.book import *
1366 Distributions:
1367 - updated to latest version of numpy
1368 - removed WordNet installation instructions as WordNet is now included in corpus distribution
1369 - added pylab (matplotlib)
1370
1371 Version 0.7.5 2007-05-16
1372
1373 Code:
1374 - improved WordNet and WordNet-Similarity interface
1375 - the Lancaster Stemmer (contributed by Steven Tomcavage)
1376 Corpora:
1377 - Web text samples
1378 - BioCreAtIvE-PPI - a corpus for protein-protein interactions
1379 - Switchboard Telephone Speech Corpus Sample (via Talkbank)
1380 - CMU Problem Reports Corpus sample
1381 - CONLL2002 POS+NER data
1382 - Patient Information Leaflet corpus
1383 - WordNet 3.0 data files
1384 - English wordlists: basic English, frequent words
1385 Tutorials:
1386 - more improvements to text and images
1387
1388 Version 0.7.4 2007-05-01
1389
1390 Code:
1391 - Indian POS tagged corpus reader: corpora.indian
1392 - Sinica Treebank corpus reader: corpora.sinica_treebank
1393 - new web corpus reader corpora.web
1394 - tag package now supports pickling
1395 - added function to utilities.py to guess character encoding
1396 Corpora:
1397 - Rotokas texts from Stuart Robinson
1398 - POS-tagged corpora for several Indian languages (Bangla, Hindi, Marathi, Telugu) from A Kumaran
1399 Tutorials:
1400 - Substantial work on Part II of book on structured programming, parsing and grammar
1401 - More bibliographic citations
1402 - Improvements in typesetting, cross references
1403 - Redimensioned images and tables for better use of page space
1404 - Moved project list to wiki
1405 Contrib:
1406 - validation of toolbox entries using chunking
1407 - improved classifiers
1408 Distribution:
1409 - updated for Python 2.5.1, Numpy 1.0.2
1410
1411 Version 0.7.3 2007-04-02
1412
1413 * Code:
1414 - made chunk.Regexp.parse() more flexible about its input
1415 - developed new syntax for PCFG grammars, e.g. A -> B C [0.3] | D [0.7]
1416 - fixed CFG parser to support grammars with slash categories
1417 - moved beta classify package from main NLTK to contrib
1418 - Brill taggers loaded correctly
1419 - misc bugfixes
1420 * Corpora:
1421 - Shakespeare XML corpus sample and corpus reader
1422 * Tutorials:
1423 - improvements to prose, exercises, plots, images
1424 - expanded and reorganized tutorial on structured programming
1425 - formatting improvements for Python listings
1426 - improved plots (using pylab)
1427 - categorization of problems by difficulty
1428 Contrib:
1429 - more work on kimmo lexicon and grammar
1430 - more work on classifiers
1431
1432 Version 0.7.2 2007-03-01
1433
1434 * Code:
1435 - simple feature detectors (detect module)
1436 - fixed problem when token generators are passed to a parser (parse package)
1437 - fixed bug in Grammar.productions() (identified by Lucas Champollion and Mitch Marcus)
1438 - fixed import bug in category.GrammarFile.earley_parser
1439 - added utilities.OrderedDict
1440 - initial port of old NLTK classifier package (by Sam Huston)
1441 - UDHR corpus reader
1442 * Corpora:
1443 - added UDHR corpus (Universal Declaration of Human Rights)
1444 with 10k text samples in 300+ languages
1445 * Tutorials:
1446 - improved images
1447 - improved book formatting, including new support for:
1448 - javascript to copy program examples to clipboard in HTML version,
1449 - bibliography, chapter cross-references, colorization, index, table-of-contents
1450
1451 * Contrib:
1452 - new Kimmo system: contrib.mit.six863.kimmo (Rob Speer)
1453 - fixes for: contrib.fsa (Rob Speer)
1454 - demonstration of text classifiers trained on UDHR corpus for
1455 language identification: contrib.langid (Sam Huston)
1456 - new Lambek calculus system: contrib.lambek
1457 - new tree implementation based on elementtree: contrib.tree
1458
1459 Version 0.7.1 2007-01-14
1460
1461 * Code:
1462 - bugfixes (HMM, WordNet)
1463
1464 Version 0.7 2006-12-22
1465
1466 * Code:
1467 - bugfixes, including fixed bug in Brown corpus reader
1468 - cleaned up wordnet 2.1 interface code and similarity measures
1469 - support for full Penn treebank format contributed by Yoav Goldberg
1470 * Tutorials:
1471 - expanded tutorials on advanced parsing and structured programming
1472 - checked all doctest code
1473 - improved images for chart parsing
1474
1475 Version 0.7b1 2006-12-06
1476
1477 * Code:
1478 - expanded semantic interpretation package
1479 - new high-level chunking interface, with cascaded chunking
1480 - split chunking code into new chunk package
1481 - updated wordnet package to support version 2.1 of Wordnet.
1482 - prototyped basic wordnet similarity measures
1483 (path distance, Wu + Palmer and Leacock + Chodorow, Resnik similarity measures.)
1484 - bugfixes (tag.Window, tag.ngram)
1485 - more doctests
1486 * Contrib:
1487 - toolbox language settings module
1488 * Tutorials:
1489 - rewrite of chunking chapter, switched from Treebank to CoNLL format as main focus,
1490 simplified evaluation framework, added ngram chunking section
1491 - substantial updates throughout (esp programming and semantics chapters)
1492 * Corpora:
1493 - Chat-80 Prolog data files provided as corpora, plus corpus reader
1494
1495 Version 0.7a2 2006-11-13
1496
1497 * Code:
1498 - more doctest
1499 - code to read Chat-80 data
1500 - HMM bugfix
1501 * Tutorials:
1502 - continued updates and polishing
1503 * Corpora:
1504 - toolbox MDF sample data
1505
1506 Version 0.7a1 2006-10-29
1507
1508 * Code:
1509 - new toolbox module (Greg Aumann)
1510 - new semantics package (Ewan Klein)
1511 - bugfixes
1512 * Tutorials
1513 - substantial revision, especially in preface, introduction, words,
1514 and semantics chapters.
1515
1516 Version 0.6.6 2006-10-06
1517
1518 * Code:
1519 - bugfixes (probability, shoebox, draw)
1520 * Contrib:
1521 - new work on shoebox package (Stuart Robinson)
1522 * Tutorials:
1523 - continual expansion and revision, especially on introduction to
1524 programming, advanced programming and the feature-based grammar chapters.
1525
1526 Version 0.6.5 2006-07-09
1527
1528 * Code:
1529 - improvements to shoebox module (Stuart Robinson, Greg Aumann)
1530 - incorporated feature-based parsing into core NLTK-Lite
1531 - corpus reader for Sinica treebank sample
1532 - new stemmer package
1533 * Contrib:
1534 - hole semantics implementation (Peter Wang)
1535 - Incorporating yaml
1536 - new work on feature structures, unification, lambda calculus
1537 - new work on shoebox package (Stuart Robinson, Greg Aumann)
1538 * Corpora:
1539 - Sinica treebank sample
1540 * Tutorials:
1541 - expanded discussion throughout, incl: left-recursion, trees, grammars,
1542 feature-based grammar, agreement, unification, PCFGs,
1543 baseline performance, exercises, improved display of trees
1544
1545 Version 0.6.4 2006-04-20
1546
1547 * Code:
1548 - corpus readers for Senseval 2 and TIMIT
1549 - clusterer (ported from old NLTK)
1550 - support for cascaded chunkers
1551 - bugfix suggested by Brent Payne
1552 - new SortedDict class for regression testing
1553 * Contrib:
1554 - CombinedTagger tagger and marshalling taggers, contributed by Tiago Tresoldi
1555 * Corpora:
1556 - new: Senseval 2, TIMIT sample
1557 * Tutorials:
1558 - major revisions to programming, words, tagging, chunking, and parsing tutorials
1559 - many new exercises
1560 - formatting improvements, including colorized program examples
1561 - fixed problem with testing on training data, reported by Jason Baldridge
1562
1563 Version 0.6.3 2006-03-09
1564
1565 * switch to new style classes
1566 * repair FSA model sufficiently for Kimmo module to work
1567 * port of MIT Kimmo morphological analyzer; still needs lots of code clean-up and inline docs
1568 * expanded support for shoebox format, developed with Stuart Robinson
1569 * fixed bug in indexing CFG productions, for empty right-hand-sides
1570 * efficiency improvements, suggested by Martin Ranang
1571 * replaced classeq with isinstance, for efficiency improvement, as suggested by Martin Ranang
1572 * bugfixes in chunk eval
1573 * simplified call to draw_trees
1574 * names, stopwords corpora
1575
1576 Version 0.6.2 2006-01-29
1577
1578 * Peter Spiller's concordancer
1579 * Will Hardy's implementation of Penton's paradigm visualization system
1580 * corpus readers for presidential speeches
1581 * removed NLTK dependency
1582 * generalized CFG terminals to permit full range of characters
1583 * used fully qualified names in demo code, for portability
1584 * bugfixes from Yoav Goldberg, Eduardo Pereira Habkost
1585 * fixed obscure quoting bug in tree displays and conversions
1586 * simplified demo code, fixed import bug