| 1 |
Version 3.10.0 2026-06-11 |
| 2 |
|
| 3 |
* Enforce the stricter `nltk.pathsec` security policy by default |
| 4 |
* Document the new security model and migration guidance |
| 5 |
* Harden resource loading against path traversal and SSRF/DNS-rebinding |
| 6 |
* Harden downloader path handling and block XML entity expansion |
| 7 |
* Close remaining corpus-reader security edge cases |
| 8 |
* Replace unsafe `exec()` usage in the utility CLI |
| 9 |
* Warn on unpickling user-provided pickles |
| 10 |
* Add HuggingFace datasets integration (`nltk.huggingface`) |
| 11 |
* Align TnT with Brants (2000) specifications |
| 12 |
* Fix PorterStemmer irregular-form lowercasing in NLTK mode |
| 13 |
* Fix TransitionParser sparse index dtype for scikit-learn 1.9 |
| 14 |
* Fix TextCat tie handling |
| 15 |
* Fix WordNet object comparisons for incompatible types |
| 16 |
* Cache WordNet max depth lazily for `lch_similarity()` |
| 17 |
* Fix CCG variable direction, substitution, and type-raising bugs |
| 18 |
* Fix Jaro similarity for single-character and empty-string cases |
| 19 |
* Improve CI and release-maintenance workflows |
| 20 |
|
| 21 |
Thanks to the following contributors to 3.10.0: |
| 22 |
13rac1, alvations, bowiechen, devesh-2002, ekaf, elias-ba, |
| 23 |
haosenwang1018, HyperPS, ihitamandal, jancallewaert, jhnwnstd, |
| 24 |
JuanIMartinezB, Lemm1, LinZiyuu, Mr-Neutr0n, PastelStorm, |
| 25 |
scruge1, Syzygy2048, ylwango613, yzhaoinuw |
| 26 |
|
| 27 |
Version 3.9.4 2026-03-24 |
| 28 |
|
| 29 |
* Support Python 3.14 |
| 30 |
* Fix bug in Levenshtein distance when substitution_cost > 2 |
| 31 |
* Fix bug in Treebank detokeniser re quote ordering |
| 32 |
* Fix bug in Jaro similarity for empty strings |
| 33 |
* Several security enhancements |
| 34 |
* Fix GHSA-rf74-v2fm-23pw: unbounded recursion in JSONTaggedDecoder |
| 35 |
* Implement TextTiling vocabulary introduction method (Hearst 1997) |
| 36 |
* Fix ALINE feature matrix errors and add comprehensive tests |
| 37 |
* Support multiple VerbNet versions, fix longid/shortid regex for VerbNet ids |
| 38 |
* Let downloader fallback to md5 when sha256 is unavailable |
| 39 |
* Several other minor bugfixes and code cleanups |
| 40 |
|
| 41 |
Thanks to the following contributors to 3.9.4: |
| 42 |
Min-Yen Kan, Eric Kafe, Emily Voss, bowiechen, Hrudhai01, |
| 43 |
jancallewaert, Mr-Neutr0n, pollak.peter89, ylwango613, |
| 44 |
|
| 45 |
Version 3.9.3 2026-02-21 |
| 46 |
|
| 47 |
* Fix CVE-2025-14009: secure ZIP extraction in nltk.downloader (#3468) |
| 48 |
* Block path traversal/arbitrary reads in nltk.data for protocol-less refs (#3467) |
| 49 |
* Block path traversal/abs paths in corpus readers and FS pointers (#3479, #3480) |
| 50 |
* Validate external StanfordSegmenter JARs using SHA256 (#3477) |
| 51 |
* Add optional sandbox enforcement for filestring() (#3485) |
| 52 |
* Maintenance: downloader/zipped models, CI/tooling updates |
| 53 |
|
| 54 |
Thanks to the following contributors to 3.9.3: |
| 55 |
Chris Clauss, Eric Kafe, HyperPS, purificant, Shivansh-Game, Christopher Smith |
| 56 |
|
| 57 |
Version 3.9.2 2025-10-01 |
| 58 |
|
| 59 |
* Update download checksums to use SHA256 in built index |
| 60 |
* Fix percentage escape in new-style string formatting |
| 61 |
* replace shortened URLs using goo.gl |
| 62 |
* Make Wordnet interoperable with various taggers and tagged corpora |
| 63 |
* Fix saving PerceptronTagger |
| 64 |
* Document how to reproduce old Wordnet studies |
| 65 |
* properly initialize Portuguese corpus reader |
| 66 |
* support for mixed rules conversion into Chomsky Normal Form |
| 67 |
* only import tkinter if a GUI is needed |
| 68 |
* issue #2112 with Corenlp |
| 69 |
* new environment variable NLTK_DOWNLOADER_FORCE_INTERACTIVE_SHELL |
| 70 |
* Lesk defaults to most frequent sense in case of ties |
| 71 |
|
| 72 |
Thanks to the following contributors to 3.9.2: |
| 73 |
Jose Cols, Peter de Blanc, GeneralPoxter, Eric Kafe, William LaCroix, Jason Liu, |
| 74 |
Samer Masterson, Mike014, purificant, Andrew Ernest Ritz, samertm, Ikram Ul Haq, |
| 75 |
Christopher Smith, Ryan Mannion |
| 76 |
|
| 77 |
Version 3.9.1 2024-08-19 |
| 78 |
* Fixed bug that prevented wordnet from loading |
| 79 |
|
| 80 |
Version 3.9 2024-08-18 |
| 81 |
* Fix security vulnerability CVE-2024-39705 (breaking change) |
| 82 |
* Replace pickled models (punkt, chunker, taggers) by new pickle-free "_tab" packages |
| 83 |
* No longer sort Wordnet synsets and relations (sort in calling function when required) |
| 84 |
* Only strip the last suffix in Wordnet Morphy, thus restricting synsets() results |
| 85 |
* Add Python 3.12 support |
| 86 |
* Many other minor fixes |
| 87 |
|
| 88 |
Thanks to the following contributors to 3.8.2: |
| 89 |
Tom Aarsen, Cat Lee Ball, Veralara Bernhard, Carlos Brandt, Konstantin Chernyshev, Michael Higgins, |
| 90 |
Eric Kafe, Vivek Kalyan, David Lukes, Rob Malouf, purificant, Alex Rudnick, Liling Tan, Akihiro Yamazaki. |
| 91 |
|
| 92 |
Version 3.8.1 2023-01-02 |
| 93 |
|
| 94 |
* Resolve RCE vulnerability in localhost WordNet Browser (#3100) |
| 95 |
* Remove unused tool scripts (#3099) |
| 96 |
* Resolve XSS vulnerability in localhost WordNet Browser (#3096) |
| 97 |
* Add Python 3.11 support (#3090) |
| 98 |
|
| 99 |
Thanks to the following contributors to 3.8.1: |
| 100 |
Francis Bond, John Vandenberg, Tom Aarsen |
| 101 |
|
| 102 |
Version 3.8 2022-12-12 |
| 103 |
|
| 104 |
* Refactor dispersion plot (#3082) |
| 105 |
* Provide type hints for LazyCorpusLoader variables (#3081) |
| 106 |
* Throw warning when LanguageModel is initialized with incorrect vocabulary (#3080) |
| 107 |
* Fix WordNet's all_synsets() function (#3078) |
| 108 |
* Resolve TreebankWordDetokenizer inconsistency with end-of-string contractions (#3070) |
| 109 |
* Support both iso639-3 codes and BCP-47 language tags (#3060) |
| 110 |
* Avoid DeprecationWarning in Regexp tokenizer (#3055) |
| 111 |
* Fix many doctests, add doctests to CI (#3054, #3050, #3048) |
| 112 |
* Fix bool field not being read in VerbNet (#3044) |
| 113 |
* Greatly improve time efficiency of SyllableTokenizer when tokenizing numbers (#3042) |
| 114 |
* Fix encodings of Polish udhr corpus reader (#3038) |
| 115 |
* Allow TweetTokenizer to tokenize emoji flag sequences (#3034) |
| 116 |
* Prevent LazyModule from increasing the size of nltk.__dict__ (#3033) |
| 117 |
* Fix CoreNLPServer non-default port issue (#3031) |
| 118 |
* Add "acion" suffix to the Spanish SnowballStemmer (#3030) |
| 119 |
* Allow loading WordNet without OMW (#3026) |
| 120 |
* Use input() in nltk.chat.chatbot() for Jupyter support (#3022) |
| 121 |
* Fix edit_distance_align() in distance.py (#3017) |
| 122 |
* Tackle performance and accuracy regression of sentence tokenizer since NLTK 3.6.6 (#3014) |
| 123 |
* Add the Iota operator to semantic logic (#3010) |
| 124 |
* Resolve critical errors in WordNet app (#3008) |
| 125 |
* Resolve critical error in CHILDES Corpus (#2998) |
| 126 |
* Make WordNet information_content() accept adjective satellites (#2995) |
| 127 |
* Add "strict=True" parameter to CoreNLP (#2993, #3043) |
| 128 |
* Resolve issue with WordNet's synset_from_sense_key (#2988) |
| 129 |
* Handle WordNet synsets that were lost in mapping (#2985) |
| 130 |
* Resolve TypeError in Boxer (#2979) |
| 131 |
* Add function to retrieve WordNet synonyms (#2978) |
| 132 |
* Warn about nonexistent OMW offsets instead of raising an error (#2974) |
| 133 |
* Fix missing ic argument in res, jcn and lin similarity functions of WordNet (#2970) |
| 134 |
* Add support for the extended OMW (#2946) |
| 135 |
* Fix LC cutoff policy of text tiling (#2936) |
| 136 |
* Optimize ConditionalFreqDist.__add__ performance (#2939) |
| 137 |
* Add Markdown corpus reader (#2902) |
| 138 |
|
| 139 |
Thanks to the following contributors to 3.8: |
| 140 |
Alexandre Perez-Lebel, David Lukes, Eric Kafe, Fernando Carranza, Heungson Lee, |
| 141 |
Hoyeol Kim, James Huang, Jelle Zijlstra, Louis-Justin Tallot, M.K. Pawelkiewicz, |
| 142 |
Jan Lennartz, Malinda Dilhara, Martin Kondratzky, Rob Malouf, Saud Kadiri, |
| 143 |
Siddhesh Mhadnak, Stephan Hasler, Steve Smith, Tom Aarsen, Tyler Sheaffer, |
| 144 |
Yue Zhao, cestwc, elespike, purificant, richardyy1188 |
| 145 |
|
| 146 |
Version 3.7 2022-02-09 |
| 147 |
|
| 148 |
* Improve and update the NLTK team page on nltk.org (#2855, #2941) |
| 149 |
* Drop support for Python 3.6, support Python 3.10 (#2920) |
| 150 |
|
| 151 |
Thanks to the following contributors to 3.7: |
| 152 |
Tom Aarsen |
| 153 |
|
| 154 |
Version 3.6.7 2021-12-28 |
| 155 |
|
| 156 |
* Resolve IndexError in `sent_tokenize` and `word_tokenize` (#2922) |
| 157 |
|
| 158 |
Thanks to the following contributors to 3.6.7: |
| 159 |
Tom Aarsen |
| 160 |
|
| 161 |
Version 3.6.6 2021-12-21 |
| 162 |
|
| 163 |
* Refactor `gensim.doctest` to work for gensim 4.0.0 and up (#2914) |
| 164 |
* Add Precision, Recall, F-measure, Confusion Matrix to Taggers (#2862) |
| 165 |
* Added warnings if .zip files exist without any corresponding .csv files. (#2908) |
| 166 |
* Fix `FileNotFoundError` when the `download_dir` is a non-existing nested folder (#2910) |
| 167 |
* Rename omw to omw-1.4 (#2907) |
| 168 |
* Resolve ReDoS opportunity by fixing incorrectly specified regex (#2906) |
| 169 |
* Support OMW 1.4 (#2899) |
| 170 |
* Deprecate Tree get and set node methods (#2900) |
| 171 |
* Fix broken inaugural test case (#2903) |
| 172 |
* Use Multilingual Wordnet Data from OMW with newer Wordnet versions (#2889) |
| 173 |
* Keep NLTKs "tokenize" module working with pathlib (#2896) |
| 174 |
* Make prettyprinter to be more readable (#2893) |
| 175 |
* Update links to the nltk book (#2895) |
| 176 |
* Add `CITATION.cff` to nltk (#2880) |
| 177 |
* Resolve serious ReDoS in PunktSentenceTokenizer (#2869) |
| 178 |
* Delete old CI config files (#2881) |
| 179 |
* Improve Tokenize documentation + add TokenizerI as superclass for TweetTokenizer (#2878) |
| 180 |
* Fix expected value for BLEU score doctest after changes from #2572 |
| 181 |
* Add multi Bleu functionality and tests (#2793) |
| 182 |
* Deprecate 'return_str' parameter in NLTKWordTokenizer and TreebankWordTokenizer (#2883) |
| 183 |
* Allow empty string in CFG's + more (#2888) |
| 184 |
* Partition `tree.py` module into `tree` package + pickle fix (#2863) |
| 185 |
* Fix several TreebankWordTokenizer and NLTKWordTokenizer bugs (#2877) |
| 186 |
* Rewind Wordnet data file after each lookup (#2868) |
| 187 |
* Correct __init__ call for SyntaxCorpusReader subclasses (#2872) |
| 188 |
* Documentation fixes (#2873) |
| 189 |
* Fix levenstein distance for duplicated letters (#2849) |
| 190 |
* Support alternative Wordnet versions (#2860) |
| 191 |
* Remove hundreds of formatting warnings for nltk.org (#2859) |
| 192 |
* Modernize `nltk.org/howto` pages (#2856) |
| 193 |
* Fix Bleu Score smoothing function from taking log(0) (#2839) |
| 194 |
* Update third party tools to newer versions and removing MaltParser fixed version (#2832) |
| 195 |
* Fix TypeError: _pretty() takes 1 positional argument but 2 were given in sem/drt.py (#2854) |
| 196 |
* Replace `http` with `https` in most URLs (#2852) |
| 197 |
|
| 198 |
Thanks to the following contributors to 3.6.6: |
| 199 |
Adam Hawley, BatMrE, Danny Sepler, Eric Kafe, Gavish Poddar, Panagiotis Simakis, |
| 200 |
RnDevelover, Robby Horvath, Tom Aarsen, Yuta Nakamura, Mohaned Mashaly |
| 201 |
|
| 202 |
Version 3.6.5 2021-10-11 |
| 203 |
|
| 204 |
* modernised nltk.org website |
| 205 |
* addressed LGTM.com issues |
| 206 |
* support ZWJ sequences emoji and skin tone modifer emoji in TweetTokenizer |
| 207 |
* METEOR evaluation now requires pre-tokenized input |
| 208 |
* Code linting and type hinting |
| 209 |
* implement get_refs function for DrtLambdaExpression |
| 210 |
* Enable automated CoreNLP, Senna, Prover9/Mace4, Megam, MaltParser CI tests |
| 211 |
* specify minimum regex version that supports regex.Pattern |
| 212 |
* avoid re.Pattern and regex.Pattern which fail for Python 3.6, 3.7 |
| 213 |
|
| 214 |
Thanks to the following contributors to 3.6.5: |
| 215 |
Tom Aarsen, Saibo Geng, Mohaned Mashaly, Dimitri Papadopoulos, Danny Sepler, |
| 216 |
Ahmet Yildirim, RnDevelover, yutanakamura |
| 217 |
|
| 218 |
Version 3.6.4 2021-10-01 |
| 219 |
|
| 220 |
* deprecate `nltk.usage(obj)` in favor of `help(obj)` |
| 221 |
* resolve ReDoS vulnerability in Corpus Reader |
| 222 |
* solidify performance tests |
| 223 |
* improve phone number recognition in tweet tokenizer |
| 224 |
* refactored CISTEM stemmer for German |
| 225 |
* identify NLTK Team as the author |
| 226 |
* replace travis badge with github actions badge |
| 227 |
* add SECURITY.md |
| 228 |
|
| 229 |
Thanks to the following contributors to 3.6.4: |
| 230 |
Tom Aarsen, Mohaned Mashaly, Dimitri Papadopoulos Orfanos, purificant, Danny Sepler |
| 231 |
|
| 232 |
Version 3.6.3 2021-09-19 |
| 233 |
* Dropped support for Python 3.5 |
| 234 |
* Run CI tests on Windows, too |
| 235 |
* Moved from Travis CI to GitHub Actions |
| 236 |
* Code and comment cleanups |
| 237 |
* Visualize WordNet relation graphs using Graphviz |
| 238 |
* Fixed large error in METEOR score |
| 239 |
* Apply isort, pyupgrade, black, added as pre-commit hooks |
| 240 |
* Prevent debug_decisions in Punkt from throwing IndexError |
| 241 |
* Resolved ZeroDivisionError in RIBES with dissimilar sentences |
| 242 |
* Initialize WordNet IC total counts with smoothing value |
| 243 |
* Fixed AttributeError for Arabic ARLSTem2 stemmer |
| 244 |
* Many fixes and improvements to lm language model package |
| 245 |
* Fix bug in nltk.metrics.aline, C_skip = -10 |
| 246 |
* Improvements to TweetTokenizer |
| 247 |
* Optional show arg for FreqDist.plot, ConditionalFreqDist.plot |
| 248 |
* edit_distance now computes Damerau-Levenshtein edit-distance |
| 249 |
|
| 250 |
Thanks to the following contributors to 3.6.3: |
| 251 |
Tom Aarsen, Abhijnan Bajpai, Michael Wayne Goodman, Michał Górny, Maarten ter Huurne, |
| 252 |
Manu Joseph, Eric Kafe, Ilia Kurenkov, Daniel Loney, Rob Malouf, Mohaned Mashaly, |
| 253 |
purificant, Danny Sepler, Anthony Sottile |
| 254 |
|
| 255 |
Version 3.6.2 2021-04-20 |
| 256 |
* move test code to nltk/test |
| 257 |
* clean up some doctests |
| 258 |
* fix bug in NgramAssocMeasures (order preserving fix) |
| 259 |
* fixes for compatibility with Pypy 7.3.4 |
| 260 |
|
| 261 |
Thanks to the following contributors to 3.6.2: |
| 262 |
Ruben Cartuyvels, Rob Malouf, Dalton Pearson, Danny Sepler |
| 263 |
|
| 264 |
Version 3.6 2021-04-07 |
| 265 |
* add support for Python 3.9 |
| 266 |
* add Tree.fromlist |
| 267 |
* compute Minimum Spanning Tree of unweighted graph using BFS |
| 268 |
* fix bug with infinite loop in Wordnet closure and tree |
| 269 |
* fix bug in calculating BLEU using smoothing method 4 |
| 270 |
* Wordnet synset similarities work for all pos |
| 271 |
* new Arabic light stemmer (ARLSTem2) |
| 272 |
* new syllable tokenizer (LegalitySyllableTokenizer) |
| 273 |
* remove nose in favor of pytest |
| 274 |
* misc bug fixes, code cleanups, test cleanups, efficiency improvements |
| 275 |
|
| 276 |
Thanks to the following contributors to 3.6: |
| 277 |
Tom Aarsen, K Abainia, Akshita Bhagia, Andrew Bird, Thomas Bird, |
| 278 |
Tom Conroy, Christopher Hench, Andrew Jorgensen, Eric Kafe, |
| 279 |
Ilia Kurenkov, Yeting Li, Joseph Manu, Marius Mather, Denali Molitor, |
| 280 |
Jacob Moorman, Philippe Ombredanne, Vassilis Palassopoulos, Ram Rachum, |
| 281 |
Danny Sepler, Or Sharir, Brad Solomon, Hiroki Teranishi, Constantin Weisser, |
| 282 |
Pratap Yadav, Louis Yang |
| 283 |
|
| 284 |
Version 3.5 2020-04-13 |
| 285 |
* add support for Python 3.8 |
| 286 |
* drop support for Python 2 |
| 287 |
* create NLTK's own Tokenizer class distinct from the Treebank reference tokeniser |
| 288 |
* update Vader sentiment analyser |
| 289 |
* fix JSON serialization of some PoS taggers |
| 290 |
* minor improvements in grammar.CFG, Vader, pl196x corpus reader, StringTokenizer |
| 291 |
* change implementation <= and >= for FreqDist so they are partial orders |
| 292 |
* make FreqDist iterable |
| 293 |
* correctly handle Penn Treebank trees with a unlabeled branching top node. |
| 294 |
|
| 295 |
Thanks to the following contributors to 3.5: |
| 296 |
Nicolas Darr, Gerhard Kremer, Liling Tan, Christopher Hench, Alexandre Dias, Hervé Nicol, |
| 297 |
Pierpaolo Pantone, Bonifacio de Oliveira, Maciej Gawinecki, BLKSerene, hoefling, alvations, |
| 298 |
pyfisch, srhrshr |
| 299 |
|
| 300 |
Version 3.4.5 2019-08-20 |
| 301 |
* Fixed security bug in downloader: Zip slip vulnerability - for the unlikely |
| 302 |
situation where a user configures their downloader to use a compromised server |
| 303 |
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14751) |
| 304 |
|
| 305 |
Thanks to the following contributors to 3.4.5: |
| 306 |
Mike Salvatore |
| 307 |
|
| 308 |
Version 3.4.4 2019-07-04 |
| 309 |
* fix bug in plot function (probability.py) |
| 310 |
* add improved PanLex Swadesh corpus reader |
| 311 |
|
| 312 |
Thanks to the following contributors to 3.4.4: |
| 313 |
Devashish Lal, Liling Tan |
| 314 |
|
| 315 |
Version 3.4.3 2019-06-07 |
| 316 |
|
| 317 |
* add Text.generate() |
| 318 |
* add QuadgramAssocMeasures |
| 319 |
* add SSP to tokenizers |
| 320 |
* return confidence of best tag from AveragedPerceptron |
| 321 |
* make plot methods return Axes objects |
| 322 |
* don't require list arguments to PositiveNaiveBayesClassifier.train |
| 323 |
* fix Tree classes to work with native Python copy library |
| 324 |
* fix inconsistency for NomBank |
| 325 |
* fix random seeding in LanguageModel.generate |
| 326 |
* fix ConditionalFreqDist mutation on tabulate/plot call |
| 327 |
* fix broken links in documentation |
| 328 |
* fix misc Wordnet issues |
| 329 |
* update installation instructions |
| 330 |
|
| 331 |
Thanks to the following contributors to 3.4.3: |
| 332 |
alvations, Bharat123rox, cifkao, drewmiller, free-variation, henchc |
| 333 |
irisxzhou, nick-ulle, ppartarr, simonepri, yigitsever, zhaoyanpeng |
| 334 |
|
| 335 |
Version 3.4.1 2019-04-17 |
| 336 |
|
| 337 |
* add chomsky_normal_form for CFGs |
| 338 |
* add meteor score |
| 339 |
* add minimum edit/Levenshtein distance based alignment function |
| 340 |
* allow access to collocation list via text.collocation_list() |
| 341 |
* support corenlp server options |
| 342 |
* drop support for Python 3.4 |
| 343 |
* other minor fixes |
| 344 |
|
| 345 |
Thanks to the following contributors to 3.4.1: |
| 346 |
Adrian Ellis, Andrew Martin, Ayush Kaushal, BLKSerene, Bharat |
| 347 |
Raghunathan, Franklin Chen, KMiNT21 Kevin Brown, Liling Tan, |
| 348 |
Matan Rak, Nat Quayle Nelson, Osman Zubair, Purificant, |
| 349 |
Uday Krishna, Viresh Gupta |
| 350 |
|
| 351 |
Version 3.4 2018-11-17 |
| 352 |
* Support Python 3.7 |
| 353 |
* Language Modeling incl Kneser-Ney, Witten-Bell, Good-Turing |
| 354 |
* Cistem Stemmer for German |
| 355 |
* Support Russian National Corpus incl POS tag model |
| 356 |
* Decouple sentiment and twitter packages |
| 357 |
* Minor extensions for WordNet |
| 358 |
* K-alpha |
| 359 |
* Fix warning messages for corenlp |
| 360 |
* Comprehensive code cleanups |
| 361 |
* Many other minor fixes |
| 362 |
* Switch continuous integration from Jenkins to Travis |
| 363 |
|
| 364 |
Special thanks to Ilia Kurenkov (Language Model package), Liling Tan (Python 3.7, Travis-CI), |
| 365 |
and purificant (code cleanups). Thanks also to: Afshin Sadeghi, Ales Tamchyna, Alok Debnath, |
| 366 |
aquatiko, Coykto, Denis Kataev, dnc1994, Fabian Howard, Frankie Robertson, Iaroslav Tymchenko, |
| 367 |
Jayakrishna Sahit, LBenzahia, Leonie Weißweiler, Linghao Zhang, Rohit Kumar, sahitpj, |
| 368 |
Tim Gianitsos, vagrant, 53X |
| 369 |
|
| 370 |
Version 3.3 2018-05-06 |
| 371 |
* Support Python 3.6 |
| 372 |
* New interface to CoreNLP |
| 373 |
* Support synset retrieval by sense key |
| 374 |
* Minor fixes to CoNLL Corpus Reader, AlignedSent |
| 375 |
* Fixed minor inconsistencies in APIs and API documentation |
| 376 |
* Better conformance to PEP8 |
| 377 |
* Drop moses.py (incompatible license) |
| 378 |
|
| 379 |
Special thanks to Liling Tan for leading our transition to Python 3.6. |
| 380 |
Thanks to other contributors listed here: https://github.com/nltk/nltk/blob/develop/AUTHORS.md |
| 381 |
|
| 382 |
Version 3.2.5 2017-09-24 |
| 383 |
|
| 384 |
* Arabic stemmers (ARLSTem, Snowball) |
| 385 |
* NIST MT evaluation metric and added NIST international_tokenize |
| 386 |
* Moses tokenizer |
| 387 |
* Document Russian tagger |
| 388 |
* Fix to Stanford segmenter |
| 389 |
* Improve treebank detokenizer, VerbNet, Vader |
| 390 |
* Misc code and documentation cleanups |
| 391 |
* Implement fixes suggested by LGTM |
| 392 |
|
| 393 |
Thanks to the following contributors to 3.2.5: |
| 394 |
Ali Abdullah, Lakhdar Benzahia, Henry Elder, Campion Fellin, |
| 395 |
Tsolak Ghukasyan, Thanh Ha, Jean Helie, Nelson Liu, |
| 396 |
Nathan Schneider, Chintan Shah, Fábio Silva, Liling Tan, |
| 397 |
Ziyao Wei, Zicheng Xu, Albert Au Yeung, AbdealiJK, |
| 398 |
porqupine, sbagan, xprogramer |
| 399 |
|
| 400 |
Version 3.2.4 2017-05-21 |
| 401 |
|
| 402 |
* remove load-time dependency on Python requests library |
| 403 |
* add support for Arabic in StanfordSegmenter |
| 404 |
* fix MosesDetokenizer on irregular quote tokens |
| 405 |
|
| 406 |
Thanks to the following contributors to 3.2.4: |
| 407 |
Alex Constantin, Hatem Nassrat, Liling Tan |
| 408 |
|
| 409 |
Version 3.2.3 2017-05-16 |
| 410 |
|
| 411 |
* new interface to Stanford CoreNLP Web API |
| 412 |
* improved Lancaster stemmer with customizable rules from Whoosh |
| 413 |
* improved Treebank tokenizer |
| 414 |
* improved support for GLEU score |
| 415 |
* adopt new Abstract base class style |
| 416 |
* support custom tab files for extending WordNet |
| 417 |
* make synset_from_pos_and_offset a public method |
| 418 |
* make non-English WordNet lemma lookups case-insensitive |
| 419 |
* speed up TnT tagger |
| 420 |
* speed up FreqDist and ConditionalFreqDist |
| 421 |
* support additional quotes in TreebankWordTokenizer |
| 422 |
* clean up Tk's postscript output |
| 423 |
* drop explicit support for corpora not distributed with NLTK to streamline testing |
| 424 |
* allow iterator in perceptron tagger training |
| 425 |
* allow for curly bracket quantifiers in chunk.regexp.CHUNK_TAG_PATTERN |
| 426 |
* new corpus reader for MWA subset of PPDB |
| 427 |
* improved testing framework |
| 428 |
|
| 429 |
Thanks to the following contributors to 3.2.3: |
| 430 |
Mark Amery, Carl Bolz, Abdelhak Bougouffa, Matt Chaput, Michael Goodman, |
| 431 |
Jaehoon Hwang, Naoya Kanai, Jackson Lee, Christian Meyer, Dmitrijs Milajevs, |
| 432 |
Adam Nelson, Pierpaolo Pantone, Liling Tan, Vilhjalmur Thorsteinsson, |
| 433 |
Arthur Tilley, jmhutch, Yorwba, eromoe and others |
| 434 |
|
| 435 |
Version 3.2.2 2016-12-31 |
| 436 |
* added Kondrak's Aline algorithm |
| 437 |
* added ChrF and GLEU MT evaluation metrics |
| 438 |
* added Russian pos tagger model |
| 439 |
* added Moses detokenizer |
| 440 |
* rewrite Porter Stemmer |
| 441 |
* rewrite FrameNet corpus reader |
| 442 |
(adds frame parameter to fes(), lus(), exemplars() |
| 443 |
see https://www.nltk.org/howto/framenet.html) |
| 444 |
* updated FrameNet Corpus to version 1.7 |
| 445 |
* fixes to stanford_segmenter.py, SentiText, CoNLL Corpus Reader |
| 446 |
* fixes to BLEU, naivebayes, Krippendorff's alpha, Punkt |
| 447 |
* fixes to tests for TransitionParser, Senna, edit distance |
| 448 |
* fixes to Moses Tokenizer and Detokenizer |
| 449 |
* improved TweetTokenizer |
| 450 |
* strip trailing whitespace when splitting sentences |
| 451 |
* handle inverted exclamation mark in ToktokTokenizer |
| 452 |
* resolved some issues with Python 3.5 support |
| 453 |
* improvements to testing framework |
| 454 |
* clean up dependencies |
| 455 |
|
| 456 |
Thanks to the following contributors to 3.2.2: |
| 457 |
|
| 458 |
Prasasto Adi, Mark Amery, Geoff Bacon, George Berry, Colin Carroll, Alexis Dimitriadis, |
| 459 |
Nicholas Fabina, German Ferrero, Tsolak Ghukasyan, Hyuckin David Lim, Naoya Kanai, |
| 460 |
Greg Kondrak, Igor Korolev, Tim Leslie, Rob Malouf, Heguang Miao, Dmitrijs Milajevs, |
| 461 |
Adam Nelson, Dennis O'Brien, Qi Liu, Pierpaolo Pantone, Andy Reagan, Mike Recachinas, |
| 462 |
Nathan Schneider, Jānis Šlapiņš, Richard Snape, Liling Tan, Marcus Uneson, |
| 463 |
Linghao Zhang, drevicko, SaintNazaire |
| 464 |
|
| 465 |
Version 3.2.1 2016-04-09 |
| 466 |
* Support for CCG semantics, Stanford segmenter, VADER lexicon |
| 467 |
* Fixes to BLEU score calculation, CHILDES corpus reader |
| 468 |
* Other miscellaneous fixes |
| 469 |
|
| 470 |
Thanks to the following contributors to 3.2.1: |
| 471 |
Andrew Giel, Casper Lehmann-Strøm, David Madl, Tanin Na Nakorn, |
| 472 |
Guilherme Nardari, Philippe Ombredanne, Nathan Schneider, Liling Tan, |
| 473 |
Josiah Wang, venticello |
| 474 |
|
| 475 |
Version 3.2 2016-03-03 |
| 476 |
* Fixes for Python 3.5 |
| 477 |
* Code cleanups now Python 2.6 is no longer supported |
| 478 |
* Improvements to documentation |
| 479 |
* Comprehensive use of os.path for platform-specific path handling |
| 480 |
* Support for PanLex |
| 481 |
* Support for third party download locations for NLTK data |
| 482 |
* Fix bugs in IBM method 3 smoothing and BLEU calculation |
| 483 |
* Support smoothing for BLEU score and corpus-level BLEU |
| 484 |
* Support RIBES score |
| 485 |
* Improvements to TweetTokenizer |
| 486 |
* Updates for Stanford API |
| 487 |
* Add mathematical operators to ConditionalFreqDist |
| 488 |
* Fix bug in sentiwordnet for adjectives |
| 489 |
* Merged internal implementations of Trie |
| 490 |
|
| 491 |
Thanks to the following contributors to 3.2: |
| 492 |
Santiago Castro, Jihun Choi, Graham Christensen, Andrew Drozdov, Long |
| 493 |
Duong, Kyriakos Georgiou, Michael Wayne Goodman, Clark Grubb, Tah Wei |
| 494 |
Hoon, David Kamholz, Ewan Klein, Reed Loden, Rob Malouf, Philippe |
| 495 |
Ombredanne, Josh Owen, Pierpaolo Pantone, Mike Recachinas, Elijah |
| 496 |
Rippeth, Thomas Stieglmaier, Liling Tan, Philip Tzou, Pratap Vardhan. |
| 497 |
|
| 498 |
Version 3.1 2015-10-15 |
| 499 |
* Fixes for Python 3.5 (drop support for capturing groups in regexp tokenizer) |
| 500 |
* Drop support for Python 2.6 |
| 501 |
* Adopt perceptron tagger for new default POS tagger nltk.pos_tag |
| 502 |
* Stanford Neural Dependency Parser wrapper |
| 503 |
* Sentiment analysis package incl VADER |
| 504 |
* Improvements to twitter package |
| 505 |
* Multi word expression tokenizer |
| 506 |
* Support for everygram and skipgram |
| 507 |
* consistent evaluation metric interfaces, putting reference before hypothesis |
| 508 |
* new nltk.translate module, incorporating the old align module |
| 509 |
* implement stack decoder |
| 510 |
* clean up Alignment interface |
| 511 |
* CorpusReader method to support access to license and citation |
| 512 |
* Multext East Corpus and MTECorpusReader |
| 513 |
* include six module to streamline installation on MS Windows |
| 514 |
|
| 515 |
Thanks to the following contributors to 3.1: |
| 516 |
Le Tuan Anh, Petra Barancikova, Alexander Böhm, Francis Bond, |
| 517 |
Long Duong, Anna Garbar, Matthew Honnibal, Tah Wei Hoon, Ewan Klein, |
| 518 |
Rob Malouf, Dmitrijs Milajevs, Will Monroe, Sergio Oller, Pierpaolo |
| 519 |
Pantone, Jacob Perkins, Lorenzo Rubio, Thomas Stieglmaier, Liling Tan, |
| 520 |
Pratap Vardhan |
| 521 |
|
| 522 |
Version 3.0.5 2015-09-05 |
| 523 |
* rewritten IBM models, and new IBM Model 4 and 5 implementations |
| 524 |
* new Twitter package |
| 525 |
* stabilized MaltParser API |
| 526 |
* improved regex tagger |
| 527 |
* improved documentation on contributing |
| 528 |
* minor improvements to documentation and testing |
| 529 |
|
| 530 |
Thanks to the following contributors to 3.0.5: |
| 531 |
Álvaro Justen, Dmitrijs Milajevs, Ewan Klein, Heran Lin, Justin Hammar, |
| 532 |
Liling Tan, Long Duong, Lorenzo Rubio, Pierpaolo Pantone, Tah Wei Hoon |
| 533 |
|
| 534 |
Version 3.0.4 2015-07-13 |
| 535 |
* minor bug fixes and enhancements |
| 536 |
|
| 537 |
Thanks to the following contributors to 3.0.4: |
| 538 |
Nicola Bova, Santiago Castro, Len Remmerswaal, Keith Suderman, kabayan55, |
| 539 |
pln-fing-udelar (NLP Group, Instituto de Computación, Facultad de Ingeniería, Universidad de la República, Uruguay). |
| 540 |
|
| 541 |
Version 3.0.3 2015-06-12 |
| 542 |
* bug fixes (Stanford NER, Boxer, Snowball, treebank tokenizer, |
| 543 |
dependency graph, KneserNey, BLEU) |
| 544 |
* code clean-ups |
| 545 |
* default POS tagger permits tagset to be specified |
| 546 |
* gensim illustration |
| 547 |
* tgrep implementation |
| 548 |
* added PanLex Swadesh corpora |
| 549 |
* visualisation for aligned bitext |
| 550 |
* support for Google App Engine |
| 551 |
* POSTagger renamed StanfordPOSTagger, NERTagger renamed StanfordNERTagger |
| 552 |
|
| 553 |
Thanks to the following contributors to 3.0.3: |
| 554 |
|
| 555 |
Long Duong, Pedro Fialho, Dan Garrette, Helder, Saimadhav Heblikar, |
| 556 |
Chris Inskip, David Kamholz, Dmitrijs Milajevs, Smitha Milli, |
| 557 |
Tom Mortimer-Jones, Avital Pekker, Jonathan Pool, Sam Raker, |
| 558 |
Will Roberts, Dmitry Sadovnychyi, Nathan Schneider, Anirudh W |
| 559 |
|
| 560 |
Version 3.0.2 2015-03-13 |
| 561 |
* make pretty-printing method names consistent |
| 562 |
* improvements to Portuguese stemmer |
| 563 |
* transition-based dependency parsers |
| 564 |
* dependency graph visualisation for ipython notebook |
| 565 |
* interfaces for Senna, BLLIP, python-crfsuite |
| 566 |
* NKJP corpus reader |
| 567 |
* code clean ups, minor bug fixes |
| 568 |
|
| 569 |
Thanks to the following contributors to 3.0.2: |
| 570 |
|
| 571 |
Long Duong, Saimadhav Heblikar, Helder, Mikhail Korobov, Denis Krusko, |
| 572 |
Alex Louden, Felipe Madrigal, David McClosky, Dmitrijs Milajevs, |
| 573 |
Ondrej Platek, Nathan Schneider, Dávid Márk Nemeskey, 0ssifrage, ducki13, kiwipi. |
| 574 |
|
| 575 |
Version 3.0.1 2015-01-12 |
| 576 |
* fix setup.py for new version of setuptools |
| 577 |
|
| 578 |
Version 3.0.0 2014-09-07 |
| 579 |
* minor bugfixes |
| 580 |
* added phrase extraction code by Liling Tan and Fredrik Hedman |
| 581 |
|
| 582 |
Thanks to the following contributors to 3.0.0: |
| 583 |
Mark Amery, Ivan Barria, Ingolf Becker, Francis Bond, Lars |
| 584 |
Buitinck, Cristian Capdevila, Arthur Darcet, Michelle Fullwood, |
| 585 |
Dan Garrette, Dougal Graham, Dan Garrette, Dougal Graham, Lauri |
| 586 |
Hallila, Tyler Hartley, Fredrik Hedman, Ofer Helman, Bruce Hill, |
| 587 |
Marcus Huderle, Nancy Ide, Nick Johnson, Angelos Katharopoulos, |
| 588 |
Ewan Klein, Mikhail Korobov, Chris Liechti, Peter Ljunglof, |
| 589 |
Joseph Lynch, Haejoong Lee, Peter Ljunglöf, Dean Malmgren, Rob |
| 590 |
Malouf, Thorsten Marek, Dmitrijs Milajevs, Shari A’aidil |
| 591 |
Nasruddin, Lance Nathan, Joel Nothman, Alireza Nourian, Alexander |
| 592 |
Oleynikov, Ted Pedersen, Jacob Perkins, Will Roberts, Alex |
| 593 |
Rudnick, Nathan Schneider, Geraldine Sim Wei Ying, Lynn Soe, |
| 594 |
Liling Tan, Louis Tiao, Marcus Uneson, Yu Usami, Steven Xu, Zhe |
| 595 |
Wang, Chuck Wooters, lade, isnowfy, onesandzeros, pquentin, wvanlint |
| 596 |
|
| 597 |
Version 3.0b2 2014-08-21 |
| 598 |
* minor bugfixes and clean-ups |
| 599 |
* renamed remaining parse_ methods to read_ or load_, cf issue #656 |
| 600 |
* added Paice's method of evaluating stemming algorithms |
| 601 |
|
| 602 |
Thanks to the following contributors to 3.0.0b2: Lars Buitinck, |
| 603 |
Cristian Capdevila, Lauri Hallila, Ofer Helman, Dmitrijs Milajevs, |
| 604 |
lade, Liling Tan, Steven Xu |
| 605 |
|
| 606 |
Version 3.0.0b1 2014-07-11 |
| 607 |
* Added SentiWordNet corpus and corpus reader |
| 608 |
* Fixed support for 10-column dependency file format |
| 609 |
* Changed Tree initialization to use fromstring |
| 610 |
|
| 611 |
Thanks to the following contributors to 3.0b1: Mark Amery, Ivan |
| 612 |
Barria, Ingolf Becker, Francis Bond, Lars Buitinck, Arthur Darcet, |
| 613 |
Michelle Fullwood, Dan Garrette, Dougal Graham, Dan Garrette, Dougal |
| 614 |
Graham, Tyler Hartley, Ofer Helman, Bruce Hill, Marcus Huderle, Nancy |
| 615 |
Ide, Nick Johnson, Angelos Katharopoulos, Ewan Klein, Mikhail Korobov, |
| 616 |
Chris Liechti, Peter Ljunglof, Joseph Lynch, Haejoong Lee, Peter |
| 617 |
Ljunglöf, Dean Malmgren, Rob Malouf, Thorsten Marek, Dmitrijs |
| 618 |
Milajevs, Shari A’aidil Nasruddin, Lance Nathan, Joel Nothman, Alireza |
| 619 |
Nourian, Alexander Oleynikov, Ted Pedersen, Jacob Perkins, Will |
| 620 |
Roberts, Alex Rudnick, Nathan Schneider, Geraldine Sim Wei Ying, Lynn |
| 621 |
Soe, Liling Tan, Louis Tiao, Marcus Uneson, Yu Usami, Steven Xu, Zhe |
| 622 |
Wang, Chuck Wooters, isnowfy, onesandzeros, pquentin, wvanlint |
| 623 |
|
| 624 |
Version 3.0a4 2014-05-25 |
| 625 |
* IBM Models 1-3, BLEU, Gale-Church aligner |
| 626 |
* Lesk algorithm for WSD |
| 627 |
* Open Multilingual WordNet |
| 628 |
* New implementation of Brill Tagger |
| 629 |
* Extend BNCCorpusReader to parse the whole BNC |
| 630 |
* MASC Tagged Corpus and corpus reader |
| 631 |
* Interface to Stanford Parser |
| 632 |
* Code speed-ups and clean-ups |
| 633 |
* API standardisation, including fromstring method for many objects |
| 634 |
* Improved regression testing setup |
| 635 |
* Removed PyYAML dependency |
| 636 |
|
| 637 |
Thanks to the following contributors to 3.0a4: |
| 638 |
Ivan Barria, Ingolf Becker, Francis Bond, Arthur Darcet, Dan Garrette, |
| 639 |
Ofer Helman, Dougal Graham, Nancy Ide, Ewan Klein, Mikhail Korobov, |
| 640 |
Chris Liechti, Peter Ljunglof, Joseph Lynch, Rob Malouf, Thorsten Marek, |
| 641 |
Dmitrijs Milajevs, Shari A’aidil Nasruddin, Lance Nathan, Joel Nothman, |
| 642 |
Jacob Perkins, Lynn Soe, Liling Tan, Louis Tiao, Marcus Uneson, Steven Xu, |
| 643 |
Geraldine Sim Wei Ying |
| 644 |
|
| 645 |
Version 3.0a3 2013-11-02 |
| 646 |
* support for FrameNet contributed by Chuck Wooters |
| 647 |
* support for Universal Declaration of Human Rights Corpus (udhr2) |
| 648 |
* major API changes: |
| 649 |
- Tree.node -> Tree.label() / Tree.set_label() |
| 650 |
- Chunk parser: top_node -> root_label; chunk_node -> chunk_label |
| 651 |
- WordNet properties are now access methods, e.g. Synset.definition -> Synset.definition() |
| 652 |
- relextract: show_raw_rtuple() -> rtuple(), show_clause() -> clause() |
| 653 |
* bugfix in texttiling |
| 654 |
* replaced simplify_tags with support for universal tagset (simplify_tags=True -> tagset='universal') |
| 655 |
* Punkt default behavior changed to realign sentence boundaries after trailing parenthesis and quotes |
| 656 |
* deprecated classify.svm (use scikit-learn instead) |
| 657 |
* various efficiency improvements |
| 658 |
|
| 659 |
Thanks to the following contributors to 3.0a3: |
| 660 |
Lars Buitinck, Marcus Huderle, Nick Johnson, Dougal Graham, Ewan Klein, |
| 661 |
Mikhail Korobov, Haejoong Lee, Peter Ljunglöf, Dean Malmgren, Lance Nathan, |
| 662 |
Alexander Oleynikov, Nathan Schneider, Chuck Wooters, Yu Usami, Steven Xu, |
| 663 |
pquentin, wvanlint |
| 664 |
|
| 665 |
Version 3.0a2 2013-07-12 |
| 666 |
* speed improvements in word_tokenize, GAAClusterer, TnT tagger, Baum Welch, HMM tagger |
| 667 |
* small improvements in collocation finders, probability, modelling, Porter Stemmer |
| 668 |
* bugfix in lowest common hypernyn calculation (used in path similarity measures) |
| 669 |
* code cleanups, docstring cleanups, demo fixes |
| 670 |
|
| 671 |
Thanks to the following contributors to 3.0a2: |
| 672 |
Mark Amery, Lars Buitinck, Michelle Fullwood, Dan Garrette, Dougal Graham, |
| 673 |
Tyler Hartley, Bruce Hill, Angelos Katharopoulos, Mikhail Korobov, |
| 674 |
Rob Malouf, Joel Nothman, Ted Pedersen, Will Roberts, Alex Rudnick, |
| 675 |
Steven Xu, isnowfy, onesandzeros |
| 676 |
|
| 677 |
Version 3.0a1 2013-02-14 |
| 678 |
* reinstated tkinter support (Haejoong Lee) |
| 679 |
|
| 680 |
Version 3.0a0 2013-01-14 |
| 681 |
* alpha release of first version to support Python 2.6, 2.7, and 3. |
| 682 |
|
| 683 |
Version 2.0.4 2012-11-07 |
| 684 |
* minor bugfix (removed numpy dependency) |
| 685 |
|
| 686 |
Version 2.0.3 2012-09-24 |
| 687 |
|
| 688 |
* fixed corpus/reader/util.py to support Python 2.5 |
| 689 |
* make MaltParser safe to use in parallel |
| 690 |
* fixed bug in inter-annotator agreement |
| 691 |
* updates to various doctests (nltk/test) |
| 692 |
* minor bugfixes |
| 693 |
|
| 694 |
Thanks to the following contributors to 2.0.3: |
| 695 |
Robin Cooper, Pablo Duboue, Christian Federmann, Dan Garrette, Ewan Klein, |
| 696 |
Pierre-François Laquerre, Max Leonov, Peter Ljunglöf, Nitin Madnani, Ceri Stagg |
| 697 |
|
| 698 |
Version 2.0.2 2012-07-05 |
| 699 |
|
| 700 |
* improvements to PropBank, NomBank, and SemCor corpus readers |
| 701 |
* interface to full Penn Treebank Corpus V3 (corpus.ptb) |
| 702 |
* made wordnet.lemmas case-insensitive |
| 703 |
* more flexible padding in model.ngram |
| 704 |
* minor bugfixes and documentation enhancements |
| 705 |
* better support for automated testing |
| 706 |
|
| 707 |
Thanks to the following contributors to 2.0.2: |
| 708 |
Daniel Blanchard, Mikhail Korobov, Nitin Madnani, Duncan McGreggor, |
| 709 |
Morten Neergaard, Nathan Schneider, Rico Sennrich. |
| 710 |
|
| 711 |
Version 2.0.1 2012-05-15 |
| 712 |
|
| 713 |
* moved NLTK to GitHub: https://github.com/nltk |
| 714 |
* set up integration testing: https://jenkins.shiningpanda.com/nltk/ (Morten Neergaard) |
| 715 |
* converted documentation to Sphinx format: https://www.nltk.org/api/nltk.html |
| 716 |
* dozens of minor enhancements and bugfixes: https://github.com/nltk/nltk/commits/ |
| 717 |
* dozens of fixes for conformance with PEP-8 |
| 718 |
* dozens of fixes to ensure operation with Python 2.5 |
| 719 |
* added interface to Lin's Dependency Thesaurus (Dan Blanchard) |
| 720 |
* added interface to scikit-learn classifiers (Lars Buitinck) |
| 721 |
* added segmentation evaluation measures (David Doukhan) |
| 722 |
|
| 723 |
Thanks to the following contributors to 2.0.1 (since 2.0b9, July 2010): |
| 724 |
Rami Al-Rfou', Yonatan Becker, Steven Bethard, Daniel Blanchard, Lars |
| 725 |
Buitinck, David Coles, Lucas Cooper, David Doukhan, Dan Garrette, |
| 726 |
Masato Hagiwara, Michael Hansen, Michael Heilman, Rebecca Ingram, |
| 727 |
Sudharshan Kaushik, Mikhail Korobov, Peter Ljunglof, Nitin Madnani, |
| 728 |
Rob Malouf, Tomonori Nagano, Morten Neergaard, David Nemeskey, |
| 729 |
Joel Nothman, Jacob Perkins, Alessandro Presta, Alex Rudnick, |
| 730 |
Nathan Schneider, Stefano Lattarini, Peter Stahl, Jason Yoder |
| 731 |
|
| 732 |
Version 2.0.1 (rc1) 2011-04-11 |
| 733 |
|
| 734 |
NLTK: |
| 735 |
* added interface to the Stanford POS Tagger |
| 736 |
* updates to sem.Boxer, sem.drt.DRS |
| 737 |
* allow unicode strings in grammars |
| 738 |
* allow non-string features in classifiers |
| 739 |
* modifications to HunposTagger |
| 740 |
* issues with DRS printing |
| 741 |
* fixed bigram collocation finder for window_size > 2 |
| 742 |
* doctest paths no longer presume unix-style pathname separators |
| 743 |
* fixed issue with NLTK's tokenize module colliding with the Python tokenize module |
| 744 |
* fixed issue with stemming Unicode strings |
| 745 |
* changed ViterbiParser.nbest_parse to parse |
| 746 |
* ChaSen and KNBC Japanese corpus readers |
| 747 |
* preserve case in concordance display |
| 748 |
* fixed bug in simplification of Brown tags |
| 749 |
* a version of IBM Model 1 as described in Koehn 2010 |
| 750 |
* new class AlignedSent for aligned sentence data and evaluation metrics |
| 751 |
* new nltk.util.set_proxy to allow easy configuration of HTTP proxy |
| 752 |
* improvements to downloader user interface to catch URL and HTTP errors |
| 753 |
* added CHILDES corpus reader |
| 754 |
* created special exception hierarchy for Prover9 errors |
| 755 |
* significant changes to the underlying code of the boxer interface |
| 756 |
* path-based wordnet similarity metrics use a fake root node for verbs, following the Perl version |
| 757 |
* added ability to handle multi-sentence discourses in Boxer |
| 758 |
* added the 'english' Snowball stemmer |
| 759 |
* simplifications and corrections of Earley Chart Parser rules |
| 760 |
* several changes to the feature chart parsers for correct unification |
| 761 |
* bugfixes: FreqDist.plot, FreqDist.max, NgramModel.entropy, CategorizedCorpusReader, DecisionTreeClassifier |
| 762 |
* removal of Python >2.4 language features for 2.4 compatibility |
| 763 |
* removal of deprecated functions and associated warnings |
| 764 |
* added semantic domains to wordnet corpus reader |
| 765 |
* changed wordnet similarity functions to include instance hyponyms |
| 766 |
* updated to use latest version of Boxer |
| 767 |
|
| 768 |
Data: |
| 769 |
* JEITA Public Morphologically Tagged Corpus (in ChaSen format) |
| 770 |
* KNB Annotated corpus of Japanese blog posts |
| 771 |
* Fixed some minor bugs in alvey.fcfg, and added number of parse trees in alvey_sentences.txt |
| 772 |
* added more comtrans data |
| 773 |
|
| 774 |
Documentation: |
| 775 |
* minor fixes to documentation |
| 776 |
* NLTK Japanese book (chapter 12) by Masato Hagiwara |
| 777 |
|
| 778 |
NLTK-Contrib: |
| 779 |
* Viethen and Dale referring expression algorithms |
| 780 |
|
| 781 |
|
| 782 |
Version 2.0b9 2010-07-25 |
| 783 |
|
| 784 |
NLTK: |
| 785 |
* many code and documentation cleanups |
| 786 |
* Added port of Snowball stemmers |
| 787 |
* Fixed loading of pickled tokenizers (issue 556) |
| 788 |
* DecisionTreeClassifier now handles unknown features (issue 570) |
| 789 |
* Added error messages to LogicParser |
| 790 |
* Replaced max_models with end_size to prevent Mace from hanging |
| 791 |
* Added interface to Boxer |
| 792 |
* Added nltk.corpus.semcor to give access to SemCor 3.0 corpus (issue 530) |
| 793 |
* Added support for integer- and float-valued features in maxent classifiers |
| 794 |
* Permit NgramModels to be pickled |
| 795 |
* Added Sourced Strings (see test/sourcedstring.doctest for details) |
| 796 |
* Fixed bugs in with Good-Turing and Simple Good-Turing Estimation (issue 26) |
| 797 |
* Add support for span tokenization, aka standoff annotation of segmentation (incl Punkt) |
| 798 |
* allow unicode nodes in Tree.productions() |
| 799 |
* Fixed WordNet's morphy to be consistent with the original implementation, |
| 800 |
taking the shortest returned form instead of an arbitrary one (issues 427, 487) |
| 801 |
* Fixed bug in MaxentClassifier |
| 802 |
* Accepted bugfixes for YCOE corpus reader (issue 435) |
| 803 |
* Added test to _cumulative_frequencies() to correctly handle the case when no arguments are supplied |
| 804 |
* Added a TaggerI interface to the HunPos open-source tagger |
| 805 |
* Return 0, not None, when no count is present for a lemma in WordNet |
| 806 |
* fixed pretty-printing of unicode leaves |
| 807 |
* More efficient calculation of the leftcorner relation for left corner parsers |
| 808 |
* Added two functions for graph calculations: transitive closure and inversion. |
| 809 |
* FreqDist.pop() and FreqDist.popitems() now invalidate the caches (issue 511) |
| 810 |
|
| 811 |
Data: |
| 812 |
* Added SemCor 3.0 corpus (Brown Corpus tagged with WordNet synsets) |
| 813 |
* Added LanguageID corpus (trigram counts for 451 languages) |
| 814 |
* Added grammar for a^n b^n c^n |
| 815 |
|
| 816 |
NLTK-Contrib: |
| 817 |
* minor updates |
| 818 |
|
| 819 |
Thanks to the following contributors to 2.0b9: |
| 820 |
|
| 821 |
Steven Bethard, Francis Bond, Dmitry Chichkov, Liang Dong, Dan Garrette, |
| 822 |
Simon Greenhill, Bjorn Maeland, Rob Malouf, Joel Nothman, Jacob Perkins, |
| 823 |
Alberto Planas, Alex Rudnick, Geoffrey Sampson, Kevin Scannell, Richard Sproat |
| 824 |
|
| 825 |
|
| 826 |
Version 2.0b8 2010-02-05 |
| 827 |
|
| 828 |
NLTK: |
| 829 |
* fixed copyright and license statements |
| 830 |
* removed PyYAML, and added dependency to installers and download instructions |
| 831 |
* updated to LogicParser, DRT (Dan Garrette) |
| 832 |
* WordNet similarity metrics return None instead of -1 when |
| 833 |
they fail to find a path (Steve Bethard) |
| 834 |
* shortest_path_distance uses instance hypernyms (Jordan Boyd-Graber) |
| 835 |
* clean_html improved (Bjorn Maeland) |
| 836 |
* batch_parse, batch_interpret and batch_evaluate functions allow |
| 837 |
grammar or grammar filename as argument |
| 838 |
* more Portuguese examples (portuguese_en.doctest, examples/pt.py) |
| 839 |
|
| 840 |
NLTK-Contrib: |
| 841 |
* Aligner implementations (Christopher Crowner, Torsten Marek) |
| 842 |
* ScriptTranscriber package (Richard Sproat and Kristy Hollingshead) |
| 843 |
|
| 844 |
Book: |
| 845 |
* updates for second printing, correcting errata |
| 846 |
https://nltk.googlecode.com/svn/trunk/nltk/doc/book/errata.txt |
| 847 |
|
| 848 |
Data: |
| 849 |
* added Europarl sample, with 10 docs for each of 11 langs (Nitin Madnani) |
| 850 |
* added SMULTRON sample corpus (Torsten Marek, Martin Volk) |
| 851 |
|
| 852 |
|
| 853 |
Version 2.0b7 2009-11-09 |
| 854 |
|
| 855 |
NLTK: |
| 856 |
* minor bugfixes and enhancements: data loader, inference package, FreqDist, Punkt |
| 857 |
* added Portuguese example module, similar to nltk.book for English (examples/pt.py) |
| 858 |
* added all_lemma_names() method to WordNet corpus reader |
| 859 |
* added update() and __add__() extensions to FreqDist (enhances alignment with Python 3.0 counters) |
| 860 |
* reimplemented clean_html |
| 861 |
* added test-suite runner for automatic/manual regression testing |
| 862 |
|
| 863 |
NLTK-Data: |
| 864 |
* updated Punkt models for sentence segmentation |
| 865 |
* added corpus of the works of Machado de Assis (Brazilian Portuguese) |
| 866 |
|
| 867 |
Book: |
| 868 |
* Added translation of preface into Portuguese, contributed by Tiago Tresoldi. |
| 869 |
|
| 870 |
Version 2.0b6 2009-09-20 |
| 871 |
|
| 872 |
NLTK: |
| 873 |
* minor fixes for Python 2.4 compatibility |
| 874 |
* added words() method to XML corpus reader |
| 875 |
* minor bugfixes and code clean-ups |
| 876 |
* fixed downloader to put data in %APPDATA% on Windows |
| 877 |
|
| 878 |
Data: |
| 879 |
* Updated Punkt models |
| 880 |
* Fixed utf8 encoding issues with UDHR and Stopwords Corpora |
| 881 |
* Renamed CoNLL "cat" files to "esp" (different language) |
| 882 |
* Added Alvey NLT feature-based grammar |
| 883 |
* Added Polish PL196x corpus |
| 884 |
|
| 885 |
Version 2.0b5 2009-07-19 |
| 886 |
|
| 887 |
NLTK: |
| 888 |
* minor bugfixes (incl FreqDist, Python eggs) |
| 889 |
* added reader for Europarl Corpora (contributed by Nitin Madnani) |
| 890 |
* added reader for IPI PAN Polish Corpus (contributed by Konrad Goluchowski) |
| 891 |
* fixed data.py so that it doesn't generate a warning for Windows Python 2.6 |
| 892 |
|
| 893 |
NLTK-Contrib: |
| 894 |
* updated Praat reader (contributed by Margaret Mitchell) |
| 895 |
|
| 896 |
Version 2.0b4 2009-07-10 |
| 897 |
|
| 898 |
NLTK: |
| 899 |
* switched to Apache License, Version 2.0 |
| 900 |
* minor bugfixes in semantics and inference packages |
| 901 |
* support for Python eggs |
| 902 |
* fixed stale regression tests |
| 903 |
|
| 904 |
Data: |
| 905 |
* added NomBank 1.0 |
| 906 |
* uppercased feature names in some grammars |
| 907 |
|
| 908 |
Version 2.0b3 2009-06-25 |
| 909 |
|
| 910 |
NLTK: |
| 911 |
* several bugfixes |
| 912 |
* added nombank corpus reader (Paul Bedaride) |
| 913 |
|
| 914 |
Version 2.0b2 2009-06-15 |
| 915 |
|
| 916 |
NLTK: |
| 917 |
* minor bugfixes and optimizations for parsers, updated some doctests |
| 918 |
* added bottom-up filtered left corner parsers, |
| 919 |
LeftCornerChartParser and IncrementalLeftCornerChartParser. |
| 920 |
* fixed dispersion plot bug which prevented empty plots |
| 921 |
|
| 922 |
Version 2.0b1 2009-06-09 |
| 923 |
|
| 924 |
NLTK: |
| 925 |
* major refactor of chart parser code and improved API (Peter Ljungl喃) |
| 926 |
* added new bottom-up left-corner chart parser strategy |
| 927 |
* misc bugfixes (ChunkScore, chart rules, chatbots, jcn-similarity) |
| 928 |
* improved efficiency of "import nltk" using lazy module imports |
| 929 |
* moved CCG package and ISRI Arabic stemmer from NLTK-Contrib into core NLTK |
| 930 |
* misc code cleanups |
| 931 |
|
| 932 |
Contrib: |
| 933 |
* moved out of the main NLTK distribution into a separate distribution |
| 934 |
|
| 935 |
Book: |
| 936 |
* Ongoing polishing ahead of print publication |
| 937 |
|
| 938 |
Version 0.9.9 2009-05-06 |
| 939 |
|
| 940 |
NLTK: |
| 941 |
* Finalized API for NLTK 2.0 and the book, incl dozens of small fixes |
| 942 |
* Names of the form nltk.foo.Bar now available as nltk.Bar |
| 943 |
for significant functionality; in some cases the name was modified |
| 944 |
(using old names will produce a deprecation warning) |
| 945 |
* Bugfixes in downloader, WordNet |
| 946 |
* Expanded functionality in DecisionTree |
| 947 |
* Bigram collocations extended for discontiguous bigrams |
| 948 |
* Translation toy nltk.misc.babelfish |
| 949 |
* New module nltk.help giving access to tagset documentation |
| 950 |
* Fix imports so that NLTK builds without Tkinter (Bjorn Maeland) |
| 951 |
|
| 952 |
Data: |
| 953 |
* new maxent NE chunker model |
| 954 |
* updated grammar packages for the book |
| 955 |
* data for new tagsets collection, documenting several tagsets |
| 956 |
* added lolcat translation to the Genesis collection |
| 957 |
|
| 958 |
Contrib (work in progress): |
| 959 |
* Updates to coreference package (Joseph Frazee) |
| 960 |
* New ISRI Arabic stemmer (Hosam Algasaier) |
| 961 |
* Updates to Toolbox package (Greg Aumann) |
| 962 |
|
| 963 |
Book: |
| 964 |
* Substantial editorial corrections ahead of final submission |
| 965 |
|
| 966 |
Version 0.9.8 2009-02-18 |
| 967 |
|
| 968 |
NLTK: |
| 969 |
* New off-the-shelf tokenizer, POS tagger, and named-entity tagger |
| 970 |
* New metrics package with inter-annotator agreement scores, |
| 971 |
distance metrics, rank correlation |
| 972 |
* New collocations package (Joel Nothman) |
| 973 |
* Many clean-ups to WordNet package (Steven Bethard, Jordan Boyd-Graber) |
| 974 |
* Moved old pywordnet-based WordNet package to nltk_contrib |
| 975 |
* WordNet browser (Paul Bone) |
| 976 |
* New interface to dependency treebank corpora |
| 977 |
* Moved MinimalSet class into nltk.misc package |
| 978 |
* Put NLTK applications in new nltk.app package |
| 979 |
* Many other improvements incl semantics package, toolbox, MaltParser |
| 980 |
* Misc changes to many API names in preparation for 1.0, old names deprecated |
| 981 |
* Most classes now available in the top-level namespace |
| 982 |
* Work on Python egg distribution (Brandon Rhodes) |
| 983 |
* Removed deprecated code remaining from 0.8.* versions |
| 984 |
* Fixes for Python 2.4 compatibility |
| 985 |
|
| 986 |
Data: |
| 987 |
* Corrected identifiers in Dependency Treebank corpus |
| 988 |
* Basque and Catalan Dependency Treebanks (CoNLL 2007) |
| 989 |
* PE08 Parser Evaluation data |
| 990 |
* New models for POS tagger and named-entity tagger |
| 991 |
|
| 992 |
Book: |
| 993 |
* Substantial editorial corrections |
| 994 |
|
| 995 |
Version 0.9.7 2008-12-19 |
| 996 |
|
| 997 |
NLTK: |
| 998 |
* fixed problems with accessing zipped corpora |
| 999 |
* improved design and efficiency of grammars and chart parsers |
| 1000 |
including new bottom-up combine strategy and a redesigned |
| 1001 |
Earley strategy (Peter Ljunglof) |
| 1002 |
* fixed bugs in smoothed probability distributions and added |
| 1003 |
regression tests (Peter Ljunglof) |
| 1004 |
* improvements to Punkt (Joel Nothman) |
| 1005 |
* improvements to text classifiers |
| 1006 |
* simple word-overlap RTE classifier |
| 1007 |
|
| 1008 |
Data: |
| 1009 |
* A new package of large grammars (Peter Ljunglof) |
| 1010 |
* A small gazetteer corpus and corpus reader |
| 1011 |
* Organized example grammars into separate packages |
| 1012 |
* Childrens' stories added to gutenberg package |
| 1013 |
|
| 1014 |
Contrib (work in progress): |
| 1015 |
* fixes and demonstration for named-entity feature extractors in nltk_contrib.coref |
| 1016 |
|
| 1017 |
Book: |
| 1018 |
* extensive changes throughout, including new chapter 5 on classification |
| 1019 |
and substantially revised chapter 11 on managing linguistic data |
| 1020 |
|
| 1021 |
Version 0.9.6 2008-12-07 |
| 1022 |
|
| 1023 |
NLTK: |
| 1024 |
* new WordNet corpus reader (contributed by Steven Bethard) |
| 1025 |
* incorporated dependency parsers into NLTK (was NLTK-Contrib) (contributed by Jason Narad) |
| 1026 |
* moved nltk/cfg.py to nltk/grammar.py and incorporated dependency grammars |
| 1027 |
* improved efficiency of unification algorithm |
| 1028 |
* various enhancements to the semantics package |
| 1029 |
* added plot() and tabulate() methods to FreqDist and ConditionalFreqDist |
| 1030 |
* FreqDist.keys() and list(FreqDist) provide keys reverse-sorted by value, |
| 1031 |
to avoid the confusion caused by FreqDist.sorted() |
| 1032 |
* new downloader module to support interactive data download: nltk.download() |
| 1033 |
run using "python -m nltk.downloader all" |
| 1034 |
* fixed WordNet bug that caused min_depth() to sometimes give incorrect result |
| 1035 |
* added nltk.util.Index as a wrapper around defaultdict(list) plus |
| 1036 |
a functional-style initializer |
| 1037 |
* fixed bug in Earley chart parser that caused it to break |
| 1038 |
* added basic TnT tagger nltk.tag.tnt |
| 1039 |
* new corpus reader for CoNLL dependency format (contributed by Kepa Sarasola and Iker Manterola) |
| 1040 |
* misc other bugfixes |
| 1041 |
|
| 1042 |
Contrib (work in progress): |
| 1043 |
* TIGERSearch implementation by Torsten Marek |
| 1044 |
* extensions to hole and glue semantics modules by Dan Garrette |
| 1045 |
* new coreference package by Joseph Frazee |
| 1046 |
* MapReduce interface by Xinfan Meng |
| 1047 |
|
| 1048 |
Data: |
| 1049 |
* Corpora are stored in compressed format if this will not compromise speed of access |
| 1050 |
* Swadesh Corpus of comparative wordlists in 23 languages |
| 1051 |
* Split grammar collection into separate packages |
| 1052 |
* New Basque and Spanish grammar samples (contributed by Kepa Sarasola and Iker Manterola) |
| 1053 |
* Brown Corpus sections now have meaningful names (e.g. 'a' is now 'news') |
| 1054 |
* Fixed bug that forced users to manually unzip the WordNet corpus |
| 1055 |
* New dependency-parsed version of Treebank corpus sample |
| 1056 |
* Added movie script "Monty Python and the Holy Grail" to webtext corpus |
| 1057 |
* Replaced words corpus data with a much larger list of English words |
| 1058 |
* New URL for list of available NLTK corpora |
| 1059 |
https://nltk.googlecode.com/svn/trunk/nltk_data/index.xml |
| 1060 |
|
| 1061 |
Book: |
| 1062 |
* complete rewrite of first three chapters to make the book accessible |
| 1063 |
to a wider audience |
| 1064 |
* new chapter on data-intensive language processing |
| 1065 |
* extensive reworking of most chapters |
| 1066 |
* Dropped subsection numbering; moved exercises to end of chapters |
| 1067 |
|
| 1068 |
Distributions: |
| 1069 |
* created Portfile to support Mac installation |
| 1070 |
|
| 1071 |
|
| 1072 |
Version 0.9.5 2008-08-27 |
| 1073 |
|
| 1074 |
NLTK: |
| 1075 |
* text module with support for concordancing, text generation, plotting |
| 1076 |
* book module |
| 1077 |
* Major reworking of the automated theorem proving modules (Dan Garrette) |
| 1078 |
* draw.dispersion now uses pylab |
| 1079 |
* draw.concordance GUI tool |
| 1080 |
* nltk.data supports for reading corpora and other data files from within zipfiles |
| 1081 |
* trees can be constructed from strings with Tree(s) (cf Tree.parse(s)) |
| 1082 |
|
| 1083 |
Contrib (work in progress): |
| 1084 |
* many updates to student projects |
| 1085 |
- nltk_contrib.agreement (Thomas Lippincott) |
| 1086 |
- nltk_contrib.coref (Joseph Frazee) |
| 1087 |
- nltk_contrib.depparser (Jason Narad) |
| 1088 |
- nltk_contrib.fuf (Petro Verkhogliad) |
| 1089 |
- nltk_contrib.hadoop (Xinfan Meng) |
| 1090 |
* clean-ups: deleted stale files; moved some packages to misc |
| 1091 |
|
| 1092 |
Data |
| 1093 |
* Cleaned up Gutenberg text corpora |
| 1094 |
* added Moby Dick; removed redundant copy of Blake songs. |
| 1095 |
* more tagger models |
| 1096 |
* renamed to nltk_data to facilitate installation |
| 1097 |
* stored each corpus as a zip file for quicker installation |
| 1098 |
and access, and to solve a problem with the Propbank |
| 1099 |
corpus including a file with an illegal name for MSWindows |
| 1100 |
(con.xml). |
| 1101 |
|
| 1102 |
Book: |
| 1103 |
* changed filenames to chNN format |
| 1104 |
* reworked opening chapters (work in progress) |
| 1105 |
|
| 1106 |
Distributions: |
| 1107 |
* fixed problem with mac installer that arose when Python binary |
| 1108 |
couldn't be found |
| 1109 |
* removed dependency of NLTK on nltk_data so that NLTK code can be |
| 1110 |
installed before the data |
| 1111 |
|
| 1112 |
Version 0.9.4 2008-08-01 |
| 1113 |
|
| 1114 |
NLTK: |
| 1115 |
- Expanded semantics package for first order logic, linear logic, |
| 1116 |
glue semantics, DRT, LFG (Dan Garrette) |
| 1117 |
- new WordSense class in wordnet.synset supporting access to synsets |
| 1118 |
from sense keys and accessing sense counts (Joel Nothman) |
| 1119 |
- interface to Mallet's linear chain CRF implementation (nltk.tag.crf) |
| 1120 |
- misc bugfixes incl Punkt, synsets, maxent |
| 1121 |
- improved support for chunkers incl flexible chunk corpus reader, |
| 1122 |
new rule type: ChunkRuleWithContext |
| 1123 |
- new GUI for pos-tagged concordancing nltk.draw.pos_concordance |
| 1124 |
- new GUI for developing regexp chunkers nltk.draw.rechunkparser |
| 1125 |
- added bio_sents() and bio_words() methods to ConllChunkCorpusReader in conll.py |
| 1126 |
to allow reading (word, tag, chunk_typ) tuples off of CoNLL-2000 corpus. Also |
| 1127 |
modified ConllChunkCorpusView to support these changes. |
| 1128 |
- feature structures support values with custom unification methods |
| 1129 |
- new flag on tagged corpus readers to use simplified tagsets |
| 1130 |
- new package for ngram language modeling with Katz backoff nltk.model |
| 1131 |
- added classes for single-parented and multi-parented trees that |
| 1132 |
automatically maintain parent pointers (nltk.tree.ParentedTree and |
| 1133 |
nltk.tree.MultiParentedTree) |
| 1134 |
- new WordNet browser GUI (Jussi Salmela, Paul Bone) |
| 1135 |
- improved support for lazy sequences |
| 1136 |
- added generate() method to probability distributions |
| 1137 |
- more flexible parser for converting bracketed strings to trees |
| 1138 |
- made fixes to docstrings to improve API documentation |
| 1139 |
|
| 1140 |
Contrib (work in progress) |
| 1141 |
- new NLG package, FUF/SURGE (Petro Verkhogliad) |
| 1142 |
- new dependency parser package (Jason Narad) |
| 1143 |
- new Coreference package, incl support for |
| 1144 |
ACE-2, MUC-6 and MUC-7 corpora (Joseph Frazee) |
| 1145 |
- CCG Parser (Graeme Gange) |
| 1146 |
- first order resolution theorem prover (Dan Garrette) |
| 1147 |
|
| 1148 |
Data: |
| 1149 |
- Nnw NPS Chat Corpus and corpus reader (nltk.corpus.nps_chat) |
| 1150 |
- ConllCorpusReader can now be used to read CoNLL 2004 and 2005 corpora. |
| 1151 |
- Implemented HMM-based Treebank POS tagger and phrase chunker for |
| 1152 |
nltk_contrib.coref in api.py. Pickled versions of these objects are checked |
| 1153 |
in in data/taggers and data/chunkers. |
| 1154 |
|
| 1155 |
Book: |
| 1156 |
- misc corrections in response to feedback from readers |
| 1157 |
|
| 1158 |
Version 0.9.3 2008-06-03 |
| 1159 |
|
| 1160 |
NLTK: |
| 1161 |
- modified WordNet similarity code to use pre-built information content files |
| 1162 |
- new classifier-based tagger, BNC corpus reader |
| 1163 |
- improved unicode support for corpus readers |
| 1164 |
- improved interfaces to Weka, Prover9/Mace4 |
| 1165 |
- new support for using MEGAM and SciPy to train maxent classifiers |
| 1166 |
- rewrite of Punkt sentence segmenter (Joel Nothman) |
| 1167 |
- bugfixes for WordNet information content module (Jordan Boyd-Graber) |
| 1168 |
- code clean-ups throughout |
| 1169 |
|
| 1170 |
Book: |
| 1171 |
- miscellaneous fixes in response to feedback from readers |
| 1172 |
|
| 1173 |
Contrib: |
| 1174 |
- implementation of incremental algorithm for generating |
| 1175 |
referring expressions (contributed by Margaret Mitchell) |
| 1176 |
- refactoring WordNet browser (Paul Bone) |
| 1177 |
|
| 1178 |
Corpora: |
| 1179 |
- included WordNet information content files |
| 1180 |
|
| 1181 |
Version 0.9.2 2008-03-04 |
| 1182 |
|
| 1183 |
NLTK: |
| 1184 |
- new theorem-prover and model-checker module nltk.inference, |
| 1185 |
including interface to Prover9/Mace4 (Dan Garrette, Ewan Klein) |
| 1186 |
- bugfix in Reuters corpus reader that causes Python |
| 1187 |
to complain about too many open files |
| 1188 |
- VerbNet and PropBank corpus readers |
| 1189 |
|
| 1190 |
Data: |
| 1191 |
- VerbNet Corpus version 2.1: hierarchical, verb lexicon linked to WordNet |
| 1192 |
- PropBank Corpus: predicate-argument structures, as stand-off annotation of Penn Treebank |
| 1193 |
|
| 1194 |
Contrib: |
| 1195 |
- New work on WordNet browser, incorporating a client-server model (Jussi Salmela) |
| 1196 |
|
| 1197 |
Distributions: |
| 1198 |
- Mac OS 10.5 distribution |
| 1199 |
|
| 1200 |
Version 0.9.1 2008-01-24 |
| 1201 |
|
| 1202 |
NLTK: |
| 1203 |
- new interface for text categorization corpora |
| 1204 |
- new corpus readers: RTE, Movie Reviews, Question Classification, Brown Corpus |
| 1205 |
- bugfix in ConcatenatedCorpusView that caused iteration to fail if it didn't start from the beginning of the corpus |
| 1206 |
|
| 1207 |
Data: |
| 1208 |
- Question classification data, included with permission of Li & Roth |
| 1209 |
- Reuters 21578 Corpus, ApteMod version, from CPAN |
| 1210 |
- Movie Reviews corpus (sentiment polarity), included with permission of Lillian Lee |
| 1211 |
- Corpus for Recognising Textual Entailment (RTE) Challenges 1, 2 and 3 |
| 1212 |
- Brown Corpus (reverted to original file structure: ca01-cr09) |
| 1213 |
- Penn Treebank corpus sample (simplified implementation, new readers treebank_raw and treebank_chunk) |
| 1214 |
- Minor redesign of corpus readers, to use filenames instead of "items" to identify parts of a corpus |
| 1215 |
|
| 1216 |
Contrib: |
| 1217 |
- theorem_prover: Prover9, tableau, MaltParser, Mace4, glue semantics, docs (Dan Garrette, Ewan Klein) |
| 1218 |
- drt: improved drawing, conversion to FOL (Dan Garrette) |
| 1219 |
- gluesemantics: GUI demonstration, abstracted LFG code, documentation (Dan Garrette) |
| 1220 |
- readability: various text readability scores (Thomas Jakobsen, Thomas Skardal) |
| 1221 |
- toolbox: code to normalize toolbox databases (Greg Aumann) |
| 1222 |
|
| 1223 |
Book: |
| 1224 |
- many improvements in early chapters in response to reader feedback |
| 1225 |
- updates for revised corpus readers |
| 1226 |
- moved unicode section to chapter 3 |
| 1227 |
- work on engineering.txt (not included in 0.9.1) |
| 1228 |
|
| 1229 |
Distributions: |
| 1230 |
- Fixed installation for Mac OS 10.5 (Joshua Ritterman) |
| 1231 |
- Generalize doctest_driver to work with doc_contrib |
| 1232 |
|
| 1233 |
Version 0.9 2007-10-12 |
| 1234 |
|
| 1235 |
NLTK: |
| 1236 |
- New naming of packages and modules, and more functions imported into |
| 1237 |
top-level nltk namespace, e.g. nltk.chunk.Regexp -> nltk.RegexpParser, |
| 1238 |
nltk.tokenize.Line -> nltk.LineTokenizer, nltk.stem.Porter -> nltk.PorterStemmer, |
| 1239 |
nltk.parse.ShiftReduce -> nltk.ShiftReduceParser |
| 1240 |
- processing class names changed from verbs to nouns, e.g. |
| 1241 |
StemI -> StemmerI, ParseI -> ParserI, ChunkParseI -> ChunkParserI, ClassifyI -> ClassifierI |
| 1242 |
- all tokenizers are now available as subclasses of TokenizeI, |
| 1243 |
selected tokenizers are also available as functions, e.g. wordpunct_tokenize() |
| 1244 |
- rewritten ngram tagger code, collapsed lookup tagger with unigram tagger |
| 1245 |
- improved tagger API, permitting training in the initializer |
| 1246 |
- new system for deprecating code so that users are notified of name changes. |
| 1247 |
- support for reading feature cfgs to parallel reading cfgs (parse_featcfg()) |
| 1248 |
- text classifier package, maxent (GIS, IIS), naive Bayes, decision trees, weka support |
| 1249 |
- more consistent tree printing |
| 1250 |
- wordnet's morphy stemmer now accessible via stemmer package |
| 1251 |
- RSLP Portuguese stemmer (originally developed by Viviane Moreira Orengo, reimplemented by Tiago Tresoldi) |
| 1252 |
- promoted ieer_rels.py to the sem package |
| 1253 |
- improvements to WordNet package (Jussi Salmela) |
| 1254 |
- more regression tests, and support for checking coverage of tests |
| 1255 |
- miscellaneous bugfixes |
| 1256 |
- remove numpy dependency |
| 1257 |
|
| 1258 |
Data: |
| 1259 |
- new corpus reader implementation, refactored syntax corpus readers |
| 1260 |
- new data package: corpora, grammars, tokenizers, stemmers, samples |
| 1261 |
- CESS-ESP Spanish Treebank and corpus reader |
| 1262 |
- CESS-CAT Catalan Treebank and corpus reader |
| 1263 |
- Alpino Dutch Treebank and corpus reader |
| 1264 |
- MacMorpho POS-tagged Brazilian Portuguese news text and corpus reader |
| 1265 |
- trained model for Portuguese sentence segmenter |
| 1266 |
- Floresta Portuguese Treebank version 7.4 and corpus reader |
| 1267 |
- TIMIT player audio support |
| 1268 |
|
| 1269 |
Contrib: |
| 1270 |
- BioReader (contributed by Carlos Rodriguez) |
| 1271 |
- TnT tagger (contributed by Sam Huston) |
| 1272 |
- wordnet browser (contributed by Jussi Salmela, requires wxpython) |
| 1273 |
- lpath interpreter (contributed by Haejoong Lee) |
| 1274 |
- timex -- regular expression-based temporal expression tagger |
| 1275 |
|
| 1276 |
Book: |
| 1277 |
- polishing of early chapters |
| 1278 |
- introductions to parts 1, 2, 3 |
| 1279 |
- improvements in book processing software (xrefs, avm & gloss formatting, javascript clipboard) |
| 1280 |
- updates to book organization, chapter contents |
| 1281 |
- corrections throughout suggested by readers (acknowledged in preface) |
| 1282 |
- more consistent use of US spelling throughout |
| 1283 |
- all examples redone to work with single import statement: "import nltk" |
| 1284 |
- reordered chapters: 5->7->8->9->11->12->5 |
| 1285 |
* language engineering in part 1 to broaden the appeal |
| 1286 |
of the earlier part of the book and to talk more about |
| 1287 |
evaluation and baselines at an earlier stage |
| 1288 |
* concentrate the partial and full parsing material in part 2, |
| 1289 |
and remove the specialized feature-grammar material into part 3 |
| 1290 |
|
| 1291 |
Distributions: |
| 1292 |
- streamlined mac installation (Joshua Ritterman) |
| 1293 |
- included mac distribution with ISO image |
| 1294 |
|
| 1295 |
Version 0.8 2007-07-01 |
| 1296 |
|
| 1297 |
Code: |
| 1298 |
- changed nltk.__init__ imports to explicitly import names from top-level modules |
| 1299 |
- changed corpus.util to use the 'rb' flag for opening files, to fix problems |
| 1300 |
reading corpora under MSWindows |
| 1301 |
- updated stale examples in engineering.txt |
| 1302 |
- extended feature structure interface to permit chained features, e.g. fs['F','G'] |
| 1303 |
- further misc improvements to test code plus some bugfixes |
| 1304 |
Tutorials: |
| 1305 |
- rewritten opening section of tagging chapter |
| 1306 |
- reorganized some exercises |
| 1307 |
|
| 1308 |
Version 0.8b2 2007-06-26 |
| 1309 |
|
| 1310 |
Code (major): |
| 1311 |
- new corpus package, obsoleting old corpora package |
| 1312 |
- supports caching, slicing, corpus search path |
| 1313 |
- more flexible API |
| 1314 |
- global updates so all NLTK modules use new corpus package |
| 1315 |
- moved nltk/contrib to separate top-level package nltk_contrib |
| 1316 |
- changed wordpunct tokenizer to use \w instead of a-zA-Z0-9 |
| 1317 |
as this will be more robust for languages other than English, |
| 1318 |
with implications for many corpus readers that use it |
| 1319 |
- known bug: certain re-entrant structures in featstruct |
| 1320 |
- known bug: when the LHS of an edge contains an ApplicationExpression, |
| 1321 |
variable values in the RHS bindings aren't copied over when the |
| 1322 |
fundamental rule applies |
| 1323 |
- known bug: HMM tagger is broken |
| 1324 |
Tutorials: |
| 1325 |
- global updates to NLTK and docs |
| 1326 |
- ongoing polishing |
| 1327 |
Corpora: |
| 1328 |
- treebank sample reverted to published multi-file structure |
| 1329 |
Contrib: |
| 1330 |
- DRT and Glue Semantics code (nltk_contrib.drt, nltk_contrib.gluesemantics, by Dan Garrette) |
| 1331 |
|
| 1332 |
Version 0.8b1 2007-06-18 |
| 1333 |
|
| 1334 |
Code (major): |
| 1335 |
- changed package name to nltk |
| 1336 |
- import all top-level modules into nltk, reducing need for import statements |
| 1337 |
- reorganization of sub-package structures to simplify imports |
| 1338 |
- new featstruct module, unifying old featurelite and featurestructure modules |
| 1339 |
- FreqDist now inherits from dict, fd.count(sample) becomes fd[sample] |
| 1340 |
- FreqDist initializer permits: fd = FreqDist(len(token) for token in text) |
| 1341 |
- made numpy optional |
| 1342 |
Code (minor): |
| 1343 |
- changed GrammarFile initializer to accept filename |
| 1344 |
- consistent tree display format |
| 1345 |
- fixed loading process for WordNet and TIMIT that prevented code installation if data not installed |
| 1346 |
- taken more care with unicode types |
| 1347 |
- incorporated pcfg code into cfg module |
| 1348 |
- moved cfg, tree, featstruct to top level |
| 1349 |
- new filebroker module to make handling of example grammar files more transparent |
| 1350 |
- more corpus readers (webtext, abc) |
| 1351 |
- added cfg.covers() to check that a grammar covers a sentence |
| 1352 |
- simple text-based wordnet browser |
| 1353 |
- known bug: parse/featurechart.py uses incorrect apply() function |
| 1354 |
Corpora: |
| 1355 |
- csv data file to document NLTK corpora |
| 1356 |
Contrib: |
| 1357 |
- added Glue semantics code (contrib.glue, by Dan Garrette) |
| 1358 |
- Punkt sentence segmenter port (contrib.punkt, by Willy) |
| 1359 |
- added LPath interpreter (contrib.lpath, by Haejoong Lee) |
| 1360 |
- extensive work on classifiers (contrib.classifier*, Sumukh Ghodke) |
| 1361 |
Tutorials: |
| 1362 |
- polishing on parts I, II |
| 1363 |
- more illustrations, data plots, summaries, exercises |
| 1364 |
- continuing to make prose more accessible to non-linguistic audience |
| 1365 |
- new default import that all chapters presume: from nltk.book import * |
| 1366 |
Distributions: |
| 1367 |
- updated to latest version of numpy |
| 1368 |
- removed WordNet installation instructions as WordNet is now included in corpus distribution |
| 1369 |
- added pylab (matplotlib) |
| 1370 |
|
| 1371 |
Version 0.7.5 2007-05-16 |
| 1372 |
|
| 1373 |
Code: |
| 1374 |
- improved WordNet and WordNet-Similarity interface |
| 1375 |
- the Lancaster Stemmer (contributed by Steven Tomcavage) |
| 1376 |
Corpora: |
| 1377 |
- Web text samples |
| 1378 |
- BioCreAtIvE-PPI - a corpus for protein-protein interactions |
| 1379 |
- Switchboard Telephone Speech Corpus Sample (via Talkbank) |
| 1380 |
- CMU Problem Reports Corpus sample |
| 1381 |
- CONLL2002 POS+NER data |
| 1382 |
- Patient Information Leaflet corpus |
| 1383 |
- WordNet 3.0 data files |
| 1384 |
- English wordlists: basic English, frequent words |
| 1385 |
Tutorials: |
| 1386 |
- more improvements to text and images |
| 1387 |
|
| 1388 |
Version 0.7.4 2007-05-01 |
| 1389 |
|
| 1390 |
Code: |
| 1391 |
- Indian POS tagged corpus reader: corpora.indian |
| 1392 |
- Sinica Treebank corpus reader: corpora.sinica_treebank |
| 1393 |
- new web corpus reader corpora.web |
| 1394 |
- tag package now supports pickling |
| 1395 |
- added function to utilities.py to guess character encoding |
| 1396 |
Corpora: |
| 1397 |
- Rotokas texts from Stuart Robinson |
| 1398 |
- POS-tagged corpora for several Indian languages (Bangla, Hindi, Marathi, Telugu) from A Kumaran |
| 1399 |
Tutorials: |
| 1400 |
- Substantial work on Part II of book on structured programming, parsing and grammar |
| 1401 |
- More bibliographic citations |
| 1402 |
- Improvements in typesetting, cross references |
| 1403 |
- Redimensioned images and tables for better use of page space |
| 1404 |
- Moved project list to wiki |
| 1405 |
Contrib: |
| 1406 |
- validation of toolbox entries using chunking |
| 1407 |
- improved classifiers |
| 1408 |
Distribution: |
| 1409 |
- updated for Python 2.5.1, Numpy 1.0.2 |
| 1410 |
|
| 1411 |
Version 0.7.3 2007-04-02 |
| 1412 |
|
| 1413 |
* Code: |
| 1414 |
- made chunk.Regexp.parse() more flexible about its input |
| 1415 |
- developed new syntax for PCFG grammars, e.g. A -> B C [0.3] | D [0.7] |
| 1416 |
- fixed CFG parser to support grammars with slash categories |
| 1417 |
- moved beta classify package from main NLTK to contrib |
| 1418 |
- Brill taggers loaded correctly |
| 1419 |
- misc bugfixes |
| 1420 |
* Corpora: |
| 1421 |
- Shakespeare XML corpus sample and corpus reader |
| 1422 |
* Tutorials: |
| 1423 |
- improvements to prose, exercises, plots, images |
| 1424 |
- expanded and reorganized tutorial on structured programming |
| 1425 |
- formatting improvements for Python listings |
| 1426 |
- improved plots (using pylab) |
| 1427 |
- categorization of problems by difficulty |
| 1428 |
Contrib: |
| 1429 |
- more work on kimmo lexicon and grammar |
| 1430 |
- more work on classifiers |
| 1431 |
|
| 1432 |
Version 0.7.2 2007-03-01 |
| 1433 |
|
| 1434 |
* Code: |
| 1435 |
- simple feature detectors (detect module) |
| 1436 |
- fixed problem when token generators are passed to a parser (parse package) |
| 1437 |
- fixed bug in Grammar.productions() (identified by Lucas Champollion and Mitch Marcus) |
| 1438 |
- fixed import bug in category.GrammarFile.earley_parser |
| 1439 |
- added utilities.OrderedDict |
| 1440 |
- initial port of old NLTK classifier package (by Sam Huston) |
| 1441 |
- UDHR corpus reader |
| 1442 |
* Corpora: |
| 1443 |
- added UDHR corpus (Universal Declaration of Human Rights) |
| 1444 |
with 10k text samples in 300+ languages |
| 1445 |
* Tutorials: |
| 1446 |
- improved images |
| 1447 |
- improved book formatting, including new support for: |
| 1448 |
- javascript to copy program examples to clipboard in HTML version, |
| 1449 |
- bibliography, chapter cross-references, colorization, index, table-of-contents |
| 1450 |
|
| 1451 |
* Contrib: |
| 1452 |
- new Kimmo system: contrib.mit.six863.kimmo (Rob Speer) |
| 1453 |
- fixes for: contrib.fsa (Rob Speer) |
| 1454 |
- demonstration of text classifiers trained on UDHR corpus for |
| 1455 |
language identification: contrib.langid (Sam Huston) |
| 1456 |
- new Lambek calculus system: contrib.lambek |
| 1457 |
- new tree implementation based on elementtree: contrib.tree |
| 1458 |
|
| 1459 |
Version 0.7.1 2007-01-14 |
| 1460 |
|
| 1461 |
* Code: |
| 1462 |
- bugfixes (HMM, WordNet) |
| 1463 |
|
| 1464 |
Version 0.7 2006-12-22 |
| 1465 |
|
| 1466 |
* Code: |
| 1467 |
- bugfixes, including fixed bug in Brown corpus reader |
| 1468 |
- cleaned up wordnet 2.1 interface code and similarity measures |
| 1469 |
- support for full Penn treebank format contributed by Yoav Goldberg |
| 1470 |
* Tutorials: |
| 1471 |
- expanded tutorials on advanced parsing and structured programming |
| 1472 |
- checked all doctest code |
| 1473 |
- improved images for chart parsing |
| 1474 |
|
| 1475 |
Version 0.7b1 2006-12-06 |
| 1476 |
|
| 1477 |
* Code: |
| 1478 |
- expanded semantic interpretation package |
| 1479 |
- new high-level chunking interface, with cascaded chunking |
| 1480 |
- split chunking code into new chunk package |
| 1481 |
- updated wordnet package to support version 2.1 of Wordnet. |
| 1482 |
- prototyped basic wordnet similarity measures |
| 1483 |
(path distance, Wu + Palmer and Leacock + Chodorow, Resnik similarity measures.) |
| 1484 |
- bugfixes (tag.Window, tag.ngram) |
| 1485 |
- more doctests |
| 1486 |
* Contrib: |
| 1487 |
- toolbox language settings module |
| 1488 |
* Tutorials: |
| 1489 |
- rewrite of chunking chapter, switched from Treebank to CoNLL format as main focus, |
| 1490 |
simplified evaluation framework, added ngram chunking section |
| 1491 |
- substantial updates throughout (esp programming and semantics chapters) |
| 1492 |
* Corpora: |
| 1493 |
- Chat-80 Prolog data files provided as corpora, plus corpus reader |
| 1494 |
|
| 1495 |
Version 0.7a2 2006-11-13 |
| 1496 |
|
| 1497 |
* Code: |
| 1498 |
- more doctest |
| 1499 |
- code to read Chat-80 data |
| 1500 |
- HMM bugfix |
| 1501 |
* Tutorials: |
| 1502 |
- continued updates and polishing |
| 1503 |
* Corpora: |
| 1504 |
- toolbox MDF sample data |
| 1505 |
|
| 1506 |
Version 0.7a1 2006-10-29 |
| 1507 |
|
| 1508 |
* Code: |
| 1509 |
- new toolbox module (Greg Aumann) |
| 1510 |
- new semantics package (Ewan Klein) |
| 1511 |
- bugfixes |
| 1512 |
* Tutorials |
| 1513 |
- substantial revision, especially in preface, introduction, words, |
| 1514 |
and semantics chapters. |
| 1515 |
|
| 1516 |
Version 0.6.6 2006-10-06 |
| 1517 |
|
| 1518 |
* Code: |
| 1519 |
- bugfixes (probability, shoebox, draw) |
| 1520 |
* Contrib: |
| 1521 |
- new work on shoebox package (Stuart Robinson) |
| 1522 |
* Tutorials: |
| 1523 |
- continual expansion and revision, especially on introduction to |
| 1524 |
programming, advanced programming and the feature-based grammar chapters. |
| 1525 |
|
| 1526 |
Version 0.6.5 2006-07-09 |
| 1527 |
|
| 1528 |
* Code: |
| 1529 |
- improvements to shoebox module (Stuart Robinson, Greg Aumann) |
| 1530 |
- incorporated feature-based parsing into core NLTK-Lite |
| 1531 |
- corpus reader for Sinica treebank sample |
| 1532 |
- new stemmer package |
| 1533 |
* Contrib: |
| 1534 |
- hole semantics implementation (Peter Wang) |
| 1535 |
- Incorporating yaml |
| 1536 |
- new work on feature structures, unification, lambda calculus |
| 1537 |
- new work on shoebox package (Stuart Robinson, Greg Aumann) |
| 1538 |
* Corpora: |
| 1539 |
- Sinica treebank sample |
| 1540 |
* Tutorials: |
| 1541 |
- expanded discussion throughout, incl: left-recursion, trees, grammars, |
| 1542 |
feature-based grammar, agreement, unification, PCFGs, |
| 1543 |
baseline performance, exercises, improved display of trees |
| 1544 |
|
| 1545 |
Version 0.6.4 2006-04-20 |
| 1546 |
|
| 1547 |
* Code: |
| 1548 |
- corpus readers for Senseval 2 and TIMIT |
| 1549 |
- clusterer (ported from old NLTK) |
| 1550 |
- support for cascaded chunkers |
| 1551 |
- bugfix suggested by Brent Payne |
| 1552 |
- new SortedDict class for regression testing |
| 1553 |
* Contrib: |
| 1554 |
- CombinedTagger tagger and marshalling taggers, contributed by Tiago Tresoldi |
| 1555 |
* Corpora: |
| 1556 |
- new: Senseval 2, TIMIT sample |
| 1557 |
* Tutorials: |
| 1558 |
- major revisions to programming, words, tagging, chunking, and parsing tutorials |
| 1559 |
- many new exercises |
| 1560 |
- formatting improvements, including colorized program examples |
| 1561 |
- fixed problem with testing on training data, reported by Jason Baldridge |
| 1562 |
|
| 1563 |
Version 0.6.3 2006-03-09 |
| 1564 |
|
| 1565 |
* switch to new style classes |
| 1566 |
* repair FSA model sufficiently for Kimmo module to work |
| 1567 |
* port of MIT Kimmo morphological analyzer; still needs lots of code clean-up and inline docs |
| 1568 |
* expanded support for shoebox format, developed with Stuart Robinson |
| 1569 |
* fixed bug in indexing CFG productions, for empty right-hand-sides |
| 1570 |
* efficiency improvements, suggested by Martin Ranang |
| 1571 |
* replaced classeq with isinstance, for efficiency improvement, as suggested by Martin Ranang |
| 1572 |
* bugfixes in chunk eval |
| 1573 |
* simplified call to draw_trees |
| 1574 |
* names, stopwords corpora |
| 1575 |
|
| 1576 |
Version 0.6.2 2006-01-29 |
| 1577 |
|
| 1578 |
* Peter Spiller's concordancer |
| 1579 |
* Will Hardy's implementation of Penton's paradigm visualization system |
| 1580 |
* corpus readers for presidential speeches |
| 1581 |
* removed NLTK dependency |
| 1582 |
* generalized CFG terminals to permit full range of characters |
| 1583 |
* used fully qualified names in demo code, for portability |
| 1584 |
* bugfixes from Yoav Goldberg, Eduardo Pereira Habkost |
| 1585 |
* fixed obscure quoting bug in tree displays and conversions |
| 1586 |
* simplified demo code, fixed import bug |