| 1 |
3.0.0 |
| 2 |
----- |
| 3 |
Removed trove dependency. |
| 4 |
Changed license to ASL. |
| 5 |
Re-organized package structure to support up coming work. |
| 6 |
Added perceptron classifier. |
| 7 |
|
| 8 |
2.5.3 |
| 9 |
----- |
| 10 |
Fixed bug in GIS class, a cutoff value was not passed on to the GISTrainer |
| 11 |
Fixed bug in GISTrainer, a cutoff value compare was incorrect |
| 12 |
|
| 13 |
2.5.2 |
| 14 |
----- |
| 15 |
Wrapped model reading input streams with BufferedInputStream for performance gain. |
| 16 |
|
| 17 |
2.5.1 |
| 18 |
----- |
| 19 |
Fixed bugs with real-valued feature support. |
| 20 |
Added unit tests for these bugs. |
| 21 |
|
| 22 |
2.5.0 |
| 23 |
----- |
| 24 |
Added support for real-valued features with RealBasicEventStream, |
| 25 |
RealValueFileEventStream, OnePassRealValueDataIndexer, and |
| 26 |
TwoPassRealValueDataIndexer classes. |
| 27 |
|
| 28 |
Added support for priors with the Prior interface. By default the |
| 29 |
package has been using a uniform prior when no other information is |
| 30 |
known, now you can create other priors. |
| 31 |
|
| 32 |
Set TwoPassDataIndexer to use UTF8 encoding. |
| 33 |
|
| 34 |
Made GISModel thread safe for calls to the eval() method. |
| 35 |
|
| 36 |
Refactored GISModel and GISTrainer to use common eval() method. |
| 37 |
|
| 38 |
2.4.0 |
| 39 |
----- |
| 40 |
Updated parameter datatype to re-use data structure storing which outcomes |
| 41 |
are found with that predicate/context in GISModel. (Per suggestion by |
| 42 |
Richard Northedge). |
| 43 |
|
| 44 |
Made changes to support above change in GISTrainer, GISModelReader, |
| 45 |
GISModelWriter, and OldFormatGISModelReader. |
| 46 |
|
| 47 |
Changed smoothing boolean to be parameter instead of static variable in GIS. |
| 48 |
|
| 49 |
Update sample code to reflect above change. |
| 50 |
|
| 51 |
Extracted FileEventStream from TwoPassDataIndexer. |
| 52 |
|
| 53 |
Improved javadoc. |
| 54 |
|
| 55 |
2.3.0 |
| 56 |
----- |
| 57 |
Added and updated javadoc including package descriptions. |
| 58 |
Made new trove type to improve efficiency. |
| 59 |
|
| 60 |
2.2.0 |
| 61 |
----- |
| 62 |
Added TwoPassDataIndexer to use a temorpy file for storing the events. This |
| 63 |
allows much larger models to be trained in a smaller amount of memory with no |
| 64 |
noticible decrease in performance for large models. |
| 65 |
|
| 66 |
Made minor additions to interface to allow more flexible access to model |
| 67 |
outcomes. |
| 68 |
|
| 69 |
2.1.0 (Major bug fixes) |
| 70 |
----- |
| 71 |
|
| 72 |
Fixed some bugs with the updating off the corrections paramater. |
| 73 |
Namely the expected value needed to check if a particular context was |
| 74 |
avalable with the outcome seen in training and if not add a term to |
| 75 |
the expected value of the correction constant. (Tom) |
| 76 |
|
| 77 |
Smoothing initial value wasn't in the log domain. (Tom) |
| 78 |
|
| 79 |
Loglikelihood update as returned by nextIteration wasn't in the right |
| 80 |
place so the loglikelihood value it returned was incorrect. (Tom) |
| 81 |
|
| 82 |
Added cast to integer division in model update. (Tom, fixing bug |
| 83 |
pointed out by Zhang Le) |
| 84 |
|
| 85 |
|
| 86 |
2.0 (Major improvements) |
| 87 |
--- |
| 88 |
Fixed bug where singleton events are dropped. (Tom) |
| 89 |
|
| 90 |
Added eval method so distribution could be passed in rathar then |
| 91 |
allocated durring each call. Left old interface in place but modified |
| 92 |
it to use the new eval method. Also made numfeats a class level |
| 93 |
variable. (Tom) |
| 94 |
|
| 95 |
Fixed cases where parameters which only occured with a single output |
| 96 |
weren't getting updated. Ended up getting rid of pabi and cfvals |
| 97 |
structures. These have been replaced with the data for a single event, |
| 98 |
double[] modelDistribution, and this is used to update the modifiers |
| 99 |
for a single event and then updated for each additional event. This |
| 100 |
change made it easier to initialize the modleDistribution to the |
| 101 |
uniform distribution which was necessary to fix teh above problem. |
| 102 |
Also moved the computation of modelDistribution into it's own routine |
| 103 |
which is name eval and is almost exactly the same as GISModel.eval w/o |
| 104 |
doing the context string to integer mappings. (Tom) |
| 105 |
|
| 106 |
Made correction constant non-optional. When the events all have the same |
| 107 |
number of contexts then the model tries to make the expected value of the |
| 108 |
correction constant nearly 0. This is needed because while the number of |
| 109 |
contexts may be same it is very unlikly that all context occur with all |
| 110 |
outcomes. (Tom) |
| 111 |
|
| 112 |
Made nextIteration return a double which is the log-likelihood from |
| 113 |
the previous itteration. At some point there isn't enough accuracy in |
| 114 |
a double to make further iterations useful so the routine may stop |
| 115 |
prematurly when the decrease in log-likelihood is too small. (Tom) |
| 116 |
|
| 117 |
|
| 118 |
1.2.10 |
| 119 |
------ |
| 120 |
Fixed minor bug (found by Arno Erpenbeck) in BasicContextGenerator: it |
| 121 |
was retaining whitespace in the contextual predicates. (Jason) |
| 122 |
|
| 123 |
Added error message to TrainEval's eval() method. (Jason) |
| 124 |
|
| 125 |
|
| 126 |
1.2.9 (Bug fix release) |
| 127 |
_____ |
| 128 |
|
| 129 |
Modified the cutoff loop in DataIndexer to use the increment() method |
| 130 |
of TObjectIntHashMap. (Jason) |
| 131 |
|
| 132 |
Fixed a bug (found by Chieu Hai Leong) in which the correctionConstant |
| 133 |
of GISModel was an int that was used in division. Now, |
| 134 |
correctionConstant is a double. (Jason) |
| 135 |
|
| 136 |
|
| 137 |
1.2.8 |
| 138 |
_____ |
| 139 |
|
| 140 |
Modified GISTrainer to use the new increment() and adjustValue() |
| 141 |
methods available in Trove 0.1.4 hashmaps. (Jason) |
| 142 |
|
| 143 |
Set up the GISTrainer to use an initial capacity and load factor for |
| 144 |
the big hashmaps it uses. The initial capacity is half the number of |
| 145 |
outcomes, and the load factor is 0.9. (Jason) |
| 146 |
|
| 147 |
(opennlp.maxent.DataIndexer) Do not index events with 0 active features. |
| 148 |
(Eric) |
| 149 |
|
| 150 |
Upgraded trove dependency to 0.1.1 (includes TIntArrayList, with reset()) |
| 151 |
(Eric) |
| 152 |
|
| 153 |
(opennlp.maxent.DataIndexer) |
| 154 |
Refactored event count computation so that the cutoff can be applied while |
| 155 |
events are read. This obviates the need for a separate pass over the |
| 156 |
predicates between event count computation and indexing. It also saves |
| 157 |
memory by reducing the amount of temporary data needed and by avoiding |
| 158 |
creation of instances of the Counter class. the applyCutoff() method |
| 159 |
was no longer needed and so is gone. (Eric) |
| 160 |
|
| 161 |
(opennlp.maxent.DataIndexer) |
| 162 |
Made the event count computation + cutoff application also handle the |
| 163 |
assignment of unique indexes to predicates that "make the cut." This |
| 164 |
saves a fair amount of time in the indexing process. (Eric) |
| 165 |
|
| 166 |
(opennlp.maxent.DataIndexer) |
| 167 |
Refactored the indexing implementation so that TIntArrayLists are |
| 168 |
(re-)used for constructing the array of predicate references associated |
| 169 |
with each ComparableEvent. Using the TIntArrayList instead of an |
| 170 |
ArrayList of Integers dramatically reduces the amount of garbage |
| 171 |
produced during indexing; it's also smaller. (Eric) |
| 172 |
|
| 173 |
(opennlp.maxent.DataIndexer) |
| 174 |
removed toIntArray() method, since TIntArrayList provides the same |
| 175 |
behavior without the cost of a loop over a List of Integers (Eric) |
| 176 |
|
| 177 |
(opennlp.maxent.DataIndexer) |
| 178 |
changed indexing Maps to TObjectIntHashMaps to save space in several |
| 179 |
places. (Eric) |
| 180 |
|
| 181 |
1.2.6 |
| 182 |
----- |
| 183 |
Summary: efficiency improvements for model training. |
| 184 |
|
| 185 |
Removed Colt dependency in favor of GNU Trove. (Eric) |
| 186 |
|
| 187 |
Refactored index() method in DataIndexer so that only one pass over the |
| 188 |
list of events is needed. This saves time (of course) and also space, |
| 189 |
since it's no longer necessary to allocate temporary data structures to |
| 190 |
share data between two loops. (Eric) |
| 191 |
|
| 192 |
Refactored sorting/merging algorithm for ComparableEvents so that |
| 193 |
merging can be done in place. This makes it possible to merge without |
| 194 |
copying duplicate events into sublists and so improves the indexer's |
| 195 |
ability to work on large data sets with a reasonable amount of memory. |
| 196 |
There is still more to be done in this department, however. (Eric) |
| 197 |
|
| 198 |
The output directory of the build structure is now "output" instead of |
| 199 |
"build". (Jason) |
| 200 |
|
| 201 |
1.2.4 |
| 202 |
----- |
| 203 |
Added options for doing *very* simple smoothing, in which we 'observe' |
| 204 |
features that we didn't actually see in the training data. Seems to |
| 205 |
improve performance for models with small data sets and only a few |
| 206 |
outcomes, though it conversely appears to degrade those with lots of |
| 207 |
data and lots of outcomes. |
| 208 |
|
| 209 |
Added BasicEventStream and BasicContextGenerator classes which assume |
| 210 |
that the contextual predicates and outcomes are just sitting pretty in |
| 211 |
lines. This allows the events to be stored in a file and then read in |
| 212 |
for training without scanning around producing the events everytime. |
| 213 |
|
| 214 |
Added sample application "sports" to help with testing model behavior |
| 215 |
and to act as an example to help newbies use the toolkit and build |
| 216 |
their own maxent applications. |
| 217 |
|
| 218 |
Fixed bug in TrainEval in which the number of iterations and the |
| 219 |
cutoff were swapped in the call to train the model. |
| 220 |
|
| 221 |
PerlHelp and BasicEnglishAffixes classes were moved out of Maxent so |
| 222 |
that gnu-regexp.jar is no longer needed. |
| 223 |
|
| 224 |
|
| 225 |
1.2.2 |
| 226 |
----- |
| 227 |
Added "exe" to build.xml to create stand alone jar files. Also added |
| 228 |
structure and "homepage" target so that it is easier to keep homepage |
| 229 |
up-to-date. |
| 230 |
|
| 231 |
Added IntegerPool, which manages a pool of shareable, read-only Integer |
| 232 |
objects in a fixed range. This is used in several places where Integer |
| 233 |
objects were previously created and then GCed. |
| 234 |
|
| 235 |
In various places, used Collections API features to speed things up. |
| 236 |
For example, java.util.List.toArray() will do a System.arraycopy, if |
| 237 |
given a big enough array to copy into. This is, therefore, much |
| 238 |
faster. |
| 239 |
|
| 240 |
Added getAllOutcomes() method in GISModel to show the String names of |
| 241 |
all outcomes matched up with their normalized probabilities. |
| 242 |
|
| 243 |
|
| 244 |
1.2.0 |
| 245 |
----- |
| 246 |
Changed license to LGPL. |
| 247 |
|
| 248 |
Added build.xml file to be used with the build tool Jakarta Ant. |
| 249 |
|
| 250 |
Work sponsored by Electric Knowledge: |
| 251 |
Added BasicEnglishAffixes class to perform basic morphological |
| 252 |
stemming for English. |
| 253 |
|
| 254 |
Fixed a bug pointed out by Chieu Hai Leong in which the model would |
| 255 |
not train properly in situations where the number of features was |
| 256 |
constant (and therefore, no correction features need to be computed). |
| 257 |
|
| 258 |
The top level package name has changed from quipu to opennlp. Thus, |
| 259 |
what was "quipu.maxent" is now "opennlp.maxent". See |
| 260 |
http://www.opennlp.com for more details. |
| 261 |
|
| 262 |
Lots of little tweaks to reduce memory consumption. |
| 263 |
|
| 264 |
Several changes sponsored by eTranslate Inc: |
| 265 |
|
| 266 |
* The new opennlp.maxent.io subpackage. The input/output system for |
| 267 |
models is now designed to facilitate storing and retrieving of |
| 268 |
models in different formats. At present, GISModelReaders and |
| 269 |
GISModelWriters have been supplied for reading and writing plain |
| 270 |
text and binary files and either can be gzipped or |
| 271 |
uncompressed. The OldFormatGISModelReader can be used to read old |
| 272 |
models (from Maxent 1.0), and also provides command line |
| 273 |
functionality to convert old models to a new format. Also, |
| 274 |
SuffixSensitiveGISModelReader and SuffixSensitiveGISModelWriter |
| 275 |
have been provided to allow models to be stored and retrieved |
| 276 |
appropriated based on the suffixes on their file names. |
| 277 |
|
| 278 |
* Model training no longer automatically persists the new model to |
| 279 |
disk. Instead it returns a GISModel object which can then be |
| 280 |
persisted using an object from one of the GISModelWriter classes. |
| 281 |
|
| 282 |
* Model training now relies on EventStreams rather than |
| 283 |
EventCollectors. An EventStream is fed to the DataIndexer |
| 284 |
directly without the developer needing to invoke the DataIndex |
| 285 |
class explicitly. A good way to feed an EventStream the data it |
| 286 |
needs to form events is to use a DataStream that can return |
| 287 |
Objects from a data source in a format and os-independent |
| 288 |
manner. An implementation of the DataStream interface for reading |
| 289 |
plain text files line by line is provide in the |
| 290 |
opennlp.maxent.PlainTextByLineDataStream class. |
| 291 |
|
| 292 |
In order to retain backwards compatability, the |
| 293 |
EventCollectorAsStream class is provided as a wrapper over the |
| 294 |
EventCollectors used in Maxent 1.0. |
| 295 |
|
| 296 |
* GISModel is now thread-safe. Thus, one maxent application can |
| 297 |
service multiple evaluations in parallel with a single model. |
| 298 |
|
| 299 |
* The opennlp.maxent.ModelReplacementManager class has been added to |
| 300 |
allow a maxent application to replace its current maxent model |
| 301 |
with a newly trained one in a thread-safe manner without stopping |
| 302 |
the servicing of requests. |
| 303 |
|
| 304 |
An alternative to the ModelReplacementManager is to use a |
| 305 |
DomainToModelMap object to record the mapping between different |
| 306 |
data domains to models which are optimized for them. This class |
| 307 |
allows models to be swapped in a thread-safe manner as well. |
| 308 |
|
| 309 |
* The GIS class now is a factory which invokes a new GISTrainer |
| 310 |
whenever a new model is being trained. Since GISTrainer has only |
| 311 |
local variables and methods, multiple models can be trained |
| 312 |
simultaneously on different threads. With the previous |
| 313 |
implementation, requests to train new models were forced to queue |
| 314 |
up. |
| 315 |
|
| 316 |
1.0 |
| 317 |
_____ |
| 318 |
Reworked the GIS algorithm to use efficient data structures |
| 319 |
* Tables matching things like predicates, probabilities, correction |
| 320 |
values to their outcomes now use OpenIntDoubleHashMaps. |
| 321 |
* Several functions over OpenIntDoubleHashMaps are now defined, |
| 322 |
and most of the work of the iteration loop is in fact done by |
| 323 |
these. |
| 324 |
|
| 325 |
Events with the same outcome and contextual predicates are collapsed |
| 326 |
to reduce the number of tokens which must be iterated through in |
| 327 |
several loops. The number of times each event "type" is seen is then |
| 328 |
stored in an array (numTimesEventSeen) to provide the proper values. |
| 329 |
|
| 330 |
GISModel uses less memory for models with many outcomes, and is much |
| 331 |
faster on them as well. Performance is roughly the same for models |
| 332 |
with only two outcomes. |
| 333 |
|
| 334 |
More code documentation. |
| 335 |
|
| 336 |
Fully compatible with models built using version 0.2.0. |
| 337 |
|
| 338 |
|
| 339 |
0.2.0 |
| 340 |
_____ |
| 341 |
Initial release of fully functional maxent package. |