| 1 |
Revision history for Perl extension Prog. |
| 2 |
|
| 3 |
0.75 11 Dec 2013 |
| 4 |
- 3.1.0 release of swish3 cli: |
| 5 |
* remove --lucy_highlightable and replace with -I (--Indexer_opts) |
| 6 |
* rename --aggregator_opts to --Aggregator_opts |
| 7 |
|
| 8 |
0.74 01 Oct 2013 |
| 9 |
- packaaging clean up only |
| 10 |
|
| 11 |
0.73 12 Sept 2013 |
| 12 |
- turn off Pod tests unless RELEASE_TESTING env var set. Pod::Coverage |
| 13 |
failures were because optional dependencies were not present but module |
| 14 |
load was attempted in order to walk symbol tree for pod tests. |
| 15 |
|
| 16 |
0.72 10 Sept 2013 |
| 17 |
- add TRACE_ALL to pod-coverage test to discover why it suddenly started |
| 18 |
epic fail |
| 19 |
- add explicit port to t/17-spider-server, running on 5002 |
| 20 |
|
| 21 |
0.71 1 Aug 2013 |
| 22 |
- add --null_term feature to swish3 example |
| 23 |
- add sitemap support to Spider |
| 24 |
- swish3 -W <n> sets SWISH_PARSER_WARNINGS env var |
| 25 |
|
| 26 |
0.70 10 Feb 2013 |
| 27 |
- add tests for Utils class. |
| 28 |
- Utils extension regex now matches \w+ rather than hardcoded document |
| 29 |
extensions. |
| 30 |
|
| 31 |
0.69 05 Feb 2013 |
| 32 |
- fix documentation for SWISH::Prog::InvIndex::Meta, add data() accessor |
| 33 |
method. |
| 34 |
|
| 35 |
0.68 23 Jan 2013 |
| 36 |
- fix failing test t/17-spider-server with 'use base' vs '@ISA' |
| 37 |
- force URI objects to stringify when passing in/out of cache |
| 38 |
|
| 39 |
0.67 19 Jan 2013 |
| 40 |
- Spider now uses LWP::RobotUA to respect robots.txt. Dependency on |
| 41 |
WWW::Mechanize is removed. |
| 42 |
- Spider authorization features now work. Added bona fide test suite for |
| 43 |
spidering. |
| 44 |
- expand Queue API to add remove() and clean() and internal locking on |
| 45 |
get() |
| 46 |
- Spider->modified_since feature to allow for incremental crawls |
| 47 |
- Added new class SWISH::Prog::Aggregator::Spider::Response, refactoring |
| 48 |
appropriate UA methods into Response class since WWW::Mechanize was |
| 49 |
intentionally blurring the logical distinction. |
| 50 |
- Spider->file_rules (new feature) follows same code path as |
| 51 |
Aggregator::FS. |
| 52 |
- added Utils::write_log and ::write_log_line methods for standardizing |
| 53 |
debug output |
| 54 |
|
| 55 |
0.66 8 Dec 2012 |
| 56 |
- yanked redundant test t/09 as it is now part of Search::Tools |
| 57 |
- fix t/05 spider to warn on correct missing module; fix spider count |
| 58 |
|
| 59 |
0.65 5 Nov 2012 |
| 60 |
- add more tests for MailFS aggregator |
| 61 |
- all Aggregators now set content-type to valid MIME instead of custom |
| 62 |
Swish types |
| 63 |
- Aggregator->set_parser_from_type() will now not override a defined |
| 64 |
Doc->parser() value |
| 65 |
|
| 66 |
0.64 12 Oct 2012 |
| 67 |
- add --aggregator_opts to swish3 and SWISH::Prog |
| 68 |
- add looks_like_feed() to Spider, and add dependency on XML::Feed |
| 69 |
- switch Spider::UA parent class to WWW::Mechanize::GZip |
| 70 |
|
| 71 |
0.63 11 Sept 2012 |
| 72 |
- add TagAlias support to Config. TagAlias is a Swish3 feature. |
| 73 |
- improve SWISH::Prog documentation |
| 74 |
- add --doc_filter to swish3, to allow for set_filter to work from an |
| 75 |
external Perl file. |
| 76 |
|
| 77 |
0.62 3 Sept 2012 |
| 78 |
- support subclassing of Indexer in start() sanity check introduced in |
| 79 |
0.61 |
| 80 |
- add InvIndex->meta_file method and avoid referring to swish.xml directly |
| 81 |
|
| 82 |
0.61 23 Aug 2012 |
| 83 |
- swish3 now catches exceptions and re-throws errors |
| 84 |
- swish3 now passes --debug flag into Searcher->new |
| 85 |
- base Indexer now performs check on existing indexes to verify that you |
| 86 |
aren't trying to run one format's Indexer against a different format's |
| 87 |
index. |
| 88 |
- swish_last_start file now created by base Indexer class, not swish3 cli. |
| 89 |
|
| 90 |
0.60 28 July 2012 |
| 91 |
- add *_property methods to InvIndex::Meta |
| 92 |
- SWISH::3 now required dependency |
| 93 |
- added use_quotes and quote_char features to Aggregator::DBI, inspired by |
| 94 |
patch from Miko O'Sullivan. |
| 95 |
|
| 96 |
0.59 26 July 2012 |
| 97 |
- add FollowXInclude support for Swish3 |
| 98 |
|
| 99 |
0.58 26 June 2012 |
| 100 |
- fix Config class for FuzzyIndexingMode conversion to XML (ver3) |
| 101 |
|
| 102 |
0.57 7 May 2012 |
| 103 |
- change default max_depth in Spider to 1 (makes it actually work by |
| 104 |
default) |
| 105 |
|
| 106 |
0.56 2 Apr 2012 |
| 107 |
- make swish3 stdin/stdout utf8 safe |
| 108 |
|
| 109 |
0.55 6 Nov 2011 |
| 110 |
- fix skip test count in t/15 |
| 111 |
|
| 112 |
0.54 14 Oct 2011 |
| 113 |
- $SWISH::Prog::Utils::DefaultExtension now let's you configure default |
| 114 |
file extension |
| 115 |
- fix SWISH::Prog::Config->new() to support perldoc's claims of passing in |
| 116 |
a hashref or named attribute pairs. |
| 117 |
|
| 118 |
0.53 09 Oct 2011 |
| 119 |
- fix DBI Aggregator to make code support docs for swishtitle and |
| 120 |
swishdescription. Bug reported by Logan Bell. |
| 121 |
- fix FS Aggregator behavior to work like 2.x and not skip files with no |
| 122 |
extension. |
| 123 |
- fix bug in swish3 where index dir is not created before swish_last_start |
| 124 |
is touched. |
| 125 |
- confess() when swish.xml does not exist |
| 126 |
- base Aggregator class no longer requires an indexer in new(). |
| 127 |
|
| 128 |
0.52 26 Sept 2011 |
| 129 |
- add support for IndexFile config option |
| 130 |
- add --lucy_highlightable option to swish3 |
| 131 |
- new class SWISH::Prog::ReplaceRules for backcompat |
| 132 |
- add get_stemmer_lang() method to Config class for Snowball stemmer |
| 133 |
forward-compat. |
| 134 |
|
| 135 |
0.51 28 June 2011 |
| 136 |
- add 'Lucy' backend |
| 137 |
|
| 138 |
0.50 22 Feb 2011 |
| 139 |
- make swish3 -N flag argument optional, defaulting to |
| 140 |
indexdir/swish_last_start |
| 141 |
- error check in base Indexer for valid invindex param |
| 142 |
- misc doc fixes |
| 143 |
|
| 144 |
0.49 07 Jan 2011 |
| 145 |
- refactor FileRules feature into File::Rules module |
| 146 |
|
| 147 |
0.48 03 Nov 2010 |
| 148 |
- added progress() feature to Aggregator classes, and 'expected' param to |
| 149 |
swish3. |
| 150 |
- add CascadeMetaContext config option for Swish3 compatibility |
| 151 |
- bump SWISH::3 (optional) dependency to 0.09 |
| 152 |
- fix MailFS aggregator to preserve filesystem paths as Doc url |
| 153 |
|
| 154 |
0.47 25 July 2010 |
| 155 |
- add Search time to swish3 (3.0.7) |
| 156 |
- fix bug in Native::Searcher affecting fieldtypes |
| 157 |
|
| 158 |
0.46 22 June 2010 |
| 159 |
- add dep on Search::Query |
| 160 |
- add default_boolop feature to Native::Searcher |
| 161 |
- fix IncludeConfigFile recursionin Config->ver2_to_ver() |
| 162 |
|
| 163 |
0.45 08 May 2010 |
| 164 |
- fix swish3 to allow "swishrank" as -x option |
| 165 |
- enable -H option in swish3 |
| 166 |
- SWISH::3->slurp optimization now uses transparent gunzip feature. |
| 167 |
Requires SWISH::3 >= 0.06. |
| 168 |
- fix backcompat for DefaultContents config in Config->ver2_to_ver3() |
| 169 |
|
| 170 |
0.44 19 Mar 2010 |
| 171 |
- register MailFS aggregator as 'mailfs' for use with swish3 -S |
| 172 |
|
| 173 |
0.43 19 Mar 2010 |
| 174 |
- remove docs and method for 'sort_order' in abstract Searcher class. No |
| 175 |
subclass uses it, deferring to 'order' in the search() method opts. |
| 176 |
- FS->crawl now returns count |
| 177 |
- add MailFS aggregator class |
| 178 |
|
| 179 |
0.42 27 Feb 2010 |
| 180 |
- add invindex format to search output headers in swish3 |
| 181 |
- add parsed query to search output headers in swish3; add query() method |
| 182 |
to base Results class. |
| 183 |
- add progress_size to Aggregator and implement in FS. |
| 184 |
|
| 185 |
0.41 10 Feb 2010 |
| 186 |
- fix the check_swish() feature in Native::Indexer to only return the |
| 187 |
version number, not the whole output of swish-e -V (fixes the -D fix in |
| 188 |
0.40). |
| 189 |
|
| 190 |
0.40 09 Feb 2010 |
| 191 |
- make -D '\x03' optional in Native::Indexer based on which version of |
| 192 |
swish-e is available. The -D feature is only in 2.4.8 or newer. |
| 193 |
|
| 194 |
0.39 07 Feb 2010 |
| 195 |
- add fix for systems using Swish-e 2.6, which doesn't create a |
| 196 |
index.swish-e file. |
| 197 |
- remove the requires_external_bin from Makefile.PL since KSx and Xapian |
| 198 |
do not require swish-e but do require SWISH::Prog. Fix tests (which also |
| 199 |
check for swish-e) to SKIP the correct number of tests. |
| 200 |
|
| 201 |
0.38 05 Feb 2010 |
| 202 |
- add support for 'order', 'limit', 'start', 'max' and 'rank_scheme' in |
| 203 |
Native::Searcher. |
| 204 |
- Utils->perl_to_xml() moved to Search::Tools::XML (where it really |
| 205 |
belonged). |
| 206 |
- enable more options to swish3 |
| 207 |
|
| 208 |
0.37 01 Feb 2010 |
| 209 |
- add -D '\x03' to the Native::Indexer exec command. This mimics |
| 210 |
libswish3. |
| 211 |
- examples/swish3 now installed by default with usual make install. |
| 212 |
- added -N option to swish3, along with ok_if_newer_than feature in |
| 213 |
Aggregator. |
| 214 |
- added dependency on Rose::DateTime. |
| 215 |
- Native Indexer can now merge() and add(). |
| 216 |
- -l and -M options now supported in swish3 |
| 217 |
- yank Query.pm and QueryParser.pm. They weren't used and the idea is now |
| 218 |
fleshed out in Search::Query if we ever need it. |
| 219 |
- Searcher may now take an array of invindex values, and always returns an |
| 220 |
array ref of values. |
| 221 |
|
| 222 |
0.36 12 Jan 2009 |
| 223 |
- fix Config bug where StoreDescription was ignored in ver2_to_ver3() |
| 224 |
- switch to Module::Install |
| 225 |
- updates to example/swish3 |
| 226 |
- fix t/03_object.t to test for JSON not YAML::Syck (see RT#53275) |
| 227 |
|
| 228 |
0.35 09 Jan 2009 |
| 229 |
- fix bug in Config where $append flag was being interpreted as a |
| 230 |
MetaName/Property |
| 231 |
- fix bug in Config where ver2_to_ver3 was calling $self->new instead of |
| 232 |
$class->new when passed a filename. |
| 233 |
- fix bug in Spider where filter was not being passed correctly. |
| 234 |
|
| 235 |
0.34 07 Jan 2009 |
| 236 |
- fix mem leak test to include only those outside our control. |
| 237 |
- add version() to SWISH::Prog::Doc to pick header style per instance. |
| 238 |
|
| 239 |
0.33 02 Jan 2010 |
| 240 |
- fix bug where SWISH::3->slurp was being called with out binmode flag and |
| 241 |
.gz files were getting their NUL bytes stripped. |
| 242 |
- add test_mode feature (set with SWISH_TEST env var) |
| 243 |
- fix some memory leaks |
| 244 |
- fix JSON::XS dependency (JSON required, not JSON::XS). |
| 245 |
|
| 246 |
0.32 12 Dec 2009 |
| 247 |
- Config->ver2_to_ver3() method now supported. |
| 248 |
- fix bug in Prog.pm to allow Config passed in aggregator and Indexer |
| 249 |
passed in aggregator |
| 250 |
- removed dependency on Class::Accessor for Config.pm |
| 251 |
- refactor internal handling of Config object so there is only ever one |
| 252 |
per Prog instance |
| 253 |
- refactor internal handling of Config to support SWISH::3::Config |
| 254 |
- FileRules support now in Aggregator::FS |
| 255 |
|
| 256 |
0.31 1 Dec 2009 |
| 257 |
- POD fixes |
| 258 |
|
| 259 |
0.30 29 Nov 2009 |
| 260 |
- change KinoSearch to KSx in implementation registry |
| 261 |
- fix bug in base Searcher where InvIndex class was not loaded. |
| 262 |
|
| 263 |
0.29 8 Nov 2009 |
| 264 |
- add strip_plural option to Utils->perl_to_xml() |
| 265 |
- add count attribute to arrayref values of hashrefs in perl_to_xml() |
| 266 |
- fix method/param name strictness bug revealed by Rose::ObjectX::CAF |
| 267 |
0.03. This mostly required internal param name changes to official |
| 268 |
method names. |
| 269 |
|
| 270 |
0.28 30 Sept 2009 |
| 271 |
- add some missing calls to SUPER::init() |
| 272 |
|
| 273 |
0.27 28 Sep 2009 |
| 274 |
- make QueryParser and Query part of the official install, even though |
| 275 |
there are still TODO items |
| 276 |
- add all_metanames() method to Config |
| 277 |
- refactor to use Search::Tools and Rose::ObjectX::CAF instead of |
| 278 |
Class::Accessor::Fast |
| 279 |
|
| 280 |
0.26 26 Feb 2009 |
| 281 |
- fix off-by-one error in skip test count in t/03-object.t |
| 282 |
|
| 283 |
0.25 26 Jan 2009 |
| 284 |
- added more features to example/swish3 |
| 285 |
- add missing test file to MANIFEST |
| 286 |
- tests clean up after themselves wrt swish.xml header files |
| 287 |
|
| 288 |
0.24 20 Jan 2009 (Happy Obama Day!) |
| 289 |
- add base Searcher, Results and Result classes |
| 290 |
- refactor QueryParser to use Search::QueryParser::SQL |
| 291 |
- flesh out the InvIndex::Meta class, adding XML::Simple dependency |
| 292 |
- IMPORTANT: rename Native classes to match naming convention |
| 293 |
|
| 294 |
0.23 11 Dec 2008 |
| 295 |
- properly clean up temp Config files |
| 296 |
|
| 297 |
0.22 10 Dec 2008 |
| 298 |
- added support for SWISH::Prog::DBI |
| 299 |
- use SWISH::3 for slurping files, if available. |
| 300 |
- fix bug in SWISH::Prog::Utils->perl_to_xml where hash refs were not |
| 301 |
getting tagged. |
| 302 |
|
| 303 |
0.21 31 Oct 2008 |
| 304 |
- fix tests to skip correct numbers if swish-e not installed |
| 305 |
- change to JSON::XS and make json default format in Aggregator::Object |
| 306 |
- add perl_to_xml() in Utils |
| 307 |
- change t/04 to manually create dirs at test time |
| 308 |
|
| 309 |
0.20 23 April 2008 |
| 310 |
- complete rewrite of the API for Swish3 development |
| 311 |
|
| 312 |
0.08 22 Nov 2007 |
| 313 |
- added S::P::Mail class and example swishmail.pl script |
| 314 |
- sync all $VERSION vars to same value |
| 315 |
- allow for non-standard docclass names |
| 316 |
|
| 317 |
0.07 * S::P::Object iterator now uses more idiomatic while/next algorithm. |
| 318 |
|
| 319 |
0.06 * S::P::Object now defers to explicit 'config' passed in new(). |
| 320 |
|
| 321 |
0.05 * S::P::Object now uses JSON or YAML to serialize values instead of Data::Dump |
| 322 |
- This change is sync'd with SWISH::API::Object 0.0.6 |
| 323 |
|
| 324 |
0.04 * S::P::DBI no longer calls disconnect() on db handle in DESTROY |
| 325 |
- fixed bug with auto-defining metanames |
| 326 |
- S::P::Config always returns arrayref on directive method calls |
| 327 |
- use MIME::Types instead of File::Type |
| 328 |
- S::P::Find now uses File::Find instead of Path::Class::Iterator |
| 329 |
- S::P::Object obj_filter() must return an object of some kind |
| 330 |
- S::P::Object will not dump() scalar strings, thus avoiding extra quotes |
| 331 |
- S::P::Config added stringify() method and reverted from IO::All to |
| 332 |
File::Slurp |
| 333 |
- S::Prog added elapsed() method |
| 334 |
- S::Prog new() verbose now defaults to 1 |
| 335 |
|
| 336 |
0.03 * removed all mentions of SWISH3 |
| 337 |
- test indexes are removed as part of 'make clean' |
| 338 |
- switched from Data::Dumper to Data::Dump |
| 339 |
- added new SWISH::Prog::Object example |
| 340 |
- fixed bug in SWISH::Prog::Headers to test defined() rather than |
| 341 |
simply eval true/false |
| 342 |
- fixed bug in SWISH::Prog::DBI with accepting DBI handle in init() |
| 343 |
- changed to using IO::All instead of File::stat and File::Slurp for |
| 344 |
better UTF8 support |
| 345 |
- changed default order of init() and init_indexer() to work more |
| 346 |
intuitively. Fixed S::P::DBI accordingly. To prevent an indexer from |
| 347 |
being init'd (as in 0.02) simply override init_indexer() and don't call |
| 348 |
SUPER::init_indexer(). |
| 349 |
- new feature: SWISH::Prog::Find |
| 350 |
- renamed meta() to table_meta() in SWISH::Prog::DBI for clarity. |
| 351 |
- S::P::Config now always handles config params as arrays |
| 352 |
- Config metanames() propertynames() and propertynamesnostripchars() are |
| 353 |
no longer available |
| 354 |
- S::P::Config now requires IO::All and Config::General; added read2() |
| 355 |
method. |
| 356 |
- new feature placeholder: SWISH::Prog::Spider |
| 357 |
|
| 358 |
0.02 POD fixes |
| 359 |
- added merge() add() and mv() methods to S::P::Index and t/3merge.t and |
| 360 |
t/4add.t |
| 361 |
- added 2 more test docs |
| 362 |
- added format() accessor to S::P::Index |
| 363 |
- changed S::P::I to use Class::Accessor::Fast since we don't need to |
| 364 |
override get/set |
| 365 |
- S::P now always inits an indexer even if fh=>0 |
| 366 |
- added private _index_methods to S::P |
| 367 |
|
| 368 |
0.01 2005-11-22T21:54:37Z |
| 369 |
- original version; created by h2xs 1.22 with options -XA -n Prog |
| 370 |
|