| 1 |
%***************************************************************************% |
| 2 |
% % |
| 3 |
% Copyright (C) 2013 Linas Vepstas % |
| 4 |
% See file "LICENSE" for information about commercial use of this system % |
| 5 |
% % |
| 6 |
%***************************************************************************% |
| 7 |
|
| 8 |
% Want to match apostrophes, for abreviations (I'm I've, etc.) since these |
| 9 |
% cannot be auto-split with the current splitter. Also want to accept |
| 10 |
% hyphenated words, and word with underbars in them. |
| 11 |
%#ANY-WORD: /^[[:alnum:]_'-]+$/ |
| 12 |
%#ANY-PUNCT: /^[[:punct:]]+$/ |
| 13 |
|
| 14 |
% Match anything that doesn't match the above. |
| 15 |
% Match anything that isn't white-space. |
| 16 |
% Well ... actually, reject anything that begins or ends with |
| 17 |
% punctuation. We do this, so that tokenize can split off the |
| 18 |
% the affixes (trailing commas, etc) correctly. |
| 19 |
%#JUNK: /^[^[:punct:]][^[:space:]]+[^[:punct:]]$/ |