| 1 |
%%% spelling-doc.tex |
| 2 |
%%% Copyright 2012, 2013 Stephan Hennig |
| 3 |
%% |
| 4 |
%% This work may be distributed and/or modified under the conditions of |
| 5 |
%% the LaTeX Project Public License, either version 1.3 of this license |
| 6 |
%% or (at your option) any later version. The latest version of this |
| 7 |
%% license is in http://www.latex-project.org/lppl.txt |
| 8 |
%% and version 1.3 or later is part of all distributions of LaTeX |
| 9 |
%% version 2005/12/01 or later. |
| 10 |
%% |
| 11 |
%% See file README for more information. |
| 12 |
%% |
| 13 |
\documentclass[11pt]{article} |
| 14 |
\usepackage{fontspec} |
| 15 |
\defaultfontfeatures{Ligatures=TeX} |
| 16 |
\usepackage{multicol} |
| 17 |
\usepackage[rgb, x11names]{xcolor} |
| 18 |
\usepackage{listings} |
| 19 |
\input{\jobname-lst-lua.tex} |
| 20 |
\lstset{ |
| 21 |
basicstyle=\ttfamily, |
| 22 |
columns=spaceflexible, |
| 23 |
} |
| 24 |
% Short-cut for non-language code snippets. |
| 25 |
\lstMakeShortInline\| |
| 26 |
% Short-cut for LaTeX code snippets. |
| 27 |
\lstMakeShortInline[ |
| 28 |
language={[LaTeX]TeX}, |
| 29 |
basicstyle=\ttfamily, |
| 30 |
]° |
| 31 |
\lstdefinestyle{Lua}{ |
| 32 |
language=[5.2]Lua, |
| 33 |
keywordstyle=\bfseries\color{Blue4}, |
| 34 |
keywordstyle=[2]\bfseries\color{RoyalBlue3}, |
| 35 |
keywordstyle=[3]\bfseries\color{Purple3}, |
| 36 |
stringstyle=\bfseries\color{Coral4}, |
| 37 |
commentstyle=\itshape\color{Green4}, |
| 38 |
} |
| 39 |
\usepackage{xspace} |
| 40 |
\usepackage{array} |
| 41 |
\usepackage{booktabs} |
| 42 |
\usepackage[latin, UKenglish]{babel} |
| 43 |
\usepackage{hyperref} |
| 44 |
\hypersetup{ |
| 45 |
pdftitle={spelling}, |
| 46 |
pdfauthor={Stephan Hennig}, |
| 47 |
pdfkeywords={spell-checking, spelling, TeX, LuaTeX} |
| 48 |
} |
| 49 |
\hypersetup{ |
| 50 |
english,% For \autoref. |
| 51 |
pdfstartview={XYZ null null null},% Zoom factor is determined by viewer. |
| 52 |
colorlinks, |
| 53 |
linkcolor=RoyalBlue3, |
| 54 |
urlcolor=Chocolate4, |
| 55 |
citecolor=DeepPink2 |
| 56 |
} |
| 57 |
\usepackage{spelling} |
| 58 |
\spellingreadbad{\jobname.bad} |
| 59 |
\newcommand*{\pkg}{\textsf{spelling}} |
| 60 |
\newcommand*{\acr}[1]{\mbox{\scshape#1}} |
| 61 |
\newcommand*{\descr}[1]{〈\emph{#1}〉} |
| 62 |
\newcommand*{\cmd}[1]{\mbox{\ttfamily\textbackslash#1}} |
| 63 |
\newcommand*{\macro}[1]{\cmd{#1}\marginpar{\cmd{#1}}} |
| 64 |
\newcommand*{\latinphrase}[1]{\foreignlanguage{latin}{\emph{#1}}} |
| 65 |
\newcommand*{\lpcf}{\latinphrase{cf.}\xspace} |
| 66 |
\newcommand*{\lpeg}{\latinphrase{e.\,g.}\xspace} |
| 67 |
\newcommand*{\lpetc}{\latinphrase{etc.}\xspace} |
| 68 |
\newcommand*{\lpie}{\latinphrase{i.\,e.}\xspace} |
| 69 |
\begin{document} |
| 70 |
\author{Stephan Hennig\thanks{[email protected]}} |
| 71 |
\title{\pkg\thanks{This document describes the \pkg\ package v0.41.}} |
| 72 |
\maketitle |
| 73 |
|
| 74 |
|
| 75 |
\begin{abstract} |
| 76 |
This package supports spell-checking of \TeX\ documents compiled with |
| 77 |
the Lua\TeX\ engine. It can give visual feedback in \acr{pdf} output |
| 78 |
similar to \acr{wysiwyg} word processors. The package relies on an |
| 79 |
external spell-checker application that can check a plain text file |
| 80 |
and output a list of bad spellings. The package should work with most |
| 81 |
spell-checkers, even dumb, \TeX-unaware ones. |
| 82 |
|
| 83 |
\emph{Warning! This package is in a very early state. Everything may |
| 84 |
change!} |
| 85 |
\end{abstract} |
| 86 |
|
| 87 |
\begin{multicols}{2} |
| 88 |
\small |
| 89 |
% Set toc entries ragged right. Trick taken from tocloft.pdf. |
| 90 |
\makeatletter |
| 91 |
\renewcommand{\@tocrmarg}{2.55em plus1fil} |
| 92 |
\makeatother |
| 93 |
\tableofcontents |
| 94 |
\end{multicols} |
| 95 |
|
| 96 |
|
| 97 |
\section{Introduction} |
| 98 |
\label{sec:intro} |
| 99 |
|
| 100 |
Ther% |
| 101 |
\footnote{A footnote containing mispellings.} |
| 102 |
% |
| 103 |
are three main approaches to spell-checking \TeX\ documents: |
| 104 |
|
| 105 |
\begin{enumerate} |
| 106 |
|
| 107 |
\item checking spelling in the |.tex| source file, |
| 108 |
|
| 109 |
\item converting a |.tex| file to another format, for which a proved |
| 110 |
spell-checking solution exists, |
| 111 |
|
| 112 |
\item checking spelling after a |.tex| file has been processed by \TeX. |
| 113 |
|
| 114 |
\end{enumerate} |
| 115 |
|
| 116 |
All of these approaches have their strengths and weaknesses. This |
| 117 |
package follows the third approach, providing some unique features: |
| 118 |
|
| 119 |
\begin{itemize} |
| 120 |
|
| 121 |
\item In traditional solutions, text is extracted from typeset |
| 122 |
\acr{dvi}, \acr{ps} or \acr{pdf} files, including hyphenated words. |
| 123 |
To avoid (lots of) false positives being reported by the |
| 124 |
spell-checker, hyphenation needs to be switched off during the \TeX\ |
| 125 |
run. That is, one doesn't work on the original document any more. |
| 126 |
|
| 127 |
In contrast to that, the \pkg\ package works transparently on the |
| 128 |
original |.tex| source file. Text is extracted \emph{during} |
| 129 |
typesetting, after Lua\TeX\ has applied its catcode and macro |
| 130 |
machinery, but before hyphenation takes place. |
| 131 |
|
| 132 |
\item The \pkg\ package can highlight words with known incorrect |
| 133 |
spelling in \acr{pdf} output, giving visual feedback similar to |
| 134 |
\acr{wysiwyg} word processors.% |
| 135 |
\footnote{Currently, only colouring words is implemented.} |
| 136 |
|
| 137 |
\end{itemize} |
| 138 |
|
| 139 |
|
| 140 |
\section{Usage} |
| 141 |
\label{sec:usage} |
| 142 |
|
| 143 |
The \pkg\ package requires the Lua\TeX\ engine. All functionality of |
| 144 |
the package is implemented in Lua. The \LaTeX\ interface, which is |
| 145 |
described below, is effectively a wrapper around the Lua interface. |
| 146 |
|
| 147 |
\emph{Implementing such wrappers for other formats shouldn't be too |
| 148 |
difficult. The author is a \LaTeX-only user, though, and therefore |
| 149 |
grateful for contributions. By the way, the \LaTeX\ package needs |
| 150 |
some polishing, too, \lpeg, a key-value interface is desirable. |
| 151 |
Patches welcome!} |
| 152 |
|
| 153 |
|
| 154 |
\subsection{Work-flow} |
| 155 |
\label{sec:work-flow} |
| 156 |
|
| 157 |
Here's a short outline of how using the \pkg\ package fits into the |
| 158 |
general process of compiling a document with Lua\TeX: |
| 159 |
|
| 160 |
\begin{enumerate} |
| 161 |
|
| 162 |
\item After loading the package in the preamble of a |.tex| source file, |
| 163 |
a list of bad spellings is read from a file (if that file exists). |
| 164 |
|
| 165 |
\item During the Lua\TeX\ run, text is extracted from pages and all |
| 166 |
words are checked against the list of bad spellings. Words with a |
| 167 |
known incorrect spelling are highlighted in \acr{pdf} output. |
| 168 |
|
| 169 |
\item At the end of the Lua\TeX\ run, in addition to the \acr{pdf} file, |
| 170 |
a text file is written, containing most of the text of the typeset |
| 171 |
document. |
| 172 |
|
| 173 |
\item The text file is then checked by your favourite external |
| 174 |
spell-checker application, \lpeg, Aspell or Hunspell. The |
| 175 |
spell-checker should be able to write a list of bad spellings to a |
| 176 |
file. Otherwise, visual feedback in \acr{pdf} output won't work. |
| 177 |
|
| 178 |
\item Visually minded people may now compile their document a second |
| 179 |
time. This time, the new list of bad spellings is read-in and words |
| 180 |
with incorrect spelling found by the spell-checker should now be |
| 181 |
highlighted in \acr{pdf} output. Users can then apply the necessary |
| 182 |
corrections to the |.tex| source file. |
| 183 |
|
| 184 |
\end{enumerate} |
| 185 |
|
| 186 |
Whatever way spell-checker output is employed, users not interested in |
| 187 |
visual feedback (because their spell-checker has an interactive mode |
| 188 |
only or because they prefer grabbing bad spellings from a file directly) |
| 189 |
can also benefit from this package. Using it, Lua\TeX\ writes a pure |
| 190 |
text file that is particularly well suited as spell-checker input, |
| 191 |
because it contains no hyphenated words (and neither macros, nor active |
| 192 |
characters). That way, any spell-checker application, even \TeX-unaware |
| 193 |
ones, can be used to check spelling of \TeX\ documents. |
| 194 |
|
| 195 |
|
| 196 |
\subsection{Word lists} |
| 197 |
\label{sec:wordlists} |
| 198 |
|
| 199 |
As described above, after loading the \pkg\ package, a list of bad |
| 200 |
spellings is read from a file \descr{jobname}.|spell.bad|, if that file |
| 201 |
exists. Words found in this file are stored in an internal list of bad |
| 202 |
spellings and are later used for highlighting spelling mistakes in |
| 203 |
\acr{pdf} output. Additionally, a list of good spellings is read from a |
| 204 |
file \descr{jobname}|.spell.good|, if that file exists. Words found in |
| 205 |
the latter file are stored in an internal list of good spellings. File |
| 206 |
format for both files is one word per line. Files must be in the |
| 207 |
\acr{utf-8} encoding. Letter case is significant. |
| 208 |
|
| 209 |
A word in the document is highlighted, if it occurs in the internal list |
| 210 |
of bad spellings, but not in the internal list of good spellings. That |
| 211 |
is, known good spellings take precedence over known bad spellings. |
| 212 |
|
| 213 |
Users can load additional files containing lists of bad or good |
| 214 |
spellings with macros \macro{spellingreadbad} and |
| 215 |
\macro{spellingreadgood}. Argument to both macros is a file name. If a |
| 216 |
file cannot be found, a warning is written to the console and |log| file |
| 217 |
and compilation continues. As an example, the command |
| 218 |
|
| 219 |
\begin{lstlisting}[language={[LaTeX]TeX}] |
| 220 |
\spellingreadgood{myproject.whitelist} |
| 221 |
\end{lstlisting} |
| 222 |
% |
| 223 |
reads words from a file |myproject.whitelist| and adds them to the list |
| 224 |
of good spellings. |
| 225 |
|
| 226 |
Known good spellings can be used to deal with words wrongly reported as |
| 227 |
bad spellings by the spell-checker (false positives). But note, most |
| 228 |
spell-checkers also provide means to deal with unknown words via |
| 229 |
additional dictionaries. It is recommended to configure your |
| 230 |
spell-checker to report as few false positives as possible. |
| 231 |
|
| 232 |
|
| 233 |
\subsection{Match rules} |
| 234 |
\label{sec:matchrules} |
| 235 |
|
| 236 |
\emph{This section describes an advanced feature. You may safely skip |
| 237 |
this section upon first reading.} |
| 238 |
|
| 239 |
The \pkg\ package provides an additional way to deal with bad and good |
| 240 |
spellings, match rules. Match rules can be used to employ regular |
| 241 |
patterns within certain ‘words’. A typical example are bibliographic |
| 242 |
references like \emph{Lin86}, which are often flagged by spell-checkers, |
| 243 |
but need not be highlighted as they are generated by \TeX. |
| 244 |
|
| 245 |
There are two kinds of rules, bad and good rules. A rule is a Lua |
| 246 |
function whose boolean return value indicates whether a word |
| 247 |
\emph{matches} the rule. A bad rule should return a true value for all |
| 248 |
strings identified as bad spellings, otherwise a false value. A good |
| 249 |
rule should return a true value for all strings identified as good |
| 250 |
spellings, otherwise a false value. A word in the document is |
| 251 |
highlighted if it matches any bad rule, but no good rule. |
| 252 |
|
| 253 |
Function arguments are a \emph{raw} string and a \emph{stripped} string. |
| 254 |
The raw string is a string representing a word as it is found in the |
| 255 |
document possibly surrounded by punctuation characters. The stripped |
| 256 |
string is the same string with surrounding punctuation already stripped. |
| 257 |
|
| 258 |
As an example, the rule in \autoref{lst:mr-three-letter-words} matches |
| 259 |
all words consisting of exactly three letters. The function matches the |
| 260 |
stripped string against the Lua string pattern |^%a%a%a$| via function |
| 261 |
|unicode.utf8.find| from the Selene Unicode library. The latter |
| 262 |
function is a \acr{utf-8} capable version of Lua's built-in function |
| 263 |
|string.find|. It returns |nil| (a false value) if there has been no |
| 264 |
match and a number (a true value) if there has been a match. The |
| 265 |
pattern |%a| represents a character class matching a single letter. |
| 266 |
Characters |^| and |$| are anchors for the beginning and the end of the |
| 267 |
string in question. Note, pattern |%a%a%a| without anchors would match |
| 268 |
any string containing three letters in a row. More information about |
| 269 |
Lua string patterns can be found in the Lua reference manual% |
| 270 |
\footnote{\url{http://www.lua.org/manual/5.2/manual.html\#6.4}}% |
| 271 |
% |
| 272 |
, the Selene Unicode library documentation% |
| 273 |
\footnote{\url{https://github.com/LuaDist/slnunicode/blob/master/unitest}} |
| 274 |
% |
| 275 |
and in the Unicode standard% |
| 276 |
\footnote{\url{http://www.unicode.org/Public/4.0-Update1/UCD-4.0.1.html\#General_Category_Values}}% |
| 277 |
. |
| 278 |
|
| 279 |
\suppressfloats[b] |
| 280 |
|
| 281 |
\begin{lstlisting}[style=Lua, float, label=lst:mr-three-letter-words, caption={Matching three-letter words.}] |
| 282 |
function three_letter_words(raw, stripped) |
| 283 |
return unicode.utf8.find(stripped, '^%a%a%a$') |
| 284 |
end |
| 285 |
\end{lstlisting} |
| 286 |
|
| 287 |
\autoref{lst:mr-double-punctuation} shows a rule matching all ‘words’ |
| 288 |
containing double punctuation. Note, how the raw string is examined |
| 289 |
instead of the stripped one. |
| 290 |
|
| 291 |
\begin{lstlisting}[style=Lua, float, label=lst:mr-double-punctuation, caption={Matching double punctuation.}] |
| 292 |
function double_punctuation(raw, stripped) |
| 293 |
return unicode.utf8.find(raw, '%p%p') |
| 294 |
end |
| 295 |
\end{lstlisting} |
| 296 |
|
| 297 |
The rule in \autoref{lst:mr-bibtex-alpha} combines the results of three |
| 298 |
string searches to match bibliographic references as generated by the |
| 299 |
Bib\TeX\ style \emph{alpha}. |
| 300 |
|
| 301 |
\begin{lstlisting}[style=Lua, float, label=lst:mr-bibtex-alpha, caption={Matching references generated by the Bib\TeX\ style \emph{alpha}.}] |
| 302 |
function bibtex_alpha(raw, stripped) |
| 303 |
return unicode.utf8.find(stripped, '^%u%l%l?%d%d$') |
| 304 |
or unicode.utf8.find(stripped, '^%u%u%u?%u?%d%d$') |
| 305 |
or unicode.utf8.find(stripped, '^%u%u%u%+%d%d$') |
| 306 |
end |
| 307 |
\end{lstlisting} |
| 308 |
|
| 309 |
Match rules have to be provided by means of a Lua module. Such modules |
| 310 |
can be loaded with the \macro{spellingmatchrules} command. Argument is |
| 311 |
a module name. To tell bad rules from good rules, the table returned by |
| 312 |
the module must follow this convention: Function identifiers |
| 313 |
representing bad and good match rules are prefixed |bad_rule_| and |
| 314 |
|good_rule_|, resp. The rest of an identifier is actually irrelevant. |
| 315 |
Other and non-function identifiers are ignored. |
| 316 |
|
| 317 |
\autoref{lst:mr-module} shows an example module declaring the rules from |
| 318 |
\autoref{lst:mr-three-letter-words} and |
| 319 |
\autoref{lst:mr-double-punctuation} as \emph{bad} match rules and the |
| 320 |
rule from \autoref{lst:mr-bibtex-alpha} as a \emph{good} match rule. |
| 321 |
Note, how function identifiers are made local and how exported function |
| 322 |
identifiers are prefixed |bad_rule_| and |good_rule_|, while local |
| 323 |
function identifiers have no prefixes. When the module resides in a |
| 324 |
file named |myproject.rules.lua|, it can be loaded in the preamble of a |
| 325 |
document via |
| 326 |
\begin{lstlisting}[language={[LaTeX]TeX}] |
| 327 |
\spellingmatchrules{myproject.rules} |
| 328 |
\end{lstlisting} |
| 329 |
|
| 330 |
\begin{lstlisting}[style=Lua, float=p, label=lst:mr-module, caption={A Lua module containing two bad and one good match rule.}] |
| 331 |
-- Module table. |
| 332 |
local M = {} |
| 333 |
|
| 334 |
-- Import Selene Unicode library. |
| 335 |
local unicode = require('unicode') |
| 336 |
-- Add short-cut. |
| 337 |
local Ufind = unicode.utf8.find |
| 338 |
|
| 339 |
-- Local function matching three letter words. |
| 340 |
local function three_letter_words(raw, stripped) |
| 341 |
return Ufind(stripped, '^%a%a%a$') |
| 342 |
end |
| 343 |
-- Make this a bad rule. |
| 344 |
M.bad_rule_three_letter_words = three_letter_words |
| 345 |
|
| 346 |
local function double_punctuation(raw, stripped) |
| 347 |
return Ufind(raw, '%p%p') |
| 348 |
end |
| 349 |
M.bad_rule_double_punctuation = double_punctuation |
| 350 |
|
| 351 |
local function bibtex_alpha(raw, stripped) |
| 352 |
return Ufind(stripped, '^%u%l%l?%d%d$') |
| 353 |
or Ufind(stripped, '^%u%u%u?%u?%d%d$') |
| 354 |
or Ufind(stripped, '^%u%u%u%+%d%d$') |
| 355 |
end |
| 356 |
M.good_rule_bibtex_alpha = bibtex_alpha |
| 357 |
|
| 358 |
-- Export module table. |
| 359 |
return M |
| 360 |
\end{lstlisting} |
| 361 |
|
| 362 |
How are match rules and lists of bad and good spellings related? |
| 363 |
Internally, the lists of bad and good spellings are referred to by two |
| 364 |
special default match rules, that look-up raw and stripped strings and |
| 365 |
return a true value if either argument has been found in the |
| 366 |
corresponding list. Since good rules take precedence over bad rules, an |
| 367 |
entry in the list of good spellings takes precedence over any |
| 368 |
user-supplied bad rule. Likewise, any user-supplied good rule takes |
| 369 |
precedence over an entry in the list of bad spellings. |
| 370 |
|
| 371 |
\paragraph{Some final remarks on match rules} It must be stressed that |
| 372 |
the boolean return value of a match rule \emph{does not} indicate |
| 373 |
whether a spelling is bad or good, but whether a word matches a certain |
| 374 |
rule or not. Whether it's a bad or a good spelling, depends on the name |
| 375 |
of the match rule in the module table. |
| 376 |
|
| 377 |
Match rules are only called upon the first occurrence of a spelling in a |
| 378 |
document. The information, whether a spelling needs to be highlighted, |
| 379 |
is stored in a cache table. Subsequent occurrences of a spelling just |
| 380 |
need a table look-up to determine highlighting status. For that reason, |
| 381 |
it is safe to do relatively expensive operations within a match rule |
| 382 |
without affecting compilation time too much. Nevertheless, match rules |
| 383 |
should be stated as efficient as possible.% |
| 384 |
\footnote{Some Lua performance tips can be found in the book \emph{Lua |
| 385 |
Programming Gems} by Figueiredo, Celes and Ierusalimschy |
| 386 |
\emph{(eds.)}, 2008, ch.~2. That chapter is also available online at |
| 387 |
\url{http://www.lua.org/gems/}.} |
| 388 |
|
| 389 |
When written without care, match rules can easily produce false |
| 390 |
positives as well as false negatives. While false positives in bad |
| 391 |
rules and false negatives in good rules can easily be spotted due to the |
| 392 |
unexpected highlighting of words, the other cases are more problematic. |
| 393 |
To avoid all kinds of false results, match rules should be stated as |
| 394 |
specific as possible. |
| 395 |
|
| 396 |
|
| 397 |
\subsection{Highlighting spellling mistakes} |
| 398 |
\label{sec:highlight} |
| 399 |
|
| 400 |
\paragraph{Enabling/disabling} Highlighting spelling mistakes (words |
| 401 |
with known incorrect spelling) in \acr{pdf} output can be toggled on and |
| 402 |
off with command \macro{spellinghighlight}. If the argument is |on|, |
| 403 |
highlighting is enabled. For other arguments, highlighting is disabled. |
| 404 |
Highlighting is enabled, by default. |
| 405 |
|
| 406 |
\paragraph{Colour} The colour used for highlighting bad spellings can be |
| 407 |
determined by command \cmd{spellinghighlightcolor}. Argument is a |
| 408 |
colour statement in the \acr{pdf} language. As an example, the colour |
| 409 |
red in the \acr{rgb} colour space is represented by the statement % |
| 410 |
|1 0 0 rg|. In the \acr{cmyk} colour space, a reddish colour is |
| 411 |
represented by |0 1 1 0 k|. Default colour used for highlighting is % |
| 412 |
|1 0 0 rg|, \lpie, red in the \acr{rgb} colour space. |
| 413 |
|
| 414 |
|
| 415 |
\subsection{Text output} |
| 416 |
\label{sec:textoutput} |
| 417 |
|
| 418 |
\paragraph{Text file} After loading the \pkg\ package, at the end of the |
| 419 |
Lua\TeX\ run, a text file is written that contains most of the document |
| 420 |
text. The text file is no close text rendering of the typeset document, |
| 421 |
but serves as input for your favourite spell-checker application. It |
| 422 |
contains the document text in a simple format: paragraphs separated by |
| 423 |
blank lines. A paragraph is anything that, during typesetting, starts |
| 424 |
with a |local_par| whatsit node in the node list representing a typeset |
| 425 |
page of the original document, \lpeg, paragraphs in running text, |
| 426 |
footnotes, marginal notes, (in-lined) °\parbox° commands or cells from |
| 427 |
°p°-like table columns \lpetc |
| 428 |
|
| 429 |
Paragraphs consist of words separated by spaces. A word is the textual |
| 430 |
representation of a chain of consecutive nodes of type |glyph|, |disc| |
| 431 |
or |kern|. Boxes are processed transparently. That is, the \pkg\ |
| 432 |
package (highly imperfectly) tries to recognise as a single word what in |
| 433 |
typeset output looks like a single word. As an example, the \LaTeX\ |
| 434 |
code |
| 435 |
|
| 436 |
\begin{center} |
| 437 |
\begin{tabular}{c} |
| 438 |
\begin{lstlisting}[language={[LaTeX]TeX}] |
| 439 |
foo\mbox{'s bar}s |
| 440 |
\end{lstlisting} |
| 441 |
\end{tabular} |
| 442 |
\end{center} |
| 443 |
which is typeset as |
| 444 |
|
| 445 |
\begin{center} |
| 446 |
foo\mbox{'s bar}s |
| 447 |
\end{center} |
| 448 |
is considered two words \textit{foo's} and \textit{bars}, instead of the |
| 449 |
four words \textit{foo}, \textit{'s}, \textit{bar} and~\textit{s}.% |
| 450 |
\footnote{This document has been compiled with a custom list of bad |
| 451 |
spellings, which is why the word \emph{foo's} should be highlighted.} |
| 452 |
|
| 453 |
\paragraph{Enabling/disabling} Text output can be toggled on and off |
| 454 |
with command \macro{spellingoutput}. If the argument is |on|, text |
| 455 |
output is enabled. For other arguments, text output is disabled. Text |
| 456 |
output is enabled, by default. |
| 457 |
|
| 458 |
\paragraph{File name} \hspace{0pt plus 5em} Text output file name can be |
| 459 |
configured via command \macro{spellingoutputname}. Argument is the new |
| 460 |
file name. Default text output file name is |
| 461 |
\descr{jobname}|.spell.txt|. |
| 462 |
|
| 463 |
\paragraph{Line length} In text output, paragraphs can either be put on |
| 464 |
a single line or broken into lines of a fixed length. The behaviour can |
| 465 |
be controlled via command \macro{spellingoutputlinelength}. Argument is |
| 466 |
a number. If the number is less than~1, paragraphs are put on a single |
| 467 |
line. For larger arguments, the number specifies maximum line length. |
| 468 |
Note, lines are broken at spaces only. Words longer than maximum line |
| 469 |
length are put on a single line exceeding maximum line length. Default |
| 470 |
line length is~72. |
| 471 |
|
| 472 |
|
| 473 |
\subsection{Text extraction} |
| 474 |
\label{sec:textextraction} |
| 475 |
|
| 476 |
\paragraph{Enabling/disabling} Text extraction can be enabled and |
| 477 |
disabled in the document via command \macro{spellingextract}. If the |
| 478 |
argument is |on|, text extraction is enabled. For other arguments, text |
| 479 |
extraction is disabled. The command should be used in vertical mode, |
| 480 |
\lpie, outside paragraphs. If text extraction is disabled in the |
| 481 |
document preamble, an empty text file is written at the end of the |
| 482 |
Lua\TeX\ run. Text extraction is enabled, by default. |
| 483 |
|
| 484 |
Note, text extraction and visual feedback are orthogonal features. That |
| 485 |
is, if text extraction is disabled for part of a document, \lpeg, a long |
| 486 |
table, words with a known incorrect spelling are still highlighted in |
| 487 |
that part. |
| 488 |
|
| 489 |
|
| 490 |
\subsection{Code point mapping} |
| 491 |
\label{sec:cp-mapping} |
| 492 |
|
| 493 |
As explained in \autoref{sec:textoutput}, the text file written at the |
| 494 |
end of the Lua\TeX\ run is in the \acr{utf-8} encoding. Unicode |
| 495 |
contains a wealth of code points with a special meaning, such as |
| 496 |
ligatures, alternative letters, symbols \lpetc Unfortunately, not all |
| 497 |
spell-checker applications are smart enough to correctly interpret all |
| 498 |
Unicode code points that may occur in a document. For that reason, a |
| 499 |
code point mapping feature has been implemented that allows for mapping |
| 500 |
certain Unicode code points that may appear in a node list to arbitrary |
| 501 |
strings in text output. A typical example is to map ligatures to the |
| 502 |
characters corresponding to their constituting letters. The default |
| 503 |
mappings applied can be found in \autoref{tab:cp-mapping}. |
| 504 |
|
| 505 |
\begin{table} |
| 506 |
\begin{minipage}{1.0\linewidth} |
| 507 |
\centering |
| 508 |
|
| 509 |
\newcommand*{\coltitle}[2]{% |
| 510 |
\normalfont% |
| 511 |
\vbox{ |
| 512 |
\hbox{\strut#1} |
| 513 |
\hbox{\strut#2} |
| 514 |
}% |
| 515 |
} |
| 516 |
|
| 517 |
\begin{tabular}{>{\ttfamily}l>{\fontspec{Linux Libertine |
| 518 |
O}}l>{\ttfamily}l>{\ttfamily}l} |
| 519 |
\normalfont Unicode name & \coltitle{sample}{glyph\footnote{Sample |
| 520 |
glyphs are taken from font \emph{Linux Libertine~O}.}} & |
| 521 |
\coltitle{code}{point} & \coltitle{target}{characters}\\ |
| 522 |
\addlinespace |
| 523 |
\toprule |
| 524 |
\addlinespace |
| 525 |
|
| 526 |
LATIN CAPITAL LIGATURE IJ & ^^^^0132 & 0x0132 & IJ \\ |
| 527 |
LATIN SMALL LIGATURE IJ & ^^^^0133 & 0x0133 & ij \\ |
| 528 |
LATIN CAPITAL LIGATURE OE & ^^^^0152 & 0x0152 & OE \\ |
| 529 |
LATIN SMALL LIGATURE OE & ^^^^0153 & 0x0153 & oe \\ |
| 530 |
LATIN SMALL LETTER LONG S & ^^^^017f & 0x017f & s \\ |
| 531 |
\addlinespace |
| 532 |
LATIN SMALL LIGATURE FF & ^^^^fb00 & 0xfb00 & ff \\ |
| 533 |
LATIN SMALL LIGATURE FI & ^^^^fb01 & 0xfb01 & fi \\ |
| 534 |
LATIN SMALL LIGATURE FL & ^^^^fb02 & 0xfb02 & fl \\ |
| 535 |
LATIN SMALL LIGATURE FFI & ^^^^fb03 & 0xfb03 & ffi \\ |
| 536 |
LATIN SMALL LIGATURE FFL & ^^^^fb04 & 0xfb04 & ffl \\ |
| 537 |
LATIN SMALL LIGATURE LONG S T & ^^^^fb05 & 0xfb05 & st \\ |
| 538 |
LATIN SMALL LIGATURE ST & ^^^^fb06 & 0xfb06 & st \\ |
| 539 |
\end{tabular} |
| 540 |
|
| 541 |
\caption{Default code point mappings.} |
| 542 |
\label{tab:cp-mapping} |
| 543 |
|
| 544 |
\end{minipage} |
| 545 |
\end{table} |
| 546 |
|
| 547 |
Additional mappings can be declared by command \macro{spellingmapping}. |
| 548 |
This command takes two arguments, a number that refers to the Unicode |
| 549 |
code point, and a sequence of arbitrary characters that is the mapping |
| 550 |
target. The code point number must be in a format that can be parsed by |
| 551 |
Lua. The characters must be in the \acr{utf-8} encoding. |
| 552 |
|
| 553 |
New mappings only have effect on the following document text. The |
| 554 |
command should therefore be used in the document preamble. As an |
| 555 |
example, the character |A| can be mapped to |Z| and \latinphrase{vice |
| 556 |
versa} with the following code: |
| 557 |
|
| 558 |
\begin{lstlisting}[language={[LaTeX]TeX}] |
| 559 |
\spellingmapping{65}{Z}% A => Z |
| 560 |
\spellingmapping{90}{A}% Z => A |
| 561 |
\end{lstlisting} |
| 562 |
|
| 563 |
Another command \macro{spellingclearallmappings} can be used to remove |
| 564 |
all existing code point mappings. |
| 565 |
|
| 566 |
|
| 567 |
\subsection{Tables} |
| 568 |
\label{sec:tables} |
| 569 |
|
| 570 |
How do tables fit into the simple text file format that has only |
| 571 |
paragraphs and blank lines as described in \autoref{sec:textoutput}? |
| 572 |
What is a paragraph with regards to tables? A whole table? A row? A |
| 573 |
single cell? |
| 574 |
|
| 575 |
By default, only text from cells in °p°(aragraph)-like columns is put on |
| 576 |
their own paragraph, because the corresponding node list branches |
| 577 |
contain a |local_par| whatsit node (\lpcf \autoref{sec:textoutput}). |
| 578 |
The behaviour can be changed with the \macro{spellingtablepar} command. |
| 579 |
This command takes as argument a number. If the argument is~0, the |
| 580 |
behaviour is described as above. If the argument is~1, a blank line is |
| 581 |
inserted before and after every table row (but at most once between |
| 582 |
table rows). If the argument is~2, a blank line is inserted before and |
| 583 |
after every table cell. By default, no blank lines are inserted. |
| 584 |
|
| 585 |
|
| 586 |
\section{LanguageTool support} |
| 587 |
\label{sec:languagetool} |
| 588 |
|
| 589 |
Installing spell-checkers and dictionaries can be a difficult task if |
| 590 |
there are no pre-built packages available for an architecture. That's |
| 591 |
one reason the \pkg\ package is rather spell-checker agnostic and the |
| 592 |
manual doesn't recommend a particular spell-checker application. |
| 593 |
Another reason is, there is no best spell-checker. The only |
| 594 |
recommendation the author makes is not to trust in one spell-checker, |
| 595 |
but to use multiple spell-checkers at the same time, with different |
| 596 |
dictionaries or, better yet, different checking engines under the hood. |
| 597 |
|
| 598 |
Among the set of options available, LanguageTool% |
| 599 |
\footnote{\url{http://www.languagetool.org/}}% |
| 600 |
% |
| 601 |
, a style and grammar checker that can also check spelling since |
| 602 |
version~1.8, deserves some notice for its portability, ease of |
| 603 |
installation and active development. For these reasons, the \pkg\ |
| 604 |
package provides explicit LanguageTool support. LanguageTool uses |
| 605 |
Hunspell as the spell-checking engine, augmenting results with a rule |
| 606 |
based engine and a morphological analyser (depending on the language). |
| 607 |
The \pkg\ package can parse LanguageTool's error reports in the |
| 608 |
\acr{xml} format, pick those errors that are spelling related and use |
| 609 |
them to highlight bad spellings.% |
| 610 |
\footnote{Highlighting style and grammar errors found by LanguageTool |
| 611 |
should be possible, but requires major restructuring of the \pkg\ |
| 612 |
package.} |
| 613 |
|
| 614 |
|
| 615 |
\subsection{Installation} |
| 616 |
\label{sec:lt-installation} |
| 617 |
|
| 618 |
Here are some brief installation instructions for the stand-alone |
| 619 |
version of LanguageTool (tested with LanguageTool~2.1). The stand-alone |
| 620 |
version contains a \acr{gui} as well as a command-line interface. For |
| 621 |
the \pkg\ package, the latter is needed. |
| 622 |
|
| 623 |
\begin{enumerate} |
| 624 |
|
| 625 |
\item LanguageTool is primarily written in Java. Make sure a recent |
| 626 |
Java Runtime Environment (\acr{jre}) is installed. |
| 627 |
|
| 628 |
\item\label{enum:run-java} Open a command-line and type |
| 629 |
|
| 630 |
\begin{lstlisting} |
| 631 |
java -version |
| 632 |
\end{lstlisting} |
| 633 |
% |
| 634 |
If you get an error message, find out the full path to the Java |
| 635 |
executable (called |java.exe| on Windows) for later reference. |
| 636 |
|
| 637 |
\item Download the stand-alone version of LanguageTool (should be a |
| 638 |
\acr{zip} archive). |
| 639 |
|
| 640 |
\item Uncompress the downloaded archive to a location of your choice. |
| 641 |
|
| 642 |
\item Open a command-line in the directory containing file |
| 643 |
|languagetool-commandline.jar| and type |
| 644 |
|
| 645 |
\begin{lstlisting}[escapeinside=°°] |
| 646 |
°\descr{path to}°/java -jar languagetool-commandline.jar --help |
| 647 |
\end{lstlisting} |
| 648 |
% |
| 649 |
Prepending the path to the Java executable is optional, depending on |
| 650 |
the result in step~\ref{enum:run-java}. If you now see a list of |
| 651 |
LanguageTool's command-line options rush by, all is well. |
| 652 |
|
| 653 |
\item For easier access to LanguageTool, create a small batch script and |
| 654 |
put that somewhere into the |PATH|. |
| 655 |
|
| 656 |
\begin{itemize} |
| 657 |
|
| 658 |
\item For users of unixoide systems, the script might look like |
| 659 |
|
| 660 |
\begin{lstlisting}[escapeinside=°°] |
| 661 |
#!/bin/sh |
| 662 |
°\descr{path to}°/java -jar °\descr{path to}°/languagetool-commandline.jar $* |
| 663 |
\end{lstlisting} |
| 664 |
% |
| 665 |
where \texttt{\descr{path to}} should point to the Java executable |
| 666 |
(optional) and file |languagetool-commandline.jar| (mandatory). If |
| 667 |
the script is named |lt.sh|, you should be able to run LanguageTool |
| 668 |
on the command shell by typing, \lpeg, |
| 669 |
|
| 670 |
\begin{lstlisting} |
| 671 |
sh lt.sh --version |
| 672 |
\end{lstlisting} |
| 673 |
% |
| 674 |
Don't forget to put the script into the |PATH|! For other ways of |
| 675 |
making scripts executable, please consult the operating system |
| 676 |
documentation. |
| 677 |
|
| 678 |
\item For Windows users, the script might look like |
| 679 |
|
| 680 |
\begin{lstlisting}[escapeinside=°°] |
| 681 |
@echo off |
| 682 |
°\descr{path to}°\java -jar °\descr{path to}°\languagetool-commandline.jar %* |
| 683 |
\end{lstlisting} |
| 684 |
% |
| 685 |
where \texttt{\descr{path to}} should point to the Java executable |
| 686 |
(optional) and file |languagetool-commandline.jar| (mandatory). If |
| 687 |
the script is named |lt.bat|, you should be able to run LanguageTool |
| 688 |
on the command-line by typing, \lpeg, |
| 689 |
|
| 690 |
\begin{lstlisting} |
| 691 |
lt --version |
| 692 |
\end{lstlisting} |
| 693 |
% |
| 694 |
Don't forget to put the script into the |PATH|! |
| 695 |
|
| 696 |
\end{itemize} |
| 697 |
|
| 698 |
\end{enumerate} |
| 699 |
|
| 700 |
|
| 701 |
\subsection{Usage} |
| 702 |
\label{sec:lt-usage} |
| 703 |
|
| 704 |
The results of checking a text file with LanguageTool are written to an |
| 705 |
error report, either in a human readable format or in a machine friendly |
| 706 |
\acr{xml} format. The \pkg\ package can only parse the latter format. |
| 707 |
When it was said in \autoref{sec:wordlists} that the \pkg\ package reads |
| 708 |
files \descr{jobname}|.spell.bad| and \descr{jobname}|.spell.good|, if |
| 709 |
they exist, that was not the whole truth. Additionally, a file |
| 710 |
\descr{jobname}|.spell.xml| is read, if it exists. This file should |
| 711 |
contain a LanguageTool error report in the \acr{xml} format. Additional |
| 712 |
LanguageTool \acr{xml} error reports can be loaded via the |
| 713 |
\macro{spellingreadLT} command. Argument is a file name. Macros |
| 714 |
|\spellingreadLT|, |\spellingreadbad| and |\spellingreadgood| can be |
| 715 |
used in combination in a \TeX\ file. |
| 716 |
|
| 717 |
To check a text file and create an error report in the \acr{xml} format, |
| 718 |
LanguageTool can be called on the command-line like this |
| 719 |
\begin{lstlisting}[escapeinside=°°] |
| 720 |
lt °\descr{options}° °\descr{input file}° > °\descr{error report}° |
| 721 |
\end{lstlisting} |
| 722 |
where \texttt{\descr{options}} is a list of options described below, |
| 723 |
\texttt{\descr{input file}} is the text file written by the \pkg\ |
| 724 |
package in the first Lua\TeX\ run and \texttt{\descr{error report}} is |
| 725 |
the file containing the error report. Note, how standard output is |
| 726 |
redirected to a file via the |>| operator. By default, LanguageTool |
| 727 |
writes error reports to standard output, that is, the command-line. |
| 728 |
Redirection is a feature most operating systems provide. |
| 729 |
|
| 730 |
\begin{itemize} |
| 731 |
|
| 732 |
\item Option |-l| determines the language (variant) of the file to |
| 733 |
check. As an example, language variant US English can be selected via |
| 734 |
|-l en-US|. The full list of languages supported by LanguageTool can |
| 735 |
be requested via option |--list|. |
| 736 |
|
| 737 |
\item Option |-c| determines the encoding of the input file. Since the |
| 738 |
text file written by the \pkg\ package is in the \acr{utf-8} encoding, |
| 739 |
this part should be |-c utf-8|. |
| 740 |
|
| 741 |
\item By default, LanguageTool outputs error reports in a human readable |
| 742 |
format. The \pkg\ package can only parse error reports in the |
| 743 |
\acr{xml} format. If the |--api| option is present, LanguageTool |
| 744 |
outputs \acr{xml} data. |
| 745 |
|
| 746 |
\item Users that don't want to highlight bad spellings, but prefer to |
| 747 |
study the list of bad spellings themselves, should refer to the |-u| |
| 748 |
option. But note, that with the latter option present, LanguageTool |
| 749 |
doesn't output pure \acr{xml} any more, even if the |--api| option is |
| 750 |
present. Make sure such error reports aren't read by the \pkg\ |
| 751 |
package. |
| 752 |
|
| 753 |
\item If the |--help| option is present, LanguageTool shows more |
| 754 |
information about command-line options. |
| 755 |
|
| 756 |
\end{itemize} |
| 757 |
|
| 758 |
As an example, to compile a \LaTeX\ file |myletter.tex| written in |
| 759 |
French that uses the \pkg\ package with standard settings to highlight |
| 760 |
bad spellings and to use LanguageTool as a spell-checker, the following |
| 761 |
commands should be typed on the command-line: |
| 762 |
|
| 763 |
\begin{lstlisting} |
| 764 |
lualatex myletter |
| 765 |
lt --api -c utf-8 -l fr myletter.spell.txt > myletter.spell.xml |
| 766 |
lualatex myletter |
| 767 |
\end{lstlisting} |
| 768 |
|
| 769 |
|
| 770 |
\section{Bugs} |
| 771 |
\label{sec:bugs} |
| 772 |
|
| 773 |
Note, this package is in a very early state. Expect bugs! Package |
| 774 |
development is hosted at |
| 775 |
\href{http://github.com/sh2d/spelling/}{\bfseries GitHub}. The full |
| 776 |
list of known bugs and feature requests can be found in the |
| 777 |
\href{http://github.com/sh2d/spelling/issues/}{\bfseries issue tracker}. |
| 778 |
New bugs should be reported there. |
| 779 |
|
| 780 |
The most user-visible issues are listed below: |
| 781 |
|
| 782 |
\begin{itemize} |
| 783 |
|
| 784 |
\item There's no support for the Plain \TeX\ or Con\TeX\ formats other |
| 785 |
than the \acr{API} of the package's Lua modules, yet |
| 786 |
(\href{https://github.com/sh2d/spelling/issues/1}{issue~1}). |
| 787 |
|
| 788 |
\item Macros provided by the \LaTeX\ package have very long names. A |
| 789 |
\emph{key-value} package option interface would be much more |
| 790 |
user-friendly |
| 791 |
(\href{https://github.com/sh2d/spelling/issues/2}{issue~2}). |
| 792 |
|
| 793 |
\item There are a couple of issues with text extraction and highlighting |
| 794 |
incorrect spellings: |
| 795 |
|
| 796 |
\begin{itemize} |
| 797 |
|
| 798 |
\item Text in head and foot lines is neither extracted nor highlighted |
| 799 |
(\href{https://github.com/sh2d/spelling/issues/7}{issue~7}). |
| 800 |
|
| 801 |
\item The first word starting right after an |hlist|, \lpeg, the first |
| 802 |
word within an |\mbox|, is never highlighted. It is extracted and |
| 803 |
written to the text file, though. This might affect acronyms, names |
| 804 |
\lpetc (\href{https://github.com/sh2d/spelling/issues/6}{issue~6}). |
| 805 |
|
| 806 |
\item Bad spellings that are hyphenated at a page break are not |
| 807 |
highlighted |
| 808 |
(\href{https://github.com/sh2d/spelling/issues/10}{issue~10}). |
| 809 |
|
| 810 |
\end{itemize} |
| 811 |
|
| 812 |
|
| 813 |
\end{itemize} |
| 814 |
|
| 815 |
Patches welcome! |
| 816 |
|
| 817 |
\bigskip |
| 818 |
\emph{Happy \TeX ing!} |
| 819 |
|
| 820 |
|
| 821 |
\end{document} |
| 822 |
|
| 823 |
|
| 824 |
|
| 825 |
%%% Local Variables: |
| 826 |
%%% mode: latex |
| 827 |
%%% TeX-PDF-mode: t |
| 828 |
%%% TeX-master: t |
| 829 |
%%% coding: utf-8 |
| 830 |
%%% End: |