| 1 |
% Copyright 2019 by Till Tantau |
| 2 |
% Copyright 2019 by Jonathan P. Spratte |
| 3 |
% |
| 4 |
% This file may be distributed and/or modified |
| 5 |
% |
| 6 |
% 1. under the LaTeX Project Public License and/or |
| 7 |
% 2. under the GNU Free Documentation License. |
| 8 |
% |
| 9 |
% See the file doc/generic/pgf/licenses/LICENSE for more details. |
| 10 |
|
| 11 |
|
| 12 |
\section{Parser Module}% |
| 13 |
\label{section-module-parser}% |
| 14 |
|
| 15 |
\begin{pgfmodule}{parser}% |
| 16 |
This module defines some commands for creating a simple letter-by-letter |
| 17 |
parser. |
| 18 |
\end{pgfmodule} |
| 19 |
|
| 20 |
\begin{package}{pgfparser} |
| 21 |
Because the parser module is almost independent of the rest of \pgfname, it |
| 22 |
can also be used as a standalone package with minimal dependencies. |
| 23 |
\end{package} |
| 24 |
|
| 25 |
This module provides commands for defining a parser that scans some given text |
| 26 |
letter-by-letter. For each letter, some code is executed and, possibly a |
| 27 |
state-switch occurs. The code for each letter might take mandatory or optional |
| 28 |
arguments. The parsing process ends when a final state has been reached, and |
| 29 |
optionally some code is executed afterwards. Each newly defined parser by |
| 30 |
default ignores space tokens, if you want to change that you'll have to |
| 31 |
explicitly define an action for blank spaces (with |\pgfparserdef|). |
| 32 |
|
| 33 |
\begin{command}{\pgfparserparse\marg{parser name}\meta{text}}% |
| 34 |
This command is used to parse the \meta{text} using the (previously defined) |
| 35 |
parser named \meta{parser name}. |
| 36 |
|
| 37 |
The \meta{text} is not contained in curly braces, rather it is all the text |
| 38 |
that follows. The end of the text is determined implicitly, namely when the |
| 39 |
final state of the parser has been reached. If you defined a final action for |
| 40 |
the parser using |\pgfparserdeffinal| it is executed now. |
| 41 |
|
| 42 |
The parser works as follows: At any moment, it is in a certain |
| 43 |
\emph{state}, initially this state is called |initial|. Then, the first |
| 44 |
letter of the \meta{text} is examined (using the |\futurelet| command). For |
| 45 |
each possible state and each possible letter, some action code is stored in |
| 46 |
the parser in a table. This code is then executed. This code may, but need |
| 47 |
not, trigger a \emph{state switch}, causing a new state to be set. The |
| 48 |
parser then moves on to the next character of the text and repeats the |
| 49 |
whole procedure, unless it is in the state |final|, which causes the |
| 50 |
parsing process to stop immediately. |
| 51 |
|
| 52 |
In the following example, the parser counts the number of |a|'s in the |
| 53 |
\text{text}, ignoring any |b|'s. The \meta{text} ends with the first~|c|. |
| 54 |
% |
| 55 |
\begin{codeexample}[preamble={\usepgfmodule{parser}}] |
| 56 |
\newcount\mycount |
| 57 |
\pgfparserdef{myparser}{initial}{the letter a}% |
| 58 |
{\advance\mycount by 1\relax}% |
| 59 |
\pgfparserdef{myparser}{initial}{the letter b}% |
| 60 |
{} % do nothing |
| 61 |
\pgfparserdef{myparser}{initial}{the letter c}% |
| 62 |
{\pgfparserswitch{final}}% done! |
| 63 |
|
| 64 |
\pgfparserparse{myparser}aabaabababbbbbabaabcccc |
| 65 |
There are \the\mycount\ a's. |
| 66 |
\end{codeexample}% |
| 67 |
% |
| 68 |
\end{command}% |
| 69 |
|
| 70 |
\begin{command}% |
| 71 |
{% |
| 72 |
\pgfparserdef\marg{parser name}\marg{state}\meta{symbol meaning}% |
| 73 |
\oarg{arguments}\marg{action}% |
| 74 |
}% |
| 75 |
This command should be used repeatedly to define a parser named |
| 76 |
\meta{parser name}. With a call to this command you specify that the |
| 77 |
\meta{parser name} should do the following: When it is in state |
| 78 |
\meta{state} and reads the letter \meta{symbol meaning}, perform the code |
| 79 |
stored in \meta{action}. |
| 80 |
|
| 81 |
The \meta{symbol meaning} must be the text that results from applying the |
| 82 |
\TeX\ command |\meaning| to the given character. For instance, |\meaning a| |
| 83 |
yields |the letter a|, while |\meaning 1| yields |the character 1|. A space |
| 84 |
yields |blank space |. Alternatively you can give the symbol you want without |
| 85 |
surrounding it in braces. So both |
| 86 |
|\pgfparserdef{myparser}{initial}{the letter a}{foo}| |
| 87 |
and |
| 88 |
|\pgfparserdef{myparser}{initial}a{foo}| |
| 89 |
define an \meta{action} for |the letter a|. This short form works for most |
| 90 |
tokens, but not for a space (in which case you can use |
| 91 |
|\pgfparserdef|\allowbreak|{myparser}|\allowbreak|{initial}|\allowbreak |
| 92 |
|{blank space}|\allowbreak|{foo}|% |
| 93 |
), and opening braces |
| 94 |
(in which case you can use |
| 95 |
|\pgfparserdef|\allowbreak|{myparser}|\allowbreak|{initial}|\allowbreak |
| 96 |
|{\meaning\bgroup}|\allowbreak|{foo}|% |
| 97 |
, and one might prefer |
| 98 |
to use |
| 99 |
|\pgfparserdef|\allowbreak|{myparser}|\allowbreak|{initial}|\allowbreak |
| 100 |
|{\meaning\egroup}|\allowbreak|{foo}| |
| 101 |
for closing braces as well). You can as well define an action for a macro's |
| 102 |
meaning (note that macros with different names can have the same meaning), so |
| 103 |
things like |
| 104 |
|\pgfparserdef|\allowbreak|{myparser}|\allowbreak|{initial}\texttt{foo}| |
| 105 |
are possible as well. |
| 106 |
|
| 107 |
The \meta{action} might require arguments which you can specify in the |
| 108 |
optional \meta{arguments} string. The argument string can contain up to nine |
| 109 |
argument specifications of the following types: |
| 110 |
|
| 111 |
\begingroup |
| 112 |
\def\argdesc#1#2% |
| 113 |
{% |
| 114 |
\par |
| 115 |
\begingroup |
| 116 |
\setbox0\hbox{\texttt #1}\makebox[1.5em][l]{\usebox0}% |
| 117 |
\ifdim\wd0>1.5em\\\null\hspace{1.5em}\fi |
| 118 |
\parbox[t]{\dimexpr\linewidth-1.5em\relax}{#2}% |
| 119 |
\endgroup |
| 120 |
}% |
| 121 |
\argdesc{m}% |
| 122 |
{a normal mandatory argument}% |
| 123 |
\argdesc{r\meta{delim}}% |
| 124 |
{a mandatory argument which is read up to the \meta{delim}}% |
| 125 |
\argdesc{o}% |
| 126 |
{an optional argument in |[]| defaulting to a special mark}% |
| 127 |
\argdesc{O\marg{default}}% |
| 128 |
{like |o| but defaulting to \meta{default}}% |
| 129 |
\argdesc{d\meta{delim1}\meta{delim2}}% |
| 130 |
{% |
| 131 |
an optional argument in \meta{delim1} and \meta{delim2} defaulting to a |
| 132 |
special mark% |
| 133 |
}% |
| 134 |
\argdesc{D\meta{delim1}\meta{delim2}\marg{default}}% |
| 135 |
{like |d| but defaulting to \meta{default}}% |
| 136 |
\argdesc{t\meta{token}}% |
| 137 |
{% |
| 138 |
tests whether the next letter is \meta{token}, if so gobbles it and |
| 139 |
the argument is set to a special mark.% |
| 140 |
}% |
| 141 |
\endgroup |
| 142 |
|
| 143 |
So if you want to define an \meta{action} that takes two mandatory arguments |
| 144 |
you use |[mm]|, if it should take an optional star, one optional argument in |
| 145 |
brackets that returns a marker if it's not used, one mandatory and finally an |
| 146 |
optional argument in parentheses that defaults to |something| you use |
| 147 |
|[t*omD(){something}]| as the argument string. If the argument should be |
| 148 |
anything up to a semicolon, you use |[r;]|. Spaces before argument |
| 149 |
specifications in the string are ignored. So |[r m]| will be one argument and |
| 150 |
read anything up to an |m|. Also spaces before any argument in the parsed |
| 151 |
letters are ignored, so if |a| was setup to take an optional argument the |
| 152 |
argument would be detected in |a []|. Like with normal \LaTeXe\ optional |
| 153 |
arguments you have to protect nested brackets: |[a[bc]d]| would be read as |
| 154 |
|a[bc| with a trailing |d]|, \emph{not} as |a[bc]d|. You'd have to use |
| 155 |
|[{a[bc]d}]| to get it correct. |
| 156 |
|
| 157 |
Inside the \meta{action} you can perform almost any kind of code. This code |
| 158 |
will not be surrounded by a scope, so its effect persists after the parsing |
| 159 |
is done. However, each time after the \meta{action} is executed, control |
| 160 |
goes back to the parser. You should not launch a parser inside the |
| 161 |
\meta{action} code, unless you put it in a scope. |
| 162 |
|
| 163 |
When you use |all| as the \meta{state}, the \meta{action} is performed |
| 164 |
in all states as a fallback, whenever \meta{symbol meaning} is encountered. |
| 165 |
This means that when you do not specify anything explicitly for a state and a |
| 166 |
letter, but you do specify something for |all| and this letter, then the |
| 167 |
specified \meta{action} will be used. |
| 168 |
|
| 169 |
When the parser encounters a letter for which nothing is specified in the |
| 170 |
current state (neither directly nor indirectly via |all|), an error occurs. |
| 171 |
Additionally you can specify an action that is executed after the error is |
| 172 |
thrown using |\pgfparserdefunknown|. To suppress these errors (but not the |
| 173 |
action specified with |\pgfparserdefunknown|) you can use the |
| 174 |
|/pgfparser/silent| key or the |silent| key of the current \meta{parser name}. |
| 175 |
\end{command}% |
| 176 |
|
| 177 |
\begin{command}% |
| 178 |
{% |
| 179 |
\pgfparserlet |
| 180 |
\marg{parser name 1}\marg{state 1}\meta{symbol meaning 1}% |
| 181 |
\oarg{opt 1}\oarg{opt 2}\meta{symbol meaning 2}% |
| 182 |
}% |
| 183 |
If none of the optional arguments are given in the following explanation |
| 184 |
\meta{parser name 2} and \meta{state 2} are the same as |
| 185 |
\meta{parser name 1} and \meta{state 1}. If only the first is given |
| 186 |
\meta{state 2} equals \meta{opt 1}. If both are given \meta{parser name 2} |
| 187 |
equals \meta{opt 1} and \meta{state 2} equals \meta{opt 2}. |
| 188 |
|
| 189 |
Defines an action for \meta{parser name 1} in \meta{state 1} for the |
| 190 |
\meta{symbol meaning 1} to do the same as the action of \meta{parser name 2} |
| 191 |
in \meta{state 2} for the \meta{symbol meaning 2}. For \meta{symbol meaning 1} |
| 192 |
and \meta{symbol meaning 2} the same parsing rules apply as for \meta{symbol |
| 193 |
meaning} in |\pgfparserdef| so you either give the meaning in braces or just |
| 194 |
the symbol. |
| 195 |
\end{command}% |
| 196 |
|
| 197 |
\begin{command}{\pgfparserdefunknown\marg{parser name}\marg{state}\marg{action}}% |
| 198 |
With this macro you can define an \meta{action} for the \meta{parser name} |
| 199 |
parser in \meta{state} if no action was defined for the letter which was |
| 200 |
encountered. |
| 201 |
\end{command}% |
| 202 |
|
| 203 |
\begin{command}{\pgfparserdeffinal\marg{parser name}\marg{action}}% |
| 204 |
Every parser can call a final \meta{action} after the state was switched to |
| 205 |
|final|. This \meta{action} is executed after everything else, so you can use |
| 206 |
something that grabs more arguments if you want to. |
| 207 |
\end{command}% |
| 208 |
|
| 209 |
\begin{command}{\pgfparserswitch\marg{state}}% |
| 210 |
This command can be called inside the action code of a parser to cause a |
| 211 |
state switch to \meta{state}. |
| 212 |
\end{command}% |
| 213 |
|
| 214 |
\begin{command}{\pgfparserifmark\marg{arg}\marg{true}\marg{false}}% |
| 215 |
Remember that some of the optional argument types set special marks? With |
| 216 |
|\pgfparserifmark| you can test whether \meta{arg} is such a mark. So if |
| 217 |
there was no optional argument for the argument types |o| and |d| the |
| 218 |
\meta{true} branch will be executed, else the \meta{false} branch. For the |t| |
| 219 |
type argument the \meta{true} branch is executed if the token was encountered. |
| 220 |
\end{command}% |
| 221 |
|
| 222 |
\begin{command}{\pgfparserreinsert}% |
| 223 |
You can use this as the final macro in an action of |\pgfparserdef| or |
| 224 |
|\pgfparserdefunknown|. This has the effect that the contents of |
| 225 |
|\pgfparserletter| will be parsed next. Without any redefinition the result |
| 226 |
will be that the last token will be parsed again. You can change the |
| 227 |
definition of |\pgfparserletter| just before |\pgfparserreinsert| as well to |
| 228 |
parse some specific tokens next. |
| 229 |
\end{command}% |
| 230 |
|
| 231 |
\begin{command}{\pgfparserstate}% |
| 232 |
Expands to the current state of the parser. |
| 233 |
\end{command}% |
| 234 |
|
| 235 |
\begin{command}{\pgfparsertoken}% |
| 236 |
This is the macro which is let to the following token with |\futurelet|. You |
| 237 |
can use it inside an action code. |
| 238 |
\end{command}% |
| 239 |
|
| 240 |
\begin{command}{\pgfparserletter}% |
| 241 |
This macro stores the letter to which |\pgfparsertoken| was let. So if |
| 242 |
you'd use |\pgfparserparse{foo}a| this macro would be defined with |
| 243 |
|\def\pgfparserletter{a}|. This definition is done before any action code is |
| 244 |
executed. There are four special cases: If the next token is of category code |
| 245 |
1, 2, 6, or 10, so with standard category codes the tokens |{|, |}|, |#|, and |
| 246 |
\textvisiblespace\ (a space), it would be treated differently. In those cases |
| 247 |
this macro expands to |\bgroup|, |\egroup|, |##|, and \textvisiblespace\ for |
| 248 |
the categories 1, 2, 6, and 10, respectively. |
| 249 |
\end{command}% |
| 250 |
|
| 251 |
\begin{command}{\pgfparserset\marg{key list}}% |
| 252 |
The |pgfparser| module has a few keys you can access through this macro. It |
| 253 |
is just a shortcut for |\pgfset{/pgfparser/.cd,#1}|. The available keys are |
| 254 |
listed in subsection~\ref{sec:parser:keys}. |
| 255 |
\end{command}% |
| 256 |
|
| 257 |
|
| 258 |
\subsection{Keys of the Parser Module} |
| 259 |
\label{sec:parser:keys} |
| 260 |
|
| 261 |
\begin{key}{/pgfparser/silent=\meta{boolean} (initially false)}% |
| 262 |
If |true| then no error will be thrown when a letter is parsed for which no |
| 263 |
action is specified, silently ignoring it. This holds true for every parser. |
| 264 |
\end{key}% |
| 265 |
|
| 266 |
\begin{key}{/pgfparser/status=\meta{boolean} (initially false)}% |
| 267 |
If |true| every parser prints a status message for every action executed. This |
| 268 |
might help in debugging and understanding what the parser does. |
| 269 |
\end{key}% |
| 270 |
|
| 271 |
Additionally to those keys for every \meta{parser name} for which |
| 272 |
|\pgfparserdef|, |\pgfparserdefunknown| or |\pgfparserlet| was run at least once |
| 273 |
the following will be defined: |
| 274 |
|
| 275 |
\begin{key}{/pgfparser/\meta{parser name}/silent=\meta{boolean} (initially false)}% |
| 276 |
If |true| the parser \meta{parser name} will silently ignore undefined |
| 277 |
letters. This is an individual equivalent of |/pgfparser/silent| for each |
| 278 |
defined parser. |
| 279 |
\end{key}% |
| 280 |
|
| 281 |
|
| 282 |
\subsection{Examples} |
| 283 |
|
| 284 |
The following example counts the different letters appearing in a more or less |
| 285 |
random string of letters. Every letter is counted only once, this is achieved |
| 286 |
by defining a new action for every encountered unknown letter that does |
| 287 |
nothing. We can define such rule without knowing which letter is used, because |
| 288 |
|\pgfparsertoken| has the same meaning as that letter. |
| 289 |
% |
| 290 |
\begin{codeexample}[ |
| 291 |
preamble={\usepgfmodule{parser}}, |
| 292 |
pre={\newcount\mycount}, |
| 293 |
] |
| 294 |
\mycount=0 |
| 295 |
% using the shortcut syntax of just placing ; after the state |
| 296 |
\pgfparserdef{different letters}{all};{\pgfparserswitch{final}}% |
| 297 |
\pgfparserdefunknown{different letters}{all}% |
| 298 |
{\pgfparserdef{different letters}{all}\pgfparsertoken{}\advance\mycount1}% |
| 299 |
\pgfparserdeffinal{different letters}% |
| 300 |
{\the\mycount\ different letters found}% |
| 301 |
% don't throw errors for unknown letters |
| 302 |
\pgfparserset{different letters/silent=true}% |
| 303 |
|
| 304 |
\pgfparserparse{different letters}udiaternxqlchudiea; |
| 305 |
\end{codeexample}% |
| 306 |
|
| 307 |
Next we want to try something that uses some of the different argument types |
| 308 |
available. |
| 309 |
% |
| 310 |
\begin{codeexample}[preamble={\usepgfmodule{parser}}] |
| 311 |
% using the long syntax of \pgfparserdef |
| 312 |
\pgfparserdef{arguments}{initial}{the letter a}[d()] |
| 313 |
{\pgfparserifmark{#1}{\textcolor{red}{\textit{use}}}{\textbf{#1}} }% |
| 314 |
% using the shortcut syntax |
| 315 |
\pgfparserdef{arguments}{initial}t[m]{\texttt{#1} }% |
| 316 |
\pgfparserdef{arguments}{initial}c[t*O{blue}m] |
| 317 |
{\pgfparserifmark{#1}{#3}{\textcolor{#2}{#3}}}% |
| 318 |
\pgfparserdef{arguments}{all};{\pgfparserswitch{final}}% |
| 319 |
|
| 320 |
\pgfparserparse{arguments}t{nobody}a(will)ac[green]{P}c*{arse}c{r}; |
| 321 |
\end{codeexample}% |