Debian Codegraph @

texmf-dist/doc/generic/pgf/pgfmanual-en-module-parser.tex

raw
1 % Copyright 2019 by Till Tantau
2 % Copyright 2019 by Jonathan P. Spratte
3 %
4 % This file may be distributed and/or modified
5 %
6 % 1. under the LaTeX Project Public License and/or
7 % 2. under the GNU Free Documentation License.
8 %
9 % See the file doc/generic/pgf/licenses/LICENSE for more details.
10
11
12 \section{Parser Module}%
13 \label{section-module-parser}%
14
15 \begin{pgfmodule}{parser}%
16 This module defines some commands for creating a simple letter-by-letter
17 parser.
18 \end{pgfmodule}
19
20 \begin{package}{pgfparser}
21 Because the parser module is almost independent of the rest of \pgfname, it
22 can also be used as a standalone package with minimal dependencies.
23 \end{package}
24
25 This module provides commands for defining a parser that scans some given text
26 letter-by-letter. For each letter, some code is executed and, possibly a
27 state-switch occurs. The code for each letter might take mandatory or optional
28 arguments. The parsing process ends when a final state has been reached, and
29 optionally some code is executed afterwards. Each newly defined parser by
30 default ignores space tokens, if you want to change that you'll have to
31 explicitly define an action for blank spaces (with |\pgfparserdef|).
32
33 \begin{command}{\pgfparserparse\marg{parser name}\meta{text}}%
34 This command is used to parse the \meta{text} using the (previously defined)
35 parser named \meta{parser name}.
36
37 The \meta{text} is not contained in curly braces, rather it is all the text
38 that follows. The end of the text is determined implicitly, namely when the
39 final state of the parser has been reached. If you defined a final action for
40 the parser using |\pgfparserdeffinal| it is executed now.
41
42 The parser works as follows: At any moment, it is in a certain
43 \emph{state}, initially this state is called |initial|. Then, the first
44 letter of the \meta{text} is examined (using the |\futurelet| command). For
45 each possible state and each possible letter, some action code is stored in
46 the parser in a table. This code is then executed. This code may, but need
47 not, trigger a \emph{state switch}, causing a new state to be set. The
48 parser then moves on to the next character of the text and repeats the
49 whole procedure, unless it is in the state |final|, which causes the
50 parsing process to stop immediately.
51
52 In the following example, the parser counts the number of |a|'s in the
53 \text{text}, ignoring any |b|'s. The \meta{text} ends with the first~|c|.
54 %
55 \begin{codeexample}[preamble={\usepgfmodule{parser}}]
56 \newcount\mycount
57 \pgfparserdef{myparser}{initial}{the letter a}%
58 {\advance\mycount by 1\relax}%
59 \pgfparserdef{myparser}{initial}{the letter b}%
60 {} % do nothing
61 \pgfparserdef{myparser}{initial}{the letter c}%
62 {\pgfparserswitch{final}}% done!
63
64 \pgfparserparse{myparser}aabaabababbbbbabaabcccc
65 There are \the\mycount\ a's.
66 \end{codeexample}%
67 %
68 \end{command}%
69
70 \begin{command}%
71 {%
72 \pgfparserdef\marg{parser name}\marg{state}\meta{symbol meaning}%
73 \oarg{arguments}\marg{action}%
74 }%
75 This command should be used repeatedly to define a parser named
76 \meta{parser name}. With a call to this command you specify that the
77 \meta{parser name} should do the following: When it is in state
78 \meta{state} and reads the letter \meta{symbol meaning}, perform the code
79 stored in \meta{action}.
80
81 The \meta{symbol meaning} must be the text that results from applying the
82 \TeX\ command |\meaning| to the given character. For instance, |\meaning a|
83 yields |the letter a|, while |\meaning 1| yields |the character 1|. A space
84 yields |blank space |. Alternatively you can give the symbol you want without
85 surrounding it in braces. So both
86 |\pgfparserdef{myparser}{initial}{the letter a}{foo}|
87 and
88 |\pgfparserdef{myparser}{initial}a{foo}|
89 define an \meta{action} for |the letter a|. This short form works for most
90 tokens, but not for a space (in which case you can use
91 |\pgfparserdef|\allowbreak|{myparser}|\allowbreak|{initial}|\allowbreak
92 |{blank space}|\allowbreak|{foo}|%
93 ), and opening braces
94 (in which case you can use
95 |\pgfparserdef|\allowbreak|{myparser}|\allowbreak|{initial}|\allowbreak
96 |{\meaning\bgroup}|\allowbreak|{foo}|%
97 , and one might prefer
98 to use
99 |\pgfparserdef|\allowbreak|{myparser}|\allowbreak|{initial}|\allowbreak
100 |{\meaning\egroup}|\allowbreak|{foo}|
101 for closing braces as well). You can as well define an action for a macro's
102 meaning (note that macros with different names can have the same meaning), so
103 things like
104 |\pgfparserdef|\allowbreak|{myparser}|\allowbreak|{initial}\texttt{foo}|
105 are possible as well.
106
107 The \meta{action} might require arguments which you can specify in the
108 optional \meta{arguments} string. The argument string can contain up to nine
109 argument specifications of the following types:
110
111 \begingroup
112 \def\argdesc#1#2%
113 {%
114 \par
115 \begingroup
116 \setbox0\hbox{\texttt #1}\makebox[1.5em][l]{\usebox0}%
117 \ifdim\wd0>1.5em\\\null\hspace{1.5em}\fi
118 \parbox[t]{\dimexpr\linewidth-1.5em\relax}{#2}%
119 \endgroup
120 }%
121 \argdesc{m}%
122 {a normal mandatory argument}%
123 \argdesc{r\meta{delim}}%
124 {a mandatory argument which is read up to the \meta{delim}}%
125 \argdesc{o}%
126 {an optional argument in |[]| defaulting to a special mark}%
127 \argdesc{O\marg{default}}%
128 {like |o| but defaulting to \meta{default}}%
129 \argdesc{d\meta{delim1}\meta{delim2}}%
130 {%
131 an optional argument in \meta{delim1} and \meta{delim2} defaulting to a
132 special mark%
133 }%
134 \argdesc{D\meta{delim1}\meta{delim2}\marg{default}}%
135 {like |d| but defaulting to \meta{default}}%
136 \argdesc{t\meta{token}}%
137 {%
138 tests whether the next letter is \meta{token}, if so gobbles it and
139 the argument is set to a special mark.%
140 }%
141 \endgroup
142
143 So if you want to define an \meta{action} that takes two mandatory arguments
144 you use |[mm]|, if it should take an optional star, one optional argument in
145 brackets that returns a marker if it's not used, one mandatory and finally an
146 optional argument in parentheses that defaults to |something| you use
147 |[t*omD(){something}]| as the argument string. If the argument should be
148 anything up to a semicolon, you use |[r;]|. Spaces before argument
149 specifications in the string are ignored. So |[r m]| will be one argument and
150 read anything up to an |m|. Also spaces before any argument in the parsed
151 letters are ignored, so if |a| was setup to take an optional argument the
152 argument would be detected in |a []|. Like with normal \LaTeXe\ optional
153 arguments you have to protect nested brackets: |[a[bc]d]| would be read as
154 |a[bc| with a trailing |d]|, \emph{not} as |a[bc]d|. You'd have to use
155 |[{a[bc]d}]| to get it correct.
156
157 Inside the \meta{action} you can perform almost any kind of code. This code
158 will not be surrounded by a scope, so its effect persists after the parsing
159 is done. However, each time after the \meta{action} is executed, control
160 goes back to the parser. You should not launch a parser inside the
161 \meta{action} code, unless you put it in a scope.
162
163 When you use |all| as the \meta{state}, the \meta{action} is performed
164 in all states as a fallback, whenever \meta{symbol meaning} is encountered.
165 This means that when you do not specify anything explicitly for a state and a
166 letter, but you do specify something for |all| and this letter, then the
167 specified \meta{action} will be used.
168
169 When the parser encounters a letter for which nothing is specified in the
170 current state (neither directly nor indirectly via |all|), an error occurs.
171 Additionally you can specify an action that is executed after the error is
172 thrown using |\pgfparserdefunknown|. To suppress these errors (but not the
173 action specified with |\pgfparserdefunknown|) you can use the
174 |/pgfparser/silent| key or the |silent| key of the current \meta{parser name}.
175 \end{command}%
176
177 \begin{command}%
178 {%
179 \pgfparserlet
180 \marg{parser name 1}\marg{state 1}\meta{symbol meaning 1}%
181 \oarg{opt 1}\oarg{opt 2}\meta{symbol meaning 2}%
182 }%
183 If none of the optional arguments are given in the following explanation
184 \meta{parser name 2} and \meta{state 2} are the same as
185 \meta{parser name 1} and \meta{state 1}. If only the first is given
186 \meta{state 2} equals \meta{opt 1}. If both are given \meta{parser name 2}
187 equals \meta{opt 1} and \meta{state 2} equals \meta{opt 2}.
188
189 Defines an action for \meta{parser name 1} in \meta{state 1} for the
190 \meta{symbol meaning 1} to do the same as the action of \meta{parser name 2}
191 in \meta{state 2} for the \meta{symbol meaning 2}. For \meta{symbol meaning 1}
192 and \meta{symbol meaning 2} the same parsing rules apply as for \meta{symbol
193 meaning} in |\pgfparserdef| so you either give the meaning in braces or just
194 the symbol.
195 \end{command}%
196
197 \begin{command}{\pgfparserdefunknown\marg{parser name}\marg{state}\marg{action}}%
198 With this macro you can define an \meta{action} for the \meta{parser name}
199 parser in \meta{state} if no action was defined for the letter which was
200 encountered.
201 \end{command}%
202
203 \begin{command}{\pgfparserdeffinal\marg{parser name}\marg{action}}%
204 Every parser can call a final \meta{action} after the state was switched to
205 |final|. This \meta{action} is executed after everything else, so you can use
206 something that grabs more arguments if you want to.
207 \end{command}%
208
209 \begin{command}{\pgfparserswitch\marg{state}}%
210 This command can be called inside the action code of a parser to cause a
211 state switch to \meta{state}.
212 \end{command}%
213
214 \begin{command}{\pgfparserifmark\marg{arg}\marg{true}\marg{false}}%
215 Remember that some of the optional argument types set special marks? With
216 |\pgfparserifmark| you can test whether \meta{arg} is such a mark. So if
217 there was no optional argument for the argument types |o| and |d| the
218 \meta{true} branch will be executed, else the \meta{false} branch. For the |t|
219 type argument the \meta{true} branch is executed if the token was encountered.
220 \end{command}%
221
222 \begin{command}{\pgfparserreinsert}%
223 You can use this as the final macro in an action of |\pgfparserdef| or
224 |\pgfparserdefunknown|. This has the effect that the contents of
225 |\pgfparserletter| will be parsed next. Without any redefinition the result
226 will be that the last token will be parsed again. You can change the
227 definition of |\pgfparserletter| just before |\pgfparserreinsert| as well to
228 parse some specific tokens next.
229 \end{command}%
230
231 \begin{command}{\pgfparserstate}%
232 Expands to the current state of the parser.
233 \end{command}%
234
235 \begin{command}{\pgfparsertoken}%
236 This is the macro which is let to the following token with |\futurelet|. You
237 can use it inside an action code.
238 \end{command}%
239
240 \begin{command}{\pgfparserletter}%
241 This macro stores the letter to which |\pgfparsertoken| was let. So if
242 you'd use |\pgfparserparse{foo}a| this macro would be defined with
243 |\def\pgfparserletter{a}|. This definition is done before any action code is
244 executed. There are four special cases: If the next token is of category code
245 1, 2, 6, or 10, so with standard category codes the tokens |{|, |}|, |#|, and
246 \textvisiblespace\ (a space), it would be treated differently. In those cases
247 this macro expands to |\bgroup|, |\egroup|, |##|, and \textvisiblespace\ for
248 the categories 1, 2, 6, and 10, respectively.
249 \end{command}%
250
251 \begin{command}{\pgfparserset\marg{key list}}%
252 The |pgfparser| module has a few keys you can access through this macro. It
253 is just a shortcut for |\pgfset{/pgfparser/.cd,#1}|. The available keys are
254 listed in subsection~\ref{sec:parser:keys}.
255 \end{command}%
256
257
258 \subsection{Keys of the Parser Module}
259 \label{sec:parser:keys}
260
261 \begin{key}{/pgfparser/silent=\meta{boolean} (initially false)}%
262 If |true| then no error will be thrown when a letter is parsed for which no
263 action is specified, silently ignoring it. This holds true for every parser.
264 \end{key}%
265
266 \begin{key}{/pgfparser/status=\meta{boolean} (initially false)}%
267 If |true| every parser prints a status message for every action executed. This
268 might help in debugging and understanding what the parser does.
269 \end{key}%
270
271 Additionally to those keys for every \meta{parser name} for which
272 |\pgfparserdef|, |\pgfparserdefunknown| or |\pgfparserlet| was run at least once
273 the following will be defined:
274
275 \begin{key}{/pgfparser/\meta{parser name}/silent=\meta{boolean} (initially false)}%
276 If |true| the parser \meta{parser name} will silently ignore undefined
277 letters. This is an individual equivalent of |/pgfparser/silent| for each
278 defined parser.
279 \end{key}%
280
281
282 \subsection{Examples}
283
284 The following example counts the different letters appearing in a more or less
285 random string of letters. Every letter is counted only once, this is achieved
286 by defining a new action for every encountered unknown letter that does
287 nothing. We can define such rule without knowing which letter is used, because
288 |\pgfparsertoken| has the same meaning as that letter.
289 %
290 \begin{codeexample}[
291 preamble={\usepgfmodule{parser}},
292 pre={\newcount\mycount},
293 ]
294 \mycount=0
295 % using the shortcut syntax of just placing ; after the state
296 \pgfparserdef{different letters}{all};{\pgfparserswitch{final}}%
297 \pgfparserdefunknown{different letters}{all}%
298 {\pgfparserdef{different letters}{all}\pgfparsertoken{}\advance\mycount1}%
299 \pgfparserdeffinal{different letters}%
300 {\the\mycount\ different letters found}%
301 % don't throw errors for unknown letters
302 \pgfparserset{different letters/silent=true}%
303
304 \pgfparserparse{different letters}udiaternxqlchudiea;
305 \end{codeexample}%
306
307 Next we want to try something that uses some of the different argument types
308 available.
309 %
310 \begin{codeexample}[preamble={\usepgfmodule{parser}}]
311 % using the long syntax of \pgfparserdef
312 \pgfparserdef{arguments}{initial}{the letter a}[d()]
313 {\pgfparserifmark{#1}{\textcolor{red}{\textit{use}}}{\textbf{#1}} }%
314 % using the shortcut syntax
315 \pgfparserdef{arguments}{initial}t[m]{\texttt{#1} }%
316 \pgfparserdef{arguments}{initial}c[t*O{blue}m]
317 {\pgfparserifmark{#1}{#3}{\textcolor{#2}{#3}}}%
318 \pgfparserdef{arguments}{all};{\pgfparserswitch{final}}%
319
320 \pgfparserparse{arguments}t{nobody}a(will)ac[green]{P}c*{arse}c{r};
321 \end{codeexample}%