Debian Codegraph @

doc/scanner.html

raw
1
2
3 <html>
4 <head>
5 <title>scanner</title>
6 <link rel="stylesheet" href="doc.css" type="text/css"/>
7 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
8 </head>
9 <body>
10
11 <div id="container">
12
13 <div id="product">
14 <p/>
15 <div id="product_logo">
16 <a href="http://www.lua.org"> <img alt="Lua logo" src="lua.gif"/> </a>
17 </div>
18 <div id="product_name"><big><strong>scanner</strong></big></div>
19 <div id="product_description"> Lua 5.1 lexical patterns </div>
20 <br/>
21
22 <div id="product_description"> <small><em> <b>Version:</b> 0.1.2 </em></small></div>
23 <div id="product_description"> <small><em><b>Generated:</b> November 26, 2007 </em></small> </div>
24
25 </div> <!-- id="product" -->
26
27 <div id="main">
28
29 <div id="navigation">
30 <h1><a href="index.html">Modules</a></h1>
31 <ul>
32 <li><a href="grammar.html">grammar</a></li>
33 <li><a href="parser.html">parser</a></li>
34 <li><a href="scanner.html">scanner</a></li>
35 </ul>
36 <br/>
37 <h1>scanner</h1>
38 <ul>
39
40 <!--
41 <li><a href="#section_Description"> Description </a>
42
43 </li>
44 -->
45 <li><a href="#section_Description"> Description </a>
46 </li>
47
48 <!--
49 <li><a href="#section_Dependencies"> Dependencies </a>
50
51 </li>
52 -->
53 <li><a href="#section_Dependencies"> Dependencies </a>
54 </li>
55
56 <!--
57 <li><a href="#section_Example"> Example </a>
58
59 </li>
60 -->
61 <li><a href="#section_Example"> Example </a>
62 </li>
63
64
65 <li><a href="#variables"> Variables </a>
66 <ul>
67
68 <li><code><a href="#variable_ANY"> ANY </a></code></li>
69
70 <li><code><a href="#variable_BANG"> BANG </a></code></li>
71
72 <li><code><a href="#variable_BOF"> BOF </a></code></li>
73
74 <li><code><a href="#variable_COMMENT"> COMMENT </a></code></li>
75
76 <li><code><a href="#variable_EOF"> EOF </a></code></li>
77
78 <li><code><a href="#variable_IDENTIFIER"> IDENTIFIER </a></code></li>
79
80 <li><code><a href="#variable_IGNORED"> IGNORED </a></code></li>
81
82 <li><code><a href="#variable_KEYWORD"> KEYWORD </a></code></li>
83
84 <li><code><a href="#variable_keywords"> keywords </a></code></li>
85
86 <li><code><a href="#variable_NUMBER"> NUMBER </a></code></li>
87
88 <li><code><a href="#variable_SPACE"> SPACE </a></code></li>
89
90 <li><code><a href="#variable_STRING"> STRING </a></code></li>
91
92 <li><code><a href="#variable_SYMBOL"> SYMBOL </a></code></li>
93
94 <li><code><a href="#variable_symbols"> symbols </a></code></li>
95
96 <li><code><a href="#variable_TOKEN"> TOKEN </a></code></li>
97
98 </ul>
99 </li>
100
101
102 <li><a href="#functions"> Functions </a>
103 <ul>
104
105 <li><code><a href="#function_comment2text"> comment2text </a></code></li>
106
107 <li><code><a href="#function_error"> error </a></code></li>
108
109 <li><code><a href="#function_string2text"> string2text </a></code></li>
110
111 <li><code><a href="#function_text2comment"> text2comment </a></code></li>
112
113 <li><code><a href="#function_text2string"> text2string </a></code></li>
114
115 </ul>
116 </li>
117
118
119 </ul>
120 </div> <!-- id="navigation" -->
121
122 <div id="content">
123
124
125
126 <p/>
127
128
129 <p/>
130 <h1><a name="section_Description"></a>Description</h1>
131 <p/>
132 This module exports several Lua lexical patterns, all implemented in <a href="http://www.inf.puc-rio.br/~roberto/lpeg.html">LPeg</a>. None of them have captures.
133 <p/>
134 <h1><a name="section_Dependencies"></a>Dependencies</h1>
135 <ul>
136 <li><a href="http://www.inf.puc-rio.br/~roberto/lpeg.html">LPeg</a></li></ul>
137 <p/>
138 <h1><a name="section_Example"></a>Example</h1>
139 <p/>
140 The following example lists all the tokens in a Lua script:
141 <p/>
142 <pre class="example">
143 <font color="#0000FF"><b>local</b></font> lpeg = <b><font color ="#800040">require</font></b> <font color="#008000">'lpeg'</font>
144 <font color="#0000FF"><b>local</b></font> scanner = <b><font color ="#800040">require</font></b> <font color="#008000">'leg.scanner'</font>
145
146 <font color="#808080"> -- this pattern captures all tokens, ignores spaces and comments,</font>
147 <font color="#808080"> -- and matches with end of file, giving an error otherwise</font>
148 patt = (<b>lpeg.C</b>(scanner.TOKEN) + scanner.SPACE + scanner.COMMENT)^<font color="#B00000"><i>0</i></font>
149 * (scanner.EOF + <b>scanner.error</b><font color="#008000">'invalid character'</font>)
150 patt = <b>lpeg.Ct</b>(patt)
151
152 <font color="#808080"> -- opens the file passed as parameter and tries to match with patt</font>
153 f = <b><font color ="#800040">assert</font></b>(<font color = "#800040"><b>io.open</b></font>(arg[<font color="#B00000"><i>1</i></font>]))
154
155 <font color="#808080"> -- a table storing all the tokens</font>
156 ALL = <b>patt:match</b>(<b>f:read</b><font color="#008000">'*a'</font>)
157 <b>f:close</b>()
158
159 <font color="#808080"> -- dumps all tokens on screen</font>
160 <font color="#0000FF"><b>for</b></font> _, tk <font color="#0000FF"><b>in</b></font> <b><font color ="#800040">ipairs</font></b>(ALL) <font color="#0000FF"><b>do</b></font>
161 <b><font color ="#800040">print</font></b>(tk)
162 <font color="#0000FF"><b>end</b></font>
163 </pre>
164 <p/>
165 <p/>
166 <p/>
167
168
169
170 <hr/>
171 <h1><a name="variables"></a> Variables </h1>
172 <table border="0" width="95%">
173
174 <tr>
175 <a name="variable_ANY"></a>
176 <td> <pre class="example"><big><strong>ANY</strong></big> = <small>LPeg pattern</small> </pre> </td>
177 <td> Matches any <a href="#variable_TOKEN">token</a>, <a href="#variable_COMMENT">comment</a> or <a href="#variable_SPACE">space</a>.
178 </td>
179 </tr>
180
181 <tr>
182 <a name="variable_BANG"></a>
183 <td> <pre class="example"><big><strong>BANG</strong></big> = <small>LPeg pattern</small> </pre> </td>
184 <td> Matches UNIX's shebang (e.g. <code>#!/usr/bin/lua</code>).
185 </td>
186 </tr>
187
188 <tr>
189 <a name="variable_BOF"></a>
190 <td> <pre class="example"><big><strong>BOF</strong></big> = <small>LPeg pattern</small> </pre> </td>
191 <td> Matches the beginning of a file.
192 </td>
193 </tr>
194
195 <tr>
196 <a name="variable_COMMENT"></a>
197 <td> <pre class="example"><big><strong>COMMENT</strong></big> = <small>LPeg pattern</small> </pre> </td>
198 <td> Matches any type of comment.
199 </td>
200 </tr>
201
202 <tr>
203 <a name="variable_EOF"></a>
204 <td> <pre class="example"><big><strong>EOF</strong></big> = <small>LPeg pattern</small> </pre> </td>
205 <td> Matches the end of a file.
206 </td>
207 </tr>
208
209 <tr>
210 <a name="variable_IDENTIFIER"></a>
211 <td> <pre class="example"><big><strong>IDENTIFIER</strong></big> = <small>LPeg pattern</small> </pre> </td>
212 <td> Matches any Lua identifier.
213 </td>
214 </tr>
215
216 <tr>
217 <a name="variable_IGNORED"></a>
218 <td> <pre class="example"><big><strong>IGNORED</strong></big> = <small>LPeg pattern</small> </pre> </td>
219 <td> Matches everything ignored by the parser.
220 </td>
221 </tr>
222
223 <tr>
224 <a name="variable_KEYWORD"></a>
225 <td> <pre class="example"><big><strong>KEYWORD</strong></big> = <small>LPeg pattern</small> </pre> </td>
226 <td> A pattern which matches any Lua keyword.
227 </td>
228 </tr>
229
230 <tr>
231 <a name="variable_keywords"></a>
232 <td> <pre class="example"><big><strong>keywords</strong></big> = <small>{}</small> </pre> </td>
233 <td> <br/>
234 A table with Lua keyword-matching patterns, with the keywords in uppercase
235 as keys.
236 <p/>
237 Examples: <code>keywords.WHILE</code>, <code>keywords['ELSEIF']</code>.
238 <p/>
239 </td>
240 </tr>
241
242 <tr>
243 <a name="variable_NUMBER"></a>
244 <td> <pre class="example"><big><strong>NUMBER</strong></big> = <small>LPeg pattern</small> </pre> </td>
245 <td> Matches any Lua number.
246 </td>
247 </tr>
248
249 <tr>
250 <a name="variable_SPACE"></a>
251 <td> <pre class="example"><big><strong>SPACE</strong></big> = <small>LPeg pattern</small> </pre> </td>
252 <td> Matches any space character.
253 </td>
254 </tr>
255
256 <tr>
257 <a name="variable_STRING"></a>
258 <td> <pre class="example"><big><strong>STRING</strong></big> = <small>LPeg pattern</small> </pre> </td>
259 <td> Matches any Lua string.
260 </td>
261 </tr>
262
263 <tr>
264 <a name="variable_SYMBOL"></a>
265 <td> <pre class="example"><big><strong>SYMBOL</strong></big> = <small>LPeg pattern</small> </pre> </td>
266 <td> A pattern which matches any Lua symbol.
267 </td>
268 </tr>
269
270 <tr>
271 <a name="variable_symbols"></a>
272 <td> <pre class="example"><big><strong>symbols</strong></big> = <small>{}</small> </pre> </td>
273 <td> <br/>
274 A table with Lua symbol-matching patterns, with the symbols themselves as
275 keys.
276 <p/>
277 Examples: <code>symbols['{']</code>, <code>symbols['+']</code>.
278 <p/>
279 </td>
280 </tr>
281
282 <tr>
283 <a name="variable_TOKEN"></a>
284 <td> <pre class="example"><big><strong>TOKEN</strong></big> = <small>LPeg pattern</small> </pre> </td>
285 <td> Matches any Lua <a href="#variable_IDENTIFIER">identifier</a>, <a href="#variable_KEYWORD">keyword</a>, <a href="#variable_SYMBOL">symbol</a>, <a href="#variable_NUMBER">number</a> or <a href="#variable_STRING">string</a>.
286 </td>
287 </tr>
288
289 </table>
290
291
292
293 <hr/>
294 <h1><a name="functions"></a> Functions </h1>
295 <table border="0" width="95%">
296
297 <tr>
298 <!-- <td> <pre class="example"><big><strong><a href="#functions_comment2text">comment2text</a></strong></big> (comment)</pre> </td> -->
299 <td> <code><big><strong> <a href="#function_comment2text">comment2text</a></strong></big> (comment) </code> </td>
300 <td> Strips all prefixing <code>--</code> and enclosing <code>--[=*[</code> from comment tokens. </td>
301 </tr>
302
303 <tr>
304 <!-- <td> <pre class="example"><big><strong><a href="#functions_error">error</a></strong></big> (msg)</pre> </td> -->
305 <td> <code><big><strong> <a href="#function_error">error</a></strong></big> (msg) </code> </td>
306 <td> Returns a function which throws lexical errors. </td>
307 </tr>
308
309 <tr>
310 <!-- <td> <pre class="example"><big><strong><a href="#functions_string2text">string2text</a></strong></big> (str)</pre> </td> -->
311 <td> <code><big><strong> <a href="#function_string2text">string2text</a></strong></big> (str) </code> </td>
312 <td> Strips all enclosing <code>'</code>, <code>"</code>, and <code>[=*[</code> from string tokens, and processes escape characters. </td>
313 </tr>
314
315 <tr>
316 <!-- <td> <pre class="example"><big><strong><a href="#functions_text2comment">text2comment</a></strong></big> (text)</pre> </td> -->
317 <td> <code><big><strong> <a href="#function_text2comment">text2comment</a></strong></big> (text) </code> </td>
318 <td> Encloses the text with comment markers. </td>
319 </tr>
320
321 <tr>
322 <!-- <td> <pre class="example"><big><strong><a href="#functions_text2string">text2string</a></strong></big> (text)</pre> </td> -->
323 <td> <code><big><strong> <a href="#function_text2string">text2string</a></strong></big> (text) </code> </td>
324 <td> Transforms a text into a syntactically valid Lua string. Similar to <code>string.format</code> with the <code>'%q'</code> option, but inserting escape numbers and escape codes where applicable. </td>
325 </tr>
326
327 </table>
328
329
330 <p/><a name="function_comment2text"></a>
331 <hr/><code><big>comment2text (comment)</big></code>
332 <ul>Strips all prefixing <code>--</code> and enclosing <code>--[=*[</code> from comment tokens.
333 <p/>
334 <strong>Parameters:</strong><ul>
335 <li><code>comment</code>: the comment to strip.</li></ul>
336 <p/>
337 <strong>Returns:</strong><ul>
338 <li>the text without comment marking syntax.</li></ul></ul>
339
340 <p/><a name="function_error"></a>
341 <hr/><code><big>error (msg)</big></code>
342 <ul>Returns a function which throws lexical errors.
343 <p/>
344 <strong>Parameters:</strong><ul>
345 <li><code>msg</code>: the message to be concatenated to the error message.</li></ul>
346 <p/>
347 <strong>Returns:</strong><ul>
348 <li>A function built to be used as a <a href="http://www.inf.puc-rio.br/~roberto/lpeg.html#lpeg">LPeg pattern</a>, which will throw an error when matched.</li></ul>
349 <p/>
350 Usage example:
351 <pre class="example">
352 patt = intended_pattern^<font color="#B00000"><i>0</i></font> * (EOF + <b><font color ="#800040">error</font></b><font color="#008000">'character after EOF'</font>)
353 </pre>
354 <p/>
355 It may also be used as a normal function:
356 <pre class="example">
357 <font color="#0000FF"><b>function</b></font> (subject, i)
358 <font color="#0000FF"><b>if</b></font> bad_condition <font color="#0000FF"><b>then</b></font>
359 <b><font color ="#800040">error</font></b><font color="#008000">'bad condition'</font>(subject, i)
360 <font color="#0000FF"><b>end</b></font>
361 <font color="#0000FF"><b>end</b></font>
362 </pre></ul>
363
364 <p/><a name="function_string2text"></a>
365 <hr/><code><big>string2text (str)</big></code>
366 <ul>Strips all enclosing <code>'</code>, <code>"</code>, and <code>[=*[</code> from string tokens, and processes escape characters.
367 <p/>
368 <strong>Parameters:</strong><ul>
369 <li><code>str</code>: the string to strip.</li></ul>
370 <p/>
371 <strong>Returns:</strong><ul>
372 <li>the text without string enclosers.</li></ul></ul>
373
374 <p/><a name="function_text2comment"></a>
375 <hr/><code><big>text2comment (text)</big></code>
376 <ul>Encloses the text with comment markers.
377 <p/>
378 <strong>Parameters:</strong><ul>
379 <li><code>text</code>: the text to comment.</li></ul>
380 <p/>
381 <strong>Returns:</strong><ul>
382 <li>the text with comment marking syntax.</li></ul></ul>
383
384 <p/><a name="function_text2string"></a>
385 <hr/><code><big>text2string (text)</big></code>
386 <ul>Transforms a text into a syntactically valid Lua string. Similar to <code>string.format</code> with the <code>'%q'</code> option, but inserting escape numbers and escape codes where applicable.
387 <p/>
388 <strong>Parameters</strong><ul>
389 <li><code>text</code>: a string containing the text.</li></ul>
390 <p/>
391 <strong>Returns:</strong><ul>
392 <li>a string, similar to string.format with option <code>'%q'</code>.</li></ul></ul>
393
394 <hr/>
395
396
397
398
399 </div> <!-- id="content" -->
400 </div> <!-- id="main" -->
401 </div> <!-- id="container" -->
402 </body>
403 </html>