| 1 |
|
| 2 |
|
| 3 |
<html> |
| 4 |
<head> |
| 5 |
<title>scanner</title> |
| 6 |
<link rel="stylesheet" href="doc.css" type="text/css"/> |
| 7 |
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> |
| 8 |
</head> |
| 9 |
<body> |
| 10 |
|
| 11 |
<div id="container"> |
| 12 |
|
| 13 |
<div id="product"> |
| 14 |
<p/> |
| 15 |
<div id="product_logo"> |
| 16 |
<a href="http://www.lua.org"> <img alt="Lua logo" src="lua.gif"/> </a> |
| 17 |
</div> |
| 18 |
<div id="product_name"><big><strong>scanner</strong></big></div> |
| 19 |
<div id="product_description"> Lua 5.1 lexical patterns </div> |
| 20 |
<br/> |
| 21 |
|
| 22 |
<div id="product_description"> <small><em> <b>Version:</b> 0.1.2 </em></small></div> |
| 23 |
<div id="product_description"> <small><em><b>Generated:</b> November 26, 2007 </em></small> </div> |
| 24 |
|
| 25 |
</div> <!-- id="product" --> |
| 26 |
|
| 27 |
<div id="main"> |
| 28 |
|
| 29 |
<div id="navigation"> |
| 30 |
<h1><a href="index.html">Modules</a></h1> |
| 31 |
<ul> |
| 32 |
<li><a href="grammar.html">grammar</a></li> |
| 33 |
<li><a href="parser.html">parser</a></li> |
| 34 |
<li><a href="scanner.html">scanner</a></li> |
| 35 |
</ul> |
| 36 |
<br/> |
| 37 |
<h1>scanner</h1> |
| 38 |
<ul> |
| 39 |
|
| 40 |
<!-- |
| 41 |
<li><a href="#section_Description"> Description </a> |
| 42 |
|
| 43 |
</li> |
| 44 |
--> |
| 45 |
<li><a href="#section_Description"> Description </a> |
| 46 |
</li> |
| 47 |
|
| 48 |
<!-- |
| 49 |
<li><a href="#section_Dependencies"> Dependencies </a> |
| 50 |
|
| 51 |
</li> |
| 52 |
--> |
| 53 |
<li><a href="#section_Dependencies"> Dependencies </a> |
| 54 |
</li> |
| 55 |
|
| 56 |
<!-- |
| 57 |
<li><a href="#section_Example"> Example </a> |
| 58 |
|
| 59 |
</li> |
| 60 |
--> |
| 61 |
<li><a href="#section_Example"> Example </a> |
| 62 |
</li> |
| 63 |
|
| 64 |
|
| 65 |
<li><a href="#variables"> Variables </a> |
| 66 |
<ul> |
| 67 |
|
| 68 |
<li><code><a href="#variable_ANY"> ANY </a></code></li> |
| 69 |
|
| 70 |
<li><code><a href="#variable_BANG"> BANG </a></code></li> |
| 71 |
|
| 72 |
<li><code><a href="#variable_BOF"> BOF </a></code></li> |
| 73 |
|
| 74 |
<li><code><a href="#variable_COMMENT"> COMMENT </a></code></li> |
| 75 |
|
| 76 |
<li><code><a href="#variable_EOF"> EOF </a></code></li> |
| 77 |
|
| 78 |
<li><code><a href="#variable_IDENTIFIER"> IDENTIFIER </a></code></li> |
| 79 |
|
| 80 |
<li><code><a href="#variable_IGNORED"> IGNORED </a></code></li> |
| 81 |
|
| 82 |
<li><code><a href="#variable_KEYWORD"> KEYWORD </a></code></li> |
| 83 |
|
| 84 |
<li><code><a href="#variable_keywords"> keywords </a></code></li> |
| 85 |
|
| 86 |
<li><code><a href="#variable_NUMBER"> NUMBER </a></code></li> |
| 87 |
|
| 88 |
<li><code><a href="#variable_SPACE"> SPACE </a></code></li> |
| 89 |
|
| 90 |
<li><code><a href="#variable_STRING"> STRING </a></code></li> |
| 91 |
|
| 92 |
<li><code><a href="#variable_SYMBOL"> SYMBOL </a></code></li> |
| 93 |
|
| 94 |
<li><code><a href="#variable_symbols"> symbols </a></code></li> |
| 95 |
|
| 96 |
<li><code><a href="#variable_TOKEN"> TOKEN </a></code></li> |
| 97 |
|
| 98 |
</ul> |
| 99 |
</li> |
| 100 |
|
| 101 |
|
| 102 |
<li><a href="#functions"> Functions </a> |
| 103 |
<ul> |
| 104 |
|
| 105 |
<li><code><a href="#function_comment2text"> comment2text </a></code></li> |
| 106 |
|
| 107 |
<li><code><a href="#function_error"> error </a></code></li> |
| 108 |
|
| 109 |
<li><code><a href="#function_string2text"> string2text </a></code></li> |
| 110 |
|
| 111 |
<li><code><a href="#function_text2comment"> text2comment </a></code></li> |
| 112 |
|
| 113 |
<li><code><a href="#function_text2string"> text2string </a></code></li> |
| 114 |
|
| 115 |
</ul> |
| 116 |
</li> |
| 117 |
|
| 118 |
|
| 119 |
</ul> |
| 120 |
</div> <!-- id="navigation" --> |
| 121 |
|
| 122 |
<div id="content"> |
| 123 |
|
| 124 |
|
| 125 |
|
| 126 |
<p/> |
| 127 |
|
| 128 |
|
| 129 |
<p/> |
| 130 |
<h1><a name="section_Description"></a>Description</h1> |
| 131 |
<p/> |
| 132 |
This module exports several Lua lexical patterns, all implemented in <a href="http://www.inf.puc-rio.br/~roberto/lpeg.html">LPeg</a>. None of them have captures. |
| 133 |
<p/> |
| 134 |
<h1><a name="section_Dependencies"></a>Dependencies</h1> |
| 135 |
<ul> |
| 136 |
<li><a href="http://www.inf.puc-rio.br/~roberto/lpeg.html">LPeg</a></li></ul> |
| 137 |
<p/> |
| 138 |
<h1><a name="section_Example"></a>Example</h1> |
| 139 |
<p/> |
| 140 |
The following example lists all the tokens in a Lua script: |
| 141 |
<p/> |
| 142 |
<pre class="example"> |
| 143 |
<font color="#0000FF"><b>local</b></font> lpeg = <b><font color ="#800040">require</font></b> <font color="#008000">'lpeg'</font> |
| 144 |
<font color="#0000FF"><b>local</b></font> scanner = <b><font color ="#800040">require</font></b> <font color="#008000">'leg.scanner'</font> |
| 145 |
|
| 146 |
<font color="#808080"> -- this pattern captures all tokens, ignores spaces and comments,</font> |
| 147 |
<font color="#808080"> -- and matches with end of file, giving an error otherwise</font> |
| 148 |
patt = (<b>lpeg.C</b>(scanner.TOKEN) + scanner.SPACE + scanner.COMMENT)^<font color="#B00000"><i>0</i></font> |
| 149 |
* (scanner.EOF + <b>scanner.error</b><font color="#008000">'invalid character'</font>) |
| 150 |
patt = <b>lpeg.Ct</b>(patt) |
| 151 |
|
| 152 |
<font color="#808080"> -- opens the file passed as parameter and tries to match with patt</font> |
| 153 |
f = <b><font color ="#800040">assert</font></b>(<font color = "#800040"><b>io.open</b></font>(arg[<font color="#B00000"><i>1</i></font>])) |
| 154 |
|
| 155 |
<font color="#808080"> -- a table storing all the tokens</font> |
| 156 |
ALL = <b>patt:match</b>(<b>f:read</b><font color="#008000">'*a'</font>) |
| 157 |
<b>f:close</b>() |
| 158 |
|
| 159 |
<font color="#808080"> -- dumps all tokens on screen</font> |
| 160 |
<font color="#0000FF"><b>for</b></font> _, tk <font color="#0000FF"><b>in</b></font> <b><font color ="#800040">ipairs</font></b>(ALL) <font color="#0000FF"><b>do</b></font> |
| 161 |
<b><font color ="#800040">print</font></b>(tk) |
| 162 |
<font color="#0000FF"><b>end</b></font> |
| 163 |
</pre> |
| 164 |
<p/> |
| 165 |
<p/> |
| 166 |
<p/> |
| 167 |
|
| 168 |
|
| 169 |
|
| 170 |
<hr/> |
| 171 |
<h1><a name="variables"></a> Variables </h1> |
| 172 |
<table border="0" width="95%"> |
| 173 |
|
| 174 |
<tr> |
| 175 |
<a name="variable_ANY"></a> |
| 176 |
<td> <pre class="example"><big><strong>ANY</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 177 |
<td> Matches any <a href="#variable_TOKEN">token</a>, <a href="#variable_COMMENT">comment</a> or <a href="#variable_SPACE">space</a>. |
| 178 |
</td> |
| 179 |
</tr> |
| 180 |
|
| 181 |
<tr> |
| 182 |
<a name="variable_BANG"></a> |
| 183 |
<td> <pre class="example"><big><strong>BANG</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 184 |
<td> Matches UNIX's shebang (e.g. <code>#!/usr/bin/lua</code>). |
| 185 |
</td> |
| 186 |
</tr> |
| 187 |
|
| 188 |
<tr> |
| 189 |
<a name="variable_BOF"></a> |
| 190 |
<td> <pre class="example"><big><strong>BOF</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 191 |
<td> Matches the beginning of a file. |
| 192 |
</td> |
| 193 |
</tr> |
| 194 |
|
| 195 |
<tr> |
| 196 |
<a name="variable_COMMENT"></a> |
| 197 |
<td> <pre class="example"><big><strong>COMMENT</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 198 |
<td> Matches any type of comment. |
| 199 |
</td> |
| 200 |
</tr> |
| 201 |
|
| 202 |
<tr> |
| 203 |
<a name="variable_EOF"></a> |
| 204 |
<td> <pre class="example"><big><strong>EOF</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 205 |
<td> Matches the end of a file. |
| 206 |
</td> |
| 207 |
</tr> |
| 208 |
|
| 209 |
<tr> |
| 210 |
<a name="variable_IDENTIFIER"></a> |
| 211 |
<td> <pre class="example"><big><strong>IDENTIFIER</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 212 |
<td> Matches any Lua identifier. |
| 213 |
</td> |
| 214 |
</tr> |
| 215 |
|
| 216 |
<tr> |
| 217 |
<a name="variable_IGNORED"></a> |
| 218 |
<td> <pre class="example"><big><strong>IGNORED</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 219 |
<td> Matches everything ignored by the parser. |
| 220 |
</td> |
| 221 |
</tr> |
| 222 |
|
| 223 |
<tr> |
| 224 |
<a name="variable_KEYWORD"></a> |
| 225 |
<td> <pre class="example"><big><strong>KEYWORD</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 226 |
<td> A pattern which matches any Lua keyword. |
| 227 |
</td> |
| 228 |
</tr> |
| 229 |
|
| 230 |
<tr> |
| 231 |
<a name="variable_keywords"></a> |
| 232 |
<td> <pre class="example"><big><strong>keywords</strong></big> = <small>{}</small> </pre> </td> |
| 233 |
<td> <br/> |
| 234 |
A table with Lua keyword-matching patterns, with the keywords in uppercase |
| 235 |
as keys. |
| 236 |
<p/> |
| 237 |
Examples: <code>keywords.WHILE</code>, <code>keywords['ELSEIF']</code>. |
| 238 |
<p/> |
| 239 |
</td> |
| 240 |
</tr> |
| 241 |
|
| 242 |
<tr> |
| 243 |
<a name="variable_NUMBER"></a> |
| 244 |
<td> <pre class="example"><big><strong>NUMBER</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 245 |
<td> Matches any Lua number. |
| 246 |
</td> |
| 247 |
</tr> |
| 248 |
|
| 249 |
<tr> |
| 250 |
<a name="variable_SPACE"></a> |
| 251 |
<td> <pre class="example"><big><strong>SPACE</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 252 |
<td> Matches any space character. |
| 253 |
</td> |
| 254 |
</tr> |
| 255 |
|
| 256 |
<tr> |
| 257 |
<a name="variable_STRING"></a> |
| 258 |
<td> <pre class="example"><big><strong>STRING</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 259 |
<td> Matches any Lua string. |
| 260 |
</td> |
| 261 |
</tr> |
| 262 |
|
| 263 |
<tr> |
| 264 |
<a name="variable_SYMBOL"></a> |
| 265 |
<td> <pre class="example"><big><strong>SYMBOL</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 266 |
<td> A pattern which matches any Lua symbol. |
| 267 |
</td> |
| 268 |
</tr> |
| 269 |
|
| 270 |
<tr> |
| 271 |
<a name="variable_symbols"></a> |
| 272 |
<td> <pre class="example"><big><strong>symbols</strong></big> = <small>{}</small> </pre> </td> |
| 273 |
<td> <br/> |
| 274 |
A table with Lua symbol-matching patterns, with the symbols themselves as |
| 275 |
keys. |
| 276 |
<p/> |
| 277 |
Examples: <code>symbols['{']</code>, <code>symbols['+']</code>. |
| 278 |
<p/> |
| 279 |
</td> |
| 280 |
</tr> |
| 281 |
|
| 282 |
<tr> |
| 283 |
<a name="variable_TOKEN"></a> |
| 284 |
<td> <pre class="example"><big><strong>TOKEN</strong></big> = <small>LPeg pattern</small> </pre> </td> |
| 285 |
<td> Matches any Lua <a href="#variable_IDENTIFIER">identifier</a>, <a href="#variable_KEYWORD">keyword</a>, <a href="#variable_SYMBOL">symbol</a>, <a href="#variable_NUMBER">number</a> or <a href="#variable_STRING">string</a>. |
| 286 |
</td> |
| 287 |
</tr> |
| 288 |
|
| 289 |
</table> |
| 290 |
|
| 291 |
|
| 292 |
|
| 293 |
<hr/> |
| 294 |
<h1><a name="functions"></a> Functions </h1> |
| 295 |
<table border="0" width="95%"> |
| 296 |
|
| 297 |
<tr> |
| 298 |
<!-- <td> <pre class="example"><big><strong><a href="#functions_comment2text">comment2text</a></strong></big> (comment)</pre> </td> --> |
| 299 |
<td> <code><big><strong> <a href="#function_comment2text">comment2text</a></strong></big> (comment) </code> </td> |
| 300 |
<td> Strips all prefixing <code>--</code> and enclosing <code>--[=*[</code> from comment tokens. </td> |
| 301 |
</tr> |
| 302 |
|
| 303 |
<tr> |
| 304 |
<!-- <td> <pre class="example"><big><strong><a href="#functions_error">error</a></strong></big> (msg)</pre> </td> --> |
| 305 |
<td> <code><big><strong> <a href="#function_error">error</a></strong></big> (msg) </code> </td> |
| 306 |
<td> Returns a function which throws lexical errors. </td> |
| 307 |
</tr> |
| 308 |
|
| 309 |
<tr> |
| 310 |
<!-- <td> <pre class="example"><big><strong><a href="#functions_string2text">string2text</a></strong></big> (str)</pre> </td> --> |
| 311 |
<td> <code><big><strong> <a href="#function_string2text">string2text</a></strong></big> (str) </code> </td> |
| 312 |
<td> Strips all enclosing <code>'</code>, <code>"</code>, and <code>[=*[</code> from string tokens, and processes escape characters. </td> |
| 313 |
</tr> |
| 314 |
|
| 315 |
<tr> |
| 316 |
<!-- <td> <pre class="example"><big><strong><a href="#functions_text2comment">text2comment</a></strong></big> (text)</pre> </td> --> |
| 317 |
<td> <code><big><strong> <a href="#function_text2comment">text2comment</a></strong></big> (text) </code> </td> |
| 318 |
<td> Encloses the text with comment markers. </td> |
| 319 |
</tr> |
| 320 |
|
| 321 |
<tr> |
| 322 |
<!-- <td> <pre class="example"><big><strong><a href="#functions_text2string">text2string</a></strong></big> (text)</pre> </td> --> |
| 323 |
<td> <code><big><strong> <a href="#function_text2string">text2string</a></strong></big> (text) </code> </td> |
| 324 |
<td> Transforms a text into a syntactically valid Lua string. Similar to <code>string.format</code> with the <code>'%q'</code> option, but inserting escape numbers and escape codes where applicable. </td> |
| 325 |
</tr> |
| 326 |
|
| 327 |
</table> |
| 328 |
|
| 329 |
|
| 330 |
<p/><a name="function_comment2text"></a> |
| 331 |
<hr/><code><big>comment2text (comment)</big></code> |
| 332 |
<ul>Strips all prefixing <code>--</code> and enclosing <code>--[=*[</code> from comment tokens. |
| 333 |
<p/> |
| 334 |
<strong>Parameters:</strong><ul> |
| 335 |
<li><code>comment</code>: the comment to strip.</li></ul> |
| 336 |
<p/> |
| 337 |
<strong>Returns:</strong><ul> |
| 338 |
<li>the text without comment marking syntax.</li></ul></ul> |
| 339 |
|
| 340 |
<p/><a name="function_error"></a> |
| 341 |
<hr/><code><big>error (msg)</big></code> |
| 342 |
<ul>Returns a function which throws lexical errors. |
| 343 |
<p/> |
| 344 |
<strong>Parameters:</strong><ul> |
| 345 |
<li><code>msg</code>: the message to be concatenated to the error message.</li></ul> |
| 346 |
<p/> |
| 347 |
<strong>Returns:</strong><ul> |
| 348 |
<li>A function built to be used as a <a href="http://www.inf.puc-rio.br/~roberto/lpeg.html#lpeg">LPeg pattern</a>, which will throw an error when matched.</li></ul> |
| 349 |
<p/> |
| 350 |
Usage example: |
| 351 |
<pre class="example"> |
| 352 |
patt = intended_pattern^<font color="#B00000"><i>0</i></font> * (EOF + <b><font color ="#800040">error</font></b><font color="#008000">'character after EOF'</font>) |
| 353 |
</pre> |
| 354 |
<p/> |
| 355 |
It may also be used as a normal function: |
| 356 |
<pre class="example"> |
| 357 |
<font color="#0000FF"><b>function</b></font> (subject, i) |
| 358 |
<font color="#0000FF"><b>if</b></font> bad_condition <font color="#0000FF"><b>then</b></font> |
| 359 |
<b><font color ="#800040">error</font></b><font color="#008000">'bad condition'</font>(subject, i) |
| 360 |
<font color="#0000FF"><b>end</b></font> |
| 361 |
<font color="#0000FF"><b>end</b></font> |
| 362 |
</pre></ul> |
| 363 |
|
| 364 |
<p/><a name="function_string2text"></a> |
| 365 |
<hr/><code><big>string2text (str)</big></code> |
| 366 |
<ul>Strips all enclosing <code>'</code>, <code>"</code>, and <code>[=*[</code> from string tokens, and processes escape characters. |
| 367 |
<p/> |
| 368 |
<strong>Parameters:</strong><ul> |
| 369 |
<li><code>str</code>: the string to strip.</li></ul> |
| 370 |
<p/> |
| 371 |
<strong>Returns:</strong><ul> |
| 372 |
<li>the text without string enclosers.</li></ul></ul> |
| 373 |
|
| 374 |
<p/><a name="function_text2comment"></a> |
| 375 |
<hr/><code><big>text2comment (text)</big></code> |
| 376 |
<ul>Encloses the text with comment markers. |
| 377 |
<p/> |
| 378 |
<strong>Parameters:</strong><ul> |
| 379 |
<li><code>text</code>: the text to comment.</li></ul> |
| 380 |
<p/> |
| 381 |
<strong>Returns:</strong><ul> |
| 382 |
<li>the text with comment marking syntax.</li></ul></ul> |
| 383 |
|
| 384 |
<p/><a name="function_text2string"></a> |
| 385 |
<hr/><code><big>text2string (text)</big></code> |
| 386 |
<ul>Transforms a text into a syntactically valid Lua string. Similar to <code>string.format</code> with the <code>'%q'</code> option, but inserting escape numbers and escape codes where applicable. |
| 387 |
<p/> |
| 388 |
<strong>Parameters</strong><ul> |
| 389 |
<li><code>text</code>: a string containing the text.</li></ul> |
| 390 |
<p/> |
| 391 |
<strong>Returns:</strong><ul> |
| 392 |
<li>a string, similar to string.format with option <code>'%q'</code>.</li></ul></ul> |
| 393 |
|
| 394 |
<hr/> |
| 395 |
|
| 396 |
|
| 397 |
|
| 398 |
|
| 399 |
</div> <!-- id="content" --> |
| 400 |
</div> <!-- id="main" --> |
| 401 |
</div> <!-- id="container" --> |
| 402 |
</body> |
| 403 |
</html> |