| 1 |
A formal grammar selects tokens only by their classifications: for example, if |
| 2 |
a rule mentions the terminal symbol `integer constant', it means that em(any) |
| 3 |
integer constant is grammatically valid in that position. The precise value of |
| 4 |
the constant is irrelevant when input is being parsed: if `tt(x + 4)' is |
| 5 |
grammatically correct then `tt(x + 1)' or `tt(x + 3989)' is also |
| 6 |
grammatically correct. |
| 7 |
|
| 8 |
But actual values are very important for understanding the input's meaning. A |
| 9 |
compiler is useless if it fails to distinguish between 4, 1 and 3989 as |
| 10 |
constants that are being used in a program. Therefore, each token in a b() |
| 11 |
grammar has both a token type and a em(semantic value). See section |
| 12 |
ref(SEMANTICS) for details. |
| 13 |
|
| 14 |
A token is a terminal symbol defined in the grammar, such as tt(INTEGER), |
| 15 |
tt(IDENTIFIER) or 'tt(,)'. Tokens are used by the parser to make decisions |
| 16 |
about the continuation of the parsing process, but for that only the mere |
| 17 |
tokens are required, and nothing else. |
| 18 |
|
| 19 |
Semantic values represent any additionally available information about tokens, |
| 20 |
such as the values of integers, or the names of identifiers. (A token such as |
| 21 |
'tt(,)' which is just punctuation normally doesn't have a particular semantic |
| 22 |
value.) |
| 23 |
|
| 24 |
For example, an token might be an tt(INTEGER), having the semantic value |
| 25 |
4. Another tt(INTEGER) token could have semantic value 3989. When a grammar |
| 26 |
rule indicates that tt(INTEGER) is allowed, either of these token/value |
| 27 |
combinations is acceptable because each token is tt(INTEGER). When the parser |
| 28 |
accepts the token, it may store the token's semantic value for future |
| 29 |
reference on a em(semantic value stack). |
| 30 |
|
| 31 |
Nonterminal symbols often also have semantic values. For example, in a |
| 32 |
calculator, an expression typically has a semantic value that is a number. In |
| 33 |
a compiler for a programming language, an expression typically has a semantic |
| 34 |
value that is a tree structure describing the meaning of the expression. |
| 35 |
|