| 1 |
em(Symbols) are the building blocks of b() grammars. |
| 2 |
|
| 3 |
A em(terminal symbol) (also known as a em(token)) represents a class of |
| 4 |
syntactically equivalent symbols. Tokens are represented in b()'s parser class |
| 5 |
by a number, defined in an enum. The parser's tt(lex) member function returns |
| 6 |
a token value indicating what kind of token has been read. You don't need to |
| 7 |
know what the code value is; instead, its symbol should always be used. By |
| 8 |
convention, it contains uppercase characters. |
| 9 |
|
| 10 |
em(Nonterminal symbol) define concepts of grammars. The symbol name is used in |
| 11 |
writing grammar rules. By convention, it contains lowercase characters. |
| 12 |
|
| 13 |
Symbol names consist of letters, digits (not at the beginning), and |
| 14 |
underscores. B() does not support periods in symbol names |
| 15 |
(users familiar with Bison may observe that Bison em(does) support periods in |
| 16 |
symbol names, but the Bison's user guide states that `periods make sense only |
| 17 |
in nonterminals'. Even so, it appears that periods in symbols are hardly ever |
| 18 |
used). |
| 19 |
|
| 20 |
There are two ways terminal symbols can be referred to: |
| 21 |
itemization( |
| 22 |
it() A em(named token) is an identifier, like an identifier in |
| 23 |
bf(C++). Each token name must be defined with a b() directive such as |
| 24 |
tt(%token). See section ref(TOKTYPENAMES). |
| 25 |
it() A tt(character token) (or tt(literal character token)) is written in |
| 26 |
the grammar using bf(C++)'s character constants syntax; for example, 'tt(+)' |
| 27 |
is a character token. A character token doesn't need to be declared unless you |
| 28 |
need to specify its semantic value data type (cf. section ref(SEMANTICS)), |
| 29 |
associativity, or precedence (cf. section ref(PRECEDENCE)). |
| 30 |
) |
| 31 |
|
| 32 |
By convention, a character token is only used to represent that |
| 33 |
particular character. Thus, the token 'tt(+)' is represents the character |
| 34 |
`tt(+)' as a token. |
| 35 |
|
| 36 |
All common escape sequences that can be used in bf(C++)'s character |
| 37 |
constants can be used in b() as well. Be careful not to use the tt(0) |
| 38 |
character as a character literal because its ASCII code, zero, is the code |
| 39 |
tt(lex) returns to indicate end-of-input (see section ref(LEX)). If your |
| 40 |
program em(must) be able to return 0-byte characters, define a special token |
| 41 |
(e.g., tt(ZERO_BYTE)) and return that token instead. |
| 42 |
|
| 43 |
Note that em(literal string tokens), formally supported in Bison, are |
| 44 |
em(not) supported by b(). Again, such tokens are hardly ever encountered, and |
| 45 |
lexical scanner generators (like bf(flex)(1) and bf(flexc++)(1)) do not |
| 46 |
support them. Common practice is to define a symbolic name for a literal |
| 47 |
string token. So, a token like tt(EQ) may be defined in the grammar file, with |
| 48 |
the lexical scanner returning tt(EQ) when it matches tt(==). |
| 49 |
|
| 50 |
The value returned by the parser's tt(lex) member is always a terminal |
| 51 |
token. The numeric code for a character token is simply the ASCII code of that |
| 52 |
character, and tt(lex) can simply return that character constant as a token |
| 53 |
value. Each named token becomes a bf(C++) enumeration value in the parser base |
| 54 |
class header file, and tt(lex) can return such enumeration identifiers as |
| 55 |
well. When using an externally defined lexical scanner, that lexical scanner |
| 56 |
should include the parser's base class header file, and it should return |
| 57 |
either character constants or the token identifiers defined in that header |
| 58 |
file. So, if (%token NUM) was defined in the parser class tt(Parser), then the |
| 59 |
lexical scanner may return tt(Parser::NUM). |
| 60 |
|
| 61 |
The symbol `tt(error)' is a reserved em(terminal) symbol reserved by b() for |
| 62 |
error recovery purposes (see chapter ref(RECOVERY)). The tt(error) symbol |
| 63 |
should not be used for other purposes. In particular, the parser's member |
| 64 |
function tt(lex) should never return tt(error). Several more identifiers |
| 65 |
should not be used as terminal symbols either. See section ref(IMPROPER) for |
| 66 |
an overview. |
| 67 |
|
| 68 |
|