Debian Codegraph @

documentation/manual/demos/rpnlex.yo

raw
1 The lexical analyzer's job is to convert characters or sequences of characters
2 read from the input stream into tokens. B()'s parser obtains its tokens by
3 calling its lexical analyzer member (tt(lex)), which is a predefined member
4 of the parser class. See section ref(LEX).
5
6 Our calculator only needs a simple lexical analyzer. It skips blanks and tabs,
7 then reads numbers, returning them as tt(NUM) tokens. Any other character that
8 isn't part of a number is a separate token. Note that the token-value for such
9 a single-character token is the character itself.
10
11 The lexical analyzer function's return value is a numeric value representing a
12 token. If the token is a literal character, then its numeric code is the
13 character's tt(ASCII) code; if the token is the name of a token defined by b()
14 in a grammar specification file, then such named tokens are defined by b() in
15 a bf(C++) enumeration. In the current example, therefore, tt(NUM) becomes an
16 enumeration value, returned by the tt(lex) member as the `value' tt(NUM).
17
18 Semantic values of nonterminals are stored in the parser's data member
19 tt(d_val_) (comparable to the variable tt(yylval) used by, e.g., Bison). This
20 data member has tt(int) as its default type, but by specifying tt(%stype) in
21 the grammar file's directive section this default type is modified (to, e.g.,
22 tt(double)).
23
24 A token value of zero is returned once end-of-file is encountered (the parsing
25 function produced by b() interprets any nonpositive token value as
26 end-of-input).
27
28 Here is the lexical scanner's implementation:
29 verbinclude(rpn/parser/lex.cc)