| 1 |
/* Copyright (c) 2006, Sun Microsystems, Inc. |
| 2 |
* All rights reserved. |
| 3 |
* |
| 4 |
* Redistribution and use in source and binary forms, with or without |
| 5 |
* modification, are permitted provided that the following conditions are met: |
| 6 |
* |
| 7 |
* * Redistributions of source code must retain the above copyright notice, |
| 8 |
* this list of conditions and the following disclaimer. |
| 9 |
* * Redistributions in binary form must reproduce the above copyright |
| 10 |
* notice, this list of conditions and the following disclaimer in the |
| 11 |
* documentation and/or other materials provided with the distribution. |
| 12 |
* * Neither the name of the Sun Microsystems, Inc. nor the names of its |
| 13 |
* contributors may be used to endorse or promote products derived from |
| 14 |
* this software without specific prior written permission. |
| 15 |
* |
| 16 |
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" |
| 17 |
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE |
| 18 |
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE |
| 19 |
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE |
| 20 |
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR |
| 21 |
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF |
| 22 |
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS |
| 23 |
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN |
| 24 |
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) |
| 25 |
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF |
| 26 |
* THE POSSIBILITY OF SUCH DAMAGE. |
| 27 |
*/ |
| 28 |
|
| 29 |
This example illustrates the use of lexical states. |
| 30 |
|
| 31 |
In this directory are two grammar files Digest.jj and Faq.jj. These |
| 32 |
generate parsers that process RMAIL files that are created by the GNU |
| 33 |
emacs editor. A sample RMAIL file called sampleMailFile is also |
| 34 |
included. |
| 35 |
|
| 36 |
Digest.jj and Faq.jj are both identical in their grammar and lexical |
| 37 |
specifications. They only differ in their actions. Digest.jj takes a |
| 38 |
mail file as standard input and produces a digest version of the |
| 39 |
messages to standard output. This is what we used (before we moved |
| 40 |
over to an automatic mailing list software) to produce the weekly mail |
| 41 |
digest of the JavaCC mailing list. Faq.jj takes a mail file as |
| 42 |
standard input and produces a mail FAQ in HTML format. It produces a |
| 43 |
file "index.html" that contains all the mail headers and links to |
| 44 |
other HTML files called "1.html", "2.html", etc. that contain the 1st, |
| 45 |
2nd, etc. messages. The parser generated from Faq.jj accepts an |
| 46 |
optional integer argument that specifies the message number from where |
| 47 |
to begin processing. |
| 48 |
|
| 49 |
Type the following: |
| 50 |
|
| 51 |
javacc Digest.jj |
| 52 |
javacc Faq.jj |
| 53 |
javac *.java |
| 54 |
java Digest < sampleMailFile > digestFile |
| 55 |
java Faq < sampleMailFile |
| 56 |
|
| 57 |
And take a look at the output files created. |
| 58 |
|
| 59 |
|
| 60 |
MORE DETAILS ON HOW THE GRAMMARS WORK: |
| 61 |
|
| 62 |
The grammar specification is rather trivial. It simply says that a |
| 63 |
MailFile is is a sequence of MailMessage's followed by an end of file. |
| 64 |
And that a MailMessage is a list of one or more <SUBJECT>'s, <FROM>'s, |
| 65 |
and <DATE>'s followed by a list of zero or more <BODY>'s followed by |
| 66 |
an <END>. |
| 67 |
|
| 68 |
Essentially, there are five lexical tokens: |
| 69 |
|
| 70 |
<SUBJECT>, <FROM, <DATE> : Are the strings of the Subject, From, and |
| 71 |
Date fields of the message. |
| 72 |
|
| 73 |
<BODY> : Is a single line of the message body. |
| 74 |
|
| 75 |
<END> : is the end of message token. |
| 76 |
|
| 77 |
The lexical specification starts with a set of reusable private |
| 78 |
regular expressions EOL, TWOEOLS, and NOT_EOL. These are defined to |
| 79 |
be portable across different platforms where the end of line |
| 80 |
characters are different. |
| 81 |
|
| 82 |
In the <DEFAULT> lexical state, the token manager simply eats up |
| 83 |
characters until it sees the beginning of a message which is marked by |
| 84 |
<<EOL> "*** EOOH ***" <EOL>>. See the sample mail file for details. |
| 85 |
At this point, it switches to state <MAILHEADER>. |
| 86 |
|
| 87 |
In the state <MAILHEADER>, two consecutive end of line's indicate the |
| 88 |
end of the mail header and therefore the token manager goes to the |
| 89 |
<MAILBODY> lexical state. Also, if it sees "Subject: ", it goes to |
| 90 |
the <MAILSUBJECT> lexical state, and similarly for "From: " and "Date: ". |
| 91 |
|
| 92 |
The end of message is signified by the "^_" character, of "\u001f". |
| 93 |
The state <MAILBODY> returns to the <DEFAULT> state when it sees this |
| 94 |
character, otherwise it simply returns message body lines one by one. |
| 95 |
|
| 96 |
The general flow chart for the lexical states is shown below. It is |
| 97 |
useful to make such a diagram when building complex lexical |
| 98 |
specifications. |
| 99 |
|
| 100 |
|
| 101 |
<DEFAULT> ---> <MAILHEADER> --+--> <MAILSUBJECT> -->+ |
| 102 |
^ | ^ | | |
| 103 |
| | | | | |
| 104 |
| | | +--> <MAILFROM> ----->+ |
| 105 |
+- <MAILBODY> <--+ | | | |
| 106 |
| | | |
| 107 |
| +--> <MAILDATE> ----->+ |
| 108 |
| | |
| 109 |
| | |
| 110 |
+----------------------------+ |