Debian Codegraph @

README

raw ◀ ▶
Source reconstructed at patch state 0 of 1 — not the installed (fully patched) source.
1 Debian patch touch this file
1 NAME
2 Makefile::DOM - Simple DOM parser for Makefiles
3
4 VERSION
5 This document describes Makefile::DOM 0.008 released on 18 November
6 2014.
7
8 DESCRIPTION
9 This libary can serve as an advanced lexer for (GNU) makefiles. It
10 parses makefiles as "documents" and the parsing is lossless. The results
11 are data structures similar to DOM trees. The DOM trees hold every
12 single bit of the information in the original input files, including
13 white spaces, blank lines and makefile comments. That means it's
14 possible to reproduce the original makefiles from the DOM trees. In
15 addition, each node of the DOM trees is modifiable and so is the whole
16 tree, just like the PPI module used for Perl source parsing and the
17 HTML::TreeBuilder module used for parsing HTML source.
18
19 If you're looking for a true GNU make parser that generates an AST,
20 please see Makefile::Parser::GmakeDB instead.
21
22 The interface of "Makefile::DOM" mimics the API design of PPI. In fact,
23 I've directly stolen the source code and POD documentation of PPI::Node,
24 PPI::Element, and PPI::Dumper, with the full permission from the author
25 of PPI, Adam Kennedy.
26
27 "Makefile::DOM" tries to be independent of specific makefile's syntax.
28 The same set of DOM node types is supposed to get shared by different
29 makefile DOM generators. For example, MDOM::Document::Gmake parses GNU
30 makefiles and returns an instance of MDOM::Document, i.e., the root of
31 the DOM tree while the NMAKE makefile lexer in the future,
32 "MDOM::Document::Nmake", also returns instances of the MDOM::Document
33 class. Later, I'll also consider adding support for dmake and bsdmake.
34
35 Structure of the DOM
36 Makefile DOM (MDOM) is a structured set of a series of data types. They
37 provide a flexible document model conformed to the makefile syntax.
38 Below is a complete list of the 19 MDOM classes in the current
39 implementation where the indentation indicates the class inheritance
40 relationships.
41
42 MDOM::Element
43 MDOM::Node
44 MDOM::Unknown
45 MDOM::Assignment
46 MDOM::Command
47 MDOM::Directive
48 MDOM::Document
49 MDOM::Document::Gmake
50 MDOM::Rule
51 MDOM::Rule::Simple
52 MDOM::Rule::StaticPattern
53 MDOM::Token
54 MDOM::Token::Bare
55 MDOM::Token::Comment
56 MDOM::Token::Continuation
57 MDOM::Token::Interpolation
58 MDOM::Token::Modifier
59 MDOM::Token::Separator
60 MDOM::Token::Whitespace
61
62 It's not hard to see that all of the MDOM classes inherit from the
63 MDOM::Element class. MDOM::Token and MDOM::Node are its direct children.
64 The former represents a string token which is atomic from the
65 perspective of the lexer while the latter represents a structured node,
66 which usually has one or more children, and serves as the container for
67 other DOM::Element objects.
68
69 Next we'll show a few examples to demonstrate how to map DOM trees to
70 particular makefiles.
71
72 Case 1
73 Consider the following simple "hello, world" makefile:
74
75 all : ; echo "hello, world"
76
77 We can use the MDOM::Dumper class provided by Makefile::DOM to dump
78 out the internal structure of its corresponding MDOM tree:
79
80 MDOM::Document::Gmake
81 MDOM::Rule::Simple
82 MDOM::Token::Bare 'all'
83 MDOM::Token::Whitespace ' '
84 MDOM::Token::Separator ':'
85 MDOM::Token::Whitespace ' '
86 MDOM::Command
87 MDOM::Token::Separator ';'
88 MDOM::Token::Whitespace ' '
89 MDOM::Token::Bare 'echo "hello, world"'
90 MDOM::Token::Whitespace '\n'
91
92 In this example, speparators ":" and ";" are all instances of the
93 MDOM::Token::Separator class while spaces and new line characters
94 are all represented as MDOM::Token::Whitespace. The other two leaf
95 nodes, "all" and "echo "hello, world"" both belong to
96 MDOM::Token::Bare.
97
98 It's worth mentioning that, the space characters in the rule command
99 "echo "hello, world"" were not represented as
100 MDOM::Token::Whitespace. That's because in makefiles, the spaces in
101 commands do not make any sense to "make" in syntax; those spaces are
102 usually sent to shell programs verbatim. Therefore, the DOM parser
103 does not try to recognize those spaces specifially so as to reduce
104 memory use and the number of nodes. However, leading spaces and
105 trailing new lines will still be recognized as
106 MDOM::Token::Whitespace.
107
108 On a higher level, it's a MDOM::Rule::Simple instance holding
109 several "Token" and one MDOM::Command. On the highest level, it's
110 the root node of the whole DOM tree, i.e., an instance of
111 MDOM::Document::Gmake.
112
113 Case 2
114 Below is a relatively complex example:
115
116 a: foo.c bar.h $(baz) # hello!
117 @echo ...
118
119 It's corresponding DOM structure is
120
121 MDOM::Document::Gmake
122 MDOM::Rule::Simple
123 MDOM::Token::Bare 'a'
124 MDOM::Token::Separator ':'
125 MDOM::Token::Whitespace ' '
126 MDOM::Token::Bare 'foo.c'
127 MDOM::Token::Whitespace ' '
128 MDOM::Token::Bare 'bar.h'
129 MDOM::Token::Whitespace '\t'
130 MDOM::Token::Interpolation '$(baz)'
131 MDOM::Token::Whitespace ' '
132 MDOM::Token::Comment '# hello!'
133 MDOM::Token::Whitespace '\n'
134 MDOM::Command
135 MDOM::Token::Separator '\t'
136 MDOM::Token::Modifier '@'
137 MDOM::Token::Bare 'echo ...'
138 MDOM::Token::Whitespace '\n'
139
140 Compared to the previous example, here appears several new node
141 types.
142
143 The variable interpolation "$(baz)" on the first line of the
144 original makefile corresponds to a MDOM::Token::Interpolation node
145 in its MDOM tree. Similarly, the comment "# hello" corresponds to a
146 MDOM::Token::Comment node.
147
148 On the second line, the rule command indented by a tab character is
149 still represented by a MDOM::Command object. Its first child node
150 (or its first element) is also an MDOM::Token::Seperator instance
151 corresponding to that tab. The command modifier "@" follows the
152 "Separator" immediately, which is of type MDOM::Token::Modifier.
153
154 Case 3
155 Now let's study a sample makefile with various global structures:
156
157 a: b
158 foo = bar
159 # hello!
160
161 Here on the top level, there are three language structures: one rule
162 ""a: b"", one assignment statement "foo = bar", and one comment "#
163 hello!".
164
165 Its MDOM tree is shown below:
166
167 MDOM::Document::Gmake
168 MDOM::Rule::Simple
169 MDOM::Token::Bare 'a'
170 MDOM::Token::Separator ':'
171 MDOM::Token::Whitespace ' '
172 MDOM::Token::Bare 'b'
173 MDOM::Token::Whitespace '\n'
174 MDOM::Assignment
175 MDOM::Token::Bare 'foo'
176 MDOM::Token::Whitespace ' '
177 MDOM::Token::Separator '='
178 MDOM::Token::Whitespace ' '
179 MDOM::Token::Bare 'bar'
180 MDOM::Token::Whitespace '\n'
181 MDOM::Token::Whitespace '\t'
182 MDOM::Token::Comment '# hello!'
183 MDOM::Token::Whitespace '\n'
184
185 We can see that below the root node MDOM::Document::Gmake, there are
186 MDOM::Rule::Simple, MDOM::Assignment, and MDOM::Comment three
187 elements, as well as two MDOM::Token::Whitespace objects.
188
189 It can be observed from the examples above that the MDOM representation
190 for the makefile's lexical elements is rather loose. It only provides
191 very limited structural representation instead of making a bad guess.
192
193 OPERATIONS FOR MDOM TREES
194 Generating an MDOM tree from a GNU makefile only requires two lines of
195 Perl code:
196
197 use MDOM::Document::Gmake;
198 my $dom = MDOM::Document::Gmake->new('Makefile');
199
200 If the makefile source code being parsed is already stored in a Perl
201 variable, say, $var, then we can construct an MDOM via the following
202 code:
203
204 my $dom = MDOM::Document::Gmake->new(\$var);
205
206 Now $dom becomes the reference to the root of the MDOM tree and its type
207 is now MDOM::Document::Gmake, which is also an instance of the
208 MDOM::Node class.
209
210 Just as mentioned above, "MDOM::Node" is the container for other
211 MDOM::Element instances. So we can retrieve some element node's value
212 via its "child" method:
213
214 $node = $dom->child(3);
215 # or $node = $dom->elements(0);
216
217 And we may also use the "elements" method to obtain the values of all
218 the nodes:
219
220 @elems = $dom->elements;
221
222 For every MDOM node, its corresponding makefile source can be generated
223 by invoking its "content" method.
224
225 BUGS AND TODO
226 The current implementation of the MDOM::Document::Gmake lexer is based
227 on a hand-written state machie. Although the efficiency of the engine is
228 not bad, the code is rather complicated and messy, which hurts both
229 extensibility and maintanabilty. So it's expected to rewrite the parser
230 using some grammatical tools like the Perl 6 regex engine
231 Pugs::Compiler::Rule or a yacc-style one like Parse::Yapp.
232
233 SOURCE REPOSITORY
234 You can always get the latest source code of this module from its GitHub
235 repository:
236
237 <http://github.com/agentzh/makefile-dom-pm>
238
239 If you want a commit bit, please let me know.
240
241 AUTHOR
242 Yichun "agentzh" Zhang (章亦春) <[email protected]>
243
244 COPYRIGHT
245 Copyright 2006-2014 by Yichun "agentzh" Zhang (章亦春).
246
247 This library is free software; you can redistribute it and/or modify it
248 under the same terms as Perl itself.
249
250 SEE ALSO
251 MDOM::Document, MDOM::Document::Gmake, PPI, Makefile::Parser::GmakeDB,
252 makesimple.
253