Debian Codegraph @

texmf-dist/doc/luatex/luaxml/luaxml.tex

raw
1 \documentclass{ltxdoc}
2 % \usepackage{tgschola,url}
3 \usepackage{url}
4 \usepackage[english]{babel}
5 \usepackage{hyperref}
6 \usepackage{luacode}
7 \usepackage{framed}
8 % Version is defined in the makefile, use default values when compiled directly
9 \ifdefined\version\else
10 \def\version{v0.2c}
11 \let\gitdate\date
12 \fi
13 \newcommand\modulename[1]{\subsection{#1}\label{sec:#1}}
14 \newcommand\modulesummary[1]{#1\\}
15 \newcommand\moduleclass[1]{\subsubsection{Class: #1}}
16 \newcommand\functionname[2]{\par\noindent\textbf{#1(#2)}\\}
17 \newcommand\functionsummary[1]{#1\\\textbf{Parameters:}\\}
18 \newcommand\functionparam[2]{\texttt{#1}: #2\\}
19 \newcommand\functionreturn[1]{\textbf{Return: }\\#1\\}
20 \usepackage[default]{luaxml}
21
22 \begin{document}
23 \title{The \textsc{LuaXML} library}
24 \author{Paul Chakravarti \and Michal Hoftich}
25 \date{Version \version\\\gitdate}
26 \maketitle
27 \tableofcontents
28
29 \section{Introduction}
30
31 |LuaXML| is pure lua library for processing and serializing of the |xml| files.
32 The base code code has been written by Paul Chakravarti, with minor changes
33 which brings Lua 5.3 or HTML 5 support. On top of that, new modules for
34 accessing the |xml| files using |DOM| like methods or |CSS|
35 selectors\footnote{Thanks to Leaf Corcoran for |CSS selector| parsing code.}
36 have been added.
37
38 The documentation is divided to three parts -- first part deals with the |DOM|
39 library, second part describes the low-level libraries and the third part is
40 original documentation by Paul Chakravarti.
41 % Current release is aimed mainly as support for the odsfile package.
42 % In first release it was included with the odsfile package,
43 % but as it is general library which can be used also with other packages,
44 % I decided to distribute it as separate library.
45
46 \section{The \texttt{DOM\_Object} library}
47
48 This library can process a |xml| sources using |DOM| like functions. To load
49 it, you need to require |luaxml-domobject.lua| file. The |parse| function
50 provided by the library creates \texttt{DOM\_Object} object, which provides several
51 methods for processing the |xml| tree.
52
53 \begin{verbatim}
54 local dom = require "luaxml-domobject"
55 local document = [[
56 <html>
57 <head><title>sample</title></head>
58 <body>
59 <h1>test</h1>
60 <p>hello</p>
61 </body>
62 </html>
63 ]]
64
65 -- dom.parse returns the DOM_Object
66 local obj = dom.parse(document)
67 -- it is possible to call methods on the object
68 local root_node = obj:root_node()
69 for _, x in ipairs(root_node:get_children()) do
70 print(x:get_element_name())
71 end
72 \end{verbatim}
73
74 The details about available methods can be found in the API docs, section
75 \ref{sec:luaxml-domobject}. The above code will load a |xml| document, it will
76 get the ROOT element and print all it's children element names. The
77 \verb|DOM_Object:get_children| function returns Lua table, so it is possible to
78 loop over it using standard table functions.
79
80 \begin{framed}
81 \begin{luacode*}
82 dom = require "luaxml-domobject"
83 local document = [[
84 <html>
85 <head><title>sample</title></head>
86 <body>
87 <h1>test</h1>
88 <p>hello</p>
89 </body>
90 </html>
91 ]]
92
93 -- dom.parse returns the DOM_Object
94 obj = dom.parse(document)
95 -- it is possible to call methods on the object
96 local root_node = obj:root_node()
97 for _, x in ipairs(root_node:get_children()) do
98 tex.print(x:get_element_name().. "\\par")
99 end
100 \end{luacode*}
101 \end{framed}
102
103 \subsection{HTML parsing}
104
105 You can parse HTML documents using the \verb|DOM_Object.html_parse| function. This parser is slower
106 than the default XML parser, but it can load files that would cause errors in the XML mode.
107 It can handle wrongly nested HTML tags, inline JavaScript and CSS styles, and other HTML features
108 that would cause XML errors.
109
110 \begin{verbatim}
111 dom = require "luaxml-domobject"
112 local document = [[
113 <html>
114 <head><title>sample</title></head>
115 <body>
116 <h1>test</h1>
117 <p>hello
118 <p>another paragraph
119 <ul>
120 <li>first
121 <li>second
122 </ul>
123 </body>
124 </html>
125 ]]
126
127 -- dom.html_parse returns the DOM_Object
128 obj = dom.html_parse(document)
129 -- print names of all elements contained in body
130 for _, x in ipairs(obj:query_selector("body *")) do
131 tex.print(x:get_element_name().. "\\par")
132 end
133 \end{verbatim}
134
135
136 \begin{framed}
137 \begin{luacode*}
138 dom = require "luaxml-domobject"
139 local document = [[
140 <html>
141 <head><title>sample</title></head>
142 <body>
143 <h1>test</h1>
144 <p>hello
145 <p>another paragraph
146 <ul>
147 <li>first
148 <li>second
149 </ul>
150 </body>
151 </html>
152 ]]
153
154 -- dom.html_parse returns the DOM_Object
155 obj = dom.html_parse(document)
156 -- print names of all elements contained in body
157 for _, x in ipairs(obj:query_selector("body *")) do
158 tex.print(x:get_element_name().. "\\par")
159 end
160 \end{luacode*}
161 \end{framed}
162
163
164 \subsection{Void elements}
165
166 The \verb|DOM_Object.parse| function tries to support the HTML void elements,
167 such as \verb|<img>| or \verb|<hr>|. They cannot have closing tags, a parse
168 error occurs when the closing tags are used.
169
170 It is possible to define a different set of void elements using the second parameter
171 for \verb|DOM_Object.parse|:
172
173 \begin{verbatim}
174 obj = dom.parse(document, {custom_void = true})
175 \end{verbatim}
176
177 An empty table will disable all void elements. This setting is recommended for common |xml| documents.
178
179
180 \subsection{Node selection methods}
181 There are some other methods for element retrieving.
182
183 \subsubsection{The \texttt{DOM\_Object:get\_path} method}
184 If you want to print text content of all child elements of the body element, you can use \verb|DOM_Object:get_path|:
185
186 \begin{verbatim}
187 local path = obj:get_path("html body")
188 for _, el in ipairs(path[1]:get_children()) do
189 print(el:get_text())
190 end
191 \end{verbatim}
192
193 The \verb|DOM_Object:get_path| function always return array with all elements
194 which match the requested path, even it there is only one such element. In this
195 case, it is possible to use standard Lua table indexing to get the first and
196 only one matched element and get it's children using
197 \verb|DOM_Object:get_children| method. It the children node is an element, it's
198 text content is printed using \verb|DOM_Object:get_text|.
199
200
201
202 \begin{framed}
203 \begin{luacode*}
204 local path = obj:get_path("html body")
205
206 for _, el in ipairs(path[1]:get_children()) do
207 if el:is_element() then
208 tex.print(el:get_text().."\\par")
209 end
210 end
211 \end{luacode*}
212 \end{framed}
213
214 \subsubsection{The \texttt{DOM\_Object:query\_selector} method}
215
216 This method uses |CSS selector| syntax to select elements, similarly to JavaScript \textit{jQuery} library.
217
218 \begin{verbatim}
219 for _, el in ipairs(obj:query_selector("h1,p")) do
220 print(el:get_text())
221 end
222 \end{verbatim}
223
224
225 \begin{framed}
226 \begin{luacode*}
227 for _, el in ipairs(obj:query_selector("h1,p")) do
228 tex.print(el:get_text().."\\par")
229 end
230 \end{luacode*}
231 \end{framed}
232
233 It supports also |XML| namespaces, using \verb_namespace|element_ syntax.
234
235 \subsubsection{Supported CSS selectors}\label{sec:css_selectors}
236
237 The \verb|query_selector| method supports following CSS selectors:
238
239 \begin{description}
240 \item[Universal selector -- \texttt{*}] -- select any element.
241 \item[Type selector -- \texttt{elementname}] -- Selects all elements that have the given node name.
242 \item[Class selector -- \texttt{.classname}] -- Selects all elements that have the given class attribute.
243 \item[ID selector -- \texttt{\#idname}] -- Selects an element based on the value of its id attribute.
244 \item[Attribute selector -- \texttt{[attrname='value']}] -- Selects all elements that have the given attribute.
245 It can have the following variants:
246 \texttt{[attrname]} -- elements that contain given attribute,
247 \texttt{[attr\string|=value]} -- attribute text is exactly the value, with optional hyphen at the end,
248 \verb|[attr~=value]| -- attribute name of attr whose value is a whitespace-separated list of words, one of which is exactly value,
249 \verb|[attr^=value]| -- attribute text starts with value,
250 \texttt{[attr\$=value]} -- attribute text ends with value.
251 \item[Grouping selector -- \texttt{,}] -- This is a grouping method, it selects all the matching nodes.
252 \end{description}
253
254 \bigskip
255 \noindent It is also possible to combine selectors using \textit{combinators} to make more specific searches. Supported combinators:
256
257 \begin{description}
258 \item[Descendant combinator -- \texttt{A B}] -- match all B elements that are inside A elements.
259 \item[Child combinator -- \verb|A > B|] -- match B elements that are nested directly inside a A element.
260 \item[General sibling combinator -- \texttt{A \char`\~ ~B}] -- the second element
261 follows the first (though not necessarily immediately), and both share the
262 same parent.
263 \item[Adjacent sibling combinator -- \texttt{A + B}] -- the second element directly follows the first, and both share the same parent.
264 \end{description}
265
266 \bigskip
267 \noindent LuaXML also supports some CSS pseudo-classes. A pseudo-class is a
268 keyword added to a selector that specifies a special state of the selected
269 element. The following are supported:
270
271 \begin{description}
272 \item[:first-child] -- matches an element that is the first of its siblings.
273 \item[:first-of-type] -- matches an element that is the first of its siblings,
274 and also matches a certain type selector.
275 \item[:last-child] -- matches an element that is the last of its siblings.
276 \item[:last-of-type] -- matches an element that is the last of its siblings,
277 and also matches a certain type selector.
278 \item[:nth-child] -- matches elements based on their position in a group of siblings.
279 It can be used like this: \verb|li:nth-child(2)|.
280 \end{description}
281
282
283
284
285 \subsection{Element traversing}
286
287 \subsubsection{The \texttt{DOM\_Object:traverse\_elements} method}
288
289 It may be useful to traverse over all elements and apply a function on all of them.
290
291 \begin{verbatim}
292 obj:traverse_elements(function(node)
293 print(node:get_text())
294 end)
295 \end{verbatim}
296
297 \begin{framed}
298 \begin{luacode*}
299 obj:traverse_elements(function(node)
300 tex.print(node:get_text().."\\par")
301 end)
302 \end{luacode*}
303 \end{framed}
304
305 The \verb|get_text| method gets text from all children elements, so the first
306 line shows all text contained in the \verb|<html>| element, the second one in
307 \verb|<head>| element and so on.
308
309 \subsection{DOM modifications}
310
311 It is possible to add new elements, text nodes, or to remove them.
312
313 \begin{verbatim}
314 local headers = obj:query_selector("h1")
315 for _, header in ipairs(headers) do
316 header:remove_node()
317 end
318 -- query selector returns array, we must retrieve the first element
319 -- to get the actual body element
320 local body = obj:query_selector("body")[1]
321 local paragraph = body:create_element("p", {})
322 body:add_child_node(paragraph)
323 paragraph:add_child_node(paragraph:create_text_node("This is a second paragraph"))
324
325 for _, el in ipairs(body:get_children()) do
326 if el:is_element() then
327 print(el:get_element_name().. ": ".. el:get_text())
328 end
329 end
330 \end{verbatim}
331
332 In this example, \verb|<h1>| element is being removed from the sample document, and new
333 paragraph is added. Two paragraphs should be shown in the output:
334
335 \begin{framed}
336 \begin{luacode*}
337 local headers = obj:query_selector("h1")
338 -- query selector returns array, we must retrieve the first element
339 -- to get the actual body element
340 local body = obj:query_selector("body")[1]
341 local oldbody = body:copy_node()
342 for _, header in ipairs(headers) do
343 header:remove_node()
344 end
345 local paragraph = body:create_element("p", {})
346 body:add_child_node(paragraph)
347 paragraph:add_child_node(paragraph:create_text_node("This is a second paragraph"))
348
349 for _, el in ipairs(body:get_children()) do
350 if el:is_element() then
351 tex.print(el:get_element_name().. ": ".. el:get_text() .. "\\par")
352 end
353 end
354
355 body:replace_node(oldbody)
356 \end{luacode*}
357 \end{framed}
358
359
360 \subsubsection{Adding raw XML and HTML string}
361
362 You can also set XML or HTML markup from a string to an element using the \texttt{DOM\_Object:inner\_html} function.
363 Pass true as the second argument to parse string as XML, it is parsed as HTML otherwise.
364
365
366 \begin{verbatim}
367 local document = [[
368 <html><p>hello</p>
369 </html>
370 ]]
371 local tree = dom.html_parse(document)
372 local p = tree:query_selector("p")[1]
373 -- insert inner_html as XML
374 p:inner_html("hello <b>this</b> should be the new content")
375 print(tree:serialize())
376 \end{verbatim}
377
378 In this example, we replace contents of the first \verb|<p>| element by new content.
379
380 \begin{framed}
381 \ttfamily
382 \begin{luacode*}
383 local document = [[
384 <html><p>hello</p>
385 </html>
386 ]]
387 local tree = dom.html_parse(document)
388 local p = tree:query_selector("p")[1]
389 -- insert inner_html as XML
390 p:inner_html("hello <b>this</b> should be the new content")
391 tex.print(tree:serialize())
392 \end{luacode*}
393 \end{framed}
394
395 There are more variants of raw string methods that add the new content at specific places in the element instead of replacing contents
396 of the element:
397
398 \begin{description}
399 \item[\texttt{DOM\_Object:insert\_before\_begin}] -- before element.
400 \item[\texttt{DOM\_Object:insert\_after\_begin}] -- just inside the element, before its first child.
401 \item[\texttt{DOM\_Object:insert\_before\_end}] -- just inside the element, after its last child.
402 \item[\texttt{DOM\_Object:insert\_after\_end}] -- after the element.
403 \end{description}
404
405
406 \section{The \texttt{CssQuery} library}
407 \label{sec:cssquery_library}
408
409 This library serves mainly as a support for the
410 \texttt{DOM\_Object:query\_selector} function. It also supports adding
411 information to the DOM tree.
412
413 \subsection{Example usage}
414
415 \begin{verbatim}
416 local cssobj = require "luaxml-cssquery"
417 local domobj = require "luaxml-domobject"
418
419 local xmltext = [[
420 <html>
421 <body>
422 <h1>Header</h1>
423 <p>Some text, <i>italics</i></p>
424 </body>
425 </html>
426 ]]
427
428 local dom = domobj.parse(xmltext)
429 local css = cssobj()
430
431 css:add_selector("h1", function(obj)
432 print("header found: " .. obj:get_text())
433 end)
434
435 css:add_selector("p", function(obj)
436 print("paragraph found: " .. obj:get_text())
437 end)
438
439 css:add_selector("i", function(obj)
440 print("found italics: " .. obj:get_text())
441 end)
442
443 dom:traverse_elements(function(el)
444 -- find selectors that match the current element
445 local querylist = css:match_querylist(el)
446 -- add templates to the element
447 css:apply_querylist(el,querylist)
448 end)
449 \end{verbatim}
450
451 \begin{framed}
452 \begin{luacode*}
453 local cssobj = require "luaxml-cssquery"
454 local domobj = require "luaxml-domobject"
455 local print = function(s) tex.print(s .. "\\par") end
456
457 local xmltext = [[
458 <html>
459 <body>
460 <h1>Header</h1>
461 <p>Some text, <i>italics</i></p>
462 </body>
463 </html>
464 ]]
465
466 local dom = domobj.parse(xmltext)
467 local css = cssobj()
468
469 css:add_selector("h1", function(obj)
470 print("header found: " .. obj:get_text())
471 end)
472
473 css:add_selector("p", function(obj)
474 print("paragraph found: " .. obj:get_text())
475 end)
476
477 css:add_selector("i", function(obj)
478 print("found italics: " .. obj:get_text())
479 end)
480
481 dom:traverse_elements(function(el)
482 -- find selectors that match the current element
483 local querylist = css:match_querylist(el)
484 -- add templates to the element
485 css:apply_querylist(el,querylist)
486 end)
487 \end{luacode*}
488 \end{framed}
489
490 More complete example may be found in the \texttt{examples} directory in the
491 \texttt{LuaXML} source code
492 repository\footnote{\url{https://github.com/michal-h21/LuaXML/blob/master/examples/xmltotex.lua}}.
493
494 \section{The \texttt{luaxml-transform} library}
495
496 This library is still a bit experimental. It enables XML transformation based
497 on CSS selector templates.
498
499 It isn't nearly as powerful as XSLT, but it may suffice for simpler tasks.
500
501 \subsection{Basic example}
502
503 \begin{verbatim}
504 local transform = require "luaxml-transform"
505
506 local transformer = transform.new()
507 local xml_text = [[<section>hello <b>world</b></section>]]
508
509 -- transformatio rules
510 transformer:add_action("section", "\\section{@<.>}")
511 transformer:add_action("b", "\\textbf{@<.>}")
512
513 -- transform and print the result to the document
514 local result = transformer:parse_xml(xml_text)
515 transform.print_tex("\\verb|" .. result .. "|")
516 \end{verbatim}
517 \begin{framed}
518 \begin{luacode*}
519 local transform = require "luaxml-transform"
520
521 local transformer = transform.new()
522 local xml_text = [[<section>hello <b>world</b></section>]]
523
524 -- transformatio rules
525 transformer:add_action("section", "\\section{@<.>}")
526 transformer:add_action("b", "\\textbf{@<.>}")
527
528 -- transform and print the result to the document
529 local result = transformer:parse_xml(xml_text)
530 transform.print_tex("\\verb|" .. result .. "|")
531 \end{luacode*}
532 \end{framed}
533
534 \subsection{The Transform object }
535
536 The \texttt{luaxml-transform} library provides several functions.
537 Most important of them is \verb|new()|. It returns a Transform object,
538 that can be used for the transformations. It is possible to transform
539 XML using text templates, or Lua functions. In both cases, actions for
540 elements are selected using CSS selectors. If there is no action for
541 an element, it's text content and text from transformed child elements,
542 is placed to the output string.
543
544 There are two methods for action specification, \verb|add_action| for
545 text templates, and \verb|add_custom_action| for Lua functions.
546
547 \subsubsection{Transforming using templates}\label{sec:transform-templates}
548
549 Template actions can be added using the \verb|add_action| method:
550
551 \begin{verbatim}
552 transformer:add_action("CSS selector", "template", {parameters table})
553 \end{verbatim}
554
555 For details about CSS selectors, see the \texttt{CssQuery} library
556 (see page~\pageref{sec:cssquery_library}). Templates can contain
557 arbitrary text, with special instructions that can insert transformed
558 text contents of the element, contents of specific element, or element's
559 attributes.
560
561
562 \noindent\textbf{Instruction syntax:}
563
564 \begin{description}
565 \item[\verb|@\{attribute name\}|] insert value of an attribute
566 \item[\verb|@<.>|] insert transformed content of the element
567 \item[\texttt{\%s}] insert transformed content of the element. Shortcut for \verb|@<.>|.
568 \item[\verb|@<number>|] insert transformed content of the child element
569 selected by it's number in the list of children
570 \item[\verb|@<element name>|] insert transformed content of the named child element
571 \end{description}
572
573 \noindent\textbf{Parameters}
574
575 The parameters table can hold following values:
576
577 \begin{description}
578 \item[verbatim] -- used for source code listings and similar texts, that should keep their original formatting.
579 Special characters are not escaped, so you will want to transform the elements into verbatim or listings environment.
580 \item[separator] -- when you select element by names (\verb|@<element name>|), you can use this parameter
581 set the separator between possible multiple instances of the child element.
582 \end{description}
583
584 \noindent\textbf{Examples:}
585
586 \noindent\textbf{Process children}
587
588 \begin{verbatim}
589 local transformer = transform.new()
590 transformer:add_action("a", "@<.>")
591 -- ignore element <b>
592 transformer:add_action("b", "")
593 local result = transformer:parse_xml("<x><a>hello</a><b>, world</b></x>")
594 transform.print_tex(result)
595 \end{verbatim}
596 \begin{framed}
597 \begin{luacode*}
598
599 local transform = require "luaxml-transform"
600
601 local transformer = transform.new()
602 transformer:add_action("a", "@<.>")
603 -- ignore element <b>
604 transformer:add_action("b", "")
605 local result = transformer:parse_xml("<x><a>hello</a><b>, world</b></x>")
606 transform.print_tex(result)
607 \end{luacode*}
608 \end{framed}
609
610 \noindent\textbf{Select elements by their position}
611
612 \begin{verbatim}
613 local transformer = transform.new()
614 -- swap child elements
615 transformer:add_action("x", "@<2>, @<1>")
616 local result = transformer:parse_xml("<x><a>world</a>, <b>hello</b></x>")
617 transform.print_tex(result)
618 \end{verbatim}
619
620 \begin{framed}
621 \begin{luacode*}
622
623 local transform = require "luaxml-transform"
624
625 local transformer = transform.new()
626 transformer:add_action("x", "@<2>, @<1>")
627 local result = transformer:parse_xml("<x><a>world</a>, <b>hello</b></x>")
628 transform.print_tex(result)
629 \end{luacode*}
630 \end{framed}
631
632
633 \noindent\textbf{Select elements by name}
634 \begin{verbatim}
635 local transformer = transform.new()
636 transformer:add_action("x", "@<a>")
637 local result = transformer:parse_xml("<x><a>hello</a><b>, world</b></x>")
638 transform.print_tex(result)
639 \end{verbatim}
640 \begin{framed}
641 \begin{luacode*}
642
643 local transform = require "luaxml-transform"
644 local transformer = transform.new()
645 transformer:add_action("x", "@<a>")
646 local result = transformer:parse_xml("<x><a>hello</a><b>, world</b></x>")
647 transform.print_tex(result)
648 \end{luacode*}
649 \end{framed}
650
651 \noindent\textbf{Select attributes}
652 \begin{verbatim}
653 local transformer = transform.new()
654 transformer:add_action("b", "\\textbf{@<.>}")
655 -- this will select only <b> elements with "style" attribute
656 transformer:add_action("b[style]", "\\textcolor{@{style}}{\\textbf{@<.>}}")
657 local text = '<x><b>hello</b> <b style="red">world</b></x>'
658 local result = transformer:parse_xml(text)
659 transform.print_tex(result)
660 \end{verbatim}
661 \begin{framed}
662 \begin{luacode*}
663
664 local transform = require "luaxml-transform"
665 local transformer = transform.new()
666 transformer:add_action("b", "\\textbf{@<.>}")
667 -- this will select only <b> elements with "style" attribute
668 transformer:add_action("b[style]", "\\textcolor{@{style}}{\\textbf{@<.>}}")
669 local text = '<x><b>hello</b> <b style="red">world</b></x>'
670 local result = transformer:parse_xml(text)
671 transform.print_tex("\\verb|" .. result .. "|")
672 \end{luacode*}
673 \end{framed}
674
675 \subsubsection{Transforming using Lua functions}
676
677 You can use Lua functions for more complex transformations where simple templates don't suffice.
678
679 \begin{verbatim}
680 transformer:add_custom_action("CSS selector", function)
681 \end{verbatim}
682
683
684 \noindent\textbf{Example}
685 \begin{verbatim}
686 local transformer = transform.new()
687 local xml_text = "<x><a>world</a><b>hello, </b></x>"
688 -- load helper functions
689 local get_child_element = transform.get_child_element
690 local process_children = transform.process_children
691 -- define custom action
692 transformer:add_custom_action("x", function(el)
693 -- it basically just swaps child elements,
694 -- like in the template @<2>@<1>
695 local first = process_children(get_child_element(el, 1))
696 local second = process_children(get_child_element(el, 2))
697 return second .. first
698 end)
699 local result = transformer:parse_xml(xml_text)
700 transform.print_tex(result)
701 \end{verbatim}
702
703 \begin{framed}
704 \begin{luacode*}
705
706 local transform = require "luaxml-transform"
707 local transformer = transform.new()
708 local xml_text = "<x><a>world</a><b>hello, </b></x>"
709 -- load helper functions
710 local get_child_element = transform.get_child_element
711 local process_children = transform.process_children
712 -- define custom action
713 transformer:add_custom_action("x", function(el)
714 -- it basically just swaps child elements,
715 -- like in the template @<2>@<1>
716 local first = process_children(get_child_element(el, 1))
717 local second = process_children(get_child_element(el, 2))
718 return second .. first
719 end)
720 local result = transformer:parse_xml(xml_text)
721 transform.print_tex(result)
722 \end{luacode*}
723 \end{framed}
724
725 \subsubsection{Character handling}
726
727 You may want to escape certain characters, or replace them with
728 \LaTeX\ commands. You can use the \texttt{unicodes} table contained
729 in the Transform object:
730
731
732 \begin{verbatim}
733 local transformer = transform.new()
734 -- you must use the Unicode character code
735 transformer.unicodes[124] = "\\textbar"
736 local text = '<x>|</x>'
737 local result = transformer:parse_xml(text)
738 transform.print_tex(result)
739 \end{verbatim}
740
741 \begin{framed}
742 \begin{luacode*}
743 local transform = require "luaxml-transform"
744 local transformer = transform.new()
745 -- you must use the Unicode character code
746 transformer.unicodes[124] = "\\textbar"
747 local text = '<x>|</x>'
748 local result = transformer:parse_xml(text)
749 transform.print_tex("\\verb|" .. result .. "|")
750 \end{luacode*}
751 \end{framed}
752
753 \section{Character sets handling}
754
755 The \texttt{luaxml-encodings} library provides functions to convert texts in legacy 8-bit encodings such as WINDOWS-1250
756 or ISO-8859-2 to UTF-8. This can be useful in fixing document encoding before HTML parsing using the \texttt{luaxml-mod-html}
757 library.
758
759 \subsection{Example}
760
761
762 \begin{verbatim}
763 kpse.set_program_name "luatex"
764 local encodings = require "luaxml-encodings"
765
766 --read HTML page from the standard input
767 local text = io.read("*all")
768 -- find the character encoding in HTML metadata
769 local enc = encodings.find_html_encoding(text)
770 if enc then
771 -- local conversion table for the found encoding
772 local mapping = encodings.load_mapping(enc)
773 if mapping then
774 -- if the mapping exists, recode the HTML input and print it
775 local converted = encodings.recode(text, mapping)
776 print(converted)
777 end
778 end
779 \end{verbatim}
780
781 \section{The \texttt{luaxml.sty} Package}
782
783 The \texttt{luaxml.sty} package is designed to provide an interface for defining
784 transformation rules for XML and HTML documents using Lua and \LaTeX\ commands.
785 It allows users to declare transformation objects, apply transformation rules
786 based on CSS selectors, and process XML or HTML from files or code snippets
787 within \LaTeX\ documents.
788
789 XML and HTML documents can be inserted from files or directly via commands and environments.
790 All commands and environments intended for code input have two variants: with
791 an asterisk for inputting HTML documents and without an asterisk for inputting
792 XML documents.
793
794 \subsection{Package Options}
795
796 \begin{description}
797 \item[default] -- load HTML templates. They will be available as \verb|html|
798 option in \verb|luaxml.sty| commands and environments.
799 \end{description}
800
801 \begin{verbatim}
802 \usepackage[default]{luaxml}
803 ...
804
805 \begin{LXMLCode*}{html}
806 <p>Hello <b>world</b> and some text in <i>italics</i>.</p>
807 \end{LXMLCode*}
808 \end{verbatim}
809
810 \begin{framed}
811 \begin{LXMLCode*}{html}
812 <p>Hello <b>world</b> and some text in <i>italics</i>.</p>
813 \end{LXMLCode*}
814 \end{framed}
815
816
817 \subsection{Example of Transformation Using \LaTeX\ Commands}
818
819 \begin{verbatim}
820 \LXMLRule[sample]{h1}|\par\noindent{\large\bfseries %s\par}|
821 \LXMLRule[sample]{p}|%s\par|
822 \LXMLRule[sample]{a[href]}|\href{@{href}}{%s}|
823 %% process HTML code
824 \begin{LXMLCode*}{sample}
825 <html>
826 <h1>Hello</h1>
827 <p>Here is a link to <a href="https://tex.stackexchange.com/">TeX.sx</a>
828 </p>
829 </html>
830 \end{LXMLCode*}
831 \end{verbatim}
832 \begin{framed}
833
834 \LXMLRule[sample]{h1}|\par\noindent{\large\bfseries %s\par}|
835 \LXMLRule[sample]{p}|%s\par|
836 \LXMLRule[sample]{a[href]}|\href{@{href}}{%s}|
837 % process HTML code
838 \begin{LXMLCode*}{sample}
839 <html>
840 <h1>Hello</h1>
841 <p>Here is a link to <a href="https://tex.stackexchange.com/">TeX.sx</a>
842 </p>
843 </html>
844 \end{LXMLCode*}
845 \end{framed}
846
847
848 \subsection{Declaring Transformation Rules}
849
850
851
852 \begin{verbatim}
853 \LXMLRule[<options>]\{<CSS selector>\}|<transformation rule>|
854 \end{verbatim}
855
856 \noindent Defines a transformation rule for the current transformer. The transformation
857 is applied to elements matching the given CSS selector. You can define multiple transformers,
858 for example if you want to support multiple XML syntaxes and HTML at the same time.
859
860 \medskip
861
862 \noindent The \texttt{<options>} parameter can include:
863
864 \begin{itemize}
865 \item \texttt{verbatim}: Whether to process the rule in verbatim mode.
866 \item \texttt{transformer}: Specifies a transformer.
867 \end{itemize}
868
869 Any unknown key acts as a name of the transformer. In the following code, both
870 examples add a rule to a transformer named \texttt{sample}.
871
872 \begin{verbatim}
873 \LXMLRule[transformer=sample]{b}|\textbf{%s}|
874 \LXMLRule[sample]{i}|\textit{%s}|
875 \end{verbatim}
876
877 If you want to support only one syntax though, you don't need to specify the transformer name at all,
878 a default object will be used.
879
880 By default, spaces are collapsed. If you want to support elements where white spaces
881 should be preserved, such as HTML \verb|<pre>| element, use the \verb|verbatim| option:
882
883 \begin{verbatim}
884 \LXMLRule[verbatim]{pre}|\begin{verbatim}
885 %s
886 \end{verbatim}
887 % trick to print \end{verbatim}|
888 \verb+\end{verbatim}|+
889
890 \bigskip
891
892 The \texttt{transformation rule} must be delimited by a pair of characters that are not used in the
893 text of the rule. We use \verb+|+ in our examples, but you can use other characters if you like.
894 This is similar to how the \verb|\verb| command works. You can use the syntax
895 shown in the section~\ref{sec:transform-templates} (page~\pageref{sec:transform-templates}).
896
897 The following code defines rule that transforms the \verb|<h1>| element to a \verb|\section| command,
898 and \verb|<a>| element which has a \verb|href| attribute to \verb|\href|. URL of the link is used
899 thanks to the \verb|@{href}| rule.
900
901 \begin{verbatim}
902 \LXMLRule{h1}|{\section{%s}|
903 \LXMLRule{a[href]}|\href{@{href}}{%s}|
904 \end{verbatim}
905
906 \subsection{Content Transformation}
907
908 \begin{verbatim}
909 \LXMLSnippet[<transformer name>]{<XML code>}
910 \LXMLSnippet*[<transformer name>]{<HTML code>}
911 \end{verbatim}
912
913 \noindent The \verb|\LXMLSnippet| command processes a code snippet as XML or HTML.
914 Use the starred variant for HTML
915 input. The \texttt{<transformer name>} argument specifies the transformer object to apply
916 (default is used if empty). The code to be transformed is passed in the second
917 argument.
918
919 \medskip
920
921 \noindent{XML snippet transformation:}
922 \begin{verbatim}
923 \LXMLRule[xmlsnippet]{title}|title: %s|
924 \LXMLSnippet{<root><title>Hello</title></root>}
925 \end{verbatim}
926
927 \begin{framed}
928 \LXMLRule[xmlsnippet]{title}|title: %s|
929 \LXMLSnippet[xmlsnippet]{<root><title>Hello</title></root>}
930 \end{framed}
931
932 \noindent{HTML snippet transformation:}
933 \begin{verbatim}
934 \LXMLRule[htmlsnippet]{h1}|title: %s|
935 \LXMLSnippet*[htmlsnippet]{<div><h1>Header</h1></div>}
936 \end{verbatim}
937
938 \begin{framed}
939 \LXMLRule[htmlsnippet]{h1}|title: %s|
940 \LXMLSnippet*[htmlsnippet]{<div><h1>Header</h1></div>}
941 \end{framed}
942
943 \vtop\bgroup
944 \begin{verbatim}
945 \LXMLInputFile[<transformer>]{<XML file path>}
946 \LXMLInputFile*[<transformer>]{<HTML file path>}
947 \end{verbatim}
948
949 \noindent Processes a file as XML or HTML. Use the starred variant for HTML input. The \texttt{<transformer>} specifies the transformer object to apply (default is used if empty). The file path is passed in the second argument.
950 \egroup
951
952 \noindent\textbf{Environments}
953
954 \medskip
955 \noindent \textbf{\texttt{\textbackslash begin\{LXMLCode\}\{<transformer>\}} ... \texttt{\textbackslash end\{LXMLCode\}}}
956
957
958 \noindent Processes XML code inside the environment. The \texttt{<transformer>} specifies the transformer object to apply (default is used if empty).
959
960 \begin{verbatim}
961 \LXMLRule[xmlenv]{element}|hello: %s|
962 \begin{LXMLCode}{xmlenv}
963 <root>
964 <element>Some content</element>
965 </root>
966 \end{LXMLCode}
967 \end{verbatim}
968
969 \begin{framed}
970 \LXMLRule[xmlenv]{element}|hello: %s|
971 \begin{LXMLCode}{xmlenv}
972 <root>
973 <element>Some content</element>
974 </root>
975 \end{LXMLCode}
976 \end{framed}
977
978 \medskip
979 \noindent\textbf{\texttt{\textbackslash begin\{LXMLCode*\}\{<transformer>\}} ... \texttt{\textbackslash end\{LXMLCode*\}}}
980
981 \noindent Processes HTML code inside the environment. The \texttt{<transformer>} specifies the transformer object to apply (default is used if empty).
982
983 \begin{verbatim}
984 \LXMLRule[htmlenv]{p}|paragraph: %s|
985 \begin{LXMLCode*}{htmlenv}
986 <div>
987 <p>Some HTML content</p>
988 </div>
989 \end{LXMLCode*}
990 \end{verbatim}
991
992 \begin{framed}
993 \LXMLRule[htmlenv]{p}|paragraph: %s|
994 \begin{LXMLCode*}{htmlenv}
995 <div>
996 <p>Some HTML content</p>
997 </div>
998 \end{LXMLCode*}
999 \end{framed}
1000
1001
1002
1003 \clearpage
1004 \section{The API documentation}
1005
1006 \input{doc/api.tex}
1007
1008
1009 \section{Low-level functions usage}
1010
1011 % The processing is done with several handlers, their usage will be shown in the
1012 % following section. Full description of handlers is given in the original
1013 % documentation in section \ref{sec:handlers}.
1014
1015 % \subsection{Usage examples}
1016
1017 The original |LuaXML| library provides some low-level functions for |XML| handling.
1018 First of all, we need to load the libraries:
1019
1020 \begin{verbatim}
1021 xml = require('luaxml-mod-xml')
1022 handler = require('luaxml-mod-handler')
1023 \end{verbatim}
1024
1025
1026 The |luaxml-mod-xml| file contains the xml parser and also the serializer. In
1027 |luaxml-mod-handler|, various handlers for dealing with xml data are defined.
1028 Handlers transforms the |xml| file to data structures which can be handled from
1029 the Lua code. More information about handlers can be found in the original
1030 documentation, section \ref{sec:handlers}.
1031
1032 \subsection{The simpleTreeHandler}
1033 \begin{verbatim}
1034 sample = [[
1035 <a>
1036 <d>hello</d>
1037 <b>world.</b>
1038 <b at="Hi">another</b>
1039 </a>]]
1040 treehandler = handler.simpleTreeHandler()
1041 x = xml.xmlParser(treehandler)
1042 x:parse(sample)
1043 \end{verbatim}
1044
1045 You have to create handler object, using |handler.simpleTreeHandler()| and xml
1046 parser object using |xml.xmlParser(handler object)|. |simpleTreehandler|
1047 creates simple table hierarchy, with top root node in |treehandler.root|
1048
1049 \begin{verbatim}
1050 -- pretty printing function
1051 function printable(tb, level)
1052 level = level or 1
1053 local spaces = string.rep(' ', level*2)
1054 for k,v in pairs(tb) do
1055 if type(v) ~= "table" then
1056 print(spaces .. k..'='..v)
1057 else
1058 print(spaces .. k)
1059 level = level + 1
1060 printable(v, level)
1061 end
1062 end
1063 end
1064
1065 -- print table
1066 printable(treehandler.root)
1067 -- print xml serialization of table
1068 print(xml.serialize(treehandler.root))
1069 -- direct access to the element
1070 print(treehandler.root["a"]["b"][1])
1071 \end{verbatim}
1072
1073 This code produces the following output:
1074
1075 \begin{verbatim}
1076 output:
1077 a
1078 d=hello
1079 b
1080 1=world.
1081 2
1082 1=another
1083 _attr
1084 at=Hi
1085 <?xml version="1.0" encoding="UTF-8"?>
1086 <a>
1087 <d>hello</d>
1088 <b>world.</b>
1089 <b at="Hi">
1090 another
1091 </b>
1092 </a>
1093
1094 world.
1095 \end{verbatim}
1096
1097 First part is pretty-printed dump of Lua table structure contained in the handler, the second
1098 part is |xml| serialized from that table and the last part demonstrates direct access to particular
1099 elements.
1100
1101 Note that |simpleTreeHandler| creates tables that can be easily accessed using
1102 standard lua functions, but if the xml document is of mixed-content type\footnote{%
1103 This means that element may contain both children elements and text.}:
1104
1105 \begin{verbatim}
1106 <a>hello
1107 <b>world</b>
1108 </a>
1109 \end{verbatim}
1110
1111 \noindent then it produces wrong results. It is useful mostly for data |xml| files, not for
1112 text formats like |xhtml|.
1113
1114 \subsection{The domHandler}
1115
1116 % For complex xml documents with mixed content, |domHandler| is capable of representing any valid XML document:
1117 For complex xml documents, it is best to use the |domHandler|, which creates object which contains all information
1118 from the |xml| document.
1119
1120 \begin{verbatim}
1121 -- file dom-sample.lua
1122 -- next line enables scripts called with texlua to use luatex libraries
1123 --kpse.set_program_name("luatex")
1124 function traverseDom(current,level)
1125 local level = level or 0
1126 local spaces = string.rep(" ",level)
1127 local root= current or current.root
1128 local name = root._name or "unnamed"
1129 local xtype = root._type or "untyped"
1130 local attributes = root._attr or {}
1131 if xtype == "TEXT" then
1132 print(spaces .."TEXT : " .. root._text)
1133 else
1134 print(spaces .. xtype .. " : " .. name)
1135 end
1136 for k, v in pairs(attributes) do
1137 print(spaces .. " ".. k.."="..v)
1138 end
1139 local children = root._children or {}
1140 for _, child in ipairs(children) do
1141 traverseDom(child, level + 1)
1142 end
1143 end
1144
1145 local xml = require('luaxml-mod-xml')
1146 local handler = require('luaxml-mod-handler')
1147 local x = '<p>hello <a href="http://world.com/">world</a>, how are you?</p>'
1148 local domHandler = handler.domHandler()
1149 local parser = xml.xmlParser(domHandler)
1150 parser:parse(x)
1151 traverseDom(domHandler.root)
1152 \end{verbatim}
1153
1154 The ROOT element is stored in |domHandler.root| table, it's child nodes are stored in |_children|
1155 tables. Node type is saved in |_type| field, if the node type is |ELEMENT|, then |_name| field contains
1156 element name, |_attr| table contains element attributes. |TEXT| node contains text content in |_text|
1157 field.
1158
1159 The previous code produces following output in the terminal: % after command
1160 % |texlua dom-sample.lua| running:
1161
1162 \begin{verbatim}
1163 ROOT : unnamed
1164 ELEMENT : p
1165 TEXT : hello
1166 ELEMENT : a
1167 href=http://world.com/
1168 TEXT : world
1169 TEXT : , how are you?
1170 \end{verbatim}
1171
1172 % With \verb|domHandler|, you can process documents with mixed content, like
1173 % \verb|xhtml|, so it is a most powerful handler.
1174
1175
1176 \clearpage
1177 \part{Original \texttt{LuaXML} documentation by Paul Chakravarti}
1178 \medskip
1179
1180 \noindent This document was created automatically from the original source code comments using Pandoc\footnote{\url{https://pandoc.org}}
1181
1182 \section{Overview}
1183
1184
1185 This module provides a non-validating XML stream parser in Lua.
1186 \section{Features}
1187
1188 \begin{itemize}
1189 \item
1190 Tokenises well-formed XML (relatively robustly)
1191 \item
1192 Flexible handler based event api (see below)
1193 \item
1194 Parses all XML Infoset elements - ie.
1195 \begin{itemize}
1196 \item
1197 Tags
1198 \item
1199 Text
1200 \item
1201 Comments
1202 \item
1203 CDATA
1204 \item
1205 XML Decl
1206 \item
1207 Processing Instructions
1208 \item
1209 DOCTYPE declarations
1210 \end{itemize}
1211 \item
1212 Provides limited well-formedness checking (checks for basic syntax \&
1213 balanced tags only)
1214 \item
1215 Flexible whitespace handling (selectable)
1216 \item
1217 Entity Handling (selectable)
1218 \end{itemize}
1219 \section{Limitations}
1220
1221 \begin{itemize}
1222 \item
1223 Non-validating
1224 \item
1225 No charset handling
1226 \item
1227 No namespace support
1228 \item
1229 Shallow well-formedness checking only (fails to detect most semantic
1230 errors)
1231 \end{itemize}
1232 \section{API}
1233
1234 The parser provides a partially object-oriented API with functionality
1235 split into tokeniser and handler components.
1236
1237 The handler instance is passed to the tokeniser and receives callbacks
1238 for each XML element processed (if a suitable handler function is
1239 defined). The API is conceptually similar to the SAX API but implemented
1240 differently.
1241
1242 The following events are generated by the tokeniser
1243
1244 \begin{verbatim}
1245 handler:starttag - Start Tag
1246 handler:endtag - End Tag
1247 handler:text - Text
1248 handler:decl - XML Declaration
1249 handler:pi - Processing Instruction
1250 handler:comment - Comment
1251 handler:dtd - DOCTYPE definition
1252 handler:cdata - CDATA
1253 \end{verbatim}
1254 The function prototype for all the callback functions is
1255
1256 \begin{verbatim}
1257 callback(val,attrs,start,end)
1258 \end{verbatim}
1259 where attrs is a table and val/attrs are overloaded for specific
1260 callbacks - ie.
1261
1262 \begin{tabular}{llp{5cm}}
1263 Callback & val & attrs (table)\\
1264 \hline
1265 starttag & name & |{ attributes (name=val).. }|\\
1266 endtag & name & nil\\
1267 text & |<text>| & nil\\
1268 cdata & |<text> | & nil\\
1269 decl & "xml" & |{ attributes (name=val).. }|\\
1270 pi & pi name & \begin{verbatim}{ attributes (if present)..
1271 _text = <PI Text>
1272 }\end{verbatim}\\
1273 comment & |<text>| & nil\\
1274 dtd & root element & \begin{verbatim}{ _root = <Root Element>,
1275 _type = SYSTEM|PUBLIC,
1276 _name = <name>,
1277 _uri = <uri>,
1278 _internal = <internal dtd>
1279 }\end{verbatim}\\
1280 \end{tabular}
1281
1282 (starttag \& endtag provide the character positions of the start/end of the
1283 element)
1284
1285 XML data is passed to the parser instance through the `parse' method
1286 (Note: must be passed as single string currently)
1287
1288 \section{Options}
1289
1290 Parser options are controlled through the `self.options' table.
1291 Available options are -
1292
1293 \begin{itemize}
1294 \item
1295 stripWS
1296
1297 Strip non-significant whitespace (leading/trailing) and do not
1298 generate events for empty text elements
1299 \item
1300 expandEntities
1301
1302 Expand entities (standard entities + single char numeric entities only
1303 currently - could be extended at runtime if suitable DTD parser added
1304 elements to table (see obj.\_ENTITIES). May also be possible to expand
1305 multibyre entities for UTF--8 only
1306 \item
1307 errorHandler
1308
1309 Custom error handler function
1310 \end{itemize}
1311 NOTE: Boolean options must be set to `nil' not `0'
1312
1313 \section{Usage}
1314
1315 Create a handler instance -
1316
1317 \begin{verbatim}
1318 h = { starttag = function(t,a,s,e) .... end,
1319 endtag = function(t,a,s,e) .... end,
1320 text = function(t,a,s,e) .... end,
1321 cdata = text }
1322 \end{verbatim}
1323 (or use predefined handler - see luaxml-mod-handler.lua)
1324
1325 Create parser instance -
1326
1327 \begin{verbatim}
1328 p = xmlParser(h)
1329 \end{verbatim}
1330 Set options -
1331
1332 \begin{verbatim}
1333 p.options.xxxx = nil
1334 \end{verbatim}
1335 Parse XML data -
1336
1337 \begin{verbatim}
1338 xmlParser:parse("<?xml... ")
1339 \end{verbatim}
1340 \section{Handlers}\label{sec:handlers}
1341
1342 \subsection{Overview}
1343
1344 Standard XML event handler(s) for XML parser module (luaxml-mod-xml.lua)
1345
1346 \subsection{Features}
1347
1348 \begin{verbatim}
1349 printHandler - Generate XML event trace
1350 domHandler - Generate DOM-like node tree
1351 simpleTreeHandler - Generate 'simple' node tree
1352 simpleTeXhandler - SAX like handler with support for CSS selectros
1353 \end{verbatim}
1354 \subsection{API}
1355
1356 Must be called as handler function from xmlParser and implement XML
1357 event callbacks (see xmlParser.lua for callback API definition)
1358
1359 \subsubsection{printHandler}
1360
1361 printHandler prints event trace for debugging
1362
1363 \subsubsection{domHandler}
1364
1365 domHandler generates a DOM-like node tree structure with
1366 a single ROOT node parent - each node is a table comprising
1367 fields below.
1368
1369 \begin{verbatim}
1370 node = { _name = <Element Name>,
1371 _type = ROOT|ELEMENT|TEXT|COMMENT|PI|DECL|DTD,
1372 _attr = { Node attributes - see callback API },
1373 _parent = <Parent Node>
1374 _children = { List of child nodes - ROOT/NODE only }
1375 }
1376
1377 \end{verbatim}
1378 \subsubsection{simpleTreeHandler}
1379
1380 simpleTreeHandler is a simplified handler which attempts to generate a
1381 more `natural' table based structure which supports many common XML
1382 formats.
1383
1384 The XML tree structure is mapped directly into a recursive table
1385 structure with node names as keys and child elements as either a table
1386 of values or directly as a string value for text. Where there is only a
1387 single child element this is inserted as a named key - if there are
1388 multiple elements these are inserted as a vector (in some cases it may
1389 be preferable to always insert elements as a vector which can be
1390 specified on a per element basis in the options). Attributes are
1391 inserted as a child element with a key of `\_attr'.
1392
1393 Only Tag/Text \& CDATA elements are processed - all others are ignored.
1394
1395 This format has some limitations - primarily
1396
1397 \begin{itemize}
1398 \item Mixed-Content behaves unpredictably - the relationship between text
1399 elements and embedded tags is lost and multiple levels of mixed
1400 content does not work
1401 \item If a leaf element has both a text element and attributes then the text
1402 must be accessed through a vector (to provide a container for the
1403 attribute)
1404 \end{itemize}
1405 In general however this format is relatively useful.
1406
1407
1408 \subsection{Options}
1409
1410 \begin{verbatim}
1411 simpleTreeHandler.options.noReduce = { <tag> = bool,.. }
1412
1413 - Nodes not to reduce children vector even if only
1414 one child
1415
1416 domHandler.options.(comment|pi|dtd|decl)Node = bool
1417
1418 - Include/exclude given node types
1419 \end{verbatim}
1420 \subsection{Usage}
1421
1422 Passed as delegate in xmlParser constructor and called as callback by
1423 xmlParser:parse(xml) method.
1424
1425 \section{History}
1426
1427 This library is fork of LuaXML library originally created by Paul
1428 Chakravarti. Some files not needed for use with luatex were dropped from the distribution.
1429 Documentation was converted from original comments in the source code.
1430
1431 \section{License}
1432
1433 This code is freely distributable under the terms of the Lua license
1434 (\url{http://www.lua.org/copyright.html})
1435 \end{document}