Debian Codegraph @

ext/sxml/src/SSAX.scm

raw
1 ; Functional XML parsing framework: SAX/DOM and SXML parsers
2 ; with support for XML Namespaces and validation
3 ;
4 ; This is a package of low-to-high level lexing and parsing procedures
5 ; that can be combined to yield a SAX, a DOM, a validating parsers, or
6 ; a parser intended for a particular document type. The procedures in
7 ; the package can be used separately to tokenize or parse various
8 ; pieces of XML documents. The package supports XML Namespaces,
9 ; internal and external parsed entities, user-controlled handling of
10 ; whitespace, and validation. This module therefore is intended to be
11 ; a framework, a set of "Lego blocks" you can use to build a parser
12 ; following any discipline and performing validation to any degree. As
13 ; an example of the parser construction, this file includes a
14 ; semi-validating SXML parser.
15
16 ; The present XML framework has a "sequential" feel of SAX yet a
17 ; "functional style" of DOM. Like a SAX parser, the framework scans
18 ; the document only once and permits incremental processing. An
19 ; application that handles document elements in order can run as
20 ; efficiently as possible. _Unlike_ a SAX parser, the framework does
21 ; not require an application register stateful callbacks and surrender
22 ; control to the parser. Rather, it is the application that can drive
23 ; the framework -- calling its functions to get the current lexical or
24 ; syntax element. These functions do not maintain or mutate any state
25 ; save the input port. Therefore, the framework permits parsing of XML
26 ; in a pure functional style, with the input port being a monad (or a
27 ; linear, read-once parameter).
28
29 ; Besides the PORT, there is another monad -- SEED. Most of the
30 ; middle- and high-level parsers are single-threaded through the
31 ; seed. The functions of this framework do not process or affect the
32 ; SEED in any way: they simply pass it around as an instance of an
33 ; opaque datatype. User functions, on the other hand, can use the
34 ; seed to maintain user's state, to accumulate parsing results, etc. A
35 ; user can freely mix his own functions with those of the
36 ; framework. On the other hand, the user may wish to instantiate a
37 ; high-level parser: ssax:make-elem-parser or ssax:make-parser. In
38 ; the latter case, the user must provide functions of specific
39 ; signatures, which are called at predictable moments during the
40 ; parsing: to handle character data, element data, or processing
41 ; instructions (PI). The functions are always given the SEED, among
42 ; other parameters, and must return the new SEED.
43
44 ; From a functional point of view, XML parsing is a combined
45 ; pre-post-order traversal of a "tree" that is the XML document
46 ; itself. This down-and-up traversal tells the user about an element
47 ; when its start tag is encountered. The user is notified about the
48 ; element once more, after all element's children have been
49 ; handled. The process of XML parsing therefore is a fold over the
50 ; raw XML document. Unlike a fold over trees defined in [1], the
51 ; parser is necessarily single-threaded -- obviously as elements
52 ; in a text XML document are laid down sequentially. The parser
53 ; therefore is a tree fold that has been transformed to accept an
54 ; accumulating parameter [1,2].
55
56 ; Formally, the denotational semantics of the parser can be expressed
57 ; as
58 ; parser:: (Start-tag -> Seed -> Seed) ->
59 ; (Start-tag -> Seed -> Seed -> Seed) ->
60 ; (Char-Data -> Seed -> Seed) ->
61 ; XML-text-fragment -> Seed -> Seed
62 ; parser fdown fup fchar "<elem attrs> content </elem>" seed
63 ; = fup "<elem attrs>" seed
64 ; (parser fdown fup fchar "content" (fdown "<elem attrs>" seed))
65 ;
66 ; parser fdown fup fchar "char-data content" seed
67 ; = parser fdown fup fchar "content" (fchar "char-data" seed)
68 ;
69 ; parser fdown fup fchar "elem-content content" seed
70 ; = parser fdown fup fchar "content" (
71 ; parser fdown fup fchar "elem-content" seed)
72
73 ; Compare the last two equations with the left fold
74 ; fold-left kons elem:list seed = fold-left kons list (kons elem seed)
75
76 ; The real parser created my ssax:make-parser is slightly more complicated,
77 ; to account for processing instructions, entity references, namespaces,
78 ; processing of document type declaration, etc.
79
80
81 ; The XML standard document referred to in this module is
82 ; http://www.w3.org/TR/1998/REC-xml-19980210.html
83 ;
84 ; The present file also defines a procedure that parses the text of an
85 ; XML document or of a separate element into SXML, an
86 ; S-expression-based model of an XML Information Set. SXML is also an
87 ; Abstract Syntax Tree of an XML document. SXML is similar
88 ; but not identical to DOM; SXML is particularly suitable for
89 ; Scheme-based XML/HTML authoring, SXPath queries, and tree
90 ; transformations. See SXML.html for more details.
91 ; SXML is a term implementation of evaluation of the XML document [3].
92 ; The other implementation is context-passing.
93
94 ; The present frameworks fully supports the XML Namespaces Recommendation:
95 ; http://www.w3.org/TR/REC-xml-names/
96 ; Other links:
97 ; [1] Jeremy Gibbons, Geraint Jones, "The Under-appreciated Unfold,"
98 ; Proc. ICFP'98, 1998, pp. 273-279.
99 ; [2] Richard S. Bird, The promotion and accumulation strategies in
100 ; transformational programming, ACM Trans. Progr. Lang. Systems,
101 ; 6(4):487-504, October 1984.
102 ; [3] Ralf Hinze, "Deriving Backtracking Monad Transformers,"
103 ; Functional Pearl. Proc ICFP'00, pp. 186-197.
104
105 ; IMPORT
106 ; parser-error ssax:warn, see Handling of errors, below
107 ; functions declared in files util.scm, input-parse.scm and look-for-str.scm
108 ; char-encoding.scm for various platform-specific character-encoding functions.
109 ; From SRFI-13: string-concatenate/shared and string-concatenate-reverse/shared
110 ; If a particular implementation lacks SRFI-13 support, please
111 ; include the file srfi-13-local.scm
112
113 ; Handling of errors
114 ; This package relies on a function parser-error, which must be defined
115 ; by a user of the package. The function has the following signature:
116 ; parser-error PORT MESSAGE SPECIALISING-MSG*
117 ; Many procedures of this package call 'parser-error' whenever a
118 ; parsing, well-formedness or validation error is encountered. The
119 ; first argument is a port, which typically points to the offending
120 ; character or its neighborhood. Most of the Scheme systems let the
121 ; user query a PORT for the current position. The MESSAGE argument
122 ; indicates a failed XML production or a failed XML constraint. The
123 ; latter is referred to by its anchor name in the XML Recommendation
124 ; or XML Namespaces Recommendation. The parsing library (e.g.,
125 ; next-token, assert-curr-char) invoke 'parser-error' as well, in
126 ; exactly the same way. See input-parse.scm for more details.
127 ; See
128 ; http://pair.com/lisovsky/download/parse-error.scm
129 ; for an excellent example of such a redefined parser-error function.
130 ;
131 ; In addition, the present code invokes a function ssax:warn
132 ; ssax:warn PORT MESSAGE SPECIALISING-MSG*
133 ; to notify the user about warnings that are NOT errors but still
134 ; may alert the user.
135 ;
136 ; Again, parser-error and ssax:warn are supposed to be defined by the
137 ; user. However, if a run-test macro below is set to include
138 ; self-tests, this present code does provide the definitions for these
139 ; functions to allow tests to run.
140
141 ; Misc notes
142 ; It seems it is highly desirable to separate tests out in a dedicated
143 ; file.
144 ;
145
146 ; $Id: SSAX.scm,v 1.2 2003-07-21 12:19:43 shirok Exp $
147
148
149 ; See the Makefile in the ../tests directory
150 ; (in particular, the rule vSSAX) for an example of how
151 ; to run this code on various Scheme systems.
152 ; See SSAX examples for many samples of using this code,
153 ; again, on a variety of Scheme systems.
154 ; See http://ssax.sf.net/
155
156
157 ; The following macro runs built-in test cases -- or does not run,
158 ; depending on which of the two lines below you commented out
159 ;(define-macro run-test (lambda body '(begin #f)))
160
161 ; The following macro could've been defined just as
162 ; (define-macro run-test (lambda body `(begin (display "\n-->Test\n") ,@body)))
163 ;
164 ; Instead, it's more involved, to make up for case-insensitivity of
165 ; symbols on some Scheme systems. In Gambit, symbols are case
166 ; sensitive: (eq? 'A 'a) is #f and (eq? 'Aa (string->symbol "Aa")) is
167 ; #t. On some systems, symbols are case-insensitive and just the
168 ; opposite is true. Therefore, we introduce a notation '"ASymbol" (a
169 ; quoted string) that stands for a case-_sensitive_ ASymbol -- on any
170 ; R5RS Scheme system. This notation is valid only within the body of
171 ; run-test.
172 ; The notation is implemented by scanning the run-test's
173 ; body and replacing every occurrence of (quote "str") with the result
174 ; of (string->symbol "str"). We can do such a replacement at macro-expand
175 ; time (rather than at run time).
176
177 (define-macro run-test
178 (lambda body
179 (define (re-write body)
180 (cond
181 ((vector? body)
182 (list->vector (re-write (vector->list body))))
183 ((not (pair? body)) body)
184 ((and (eq? 'quote (car body)) (pair? (cdr body))
185 (string? (cadr body)))
186 (string->symbol (cadr body)))
187 (else (cons (re-write (car body)) (re-write (cdr body))))))
188 (cons 'begin (re-write body))))
189
190
191 ;========================================================================
192 ; Data Types
193
194 ; TAG-KIND
195 ; a symbol 'START, 'END, 'PI, 'DECL, 'COMMENT, 'CDSECT
196 ; or 'ENTITY-REF that identifies a markup token
197
198 ; UNRES-NAME
199 ; a name (called GI in the XML Recommendation) as given in an xml
200 ; document for a markup token: start-tag, PI target, attribute name.
201 ; If a GI is an NCName, UNRES-NAME is this NCName converted into
202 ; a Scheme symbol. If a GI is a QName, UNRES-NAME is a pair of
203 ; symbols: (PREFIX . LOCALPART)
204
205 ; RES-NAME
206 ; An expanded name, a resolved version of an UNRES-NAME.
207 ; For an element or an attribute name with a non-empty namespace URI,
208 ; RES-NAME is a pair of symbols, (URI-SYMB . LOCALPART).
209 ; Otherwise, it's a single symbol.
210
211 ; ELEM-CONTENT-MODEL
212 ; A symbol:
213 ; ANY - anything goes, expect an END tag.
214 ; EMPTY-TAG - no content, and no END-tag is coming
215 ; EMPTY - no content, expect the END-tag as the next token
216 ; PCDATA - expect character data only, and no children elements
217 ; MIXED
218 ; ELEM-CONTENT
219
220 ; URI-SYMB
221 ; A symbol representing a namespace URI -- or other symbol chosen
222 ; by the user to represent URI. In the former case,
223 ; URI-SYMB is created by %-quoting of bad URI characters and
224 ; converting the resulting string into a symbol.
225
226 ; NAMESPACES
227 ; A list representing namespaces in effect. An element of the list
228 ; has one of the following forms:
229 ; (PREFIX URI-SYMB . URI-SYMB) or
230 ; (PREFIX USER-PREFIX . URI-SYMB)
231 ; USER-PREFIX is a symbol chosen by the user
232 ; to represent the URI.
233 ; (#f USER-PREFIX . URI-SYMB)
234 ; Specification of the user-chosen prefix and a URI-SYMBOL.
235 ; (*DEFAULT* USER-PREFIX . URI-SYMB)
236 ; Declaration of the default namespace
237 ; (*DEFAULT* #f . #f)
238 ; Un-declaration of the default namespace. This notation
239 ; represents overriding of the previous declaration
240 ; A NAMESPACES list may contain several elements for the same PREFIX.
241 ; The one closest to the beginning of the list takes effect.
242
243 ; ATTLIST
244 ; An ordered collection of (NAME . VALUE) pairs, where NAME is
245 ; a RES-NAME or an UNRES-NAME. The collection is an ADT
246
247 ; STR-HANDLER
248 ; A procedure of three arguments: STRING1 STRING2 SEED
249 ; returning a new SEED
250 ; The procedure is supposed to handle a chunk of character data
251 ; STRING1 followed by a chunk of character data STRING2.
252 ; STRING2 is a short string, often "\n" and even ""
253
254 ; ENTITIES
255 ; An assoc list of pairs:
256 ; (named-entity-name . named-entity-body)
257 ; where named-entity-name is a symbol under which the entity was
258 ; declared, named-entity-body is either a string, or
259 ; (for an external entity) a thunk that will return an
260 ; input port (from which the entity can be read).
261 ; named-entity-body may also be #f. This is an indication that a
262 ; named-entity-name is currently being expanded. A reference to
263 ; this named-entity-name will be an error: violation of the
264 ; WFC nonrecursion.
265
266 ; XML-TOKEN -- a record
267
268 ; In Gambit, you can use the following declaration:
269 ; (define-structure xml-token kind head)
270 ; The following declaration is "standard" as it follows SRFI-9:
271 ;;(define-record-type xml-token (make-xml-token kind head) xml-token?
272 ;; (kind xml-token-kind)
273 ;; (head xml-token-head) )
274 ; No field mutators are declared as SSAX is a pure functional parser
275 ;
276 ; But to make the code more portable, we define xml-token simply as
277 ; a pair. It suffices for us. Furthermore, xml-token-kind and xml-token-head
278 ; can be defined as simple procedures. However, they are declared as
279 ; macros below for efficiency.
280
281 (define (make-xml-token kind head) (cons kind head))
282 (define xml-token? pair?)
283 (define-macro xml-token-kind (lambda (token) `(car ,token)))
284 (define-macro xml-token-head (lambda (token) `(cdr ,token)))
285
286
287 ; This record represents a markup, which is, according to the XML
288 ; Recommendation, "takes the form of start-tags, end-tags, empty-element tags,
289 ; entity references, character references, comments, CDATA section delimiters,
290 ; document type declarations, and processing instructions."
291 ;
292 ; kind -- a TAG-KIND
293 ; head -- an UNRES-NAME. For xml-tokens of kinds 'COMMENT and
294 ; 'CDSECT, the head is #f
295 ;
296 ; For example,
297 ; <P> => kind='START, head='P
298 ; </P> => kind='END, head='P
299 ; <BR/> => kind='EMPTY-EL, head='BR
300 ; <!DOCTYPE OMF ...> => kind='DECL, head='DOCTYPE
301 ; <?xml version="1.0"?> => kind='PI, head='xml
302 ; &my-ent; => kind = 'ENTITY-REF, head='my-ent
303 ;
304 ; Character references are not represented by xml-tokens as these references
305 ; are transparently resolved into the corresponding characters.
306 ;
307
308
309
310 ; XML-DECL -- a record
311
312 ; The following is Gambit-specific, see below for a portable declaration
313 ;(define-structure xml-decl elems entities notations)
314
315 ; The record represents a datatype of an XML document: the list of
316 ; declared elements and their attributes, declared notations, list of
317 ; replacement strings or loading procedures for parsed general
318 ; entities, etc. Normally an xml-decl record is created from a DTD or
319 ; an XML Schema, although it can be created and filled in in many other
320 ; ways (e.g., loaded from a file).
321 ;
322 ; elems: an (assoc) list of decl-elem or #f. The latter instructs
323 ; the parser to do no validation of elements and attributes.
324 ;
325 ; decl-elem: declaration of one element:
326 ; (elem-name elem-content decl-attrs)
327 ; elem-name is an UNRES-NAME for the element.
328 ; elem-content is an ELEM-CONTENT-MODEL.
329 ; decl-attrs is an ATTLIST, of (ATTR-NAME . VALUE) associations
330 ; !!!This element can declare a user procedure to handle parsing of an
331 ; element (e.g., to do a custom validation, or to build a hash of
332 ; IDs as they're encountered).
333 ;
334 ; decl-attr: an element of an ATTLIST, declaration of one attribute
335 ; (attr-name content-type use-type default-value)
336 ; attr-name is an UNRES-NAME for the declared attribute
337 ; content-type is a symbol: CDATA, NMTOKEN, NMTOKENS, ...
338 ; or a list of strings for the enumerated type.
339 ; use-type is a symbol: REQUIRED, IMPLIED, FIXED
340 ; default-value is a string for the default value, or #f if not given.
341 ;
342 ;
343
344 ; see a function make-empty-xml-decl to make a XML declaration entry
345 ; suitable for a non-validating parsing.
346
347
348 ;-------------------------
349 ; Utilities
350
351 ; ssax:warn PORT MESSAGE SPECIALISING-MSG*
352 ; to notify the user about warnings that are NOT errors but still
353 ; may alert the user.
354 ; Result is unspecified.
355 ; We need to define the function to allow the self-tests to run.
356 ; Normally the definition of ssax:warn is to be provided by the user.
357 (run-test
358 (define (ssax:warn port msg . other-msg)
359 (apply cerr (cons* nl "Warning: " msg other-msg)))
360 )
361
362
363 ; parser-error PORT MESSAGE SPECIALISING-MSG*
364 ; to let the user know of a syntax error or a violation of a
365 ; well-formedness or validation constraint.
366 ; Result is unspecified.
367 ; We need to define the function to allow the self-tests to run.
368 ; Normally the definition of parser-error is to be provided by the user.
369 (run-test
370 (define (parser-error port msg . specializing-msgs)
371 (apply error (cons msg specializing-msgs)))
372 )
373
374 ; The following is a function that is often used in validation tests,
375 ; to make sure that the computed result matches the expected one.
376 ; This function is a standard equal? predicate with one exception.
377 ; On Scheme systems where (string->symbol "A") and a symbol A
378 ; are the same, equal_? is precisely equal?
379 ; On other Scheme systems, we compare symbols disregarding their case.
380 ; Since this function is used only in tests, we don't have to
381 ; strive to make it efficient.
382 (run-test
383 (define (equal_? e1 e2)
384 (if (eq? 'A (string->symbol "A")) (equal? e1 e2)
385 (cond
386 ((symbol? e1)
387 (and (symbol? e2)
388 (string-ci=? (symbol->string e1) (symbol->string e2))))
389 ((pair? e1)
390 (and (pair? e2)
391 (equal_? (car e1) (car e2)) (equal_? (cdr e1) (cdr e2))))
392 ((vector? e1)
393 (and (vector? e2) (equal_? (vector->list e1) (vector->list e2))))
394 (else
395 (equal? e1 e2)))))
396 )
397
398 ; The following function, which is often used in validation tests,
399 ; lets us conveniently enter newline, CR and tab characters in a character
400 ; string.
401 ; unesc-string: ESC-STRING -> STRING
402 ; where ESC-STRING is a character string that may contain
403 ; %n -- for #\newline
404 ; %r -- for #\return
405 ; %t -- for #\tab
406 ; %% -- for #\%
407 ;
408 ; The result of unesc-string is a character string with all %-combinations
409 ; above replaced with their character equivalents
410
411 (run-test
412 (define (unesc-string str)
413 (call-with-input-string str
414 (lambda (port)
415 (let loop ((frags '()))
416 (let* ((token (next-token '() '(#\% *eof*) "unesc-string" port))
417 (cterm (read-char port))
418 (frags (cons token frags)))
419 (if (eof-object? cterm) (string-concatenate-reverse/shared frags)
420 (let ((cchar (read-char port))) ; char after #\%
421 (if (eof-object? cchar)
422 (error "unexpected EOF after reading % in unesc-string:" str)
423 (loop
424 (cons
425 (case cchar
426 ((#\n) (string #\newline))
427 ((#\r) (string char-return))
428 ((#\t) (string char-tab))
429 ((#\%) "%")
430 (else (error "bad %-char in unesc-string:" cchar)))
431 frags))))))))))
432 )
433
434
435 ; Test if a string is made of only whitespace
436 ; An empty string is considered made of whitespace as well
437 (define (string-whitespace? str)
438 (let ((len (string-length str)))
439 (cond
440 ((zero? len) #t)
441 ((= 1 len) (char-whitespace? (string-ref str 0)))
442 ((= 2 len) (and (char-whitespace? (string-ref str 0))
443 (char-whitespace? (string-ref str 1))))
444 (else
445 (let loop ((i 0))
446 (or (>= i len)
447 (and (char-whitespace? (string-ref str i))
448 (loop (++ i)))))))))
449
450 ; Find val in alist
451 ; Return (values found-el remaining-alist) or
452 ; (values #f alist)
453
454 (define (assq-values val alist)
455 (let loop ((alist alist) (scanned '()))
456 (cond
457 ((null? alist) (values #f scanned))
458 ((equal? val (caar alist))
459 (values (car alist) (append scanned (cdr alist))))
460 (else
461 (loop (cdr alist) (cons (car alist) scanned))))))
462
463 ; From SRFI-1
464 (define (fold-right kons knil lis1)
465 (let recur ((lis lis1))
466 (if (null? lis) knil
467 (let ((head (car lis)))
468 (kons head (recur (cdr lis)))))))
469
470 ; Left fold combinator for a single list
471 (define (fold kons knil lis1)
472 (let lp ((lis lis1) (ans knil))
473 (if (null? lis) ans
474 (lp (cdr lis) (kons (car lis) ans)))))
475
476
477
478 ;========================================================================
479 ; Lower-level parsers and scanners
480 ;
481 ; They deal with primitive lexical units (Names, whitespaces, tags)
482 ; and with pieces of more generic productions. Most of these parsers
483 ; must be called in appropriate context. For example, ssax:complete-start-tag
484 ; must be called only when the start-tag has been detected and its GI
485 ; has been read.
486
487 ;------------------------------------------------------------------------
488 ; Low-level parsing code
489
490 ; Skip the S (whitespace) production as defined by
491 ; [3] S ::= (#x20 | #x9 | #xD | #xA)
492 ; The procedure returns the first not-whitespace character it
493 ; encounters while scanning the PORT. This character is left
494 ; on the input stream.
495
496 (define ssax:S-chars (map ascii->char '(32 10 9 13)))
497
498 (define (ssax:skip-S port)
499 (skip-while ssax:S-chars port))
500
501
502 ; Read a Name lexem and return it as string
503 ; [4] NameChar ::= Letter | Digit | '.' | '-' | '_' | ':'
504 ; | CombiningChar | Extender
505 ; [5] Name ::= (Letter | '_' | ':') (NameChar)*
506 ;
507 ; This code supports the XML Namespace Recommendation REC-xml-names,
508 ; which modifies the above productions as follows:
509 ;
510 ; [4] NCNameChar ::= Letter | Digit | '.' | '-' | '_'
511 ; | CombiningChar | Extender
512 ; [5] NCName ::= (Letter | '_') (NCNameChar)*
513 ; As the Rec-xml-names says,
514 ; "An XML document conforms to this specification if all other tokens
515 ; [other than element types and attribute names] in the document which
516 ; are required, for XML conformance, to match the XML production for
517 ; Name, match this specification's production for NCName."
518 ; Element types and attribute names must match the production QName,
519 ; defined below.
520
521 ; Check to see if a-char may start a NCName
522 (define (ssax:ncname-starting-char? a-char)
523 (and (char? a-char)
524 (or
525 (char-alphabetic? a-char)
526 (char=? #\_ a-char))))
527
528
529 ; Read a NCName starting from the current position in the PORT and
530 ; return it as a symbol.
531 (define (ssax:read-NCName port)
532 (let ((first-char (peek-char port)))
533 (or (ssax:ncname-starting-char? first-char)
534 (parser-error port "XMLNS [4] for '" first-char "'")))
535 (string->symbol
536 (next-token-of
537 (lambda (c)
538 (cond
539 ((eof-object? c) #f)
540 ((char-alphabetic? c) c)
541 ((string-index "0123456789.-_" c) c)
542 (else #f)))
543 port)))
544
545 ; Read a (namespace-) Qualified Name, QName, from the current
546 ; position in the PORT.
547 ; From REC-xml-names:
548 ; [6] QName ::= (Prefix ':')? LocalPart
549 ; [7] Prefix ::= NCName
550 ; [8] LocalPart ::= NCName
551 ; Return: an UNRES-NAME
552 (define (ssax:read-QName port)
553 (let ((prefix-or-localpart (ssax:read-NCName port)))
554 (case (peek-char port)
555 ((#\:) ; prefix was given after all
556 (read-char port) ; consume the colon
557 (cons prefix-or-localpart (ssax:read-NCName port)))
558 (else prefix-or-localpart) ; Prefix was omitted
559 )))
560
561 ; The prefix of the pre-defined XML namespace
562 (define ssax:Prefix-XML (string->symbol "xml"))
563
564 (run-test
565 (assert (eq? '_
566 (call-with-input-string "_" ssax:read-NCName)))
567 (assert (eq? '_
568 (call-with-input-string "_" ssax:read-QName)))
569 (assert (eq? (string->symbol "_abc_")
570 (call-with-input-string "_abc_;" ssax:read-NCName)))
571 (assert (eq? (string->symbol "_abc_")
572 (call-with-input-string "_abc_;" ssax:read-QName)))
573 (assert (eq? (string->symbol "_a.b")
574 (call-with-input-string "_a.b " ssax:read-QName)))
575 (assert (equal? (cons (string->symbol "_a.b") (string->symbol "d.1-ef-"))
576 (call-with-input-string "_a.b:d.1-ef-;" ssax:read-QName)))
577 (assert (equal? (cons (string->symbol "a") (string->symbol "b"))
578 (call-with-input-string "a:b:c" ssax:read-QName)))
579
580 (assert (failed? (call-with-input-string ":abc" ssax:read-NCName)))
581 (assert (failed? (call-with-input-string "1:bc" ssax:read-NCName)))
582 )
583
584 ; Compare one RES-NAME or an UNRES-NAME with the other.
585 ; Return a symbol '<, '>, or '= depending on the result of
586 ; the comparison.
587 ; Names without PREFIX are always smaller than those with the PREFIX.
588 (define name-compare
589 (letrec ((symbol-compare
590 (lambda (symb1 symb2)
591 (cond
592 ((eq? symb1 symb2) '=)
593 ((string<? (symbol->string symb1) (symbol->string symb2))
594 '<)
595 (else '>)))))
596 (lambda (name1 name2)
597 (cond
598 ((symbol? name1) (if (symbol? name2) (symbol-compare name1 name2)
599 '<))
600 ((symbol? name2) '>)
601 ((eq? name2 ssax:largest-unres-name) '<)
602 ((eq? name1 ssax:largest-unres-name) '>)
603 ((eq? (car name1) (car name2)) ; prefixes the same
604 (symbol-compare (cdr name1) (cdr name2)))
605 (else (symbol-compare (car name1) (car name2)))))))
606
607 ; An UNRES-NAME that is postulated to be larger than anything that can occur in
608 ; a well-formed XML document.
609 ; name-compare enforces this postulate.
610 (define ssax:largest-unres-name (cons
611 (string->symbol "#LARGEST-SYMBOL")
612 (string->symbol "#LARGEST-SYMBOL")))
613
614 (run-test
615 (assert (eq? '= (name-compare 'ABC 'ABC)))
616 (assert (eq? '< (name-compare 'ABC 'ABCD)))
617 (assert (eq? '> (name-compare 'XB 'ABCD)))
618 (assert (eq? '> (name-compare '(HTML . PRE) 'PRE)))
619 (assert (eq? '< (name-compare 'HTML '(HTML . PRE))))
620 (assert (eq? '= (name-compare '(HTML . PRE) '(HTML . PRE))))
621 (assert (eq? '< (name-compare '(HTML . PRE) '(XML . PRE))))
622 (assert (eq? '> (name-compare '(HTML . PRE) '(HTML . P))))
623 (assert (eq? '< (name-compare '(HTML . PRE) ssax:largest-unres-name)))
624 (assert (eq? '< (name-compare '(ZZZZ . ZZZ) ssax:largest-unres-name)))
625 (assert (eq? '> (name-compare ssax:largest-unres-name '(ZZZZ . ZZZ) )))
626 )
627
628
629
630 ; procedure: ssax:read-markup-token PORT
631 ; This procedure starts parsing of a markup token. The current position
632 ; in the stream must be #\<. This procedure scans enough of the input stream
633 ; to figure out what kind of a markup token it is seeing. The procedure returns
634 ; an xml-token structure describing the token. Note, generally reading
635 ; of the current markup is not finished! In particular, no attributes of
636 ; the start-tag token are scanned.
637 ;
638 ; Here's a detailed break out of the return values and the position in the PORT
639 ; when that particular value is returned:
640 ; PI-token: only PI-target is read.
641 ; To finish the Processing Instruction and disregard it,
642 ; call ssax:skip-pi. ssax:read-attributes may be useful
643 ; as well (for PIs whose content is attribute-value
644 ; pairs)
645 ; END-token: The end tag is read completely; the current position
646 ; is right after the terminating #\> character.
647 ; COMMENT is read and skipped completely. The current position
648 ; is right after "-->" that terminates the comment.
649 ; CDSECT The current position is right after "<!CDATA["
650 ; Use ssax:read-cdata-body to read the rest.
651 ; DECL We have read the keyword (the one that follows "<!")
652 ; identifying this declaration markup. The current
653 ; position is after the keyword (usually a
654 ; whitespace character)
655 ;
656 ; START-token We have read the keyword (GI) of this start tag.
657 ; No attributes are scanned yet. We don't know if this
658 ; tag has an empty content either.
659 ; Use ssax:complete-start-tag to finish parsing of
660 ; the token.
661
662 (define ssax:read-markup-token ; procedure ssax:read-markup-token port
663 (let ()
664 ; we have read "<!-". Skip through the rest of the comment
665 ; Return the 'COMMENT token as an indication we saw a comment
666 ; and skipped it.
667 (define (skip-comment port)
668 (assert-curr-char '(#\-) "XML [15], second dash" port)
669 (if (not (find-string-from-port? "-->" port))
670 (parser-error port "XML [15], no -->"))
671 (make-xml-token 'COMMENT #f))
672
673 ; we have read "<![" that must begin a CDATA section
674 (define (read-cdata port)
675 (assert (string=? "CDATA[" (read-string 6 port)))
676 (make-xml-token 'CDSECT #f))
677
678 (lambda (port)
679 (assert-curr-char '(#\<) "start of the token" port)
680 (case (peek-char port)
681 ((#\/) (read-char port)
682 (begin0 (make-xml-token 'END (ssax:read-QName port))
683 (ssax:skip-S port)
684 (assert-curr-char '(#\>) "XML [42]" port)))
685 ((#\?) (read-char port) (make-xml-token 'PI (ssax:read-NCName port)))
686 ((#\!)
687 (case (peek-next-char port)
688 ((#\-) (read-char port) (skip-comment port))
689 ((#\[) (read-char port) (read-cdata port))
690 (else (make-xml-token 'DECL (ssax:read-NCName port)))))
691 (else (make-xml-token 'START (ssax:read-QName port)))))
692 ))
693
694
695 ; The current position is inside a PI. Skip till the rest of the PI
696 (define (ssax:skip-pi port)
697 (if (not (find-string-from-port? "?>" port))
698 (parser-error port "Failed to find ?> terminating the PI")))
699
700
701 ; The current position is right after reading the PITarget. We read the
702 ; body of PI and return is as a string. The port will point to the
703 ; character right sfter '?>' combination that terminates PI.
704 ; [16] PI ::= '<?' PITarget (S (Char* - (Char* '?>' Char*)))? '?>'
705
706 (define (ssax:read-pi-body-as-string port)
707 (ssax:skip-S port) ; skip WS after the PI target name
708 (string-concatenate/shared
709 (let loop ()
710 (let ((pi-fragment
711 (next-token '() '(#\?) "reading PI content" port)))
712 (if (eqv? #\> (peek-next-char port))
713 (begin
714 (read-char port)
715 (cons pi-fragment '()))
716 (cons* pi-fragment "?" (loop)))))))
717
718 (run-test
719 (assert (equal? "p1 content "
720 (call-with-input-string "<?pi1 p1 content ?>"
721 (lambda (port)
722 (ssax:read-markup-token port)
723 (ssax:read-pi-body-as-string port)))))
724 (assert (equal? "pi2? content? ?"
725 (call-with-input-string "<?pi2 pi2? content? ??>"
726 (lambda (port)
727 (ssax:read-markup-token port)
728 (ssax:read-pi-body-as-string port)))))
729 )
730
731 ;(define (ssax:read-pi-body-as-name-values port)
732
733 ; The current pos in the port is inside an internal DTD subset
734 ; (e.g., after reading #\[ that begins an internal DTD subset)
735 ; Skip until the "]>" combination that terminates this DTD
736 (define (ssax:skip-internal-dtd port)
737 (if (not (find-string-from-port? "]>" port))
738 (parser-error port
739 "Failed to find ]> terminating the internal DTD subset")))
740
741
742 ; procedure+: ssax:read-cdata-body PORT STR-HANDLER SEED
743 ;
744 ; This procedure must be called after we have read a string "<![CDATA["
745 ; that begins a CDATA section. The current position must be the first
746 ; position of the CDATA body. This function reads _lines_ of the CDATA
747 ; body and passes them to a STR-HANDLER, a character data consumer.
748 ;
749 ; The str-handler is a STR-HANDLER, a procedure STRING1 STRING2 SEED.
750 ; The first STRING1 argument to STR-HANDLER never contains a newline.
751 ; The second STRING2 argument often will. On the first invocation of
752 ; the STR-HANDLER, the seed is the one passed to ssax:read-cdata-body
753 ; as the third argument. The result of this first invocation will be
754 ; passed as the seed argument to the second invocation of the line
755 ; consumer, and so on. The result of the last invocation of the
756 ; STR-HANDLER is returned by the ssax:read-cdata-body. Note a
757 ; similarity to the fundamental 'fold' iterator.
758 ;
759 ; Within a CDATA section all characters are taken at their face value,
760 ; with only three exceptions:
761 ; CR, LF, and CRLF are treated as line delimiters, and passed
762 ; as a single #\newline to the STR-HANDLER
763 ; "]]>" combination is the end of the CDATA section.
764 ; &gt; is treated as an embedded #\> character
765 ; Note, &lt; and &amp; are not specially recognized (and are not expanded)!
766
767 (define ssax:read-cdata-body
768 (let ((cdata-delimiters (list char-return #\newline #\] #\&)))
769
770 (lambda (port str-handler seed)
771 (let loop ((seed seed))
772 (let ((fragment (next-token '() cdata-delimiters
773 "reading CDATA" port)))
774 ; that is, we're reading the char after the 'fragment'
775 (case (read-char port)
776 ((#\newline) (loop (str-handler fragment nl seed)))
777 ((#\])
778 (if (not (eqv? (peek-char port) #\]))
779 (loop (str-handler fragment "]" seed))
780 (let check-after-second-braket
781 ((seed (if (string-null? fragment) seed
782 (str-handler fragment "" seed))))
783 (case (peek-next-char port) ; after the second bracket
784 ((#\>) (read-char port) seed) ; we have read "]]>"
785 ((#\]) (check-after-second-braket
786 (str-handler "]" "" seed)))
787 (else (loop (str-handler "]]" "" seed)))))))
788 ((#\&) ; Note that #\& within CDATA may stand for itself
789 (let ((ent-ref ; it does not have to start an entity ref
790 (next-token-of (lambda (c)
791 (and (not (eof-object? c)) (char-alphabetic? c) c)) port)))
792 (cond ; "&gt;" is to be replaced with #\>
793 ((and (string=? "gt" ent-ref) (eqv? (peek-char port) #\;))
794 (read-char port)
795 (loop (str-handler fragment ">" seed)))
796 (else
797 (loop
798 (str-handler ent-ref ""
799 (str-handler fragment "&" seed)))))))
800 (else ; Must be CR: if the next char is #\newline, skip it
801 (if (eqv? (peek-char port) #\newline) (read-char port))
802 (loop (str-handler fragment nl seed)))
803 ))))))
804
805 ; a few lines of validation code
806 (run-test (letrec
807 ((consumer (lambda (fragment foll-fragment seed)
808 (cons* (if (equal? foll-fragment (string #\newline))
809 " NL" foll-fragment) fragment seed)))
810 (test (lambda (str expected-result)
811 (newline) (display "body: ") (write str)
812 (newline) (display "Result: ")
813 (let ((result
814 (reverse
815 (call-with-input-string (unesc-string str)
816 (lambda (port) (ssax:read-cdata-body port consumer '()))
817 ))))
818 (write result)
819 (assert (equal? result expected-result)))))
820 )
821 (test "]]>" '())
822 (test "abcd]]>" '("abcd" ""))
823 (test "abcd]]]>" '("abcd" "" "]" ""))
824 (test "abcd]]]]>" '("abcd" "" "]" "" "]" ""))
825 (test "abcd]]]]]>" '("abcd" "" "]" "" "]" "" "]" ""))
826 (test "abcd]]]a]]>" '("abcd" "" "]" "" "]]" "" "a" ""))
827 (test "abc%r%ndef%n]]>" '("abc" " NL" "def" " NL"))
828 (test "%r%n%r%n]]>" '("" " NL" "" " NL"))
829 (test "%r%n%r%na]]>" '("" " NL" "" " NL" "a" ""))
830 (test "%r%r%r%na]]>" '("" " NL" "" " NL" "" " NL" "a" ""))
831 (test "abc&!!!]]>" '("abc" "&" "" "" "!!!" ""))
832 (test "abc]]&gt;&gt&amp;]]]&gt;and]]>"
833 '("abc" "" "]]" "" "" ">" "" "&" "gt" "" "" "&" "amp" "" ";" "" "]" ""
834 "]]" "" "" ">" "and" ""))
835 ))
836
837
838 ; procedure+: ssax:read-char-ref PORT
839 ;
840 ; [66] CharRef ::= '&#' [0-9]+ ';'
841 ; | '&#x' [0-9a-fA-F]+ ';'
842 ;
843 ; This procedure must be called after we we have read "&#"
844 ; that introduces a char reference.
845 ; The procedure reads this reference and returns the corresponding char
846 ; The current position in PORT will be after ";" that terminates
847 ; the char reference
848 ; Faults detected:
849 ; WFC: XML-Spec.html#wf-Legalchar
850 ;
851 ; According to Secttion "4.1 Character and Entity References"
852 ; of the XML Recommendation:
853 ; "[Definition: A character reference refers to a specific character
854 ; in the ISO/IEC 10646 character set, for example one not directly
855 ; accessible from available input devices.]"
856 ; Therefore, we use a ucscode->char function to convert a character
857 ; code into the character -- *regardless* of the current character
858 ; encoding of the input stream.
859
860 (define (ssax:read-char-ref port)
861 (let* ((base
862 (cond ((eqv? (peek-char port) #\x) (read-char port) 16)
863 (else 10)))
864 (name (next-token '() '(#\;) "XML [66]" port))
865 (char-code (string->number name base)))
866 (read-char port) ; read the terminating #\; char
867 (if (integer? char-code) (ucscode->char char-code)
868 (parser-error port "[wf-Legalchar] broken for '" name "'"))))
869
870
871 ; procedure+: ssax:handle-parsed-entity PORT NAME ENTITIES
872 ; CONTENT-HANDLER STR-HANDLER SEED
873 ;
874 ; Expand and handle a parsed-entity reference
875 ; port - a PORT
876 ; name - the name of the parsed entity to expand, a symbol
877 ; entities - see ENTITIES
878 ; content-handler -- procedure PORT ENTITIES SEED
879 ; that is supposed to return a SEED
880 ; str-handler - a STR-HANDLER. It is called if the entity in question
881 ; turns out to be a pre-declared entity
882 ;
883 ; The result is the one returned by CONTENT-HANDLER or STR-HANDLER
884 ; Faults detected:
885 ; WFC: XML-Spec.html#wf-entdeclared
886 ; WFC: XML-Spec.html#norecursion
887
888 (define ssax:predefined-parsed-entities
889 `(
890 (,(string->symbol "amp") . "&")
891 (,(string->symbol "lt") . "<")
892 (,(string->symbol "gt") . ">")
893 (,(string->symbol "apos") . "'")
894 (,(string->symbol "quot") . "\"")))
895
896 (define (ssax:handle-parsed-entity port name entities
897 content-handler str-handler seed)
898 (cond ; First we check the list of the declared entities
899 ((assq name entities) =>
900 (lambda (decl-entity)
901 (let ((ent-body (cdr decl-entity)) ; mark the list to prevent recursion
902 (new-entities (cons (cons name #f) entities)))
903 (cond
904 ((string? ent-body)
905 (call-with-input-string ent-body
906 (lambda (port) (content-handler port new-entities seed))))
907 ((procedure? ent-body)
908 (let ((port (ent-body)))
909 (begin0
910 (content-handler port new-entities seed)
911 (close-input-port port))))
912 (else
913 (parser-error port "[norecursion] broken for " name))))))
914 ((assq name ssax:predefined-parsed-entities)
915 => (lambda (decl-entity)
916 (str-handler (cdr decl-entity) "" seed)))
917 (else (parser-error port "[wf-entdeclared] broken for " name))))
918
919
920
921 ; The ATTLIST Abstract Data Type
922 ; Currently is implemented as an assoc list sorted in the ascending
923 ; order of NAMES.
924
925 (define (make-empty-attlist) '())
926
927 ; Add a name-value pair to the existing attlist preserving the order
928 ; Return the new list, in the sorted ascending order.
929 ; Return #f if a pair with the same name already exists in the attlist
930
931 (define (attlist-add attlist name-value)
932 (if (null? attlist) (cons name-value attlist)
933 (case (name-compare (car name-value) (caar attlist))
934 ((=) #f)
935 ((<) (cons name-value attlist))
936 (else (cons (car attlist) (attlist-add (cdr attlist) name-value)))
937 )))
938
939 (define attlist-null? null?)
940
941 ; Given an non-null attlist, return a pair of values: the top and the rest
942 (define (attlist-remove-top attlist)
943 (values (car attlist) (cdr attlist)))
944
945 (define (attlist->alist attlist) attlist)
946 (define attlist-fold fold)
947
948 ; procedure+: ssax:read-attributes PORT ENTITIES
949 ;
950 ; This procedure reads and parses a production Attribute*
951 ; [41] Attribute ::= Name Eq AttValue
952 ; [10] AttValue ::= '"' ([^<&"] | Reference)* '"'
953 ; | "'" ([^<&'] | Reference)* "'"
954 ; [25] Eq ::= S? '=' S?
955 ;
956 ;
957 ; The procedure returns an ATTLIST, of Name (as UNRES-NAME), Value (as string)
958 ; pairs. The current character on the PORT is a non-whitespace character
959 ; that is not an ncname-starting character.
960 ;
961 ; Note the following rules to keep in mind when reading an 'AttValue'
962 ; "Before the value of an attribute is passed to the application
963 ; or checked for validity, the XML processor must normalize it as follows:
964 ; - a character reference is processed by appending the referenced
965 ; character to the attribute value
966 ; - an entity reference is processed by recursively processing the
967 ; replacement text of the entity [see ENTITIES]
968 ; [named entities amp lt gt quot apos are assumed pre-declared]
969 ; - a whitespace character (#x20, #xD, #xA, #x9) is processed by appending #x20
970 ; to the normalized value, except that only a single #x20 is appended for a
971 ; "#xD#xA" sequence that is part of an external parsed entity or the
972 ; literal entity value of an internal parsed entity
973 ; - other characters are processed by appending them to the normalized value "
974 ;
975 ;
976 ; Faults detected:
977 ; WFC: XML-Spec.html#CleanAttrVals
978 ; WFC: XML-Spec.html#uniqattspec
979
980 (define ssax:read-attributes ; ssax:read-attributes port entities
981 (let ((value-delimeters (append ssax:S-chars '(#\< #\&))))
982 ; Read the AttValue from the PORT up to the delimiter
983 ; (which can be a single or double-quote character,
984 ; or even a symbol *eof*)
985 ; 'prev-fragments' is the list of string fragments, accumulated
986 ; so far, in reverse order.
987 ; Return the list of fragments with newly read fragments
988 ; prepended.
989 (define (read-attrib-value delimiter port entities prev-fragments)
990 (let* ((new-fragments
991 (cons (next-token '() (cons delimiter value-delimeters)
992 "XML [10]" port)
993 prev-fragments))
994 (cterm (read-char port)))
995 (cond
996 ((or (eof-object? cterm) (eqv? cterm delimiter))
997 new-fragments)
998 ((eqv? cterm char-return) ; treat a CR and CRLF as a LF
999 (if (eqv? (peek-char port) #\newline) (read-char port))
1000 (read-attrib-value delimiter port entities
1001 (cons " " new-fragments)))
1002 ((memv cterm ssax:S-chars)
1003 (read-attrib-value delimiter port entities
1004 (cons " " new-fragments)))
1005 ((eqv? cterm #\&)
1006 (cond
1007 ((eqv? (peek-char port) #\#)
1008 (read-char port)
1009 (read-attrib-value delimiter port entities
1010 (cons (string (ssax:read-char-ref port)) new-fragments)))
1011 (else
1012 (read-attrib-value delimiter port entities
1013 (read-named-entity port entities new-fragments)))))
1014 (else (parser-error port "[CleanAttrVals] broken")))))
1015
1016 ; we have read "&" that introduces a named entity reference.
1017 ; read this reference and return the result of
1018 ; normalizing of the corresponding string
1019 ; (that is, read-attrib-value is applied to the replacement
1020 ; text of the entity)
1021 ; The current position will be after ";" that terminates
1022 ; the entity reference
1023 (define (read-named-entity port entities fragments)
1024 (let ((name (ssax:read-NCName port)))
1025 (assert-curr-char '(#\;) "XML [68]" port)
1026 (ssax:handle-parsed-entity port name entities
1027 (lambda (port entities fragments)
1028 (read-attrib-value '*eof* port entities fragments))
1029 (lambda (str1 str2 fragments)
1030 (if (equal? "" str2) (cons str1 fragments)
1031 (cons* str2 str1 fragments)))
1032 fragments)))
1033
1034 (lambda (port entities)
1035 (let loop ((attr-list (make-empty-attlist)))
1036 (if (not (ssax:ncname-starting-char? (ssax:skip-S port))) attr-list
1037 (let ((name (ssax:read-QName port)))
1038 (ssax:skip-S port)
1039 (assert-curr-char '(#\=) "XML [25]" port)
1040 (ssax:skip-S port)
1041 (let ((delimiter
1042 (assert-curr-char '(#\' #\" ) "XML [10]" port)))
1043 (loop
1044 (or (attlist-add attr-list
1045 (cons name
1046 (string-concatenate-reverse/shared
1047 (read-attrib-value delimiter port entities
1048 '()))))
1049 (parser-error port "[uniqattspec] broken for " name))))))))
1050 ))
1051
1052 ; a few lines of validation code
1053 (run-test (letrec
1054 ((test (lambda (str decl-entities expected-res)
1055 (newline) (display "input: ") (write str)
1056 (newline) (display "Result: ")
1057 (let ((result
1058 (call-with-input-string (unesc-string str)
1059 (lambda (port)
1060 (ssax:read-attributes port decl-entities)))))
1061 (write result) (newline)
1062 (assert (equal? result expected-res))))))
1063 (test "" '() '())
1064 (test "href='http://a%tb%r%n%r%n%nc'" '()
1065 `((,(string->symbol "href") . "http://a b c")))
1066 (test "href='http://a%tb%r%r%n%rc'" '()
1067 `((,(string->symbol "href") . "http://a b c")))
1068 (test "_1 ='12&amp;' _2= \"%r%n%t12&#10;3\">" '()
1069 `((_1 . "12&") (_2 . ,(unesc-string " 12%n3"))))
1070 (test "%tAbc='&lt;&amp;&gt;&#x0A;'%nNext='12&ent;34' />"
1071 '((ent . "&lt;xx&gt;"))
1072 `((,(string->symbol "Abc") . ,(unesc-string "<&>%n"))
1073 (,(string->symbol "Next") . "12<xx>34")))
1074 (test "%tAbc='&lt;&amp;&gt;&#x0d;'%nNext='12&ent;34' />"
1075 '((ent . "&lt;xx&gt;"))
1076 `((,(string->symbol "Abc") . ,(unesc-string "<&>%r"))
1077 (,(string->symbol "Next") . "12<xx>34")))
1078 (test "%tAbc='&lt;&amp;&gt;&#x0A;'%nNext='12&en;34' />"
1079 `((en . ,(lambda () (open-input-string "&quot;xx&apos;"))))
1080 `((,(string->symbol "Abc") . ,(unesc-string "<&>%n"))
1081 (,(string->symbol "Next") . "12\"xx'34")))
1082 (test "%tAbc='&lt;&amp;&gt;&#x0A;'%nNext='12&ent;34' />"
1083 '((ent . "&lt;&ent1;T;&gt;") (ent1 . "&amp;"))
1084 `((,(string->symbol "Abc") . ,(unesc-string "<&>%n"))
1085 (,(string->symbol "Next") . "12<&T;>34")))
1086 (assert (failed?
1087 (test "%tAbc='&lt;&amp;&gt;&#x0A;'%nNext='12&ent;34' />"
1088 '((ent . "<&ent1;T;&gt;") (ent1 . "&amp;")) '())))
1089 (assert (failed?
1090 (test "%tAbc='&lt;&amp;&gt;&#x0A;'%nNext='12&ent;34' />"
1091 '((ent . "&lt;&ent;T;&gt;") (ent1 . "&amp;")) '())))
1092 (assert (failed?
1093 (test "%tAbc='&lt;&amp;&gt;&#x0A;'%nNext='12&ent;34' />"
1094 '((ent . "&lt;&ent1;T;&gt;") (ent1 . "&ent;")) '())))
1095 (test "html:href='http://a%tb%r%n%r%n%nc'" '()
1096 `(((,(string->symbol "html") . ,(string->symbol "href"))
1097 . "http://a b c")))
1098 (test "html:href='ref1' html:src='ref2'" '()
1099 `(((,(string->symbol "html") . ,(string->symbol "href"))
1100 . "ref1")
1101 ((,(string->symbol "html") . ,(string->symbol "src"))
1102 . "ref2")))
1103 (test "html:href='ref1' xml:html='ref2'" '()
1104 `(((,(string->symbol "html") . ,(string->symbol "href"))
1105 . "ref1")
1106 ((,ssax:Prefix-XML . ,(string->symbol "html"))
1107 . "ref2")))
1108 (assert (failed? (test "html:href='ref1' html:href='ref2'" '() '())))
1109 (assert (failed? (test "html:href='<' html:href='ref2'" '() '())))
1110 (assert (failed? (test "html:href='ref1' html:href='&ref2;'" '() '())))
1111 ))
1112
1113 ; ssax:resolve-name PORT UNRES-NAME NAMESPACES apply-default-ns?
1114 ;
1115 ; Convert an UNRES-NAME to a RES-NAME given the appropriate NAMESPACES
1116 ; declarations.
1117 ; the last parameter apply-default-ns? determines if the default
1118 ; namespace applies (for instance, it does not for attribute names)
1119 ;
1120 ; Per REC-xml-names/#nsc-NSDeclared, "xml" prefix is considered pre-declared
1121 ; and bound to the namespace name "http://www.w3.org/XML/1998/namespace".
1122 ;
1123 ; This procedure tests for the namespace constraints:
1124 ; http://www.w3.org/TR/REC-xml-names/#nsc-NSDeclared
1125
1126 (define (ssax:resolve-name port unres-name namespaces apply-default-ns?)
1127 (cond
1128 ((pair? unres-name) ; it's a QNAME
1129 (cons
1130 (cond
1131 ((assq (car unres-name) namespaces) => cadr)
1132 ((eq? (car unres-name) ssax:Prefix-XML) ssax:Prefix-XML)
1133 (else
1134 (parser-error port "[nsc-NSDeclared] broken; prefix " (car unres-name))))
1135 (cdr unres-name)))
1136 (apply-default-ns? ; Do apply the default namespace, if any
1137 (let ((default-ns (assq '*DEFAULT* namespaces)))
1138 (if (and default-ns (cadr default-ns))
1139 (cons (cadr default-ns) unres-name)
1140 unres-name))) ; no default namespace declared
1141 (else unres-name))) ; no prefix, don't apply the default-ns
1142
1143
1144 (run-test
1145 (let* ((namespaces
1146 '((HTML UHTML . URN-HTML)
1147 (HTML UHTML-1 . URN-HTML)
1148 (A UHTML . URN-HTML)))
1149 (namespaces-def
1150 (cons
1151 '(*DEFAULT* DEF . URN-DEF) namespaces))
1152 (namespaces-undef
1153 (cons
1154 '(*DEFAULT* #f . #f) namespaces-def))
1155 (port (current-input-port)))
1156
1157 (assert (equal? 'ABC
1158 (ssax:resolve-name port 'ABC namespaces #t)))
1159 (assert (equal? '(DEF . ABC)
1160 (ssax:resolve-name port 'ABC namespaces-def #t)))
1161 (assert (equal? 'ABC
1162 (ssax:resolve-name port 'ABC namespaces-def #f)))
1163 (assert (equal? 'ABC
1164 (ssax:resolve-name port 'ABC namespaces-undef #t)))
1165 (assert (equal? '(UHTML . ABC)
1166 (ssax:resolve-name port '(HTML . ABC) namespaces-def #t)))
1167 (assert (equal? '(UHTML . ABC)
1168 (ssax:resolve-name port '(HTML . ABC) namespaces-def #f)))
1169 (assert (equal? `(,ssax:Prefix-XML . space)
1170 (ssax:resolve-name port
1171 `(,(string->symbol "xml") . space) namespaces-def #f)))
1172 (assert (failed?
1173 (ssax:resolve-name port '(XXX . ABC) namespaces-def #f)))
1174 ))
1175
1176
1177 ; procedure+: ssax:uri-string->symbol URI-STR
1178 ; Convert a URI-STR to an appropriate symbol
1179 (define (ssax:uri-string->symbol uri-str)
1180 (string->symbol uri-str))
1181
1182 ; procedure+: ssax:complete-start-tag TAG PORT ELEMS ENTITIES NAMESPACES
1183 ;
1184 ; This procedure is to complete parsing of a start-tag markup. The
1185 ; procedure must be called after the start tag token has been
1186 ; read. TAG is an UNRES-NAME. ELEMS is an instance of xml-decl::elems;
1187 ; it can be #f to tell the function to do _no_ validation of elements
1188 ; and their attributes.
1189 ;
1190 ; This procedure returns several values:
1191 ; ELEM-GI: a RES-NAME.
1192 ; ATTRIBUTES: element's attributes, an ATTLIST of (RES-NAME . STRING)
1193 ; pairs. The list does NOT include xmlns attributes.
1194 ; NAMESPACES: the input list of namespaces amended with namespace
1195 ; (re-)declarations contained within the start-tag under parsing
1196 ; ELEM-CONTENT-MODEL
1197
1198 ; On exit, the current position in PORT will be the first character after
1199 ; #\> that terminates the start-tag markup.
1200 ;
1201 ; Faults detected:
1202 ; VC: XML-Spec.html#enum
1203 ; VC: XML-Spec.html#RequiredAttr
1204 ; VC: XML-Spec.html#FixedAttr
1205 ; VC: XML-Spec.html#ValueType
1206 ; WFC: XML-Spec.html#uniqattspec (after namespaces prefixes are resolved)
1207 ; VC: XML-Spec.html#elementvalid
1208 ; WFC: REC-xml-names/#dt-NSName
1209
1210 ; Note, although XML Recommendation does not explicitly say it,
1211 ; xmlns and xmlns: attributes don't have to be declared (although they
1212 ; can be declared, to specify their default value)
1213
1214 ; Procedure: ssax:complete-start-tag tag-head port elems entities namespaces
1215 (define ssax:complete-start-tag
1216
1217 (let ((xmlns (string->symbol "xmlns"))
1218 (largest-dummy-decl-attr (list ssax:largest-unres-name #f #f #f)))
1219
1220 ; Scan through the attlist and validate it, against decl-attrs
1221 ; Return an assoc list with added fixed or implied attrs.
1222 ; Note that both attlist and decl-attrs are ATTLISTs, and therefore,
1223 ; sorted
1224 (define (validate-attrs port attlist decl-attrs)
1225
1226 ; Check to see decl-attr is not of use type REQUIRED. Add
1227 ; the association with the default value, if any declared
1228 (define (add-default-decl decl-attr result)
1229 (let*-values
1230 (((attr-name content-type use-type default-value)
1231 (apply values decl-attr)))
1232 (and (eq? use-type 'REQUIRED)
1233 (parser-error port "[RequiredAttr] broken for" attr-name))
1234 (if default-value
1235 (cons (cons attr-name default-value) result)
1236 result)))
1237
1238 (let loop ((attlist attlist) (decl-attrs decl-attrs) (result '()))
1239 (if (attlist-null? attlist)
1240 (attlist-fold add-default-decl result decl-attrs)
1241 (let*-values
1242 (((attr attr-others)
1243 (attlist-remove-top attlist))
1244 ((decl-attr other-decls)
1245 (if (attlist-null? decl-attrs)
1246 (values largest-dummy-decl-attr decl-attrs)
1247 (attlist-remove-top decl-attrs)))
1248 )
1249 (case (name-compare (car attr) (car decl-attr))
1250 ((<)
1251 (if (or (eq? xmlns (car attr))
1252 (and (pair? (car attr)) (eq? xmlns (caar attr))))
1253 (loop attr-others decl-attrs (cons attr result))
1254 (parser-error port "[ValueType] broken for " attr)))
1255 ((>)
1256 (loop attlist other-decls
1257 (add-default-decl decl-attr result)))
1258 (else ; matched occurrence of an attr with its declaration
1259 (let*-values
1260 (((attr-name content-type use-type default-value)
1261 (apply values decl-attr)))
1262 ; Run some tests on the content of the attribute
1263 (cond
1264 ((eq? use-type 'FIXED)
1265 (or (equal? (cdr attr) default-value)
1266 (parser-error port "[FixedAttr] broken for " attr-name)))
1267 ((eq? content-type 'CDATA) #t) ; everything goes
1268 ((pair? content-type)
1269 (or (member (cdr attr) content-type)
1270 (parser-error port "[enum] broken for " attr-name "="
1271 (cdr attr))))
1272 (else
1273 (ssax:warn port "declared content type " content-type
1274 " not verified yet")))
1275 (loop attr-others other-decls (cons attr result)))))
1276 ))))
1277
1278
1279 ; Add a new namespace declaration to namespaces.
1280 ; First we convert the uri-str to a uri-symbol and search namespaces for
1281 ; an association (_ user-prefix . uri-symbol).
1282 ; If found, we return the argument namespaces with an association
1283 ; (prefix user-prefix . uri-symbol) prepended.
1284 ; Otherwise, we prepend (prefix uri-symbol . uri-symbol)
1285 (define (add-ns port prefix uri-str namespaces)
1286 (and (equal? "" uri-str)
1287 (parser-error port "[dt-NSName] broken for " prefix))
1288 (let ((uri-symbol (ssax:uri-string->symbol uri-str)))
1289 (let loop ((nss namespaces))
1290 (cond
1291 ((null? nss)
1292 (cons (cons* prefix uri-symbol uri-symbol) namespaces))
1293 ((eq? uri-symbol (cddar nss))
1294 (cons (cons* prefix (cadar nss) uri-symbol) namespaces))
1295 (else (loop (cdr nss)))))))
1296
1297 ; partition attrs into proper attrs and new namespace declarations
1298 ; return two values: proper attrs and the updated namespace declarations
1299 (define (adjust-namespace-decl port attrs namespaces)
1300 (let loop ((attrs attrs) (proper-attrs '()) (namespaces namespaces))
1301 (cond
1302 ((null? attrs) (values proper-attrs namespaces))
1303 ((eq? xmlns (caar attrs)) ; re-decl of the default namespace
1304 (loop (cdr attrs) proper-attrs
1305 (if (equal? "" (cdar attrs)) ; un-decl of the default ns
1306 (cons (cons* '*DEFAULT* #f #f) namespaces)
1307 (add-ns port '*DEFAULT* (cdar attrs) namespaces))))
1308 ((and (pair? (caar attrs)) (eq? xmlns (caaar attrs)))
1309 (loop (cdr attrs) proper-attrs
1310 (add-ns port (cdaar attrs) (cdar attrs) namespaces)))
1311 (else
1312 (loop (cdr attrs) (cons (car attrs) proper-attrs) namespaces)))))
1313
1314 ; The body of the function
1315 (lambda (tag-head port elems entities namespaces)
1316 (let*-values
1317 (((attlist) (ssax:read-attributes port entities))
1318 ((empty-el-tag?)
1319 (begin
1320 (ssax:skip-S port)
1321 (and
1322 (eqv? #\/
1323 (assert-curr-char '(#\> #\/) "XML [40], XML [44], no '>'" port))
1324 (assert-curr-char '(#\>) "XML [44], no '>'" port))))
1325 ((elem-content decl-attrs) ; see xml-decl for their type
1326 (if elems ; elements declared: validate!
1327 (cond
1328 ((assoc tag-head elems) =>
1329 (lambda (decl-elem) ; of type xml-decl::decl-elem
1330 (values
1331 (if empty-el-tag? 'EMPTY-TAG (cadr decl-elem))
1332 (caddr decl-elem))))
1333 (else
1334 (parser-error port "[elementvalid] broken, no decl for " tag-head)))
1335 (values ; non-validating parsing
1336 (if empty-el-tag? 'EMPTY-TAG 'ANY)
1337 #f) ; no attributes declared
1338 ))
1339 ((merged-attrs) (if decl-attrs (validate-attrs port attlist decl-attrs)
1340 (attlist->alist attlist)))
1341 ((proper-attrs namespaces)
1342 (adjust-namespace-decl port merged-attrs namespaces))
1343 )
1344 ;(cerr "proper attrs: " proper-attrs nl)
1345 ; build the return value
1346 (values
1347 (ssax:resolve-name port tag-head namespaces #t)
1348 (fold-right
1349 (lambda (name-value attlist)
1350 (or
1351 (attlist-add attlist
1352 (cons (ssax:resolve-name port (car name-value) namespaces #f)
1353 (cdr name-value)))
1354 (parser-error port "[uniqattspec] after NS expansion broken for "
1355 name-value)))
1356 (make-empty-attlist)
1357 proper-attrs)
1358 namespaces
1359 elem-content)))))
1360
1361 (run-test
1362 (let* ((urn-a (string->symbol "urn:a"))
1363 (urn-b (string->symbol "urn:b"))
1364 (urn-html (string->symbol "http://w3c.org/html"))
1365 (namespaces
1366 `((#f '"UHTML" . ,urn-html)
1367 ('"A" '"UA" . ,urn-a)))
1368 (test
1369 (lambda (tag-head-name elems str)
1370 (call-with-input-string str
1371 (lambda (port)
1372 (call-with-values
1373 (lambda ()
1374 (ssax:complete-start-tag
1375 (call-with-input-string tag-head-name
1376 (lambda (port) (ssax:read-QName port)))
1377 port
1378 elems '() namespaces))
1379 list))))))
1380
1381 ; First test with no validation of elements
1382 ;(test "TAG1" #f "")
1383 (assert (equal? `('"TAG1" () ,namespaces ANY)
1384 (test "TAG1" #f ">")))
1385 (assert (equal? `('"TAG1" () ,namespaces EMPTY-TAG)
1386 (test "TAG1" #f "/>")))
1387 (assert (equal? `('"TAG1" (('"HREF" . "a")) ,namespaces EMPTY-TAG)
1388 (test "TAG1" #f "HREF='a'/>")))
1389 (assert (equal? `(('"UA" . '"TAG1") (('"HREF" . "a"))
1390 ,(cons `(*DEFAULT* '"UA" . ,urn-a) namespaces) ANY)
1391 (test "TAG1" #f "HREF='a' xmlns='urn:a'>")))
1392 (assert (equal? `('"TAG1" (('"HREF" . "a"))
1393 ,(cons '(*DEFAULT* #f . #f) namespaces) ANY)
1394 (test "TAG1" #f "HREF='a' xmlns=''>")))
1395 (assert (failed? (test "UA:TAG1" #f "HREF='a' xmlns=''/>")))
1396 (assert (equal? `(('"UA" . '"TAG1") ((('"UA" . '"HREF") . "a"))
1397 ,(cons '(*DEFAULT* #f . #f) namespaces) ANY)
1398 (test "A:TAG1" #f "A:HREF='a' xmlns=''>")))
1399 (assert (equal? `(('"UA" . '"TAG1") ((('"UA" . '"HREF") . "a"))
1400 ,(cons `(*DEFAULT* ,urn-b . ,urn-b) namespaces) ANY)
1401 (test "A:TAG1" #f "A:HREF='a' xmlns='urn:b'>")))
1402 (assert (failed? (test "B:TAG1" #f "A:HREF='a' xmlns:b=''/>")))
1403 (assert (equal? `((,urn-b . '"TAG1") ((('"UA" . '"HREF") . "a"))
1404 ,(cons `('"B" ,urn-b . ,urn-b) namespaces) ANY)
1405 (test "B:TAG1" #f "A:HREF='a' xmlns:B='urn:b'>")))
1406 (assert (equal? `((,urn-b . '"TAG1") ((('"UA" . '"HREF") . "a")
1407 ((,urn-b . '"SRC") . "b"))
1408 ,(cons `('"B" ,urn-b . ,urn-b) namespaces) ANY)
1409 (test "B:TAG1" #f
1410 "B:SRC='b' A:HREF='a' xmlns:B='urn:b'>")))
1411 (assert (equal? `((,urn-b . '"TAG1") ((('"UA" . '"HREF") . "a")
1412 ((,urn-b . '"HREF") . "b"))
1413 ,(cons `('"B" ,urn-b . ,urn-b) namespaces) ANY)
1414 (test "B:TAG1" #f
1415 "B:HREF=\"b\" A:HREF='a' xmlns:B='urn:b'>")))
1416 ; must be an error! Duplicate attr
1417 (assert (failed? (test "B:TAG1" #f
1418 "HREF=\"b\" HREF='a' xmlns:B='urn:a'/>")))
1419 ; must be an error! Duplicate attr after ns expansion
1420 (assert (failed? (test "B:TAG1" #f
1421 "B:HREF=\"b\" A:HREF='a' xmlns:B='urn:a'/>")))
1422 (assert (equal? `(('"UA" . '"TAG1") (('"HREF" . "a")
1423 (('"UA" . '"HREF") . "b"))
1424 ,(cons `(*DEFAULT* '"UA" . ,urn-a) namespaces) ANY)
1425 (test "TAG1" #f
1426 "A:HREF=\"b\" HREF='a' xmlns='urn:a'>")))
1427 (assert (equal? `('"TAG1" ((('"UHTML" . '"HREF") . "a")
1428 ((,urn-b . '"HREF") . "b"))
1429 ,(append `(
1430 ('"HTML" '"UHTML" . ,urn-html)
1431 ('"B" ,urn-b . ,urn-b))
1432 namespaces) ANY)
1433 (test "TAG1" #f
1434 "B:HREF=\"b\" xmlns:B='urn:b' xmlns:HTML='http://w3c.org/html' HTML:HREF='a' >")))
1435
1436 ; Now test the validating parsing
1437 ; No decl for tag1
1438 (assert (failed? (test "TAG1" '((TAG2 ANY ()))
1439 "B:HREF='b' xmlns:B='urn:b'>")))
1440 ; No decl for HREF elem
1441 (cond-expand
1442 ((not (or scm mit-scheme)) ; Regretfully, SCM treats '() as #f
1443 (assert (failed?
1444 (test "TAG1" '(('"TAG1" ANY ()))
1445 "B:HREF='b' xmlns:B='urn:b'>"))))
1446 (else #t))
1447 ; No decl for HREF elem
1448 (assert (failed?
1449 (test "TAG1" '(('"TAG1" ANY (('"HREF1" CDATA IMPLIED #f))))
1450 "B:HREF='b' xmlns:B='urn:b'>")))
1451 (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces EMPTY-TAG)
1452 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f))))
1453 "HREF='b'/>")))
1454 (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA)
1455 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f))))
1456 "HREF='b'>")))
1457 ; Req'd attribute not given error
1458 (assert (failed?
1459 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f))))
1460 ">")))
1461 ; Wrong content-type of the attribute
1462 (assert (failed?
1463 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" ("c") REQUIRED #f))))
1464 "HREF='b'>")))
1465 (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA)
1466 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" ("c" "b") IMPLIED #f))))
1467 "HREF='b'>")))
1468 (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA)
1469 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA IMPLIED "c"))))
1470 "HREF='b'>")))
1471 ; Bad fixed attribute
1472 (assert (failed?
1473 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA FIXED "c"))))
1474 "HREF='b'>")))
1475 (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA)
1476 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA FIXED "b"))))
1477 "HREF='b'>")))
1478 (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA)
1479 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA FIXED "b")))) ">")))
1480 (assert (equal? `('"TAG1" (('"HREF" . "b")) ,namespaces PCDATA)
1481 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA IMPLIED "b")))) ">")))
1482 (assert (equal? `('"TAG1" () ,namespaces PCDATA)
1483 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA IMPLIED #f)))) ">")))
1484 ; Undeclared attr
1485 (assert (failed?
1486 (test "TAG1"
1487 '(('"TAG1" PCDATA ((('"A" . '"HREF") CDATA IMPLIED "c"))))
1488 "HREF='b'>")))
1489 (assert (equal? `('"TAG1" (('"HREF" . "b") (('"UA" . '"HREF") . "c"))
1490 ,namespaces PCDATA)
1491 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f)
1492 (('"A" . '"HREF") CDATA IMPLIED "c"))))
1493 "HREF='b'>")))
1494 (assert (equal? `(('"UA" . '"TAG1")
1495 (('"HREF" . "b") (('"UA" . '"HREF") . "c"))
1496 ,namespaces PCDATA)
1497 (test "A:TAG1" '((('"A" . '"TAG1") PCDATA
1498 (('"HREF" NMTOKEN REQUIRED #f)
1499 (('"A" . '"HREF") CDATA IMPLIED "c"))))
1500 "HREF='b'>")))
1501 (assert (equal? `((,urn-b . '"TAG1") (('"HREF" . "b"))
1502 ,(cons `('"B" ,urn-b . ,urn-b) namespaces) PCDATA)
1503 (test "B:TAG1" '((('"B" . '"TAG1") PCDATA (('"HREF" CDATA REQUIRED #f)
1504 (('"xmlns" . '"B") CDATA IMPLIED "urn:b"))))
1505 "HREF='b'>")))
1506 (assert (equal? `((,urn-b . '"TAG1") (((,urn-b . '"HREF") . "b"))
1507 ,(cons `('"B" ,urn-b . ,urn-b) namespaces) PCDATA)
1508 (test "B:TAG1" '((('"B" . '"TAG1") PCDATA
1509 ((('"B" . '"HREF") CDATA REQUIRED #f)
1510 (('"xmlns" . '"B") CDATA IMPLIED "urn:b"))))
1511 "B:HREF='b'>")))
1512 (assert (equal? `((,urn-b . '"TAG1") (('"HREF" . "b"))
1513 ,(cons `(*DEFAULT* ,urn-b . ,urn-b) namespaces) PCDATA)
1514 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f)
1515 ('"xmlns" CDATA IMPLIED "urn:b"))))
1516 "HREF='b'>")))
1517 ; xmlns not declared
1518 (assert (equal? `((,urn-b . '"TAG1") (('"HREF" . "b"))
1519 ,(cons `(*DEFAULT* ,urn-b . ,urn-b) namespaces) PCDATA)
1520 (test "TAG1" '(('"TAG1" PCDATA (('"HREF" CDATA REQUIRED #f)
1521 )))
1522 "HREF='b' xmlns='urn:b'>")))
1523 ; xmlns:B not declared
1524 (assert (equal? `((,urn-b . '"TAG1") (((,urn-b . '"HREF") . "b"))
1525 ,(cons `('"B" ,urn-b . ,urn-b) namespaces) PCDATA)
1526 (test "B:TAG1" '((('"B" . '"TAG1") PCDATA
1527 ((('"B" . '"HREF") CDATA REQUIRED #f)
1528 )))
1529 "B:HREF='b' xmlns:B='urn:b'>")))
1530 ))
1531
1532 ; procedure+: ssax:read-external-id PORT
1533 ;
1534 ; This procedure parses an ExternalID production:
1535 ; [75] ExternalID ::= 'SYSTEM' S SystemLiteral
1536 ; | 'PUBLIC' S PubidLiteral S SystemLiteral
1537 ; [11] SystemLiteral ::= ('"' [^"]* '"') | ("'" [^']* "'")
1538 ; [12] PubidLiteral ::= '"' PubidChar* '"' | "'" (PubidChar - "'")* "'"
1539 ; [13] PubidChar ::= #x20 | #xD | #xA | [a-zA-Z0-9]
1540 ; | [-'()+,./:=?;!*#@$_%]
1541 ;
1542 ; This procedure is supposed to be called when an ExternalID is expected;
1543 ; that is, the current character must be either #\S or #\P that start
1544 ; correspondingly a SYSTEM or PUBLIC token. This procedure returns the
1545 ; SystemLiteral as a string. A PubidLiteral is disregarded if present.
1546
1547 (define (ssax:read-external-id port)
1548 (let ((discriminator (ssax:read-NCName port)))
1549 (assert-curr-char ssax:S-chars "space after SYSTEM or PUBLIC" port)
1550 (ssax:skip-S port)
1551 (let ((delimiter
1552 (assert-curr-char '(#\' #\" ) "XML [11], XML [12]" port)))
1553 (cond
1554 ((eq? discriminator (string->symbol "SYSTEM"))
1555 (begin0
1556 (next-token '() (list delimiter) "XML [11]" port)
1557 (read-char port) ; reading the closing delim
1558 ))
1559 ((eq? discriminator (string->symbol "PUBLIC"))
1560 (skip-until (list delimiter) port)
1561 (assert-curr-char ssax:S-chars "space after PubidLiteral" port)
1562 (ssax:skip-S port)
1563 (let* ((delimiter
1564 (assert-curr-char '(#\' #\" ) "XML [11]" port))
1565 (systemid
1566 (next-token '() (list delimiter) "XML [11]" port)))
1567 (read-char port) ; reading the closing delim
1568 systemid))
1569 (else
1570 (parser-error port "XML [75], " discriminator
1571 " rather than SYSTEM or PUBLIC"))))))
1572
1573
1574 ;-----------------------------------------------------------------------------
1575 ; Higher-level parsers and scanners
1576 ;
1577 ; They parse productions corresponding to the whole (document) entity
1578 ; or its higher-level pieces (prolog, root element, etc).
1579
1580
1581 ; Scan the Misc production in the context
1582 ; [1] document ::= prolog element Misc*
1583 ; [22] prolog ::= XMLDecl? Misc* (doctypedec l Misc*)?
1584 ; [27] Misc ::= Comment | PI | S
1585 ;
1586 ; The following function should be called in the prolog or epilog contexts.
1587 ; In these contexts, whitespaces are completely ignored.
1588 ; The return value from ssax:scan-Misc is either a PI-token,
1589 ; a DECL-token, a START token, or EOF.
1590 ; Comments are ignored and not reported.
1591
1592 (define (ssax:scan-Misc port)
1593 (let loop ((c (ssax:skip-S port)))
1594 (cond
1595 ((eof-object? c) c)
1596 ((not (char=? c #\<))
1597 (parser-error port "XML [22], char '" c "' unexpected"))
1598 (else
1599 (let ((token (ssax:read-markup-token port)))
1600 (case (xml-token-kind token)
1601 ((COMMENT) (loop (ssax:skip-S port)))
1602 ((PI DECL START) token)
1603 (else
1604 (parser-error port "XML [22], unexpected token of kind "
1605 (xml-token-kind token)
1606 ))))))))
1607
1608 ; procedure+: ssax:read-char-data PORT EXPECT-EOF? STR-HANDLER SEED
1609 ;
1610 ; This procedure is to read the character content of an XML document
1611 ; or an XML element.
1612 ; [43] content ::=
1613 ; (element | CharData | Reference | CDSect | PI
1614 ; | Comment)*
1615 ; To be more precise, the procedure reads CharData, expands CDSect
1616 ; and character entities, and skips comments. The procedure stops
1617 ; at a named reference, EOF, at the beginning of a PI or a start/end tag.
1618 ;
1619 ; port
1620 ; a PORT to read
1621 ; expect-eof?
1622 ; a boolean indicating if EOF is normal, i.e., the character
1623 ; data may be terminated by the EOF. EOF is normal
1624 ; while processing a parsed entity.
1625 ; str-handler
1626 ; a STR-HANDLER
1627 ; seed
1628 ; an argument passed to the first invocation of STR-HANDLER.
1629 ;
1630 ; The procedure returns two results: SEED and TOKEN.
1631 ; The SEED is the result of the last invocation of STR-HANDLER, or the
1632 ; original seed if STR-HANDLER was never called.
1633 ;
1634 ; TOKEN can be either an eof-object (this can happen only if
1635 ; expect-eof? was #t), or:
1636 ; - an xml-token describing a START tag or an END-tag;
1637 ; For a start token, the caller has to finish reading it.
1638 ; - an xml-token describing the beginning of a PI. It's up to an
1639 ; application to read or skip through the rest of this PI;
1640 ; - an xml-token describing a named entity reference.
1641 ;
1642 ; CDATA sections and character references are expanded inline and
1643 ; never returned. Comments are silently disregarded.
1644 ;
1645 ; As the XML Recommendation requires, all whitespace in character data
1646 ; must be preserved. However, a CR character (#xD) must be disregarded
1647 ; if it appears before a LF character (#xA), or replaced by a #xA character
1648 ; otherwise. See Secs. 2.10 and 2.11 of the XML Recommendation. See also
1649 ; the canonical XML Recommendation.
1650
1651 ; ssax:read-char-data port expect-eof? str-handler seed
1652 (define ssax:read-char-data
1653 (let
1654 ((terminators-usual (list #\< #\& char-return))
1655 (terminators-usual-eof (list #\< '*eof* #\& char-return))
1656
1657 (handle-fragment
1658 (lambda (fragment str-handler seed)
1659 (if (string-null? fragment) seed
1660 (str-handler fragment "" seed))))
1661 )
1662
1663 (lambda (port expect-eof? str-handler seed)
1664
1665 ; Very often, the first character we encounter is #\<
1666 ; Therefore, we handle this case in a special, fast path
1667 (if (eqv? #\< (peek-char port))
1668
1669 ; The fast path
1670 (let ((token (ssax:read-markup-token port)))
1671 (case (xml-token-kind token)
1672 ((START END) ; The most common case
1673 (values seed token))
1674 ((CDSECT)
1675 (let ((seed (ssax:read-cdata-body port str-handler seed)))
1676 (ssax:read-char-data port expect-eof? str-handler seed)))
1677 ((COMMENT) (ssax:read-char-data port expect-eof?
1678 str-handler seed))
1679 (else
1680 (values seed token))))
1681
1682
1683 ; The slow path
1684 (let ((char-data-terminators
1685 (if expect-eof? terminators-usual-eof terminators-usual)))
1686
1687 (let loop ((seed seed))
1688 (let* ((fragment
1689 (next-token '() char-data-terminators
1690 "reading char data" port))
1691 (term-char (peek-char port)) ; one of char-data-terminators
1692 )
1693 (if (eof-object? term-char)
1694 (values
1695 (handle-fragment fragment str-handler seed)
1696 term-char)
1697 (case term-char
1698 ((#\<)
1699 (let ((token (ssax:read-markup-token port)))
1700 (case (xml-token-kind token)
1701 ((CDSECT)
1702 (loop
1703 (ssax:read-cdata-body port str-handler
1704 (handle-fragment fragment str-handler seed))))
1705 ((COMMENT)
1706 (loop (handle-fragment fragment str-handler seed)))
1707 (else
1708 (values
1709 (handle-fragment fragment str-handler seed)
1710 token)))))
1711 ((#\&)
1712 (case (peek-next-char port)
1713 ((#\#) (read-char port)
1714 (loop (str-handler fragment
1715 (string (ssax:read-char-ref port))
1716 seed)))
1717 (else
1718 (let ((name (ssax:read-NCName port)))
1719 (assert-curr-char '(#\;) "XML [68]" port)
1720 (values
1721 (handle-fragment fragment str-handler seed)
1722 (make-xml-token 'ENTITY-REF name))))))
1723 (else ; This must be a CR character
1724 (if (eqv? (peek-next-char port) #\newline)
1725 (read-char port))
1726 (loop (str-handler fragment (string #\newline) seed))))
1727 ))))))))
1728
1729
1730 ; a few lines of validation code
1731 (run-test (letrec
1732 ((a-tag (make-xml-token 'START (string->symbol "BR")))
1733 (a-ref (make-xml-token 'ENTITY-REF (string->symbol "lt")))
1734 (eof-object (with-input-from-string "" read))
1735 (str-handler (lambda (fragment foll-fragment seed)
1736 (if (string-null? foll-fragment) (cons fragment seed)
1737 (cons* foll-fragment fragment seed))))
1738 (test (lambda (str expect-eof? expected-data expected-token)
1739 (newline) (display "body: ") (write str)
1740 (newline) (display "Result: ")
1741 (let*-values
1742 (((seed token)
1743 (call-with-input-string (unesc-string str)
1744 (lambda (port)
1745 (ssax:read-char-data port expect-eof? str-handler '()))))
1746 ((result) (reverse seed)))
1747 (write result)
1748 (display " ")
1749 (display token)
1750 (assert (equal? result (map unesc-string expected-data))
1751 (equal? token expected-token)))))
1752 )
1753 (test "" #t '() eof-object)
1754 (assert (failed? (test "" #f '() eof-object)))
1755 (test " " #t '(" ") eof-object)
1756 (test "<BR/>" #f '() a-tag)
1757 (test " <BR />" #f '(" ") a-tag)
1758
1759 (test " &lt;" #f '(" ") a-ref)
1760 (test " a&lt;" #f '(" a") a-ref)
1761 (test " a &lt;" #f '(" a ") a-ref)
1762
1763 (test " <!-- comment--> a a<BR/>" #f '(" " " a a") a-tag)
1764 (test " <!-- comment-->%ra a<BR/>" #f '(" " "" "%n" "a a") a-tag)
1765 (test " <!-- comment-->%r%na a<BR/>" #f '(" " "" "%n" "a a") a-tag)
1766 (test " <!-- comment-->%r%na%t%r%r%na<BR/>" #f
1767 '(" " "" "%n" "a%t" "%n" "" "%n" "a") a-tag)
1768 (test "a<!-- comment--> a a<BR/>" #f '("a" " a a") a-tag)
1769 (test "&#x21;<BR/>" #f '("" "!") a-tag)
1770 (test "&#x21;%n<BR/>" #f '("" "!" "%n") a-tag)
1771 (test "%t&#x21;%n<BR/>" #f '("%t" "!" "%n") a-tag)
1772 (test "%t&#x21;%na a<BR/>" #f '("%t" "!" "%na a") a-tag)
1773 (test "%t&#x21;%ra a<BR/>" #f '("%t" "!" "" "%n" "a a") a-tag)
1774 (test "%t&#x21;%r%na a<BR/>" #f '("%t" "!" "" "%n" "a a") a-tag)
1775
1776 (test " %ta &#x21; b <BR/>" #f '(" %ta " "!" " b ") a-tag)
1777 (test " %ta &#x20; b <BR/>" #f '(" %ta " " " " b ") a-tag)
1778
1779 (test "<![CDATA[<]]><BR/>" #f '("<") a-tag)
1780 (test "<![CDATA[]]]><BR/>" #f '("]") a-tag)
1781 (test "%t<![CDATA[<]]><BR/>" #f '("%t" "<") a-tag)
1782 (test "%t<![CDATA[<]]>a b<BR/>" #f '("%t" "<" "a b") a-tag)
1783 (test "%t<![CDATA[<]]> a b<BR/>" #f '("%t" "<" " a b") a-tag)
1784
1785 (test "%td <![CDATA[ <%r%r%n]]> a b<BR/>" #f
1786 '("%td " " <" "%n" "" "%n" " a b") a-tag)
1787 ))
1788
1789
1790
1791 ; procedure+: ssax:assert-token TOKEN KIND GI
1792 ; Make sure that TOKEN is of anticipated KIND and has anticipated GI
1793 ; Note GI argument may actually be a pair of two symbols, Namespace
1794 ; URI or the prefix, and of the localname.
1795 ; If the assertion fails, error-cont is evaluated by passing it
1796 ; three arguments: token kind gi. The result of error-cont is returned.
1797 (define (ssax:assert-token token kind gi error-cont)
1798 (or
1799 (and (xml-token? token)
1800 (eq? kind (xml-token-kind token))
1801 (equal? gi (xml-token-head token)))
1802 (error-cont token kind gi)))
1803
1804 ;========================================================================
1805 ; Highest-level parsers: XML to SXML
1806
1807 ; These parsers are a set of syntactic forms to instantiate a SSAX parser.
1808 ; A user can instantiate the parser to do the full validation, or
1809 ; no validation, or any particular validation. The user specifies
1810 ; which PI he wants to be notified about. The user tells what to do
1811 ; with the parsed character and element data. The latter handlers
1812 ; determine if the parsing follows a SAX or a DOM model.
1813
1814 ; syntax: ssax:make-pi-parser my-pi-handlers
1815 ; Create a parser to parse and process one Processing Element (PI).
1816
1817 ; my-pi-handlers
1818 ; An assoc list of pairs (PI-TAG . PI-HANDLER)
1819 ; where PI-TAG is an NCName symbol, the PI target, and
1820 ; PI-HANDLER is a procedure PORT PI-TAG SEED
1821 ; where PORT points to the first symbol after the PI target.
1822 ; The handler should read the rest of the PI up to and including
1823 ; the combination '?>' that terminates the PI. The handler should
1824 ; return a new seed.
1825 ; One of the PI-TAGs may be a symbol *DEFAULT*. The corresponding
1826 ; handler will handle PIs that no other handler will. If the
1827 ; *DEFAULT* PI-TAG is not specified, ssax:make-pi-parser will make
1828 ; one, which skips the body of the PI
1829 ;
1830 ; The output of the ssax:make-pi-parser is a procedure
1831 ; PORT PI-TAG SEED
1832 ; that will parse the current PI accoding to user-specified handlers.
1833
1834 (define-macro ssax:make-pi-parser
1835 (lambda (my-pi-handlers)
1836 `(lambda (port target seed)
1837 (case target
1838 ; Generate the body of the case statement
1839 ,@(let loop ((pi-handlers my-pi-handlers) (default #f))
1840 (cond
1841 ((null? pi-handlers)
1842 (if default `((else (,default port target seed)))
1843 '((else
1844 (ssax:warn port "Skipping PI: " target nl)
1845 (ssax:skip-pi port)
1846 seed))))
1847 ((eq? '*DEFAULT* (caar pi-handlers))
1848 (loop (cdr pi-handlers) (cdar pi-handlers)))
1849 (else
1850 (cons
1851 `((,(caar pi-handlers)) (,(cdar pi-handlers) port target seed))
1852 (loop (cdr pi-handlers) default)))))))))
1853
1854 (run-test
1855 (pp (ssax:make-pi-parser ()))
1856 (pp (ssax:make-pi-parser ((xml . (lambda (port target seed) seed)))))
1857 (pp (ssax:make-pi-parser ((xml . (lambda (port target seed) seed))
1858 (html . list)
1859 (*DEFAULT* . ssax:warn))))
1860 )
1861
1862 ; syntax: ssax:make-elem-parser my-new-level-seed my-finish-element
1863 ; my-char-data-handler my-pi-handlers
1864
1865 ; Create a parser to parse and process one element, including its
1866 ; character content or children elements. The parser is typically
1867 ; applied to the root element of a document.
1868
1869 ; my-new-level-seed
1870 ; procedure ELEM-GI ATTRIBUTES NAMESPACES EXPECTED-CONTENT SEED
1871 ; where ELEM-GI is a RES-NAME of the element
1872 ; about to be processed.
1873 ; This procedure is to generate the seed to be passed
1874 ; to handlers that process the content of the element.
1875 ; This is the function identified as 'fdown' in the denotational
1876 ; semantics of the XML parser given in the title comments to this
1877 ; file.
1878 ;
1879 ; my-finish-element
1880 ; procedure ELEM-GI ATTRIBUTES NAMESPACES PARENT-SEED SEED
1881 ; This procedure is called when parsing of ELEM-GI is finished.
1882 ; The SEED is the result from the last content parser (or
1883 ; from my-new-level-seed if the element has the empty content).
1884 ; PARENT-SEED is the same seed as was passed to my-new-level-seed.
1885 ; The procedure is to generate a seed that will be the result
1886 ; of the element parser.
1887 ; This is the function identified as 'fup' in the denotational
1888 ; semantics of the XML parser given in the title comments to this
1889 ; file.
1890 ;
1891 ; my-char-data-handler
1892 ; A STR-HANDLER
1893 ;
1894 ; my-pi-handlers
1895 ; See ssax:make-pi-handler above
1896 ;
1897
1898 ; The generated parser is a
1899 ; procedure START-TAG-HEAD PORT ELEMS ENTITIES
1900 ; NAMESPACES PRESERVE-WS? SEED
1901 ; The procedure must be called after the start tag token has been
1902 ; read. START-TAG-HEAD is an UNRES-NAME from the start-element tag.
1903 ; ELEMS is an instance of xml-decl::elems.
1904 ; See ssax:complete-start-tag::preserve-ws?
1905
1906 ; Faults detected:
1907 ; VC: XML-Spec.html#elementvalid
1908 ; WFC: XML-Spec.html#GIMatch
1909
1910
1911 (define-macro ssax:make-elem-parser
1912 (lambda (my-new-level-seed my-finish-element
1913 my-char-data-handler my-pi-handlers)
1914
1915 `(lambda (start-tag-head port elems entities namespaces
1916 preserve-ws? seed)
1917
1918 (define xml-space-gi (cons ssax:Prefix-XML
1919 (string->symbol "space")))
1920
1921 (let handle-start-tag ((start-tag-head start-tag-head)
1922 (port port) (entities entities)
1923 (namespaces namespaces)
1924 (preserve-ws? preserve-ws?) (parent-seed seed))
1925 (let*-values
1926 (((elem-gi attributes namespaces expected-content)
1927 (ssax:complete-start-tag start-tag-head port elems
1928 entities namespaces))
1929 ((seed)
1930 (,my-new-level-seed elem-gi attributes
1931 namespaces expected-content parent-seed)))
1932 (case expected-content
1933 ((EMPTY-TAG)
1934 (,my-finish-element
1935 elem-gi attributes namespaces parent-seed seed))
1936 ((EMPTY) ; The end tag must immediately follow
1937 (ssax:assert-token
1938 (and (eqv? #\< (ssax:skip-S port)) (ssax:read-markup-token port))
1939 'END start-tag-head
1940 (lambda (token exp-kind exp-head)
1941 (parser-error port "[elementvalid] broken for " token
1942 " while expecting "
1943 exp-kind exp-head)))
1944 (,my-finish-element
1945 elem-gi attributes namespaces parent-seed seed))
1946 (else ; reading the content...
1947 (let ((preserve-ws? ; inherit or set the preserve-ws? flag
1948 (cond
1949 ((assoc xml-space-gi attributes) =>
1950 (lambda (name-value)
1951 (equal? "preserve" (cdr name-value))))
1952 (else preserve-ws?))))
1953 (let loop ((port port) (entities entities)
1954 (expect-eof? #f) (seed seed))
1955 (let*-values
1956 (((seed term-token)
1957 (ssax:read-char-data port expect-eof?
1958 ,my-char-data-handler seed)))
1959 (if (eof-object? term-token)
1960 seed
1961 (case (xml-token-kind term-token)
1962 ((END)
1963 (ssax:assert-token term-token 'END start-tag-head
1964 (lambda (token exp-kind exp-head)
1965 (parser-error port "[GIMatch] broken for "
1966 term-token " while expecting "
1967 exp-kind exp-head)))
1968 (,my-finish-element
1969 elem-gi attributes namespaces parent-seed seed))
1970 ((PI)
1971 (let ((seed
1972 ((ssax:make-pi-parser ,my-pi-handlers)
1973 port (xml-token-head term-token) seed)))
1974 (loop port entities expect-eof? seed)))
1975 ((ENTITY-REF)
1976 (let ((seed
1977 (ssax:handle-parsed-entity
1978 port (xml-token-head term-token)
1979 entities
1980 (lambda (port entities seed)
1981 (loop port entities #t seed))
1982 ,my-char-data-handler
1983 seed))) ; keep on reading the content after ent
1984 (loop port entities expect-eof? seed)))
1985 ((START) ; Start of a child element
1986 (if (eq? expected-content 'PCDATA)
1987 (parser-error port "[elementvalid] broken for "
1988 elem-gi
1989 " with char content only; unexpected token "
1990 term-token))
1991 ; Do other validation of the element content
1992 (let ((seed
1993 (handle-start-tag
1994 (xml-token-head term-token)
1995 port entities namespaces
1996 preserve-ws? seed)))
1997 (loop port entities expect-eof? seed)))
1998 (else
1999 (parser-error port "XML [43] broken for "
2000 term-token))))))))
2001 )))
2002 )))
2003
2004
2005 ; syntax: ssax:make-parser user-handler-tag user-handler-proc ...
2006 ;
2007 ; Create an XML parser, an instance of the XML parsing framework.
2008 ; This will be a SAX, a DOM, or a specialized parser depending
2009 ; on the supplied user-handlers.
2010
2011 ; user-handler-tag is a symbol that identifies a procedural expression
2012 ; that follows the tag. Given below are tags and signatures of the
2013 ; corresponding procedures. Not all tags have to be specified. If some
2014 ; are omitted, reasonable defaults will apply.
2015 ;
2016
2017 ; tag: DOCTYPE
2018 ; handler-procedure: PORT DOCNAME SYSTEMID INTERNAL-SUBSET? SEED
2019 ; If internal-subset? is #t, the current position in the port
2020 ; is right after we have read #\[ that begins the internal DTD subset.
2021 ; We must finish reading of this subset before we return
2022 ; (or must call skip-internal-subset if we aren't interested in reading it).
2023 ; The port at exit must be at the first symbol after the whole
2024 ; DOCTYPE declaration.
2025 ; The handler-procedure must generate four values:
2026 ; ELEMS ENTITIES NAMESPACES SEED
2027 ; See xml-decl::elems for ELEMS. It may be #f to switch off the validation.
2028 ; NAMESPACES will typically contain USER-PREFIXes for selected URI-SYMBs.
2029 ; The default handler-procedure skips the internal subset,
2030 ; if any, and returns (values #f '() '() seed)
2031
2032 ; tag: UNDECL-ROOT
2033 ; handler-procedure: ELEM-GI SEED
2034 ; where ELEM-GI is an UNRES-NAME of the root element. This procedure
2035 ; is called when an XML document under parsing contains _no_ DOCTYPE
2036 ; declaration.
2037 ; The handler-procedure, as a DOCTYPE handler procedure above,
2038 ; must generate four values:
2039 ; ELEMS ENTITIES NAMESPACES SEED
2040 ; The default handler-procedure returns (values #f '() '() seed)
2041
2042 ; tag: DECL-ROOT
2043 ; handler-procedure: ELEM-GI SEED
2044 ; where ELEM-GI is an UNRES-NAME of the root element. This procedure
2045 ; is called when an XML document under parsing does contains the DOCTYPE
2046 ; declaration.
2047 ; The handler-procedure must generate a new SEED (and verify
2048 ; that the name of the root element matches the doctype, if the handler
2049 ; so wishes).
2050 ; The default handler-procedure is the identity function.
2051
2052 ; tag: NEW-LEVEL-SEED
2053 ; handler-procedure: see ssax:make-elem-parser, my-new-level-seed
2054
2055 ; tag: FINISH-ELEMENT
2056 ; handler-procedure: see ssax:make-elem-parser, my-finish-element
2057
2058 ; tag: CHAR-DATA-HANDLER
2059 ; handler-procedure: see ssax:make-elem-parser, my-char-data-handler
2060
2061 ; tag: PI
2062 ; handler-procedure: see ssax:make-pi-parser
2063 ; The default value is '()
2064
2065 ; The generated parser is a
2066 ; procedure PORT SEED
2067
2068 ; This procedure parses the document prolog and then exits to
2069 ; an element parser (created by ssax:make-elem-parser) to handle
2070 ; the rest.
2071 ;
2072 ; [1] document ::= prolog element Misc*
2073 ; [22] prolog ::= XMLDecl? Misc* (doctypedec | Misc*)?
2074 ; [27] Misc ::= Comment | PI | S
2075 ;
2076 ; [28] doctypedecl ::= '<!DOCTYPE' S Name (S ExternalID)? S?
2077 ; ('[' (markupdecl | PEReference | S)* ']' S?)? '>'
2078 ; [29] markupdecl ::= elementdecl | AttlistDecl
2079 ; | EntityDecl
2080 ; | NotationDecl | PI
2081 ; | Comment
2082 ;
2083
2084
2085 (define-macro ssax:make-parser
2086 (lambda user-handlers
2087
2088 ; An assoc list of user-handler-tag and default handlers
2089 (define all-handlers
2090 '((DOCTYPE .
2091 (lambda (port docname systemid internal-subset? seed)
2092 (when internal-subset?
2093 (ssax:warn port "Internal DTD subset is not currently handled ")
2094 (ssax:skip-internal-dtd port))
2095 (ssax:warn port "DOCTYPE DECL " docname " "
2096 systemid " found and skipped")
2097 (values #f '() '() seed)
2098 ))
2099 (UNDECL-ROOT .
2100 (lambda (elem-gi seed) (values #f '() '() seed)))
2101 (DECL-ROOT .
2102 (lambda (elem-gi seed) seed))
2103 (NEW-LEVEL-SEED . REQD) ; required
2104 (FINISH-ELEMENT . REQD) ; required
2105 (CHAR-DATA-HANDLER . REQD) ; required
2106 (PI . ())
2107 ))
2108
2109 ; Delete an association with the tag from alist
2110 ; exit to cont, passing the tag, tag-association, and the list
2111 ; of remaining associations.
2112 ; It's an error if the association with the tag does not exist
2113 ; in alist
2114 (define (delete-assoc alist tag cont)
2115 (let loop ((alist alist) (scanned '()))
2116 (cond
2117 ((null? alist) (error "Unknown user-handler-tag: " tag))
2118 ((eq? tag (caar alist))
2119 (cont tag (cdar alist) (append scanned (cdr alist))))
2120 (else (loop (cdr alist) (cons (car alist) scanned))))))
2121
2122 ; create an assoc list of tags and handlers
2123 ; based on the defaults and on the given handlers
2124 (define (merge-handlers declared-handlers given-handlers)
2125 (cond
2126 ((null? given-handlers) ; the arguments are exhausted...
2127 (cond
2128 ((null? declared-handlers) '())
2129 ((not (eq? 'REQD (cdar declared-handlers))) ; default value was given:
2130 (cons (car declared-handlers) ; use it
2131 (merge-handlers (cdr declared-handlers) given-handlers)))
2132 (else (error "The handler for the tag " (caar declared-handlers)
2133 " must be specified"))))
2134 ((null? (cdr given-handlers))
2135 (error "Odd number of arguments to ssax:make-parser"))
2136 (else
2137 (delete-assoc declared-handlers (car given-handlers)
2138 (lambda (tag value alist)
2139 (cons (cons tag (cadr given-handlers))
2140 (merge-handlers alist (cddr given-handlers))))))))
2141
2142 (let ((user-handlers (merge-handlers all-handlers user-handlers)))
2143
2144 (define (get-handler tag)
2145 (cond
2146 ((assq tag user-handlers) => cdr)
2147 (else (error "unknown tag: " tag))))
2148
2149
2150 `(lambda (port seed)
2151
2152 ; We must've just scanned the DOCTYPE token
2153 ; Handle the doctype declaration and exit to
2154 ; scan-for-significant-prolog-token-2, and eventually, to the
2155 ; element parser.
2156 (define (handle-decl port token-head seed)
2157 (or (eq? (string->symbol "DOCTYPE") token-head)
2158 (parser-error port "XML [22], expected DOCTYPE declaration, found "
2159 token-head))
2160 (assert-curr-char ssax:S-chars "XML [28], space after DOCTYPE" port)
2161 (ssax:skip-S port)
2162 (let*-values
2163 (((docname) (ssax:read-QName port))
2164 ((systemid)
2165 (and (ssax:ncname-starting-char? (ssax:skip-S port))
2166 (ssax:read-external-id port)))
2167 ((internal-subset?)
2168 (begin (ssax:skip-S port)
2169 (eqv? #\[ (assert-curr-char '(#\> #\[)
2170 "XML [28], end-of-DOCTYPE" port))))
2171 ((elems entities namespaces seed)
2172 (,(get-handler 'DOCTYPE) port docname systemid
2173 internal-subset? seed))
2174 )
2175 (scan-for-significant-prolog-token-2 port elems entities namespaces
2176 seed)))
2177
2178
2179 ; Scan the leading PIs until we encounter either a doctype declaration
2180 ; or a start token (of the root element)
2181 ; In the latter two cases, we exit to the appropriate continuation
2182 (define (scan-for-significant-prolog-token-1 port seed)
2183 (let ((token (ssax:scan-Misc port)))
2184 (if (eof-object? token)
2185 (parser-error port "XML [22], unexpected EOF")
2186 (case (xml-token-kind token)
2187 ((PI)
2188 (let ((seed
2189 ((ssax:make-pi-parser ,(get-handler 'PI))
2190 port (xml-token-head token) seed)))
2191 (scan-for-significant-prolog-token-1 port seed)))
2192 ((DECL) (handle-decl port (xml-token-head token) seed))
2193 ((START)
2194 (let*-values
2195 (((elems entities namespaces seed)
2196 (,(get-handler 'UNDECL-ROOT) (xml-token-head token) seed)))
2197 (element-parser (xml-token-head token) port elems
2198 entities namespaces #f seed)))
2199 (else (parser-error port "XML [22], unexpected markup "
2200 token))))))
2201
2202
2203 ; Scan PIs after the doctype declaration, till we encounter
2204 ; the start tag of the root element. After that we exit
2205 ; to the element parser
2206 (define (scan-for-significant-prolog-token-2 port elems entities
2207 namespaces seed)
2208 (let ((token (ssax:scan-Misc port)))
2209 (if (eof-object? token)
2210 (parser-error port "XML [22], unexpected EOF")
2211 (case (xml-token-kind token)
2212 ((PI)
2213 (let ((seed
2214 ((ssax:make-pi-parser ,(get-handler 'PI))
2215 port (xml-token-head token) seed)))
2216 (scan-for-significant-prolog-token-2 port elems entities
2217 namespaces seed)))
2218 ((START)
2219 (element-parser (xml-token-head token) port elems
2220 entities namespaces #f
2221 (,(get-handler 'DECL-ROOT) (xml-token-head token) seed)))
2222 (else (parser-error port "XML [22], unexpected markup "
2223 token))))))
2224
2225
2226 ; A procedure start-tag-head port elems entities namespaces
2227 ; preserve-ws? seed
2228 (define element-parser
2229 (ssax:make-elem-parser ,(get-handler 'NEW-LEVEL-SEED)
2230 ,(get-handler 'FINISH-ELEMENT)
2231 ,(get-handler 'CHAR-DATA-HANDLER)
2232 ,(get-handler 'PI)))
2233
2234 ; Get the ball rolling ...
2235 (scan-for-significant-prolog-token-1 port seed)
2236 ))))
2237
2238
2239 (run-test
2240 (letrec ((simple-parser
2241 (lambda (str doctype-fn)
2242 (call-with-input-string str
2243 (lambda (port)
2244 ((ssax:make-parser
2245 NEW-LEVEL-SEED
2246 (lambda (elem-gi attributes namespaces
2247 expected-content seed)
2248 '())
2249
2250 FINISH-ELEMENT
2251 (lambda (elem-gi attributes namespaces parent-seed seed)
2252 (let
2253 ((seed (if (null? namespaces) (reverse seed)
2254 (cons (list '*NAMESPACES* namespaces)
2255 (reverse seed)))))
2256 (let ((seed (if (attlist-null? attributes) seed
2257 (cons
2258 (cons '@
2259 (map (lambda (attr)
2260 (list (car attr) (cdr attr)))
2261 (attlist->alist attributes)))
2262 seed))))
2263 (cons (cons elem-gi seed) parent-seed))))
2264
2265 CHAR-DATA-HANDLER
2266 (lambda (string1 string2 seed)
2267 (if (string-null? string2) (cons string1 seed)
2268 (cons* string2 string1 seed)))
2269
2270 DOCTYPE
2271 (lambda (port docname systemid internal-subset? seed)
2272 (when internal-subset?
2273 (ssax:warn port
2274 "Internal DTD subset is not currently handled ")
2275 (ssax:skip-internal-dtd port))
2276 (ssax:warn port "DOCTYPE DECL " docname " "
2277 systemid " found and skipped")
2278 (doctype-fn docname seed))
2279
2280 UNDECL-ROOT
2281 (lambda (elem-gi seed)
2282 (doctype-fn elem-gi seed))
2283 )
2284 port '())))))
2285
2286 (dummy-doctype-fn (lambda (elem-gi seed) (values #f '() '() seed)))
2287 (test
2288 (lambda (str doctype-fn expected)
2289 (cout nl "Parsing: " str nl)
2290 (let ((result (simple-parser (unesc-string str) doctype-fn)))
2291 (write result)
2292 (assert (equal? result expected)))))
2293 )
2294
2295 (test "<BR/>" dummy-doctype-fn '(('"BR")))
2296 (assert (failed? (test "<BR>" dummy-doctype-fn '())))
2297 (test "<BR></BR>" dummy-doctype-fn '(('"BR")))
2298 (assert (failed? (test "<BR></BB>" dummy-doctype-fn '())))
2299
2300 (test " <A HREF='URL'> link <I>itlink </I> &amp;amp;</A>"
2301 dummy-doctype-fn
2302 '(('"A" (@ ('"HREF" "URL")) " link " ('"I" "itlink ")
2303 " " "&" "amp;")))
2304
2305 (test
2306 " <A HREF='URL' xml:space='preserve'> link <I>itlink </I> &amp;amp;</A>" dummy-doctype-fn
2307 '(('"A" (@ ('"HREF" "URL") (('"xml" . '"space") "preserve"))
2308 " link " ('"I" "itlink ") " " "&" "amp;")))
2309
2310 (test " <A HREF='URL' xml:space='preserve'> link <I xml:space='default'>itlink </I> &amp;amp;</A>" dummy-doctype-fn
2311 '(('"A" (@ ('"HREF" "URL") (('"xml" . '"space") "preserve"))
2312 " link "
2313 ('"I" (@ (('"xml" . '"space") "default")) "itlink ")
2314 " " "&" "amp;")))
2315 (test "<itemize><item>This is item 1 </item>%n<!-- Just:a comment --><item>Item 2</item>%n </itemize>" dummy-doctype-fn
2316 `(('"itemize" ('"item" "This is item 1 ")
2317 ,(unesc-string "%n") ('"item" "Item 2") ,(unesc-string "%n "))))
2318 (test " <P><![CDATA[<BR>%n<![CDATA[<BR>]]&gt;]]></P>"
2319 dummy-doctype-fn `(('"P" "<BR>" ,nl "<![CDATA[<BR>" "]]" "" ">")))
2320
2321 (test " <P><![CDATA[<BR>%r<![CDATA[<BR>]]&gt;]]></P>"
2322 dummy-doctype-fn `(('"P" "<BR>" ,nl "<![CDATA[<BR>" "]]" "" ">")))
2323
2324 (test "<?xml version='1.0'?>%n%n<Reports TStamp='1'></Reports>"
2325 dummy-doctype-fn '(('"Reports" (@ ('"TStamp" "1")))))
2326 (test "%n<?PI xxx?><!-- Comment %n -%r-->%n<?PI1 zzz?><T/>"
2327 dummy-doctype-fn '(('"T")))
2328 (test "<!DOCTYPE T SYSTEM 'system1' ><!-- comment -->%n<T/>"
2329 (lambda (elem-gi seed) (assert (equal? elem-gi ''"T"))
2330 (values #f '() '() seed))
2331 '(('"T")))
2332 (test "<!DOCTYPE T PUBLIC '//EN/T' \"system1\" [ <!ELEMENT a 'aa'> ]>%n<?pi?><T/>"
2333 (lambda (elem-gi seed) (assert (equal? elem-gi ''"T"))
2334 (values #f '() '() seed))
2335 '(('"T")))
2336 (test "<BR/>"
2337 (lambda (elem-gi seed)
2338 (values '(('"BR" EMPTY ())) '() '() seed)) '(('"BR")))
2339 (test "<BR></BR>"
2340 (lambda (elem-gi seed)
2341 (values '(('"BR" EMPTY ())) '() '() seed)) '(('"BR")))
2342 (assert (failed? (test "<BR>aa</BR>"
2343 (lambda (elem-gi seed)
2344 (values '(('"BR" EMPTY ())) '() '() seed)) '())))
2345 (test "<BR>aa</BR>"
2346 (lambda (elem-gi seed)
2347 (values '(('"BR" PCDATA ())) '() '() seed)) '(('"BR" "aa")))
2348 (assert (failed? (test "<BR>a<I>a</I></BR>"
2349 (lambda (elem-gi seed)
2350 (values '(('"BR" PCDATA ())) '() '() seed)) '())))
2351 (test "<BR>a<I>a</I></BR>"
2352 (lambda (elem-gi seed)
2353 (values '(('"BR" ANY ()) ('"I" PCDATA ())) '() '() seed))
2354 '(('"BR" "a" ('"I" "a"))))
2355
2356
2357 (test "<DIV>Example: \"&example;\"</DIV>"
2358 (lambda (elem-gi seed)
2359 (values #f '((example . "<P>An ampersand (&#38;) may be escaped numerically (&#38;#38;) or with a general entity (&amp;amp;).</P>")) '() seed))
2360 '(('"DIV" "Example: \""
2361 ('"P" "An ampersand (" "&" ") may be escaped numerically (" "&" "#38;) or with a general entity (" "&" "amp;).") "\"")))
2362 (test "<DIV>Example: \"&example;\" <P/></DIV>"
2363 (lambda (elem-gi seed)
2364 (values #f '(('"quote" . "<I>example:</I> ex")
2365 ('"example" . "<Q>&quote;!</Q>?")) '() seed))
2366 '(('"DIV" "Example: \"" ('"Q" ('"I" "example:") " ex" "!") "?"
2367 "\" " ('"P"))))
2368 (assert (failed?
2369 (test "<DIV>Example: \"&example;\" <P/></DIV>"
2370 (lambda (elem-gi seed)
2371 (values #f '(('"quote" . "<I>example:")
2372 ('"example" . "<Q>&quote;</I>!</Q>?")) '() seed))
2373 '())
2374 '(('"DIV" "Example: \"" ('"Q" ('"I" "example:") "ex" "!") "?"
2375 "\"" ('"P")))))
2376
2377 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>"
2378 (lambda (elem-gi seed)
2379 (values #f '() '() seed))
2380 '((('"URI1" . '"DIV") (@ ('"B" "B") (('"URI1" . '"B") "A"))
2381 (*NAMESPACES* (('"A" '"URI1" . '"URI1")
2382 (*DEFAULT* '"URI1" . '"URI1")))
2383 (('"URI1" . '"P")
2384 (*NAMESPACES* ((*DEFAULT* #f . #f) ('"A" '"URI1" . '"URI1")
2385 (*DEFAULT* '"URI1" . '"URI1")))
2386 ('"BR"
2387 (*NAMESPACES* ((*DEFAULT* #f . #f)
2388 ('"A" '"URI1" . '"URI1")
2389 (*DEFAULT* '"URI1" . '"URI1"))))))))
2390 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>"
2391 (lambda (elem-gi seed)
2392 (values #f '() '((#f '"UA" . '"URI1")) seed))
2393 '((('"UA" . '"DIV") (@ ('"B" "B") (('"UA" . '"B") "A"))
2394 (*NAMESPACES* (('"A" '"UA" . '"URI1")
2395 (*DEFAULT* '"UA" . '"URI1") (#f '"UA" . '"URI1")))
2396 (('"UA" . '"P")
2397 (*NAMESPACES* ((*DEFAULT* #f . #f) ('"A" '"UA" . '"URI1")
2398 (*DEFAULT* '"UA" . '"URI1") (#f '"UA" . '"URI1")))
2399 ('"BR"
2400 (*NAMESPACES* ((*DEFAULT* #f . #f) ('"A" '"UA" . '"URI1")
2401 (*DEFAULT* '"UA" . '"URI1")
2402 (#f '"UA" . '"URI1"))))))))
2403 ; uniqattr should fail
2404 (assert (failed?
2405 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>"
2406 (lambda (elem-gi seed)
2407 (values
2408 `(('"DIV" ANY (('"B" CDATA IMPLIED #f)
2409 (('"A" . '"B") CDATA IMPLIED #f)
2410 (('"C" . '"B") CDATA IMPLIED "xx")
2411 (('"xmlns" . '"C") CDATA IMPLIED "URI1")
2412 ))
2413 (('"A" . '"P") ANY ()) ('"BR" '"EMPTY" ()))
2414 '() '((#f '"UA" . '"URI1")) seed))
2415 '())))
2416 ; prefix C undeclared
2417 (assert (failed?
2418 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>"
2419 (lambda (elem-gi seed)
2420 (values
2421 '(('"DIV" ANY (('"B" CDATA IMPLIED #f)
2422 ('"xmlns" CDATA IMPLIED "URI1")
2423 (('"A" . '"B") CDATA IMPLIED #f)
2424 (('"C" . '"B") CDATA IMPLIED "xx")
2425 ))
2426 (('"A" . '"P") ANY ()) ('"BR" EMPTY ()))
2427 '() '((#f '"UA" . '"URI1")) seed))
2428 '())))
2429
2430 ; contradiction to xmlns declaration
2431 (assert (failed?
2432 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>"
2433 (lambda (elem-gi seed)
2434 (values
2435 '(('"DIV" ANY (('"B" CDATA IMPLIED #f)
2436 ('"xmlns" CDATA FIXED "URI2")
2437 (('"A" . '"B") CDATA IMPLIED #f)
2438 ))
2439 (('"A" . '"P") ANY ()) ('"BR" EMPTY ()))
2440 '() '((#f '"UA" . '"URI1")) seed))
2441 '())))
2442
2443 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>"
2444 (lambda (elem-gi seed)
2445 (values
2446 '(('"DIV" ANY (('"B" CDATA IMPLIED #f)
2447 ('"xmlns" CDATA FIXED "URI1")
2448 (('"A" . '"B") CDATA IMPLIED #f)
2449 ))
2450 (('"A" . '"P") ANY ()) ('"BR" EMPTY ()))
2451 '() '((#f '"UA" . '"URI1")) seed))
2452 '((('"UA" . '"DIV") (@ ('"B" "B") (('"UA" . '"B") "A"))
2453 (*NAMESPACES* ((*DEFAULT* '"UA" . '"URI1")
2454 ('"A" '"UA" . '"URI1") (#f '"UA" . '"URI1")))
2455 (('"UA" . '"P")
2456 (*NAMESPACES* ((*DEFAULT* #f . #f)
2457 (*DEFAULT* '"UA" . '"URI1")
2458 ('"A" '"UA" . '"URI1") (#f '"UA" . '"URI1")))
2459 ('"BR"
2460 (*NAMESPACES* ((*DEFAULT* #f . #f) (*DEFAULT* '"UA" . '"URI1")
2461 ('"A" '"UA" . '"URI1") (#f '"UA" . '"URI1"))))))))
2462
2463 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>"
2464 (lambda (elem-gi seed)
2465 (values
2466 '(('"DIV" ANY (('"B" CDATA IMPLIED #f)
2467 (('"A" . '"B") CDATA IMPLIED #f)
2468 (('"C" . '"B") CDATA IMPLIED "xx")
2469 (('"xmlns" . '"C") CDATA IMPLIED "URI2")
2470 ))
2471 (('"A" . '"P") ANY ()) ('"BR" EMPTY ()))
2472 '() '((#f '"UA" . '"URI1")) seed))
2473 '((('"UA" . '"DIV") (@ ('"B" "B") (('"UA" . '"B") "A")
2474 (('"URI2" . '"B") "xx"))
2475 (*NAMESPACES* ((*DEFAULT* '"UA" . '"URI1")
2476 ('"A" '"UA" . '"URI1")
2477 ('"C" '"URI2" . '"URI2")
2478 (#f '"UA" . '"URI1")))
2479 (('"UA" . '"P")
2480 (*NAMESPACES* ((*DEFAULT* #f . #f) (*DEFAULT* '"UA" . '"URI1")
2481 ('"A" '"UA" . '"URI1")
2482 ('"C" '"URI2" . '"URI2") (#f '"UA" . '"URI1")))
2483 ('"BR"
2484 (*NAMESPACES* ((*DEFAULT* #f . #f)
2485 (*DEFAULT* '"UA" . '"URI1")
2486 ('"A" '"UA" . '"URI1")
2487 ('"C" '"URI2" . '"URI2")
2488 (#f '"UA" . '"URI1"))))))))
2489 ))
2490
2491
2492
2493 ;========================================================================
2494 ; Highest-level parsers: XML to SXML
2495 ;
2496
2497 ; First, a few utility procedures that turned out useful
2498
2499 ; ssax:reverse-collect-str LIST-OF-FRAGS -> LIST-OF-FRAGS
2500 ; given the list of fragments (some of which are text strings)
2501 ; reverse the list and concatenate adjacent text strings.
2502 ; We can prove from the general case below that if LIST-OF-FRAGS
2503 ; has zero or one element, the result of the procedure is equal?
2504 ; to its argument. This fact justifies the shortcut evaluation below.
2505 (define (ssax:reverse-collect-str fragments)
2506 (cond
2507 ((null? fragments) '()) ; a shortcut
2508 ((null? (cdr fragments)) fragments) ; see the comment above
2509 (else
2510 (let loop ((fragments fragments) (result '()) (strs '()))
2511 (cond
2512 ((null? fragments)
2513 (if (null? strs) result
2514 (cons (string-concatenate/shared strs) result)))
2515 ((string? (car fragments))
2516 (loop (cdr fragments) result (cons (car fragments) strs)))
2517 (else
2518 (loop (cdr fragments)
2519 (cons
2520 (car fragments)
2521 (if (null? strs) result
2522 (cons (string-concatenate/shared strs) result)))
2523 '())))))))
2524
2525
2526 ; ssax:reverse-collect-str-drop-ws LIST-OF-FRAGS -> LIST-OF-FRAGS
2527 ; given the list of fragments (some of which are text strings)
2528 ; reverse the list and concatenate adjacent text strings.
2529 ; We also drop "unsignificant" whitespace, that is, whitespace
2530 ; in front, behind and between elements. The whitespace that
2531 ; is included in character data is not affected.
2532 ; We use this procedure to "intelligently" drop "insignificant"
2533 ; whitespace in the parsed SXML. If the strict compliance with
2534 ; the XML Recommendation regarding the whitespace is desired, please
2535 ; use the ssax:reverse-collect-str procedure instead.
2536
2537 (define (ssax:reverse-collect-str-drop-ws fragments)
2538 (cond
2539 ((null? fragments) '()) ; a shortcut
2540 ((null? (cdr fragments)) ; another shortcut
2541 (if (and (string? (car fragments)) (string-whitespace? (car fragments)))
2542 '() fragments)) ; remove trailing ws
2543 (else
2544 (let loop ((fragments fragments) (result '()) (strs '())
2545 (all-whitespace? #t))
2546 (cond
2547 ((null? fragments)
2548 (if all-whitespace? result ; remove leading ws
2549 (cons (string-concatenate/shared strs) result)))
2550 ((string? (car fragments))
2551 (loop (cdr fragments) result (cons (car fragments) strs)
2552 (and all-whitespace?
2553 (string-whitespace? (car fragments)))))
2554 (else
2555 (loop (cdr fragments)
2556 (cons
2557 (car fragments)
2558 (if all-whitespace? result
2559 (cons (string-concatenate/shared strs) result)))
2560 '() #t)))))))
2561
2562 ; call either ssax:reverse-collect-str or
2563 ; ssax:reverse-collect-str-drop-ws depending the attribute
2564 ; xml:space="preserve"
2565 (define (ssax:reverse-collect-str-optionally-drop-ws fragments attrs)
2566 (let ((xml-space (find (lambda (attr)
2567 (eq? (car attr) 'xml:space))
2568 attrs)))
2569 (if (and (list? xml-space) (string=? (cadr xml-space) "preserve"))
2570 (ssax:reverse-collect-str fragments)
2571 (ssax:reverse-collect-str-drop-ws fragments))))
2572
2573
2574 ; procedure: ssax:xml->sxml PORT NAMESPACE-PREFIX-ASSIG
2575 ;
2576 ; This is an instance of a SSAX parser above that returns an SXML
2577 ; representation of the XML document to be read from PORT.
2578 ; NAMESPACE-PREFIX-ASSIG is a list of (USER-PREFIX . URI-STRING)
2579 ; that assigns USER-PREFIXes to certain namespaces identified by
2580 ; particular URI-STRINGs. It may be an empty list.
2581 ; The procedure returns an SXML tree. The port points out to the
2582 ; first character after the root element.
2583
2584 (define (ssax:xml->sxml port namespace-prefix-assig)
2585 (letrec
2586 ((namespaces
2587 (map (lambda (el)
2588 (cons* #f (car el) (ssax:uri-string->symbol (cdr el))))
2589 namespace-prefix-assig))
2590
2591 (RES-NAME->SXML
2592 (lambda (res-name)
2593 (string->symbol
2594 (string-append
2595 (symbol->string (car res-name))
2596 ":"
2597 (symbol->string (cdr res-name))))))
2598
2599 )
2600 (let ((result
2601 (reverse
2602 ((ssax:make-parser
2603 NEW-LEVEL-SEED
2604 (lambda (elem-gi attributes namespaces
2605 expected-content seed)
2606 '())
2607
2608 FINISH-ELEMENT
2609 (lambda (elem-gi attributes namespaces parent-seed seed)
2610 (let* ((attrs (attlist-fold
2611 (lambda (attr accum)
2612 (cons (list
2613 (if (symbol? (car attr)) (car attr)
2614 (RES-NAME->SXML (car attr)))
2615 (cdr attr)) accum))
2616 '() attributes))
2617 (seed (ssax:reverse-collect-str-optionally-drop-ws seed attrs)))
2618 (cons
2619 (cons
2620 (if (symbol? elem-gi) elem-gi
2621 (RES-NAME->SXML elem-gi))
2622 (if (null? attrs) seed
2623 (cons (cons '@ attrs) seed)))
2624 parent-seed)))
2625
2626 CHAR-DATA-HANDLER
2627 (lambda (string1 string2 seed)
2628 (if (string-null? string2) (cons string1 seed)
2629 (cons* string2 string1 seed)))
2630
2631 DOCTYPE
2632 (lambda (port docname systemid internal-subset? seed)
2633 (when internal-subset?
2634 (ssax:warn port
2635 "Internal DTD subset is not currently handled ")
2636 (ssax:skip-internal-dtd port))
2637 (ssax:warn port "DOCTYPE DECL " docname " "
2638 systemid " found and skipped")
2639 (values #f '() namespaces seed))
2640
2641 UNDECL-ROOT
2642 (lambda (elem-gi seed)
2643 (values #f '() namespaces seed))
2644
2645 PI
2646 ((*DEFAULT* .
2647 (lambda (port pi-tag seed)
2648 (cons
2649 (list '*PI* pi-tag (ssax:read-pi-body-as-string port))
2650 seed))))
2651 )
2652 port '()))))
2653 (cons '*TOP*
2654 (if (null? namespace-prefix-assig) result
2655 (cons
2656 (list '@@ (cons '*NAMESPACES*
2657 (map (lambda (ns) (list (car ns) (cdr ns)))
2658 namespace-prefix-assig)))
2659 result)))
2660 )))
2661
2662 ; For backwards compatibility
2663 (define SSAX:XML->SXML ssax:xml->sxml)
2664
2665
2666 ; a few lines of validation code
2667 (run-test (letrec
2668 ((test (lambda (str namespace-assig expected-res)
2669 (newline) (display "input: ")
2670 (write (unesc-string str)) (newline) (display "Result: ")
2671 (let ((result
2672 (call-with-input-string (unesc-string str)
2673 (lambda (port)
2674 (ssax:xml->sxml port namespace-assig)))))
2675 (pp result)
2676 (assert (equal_? result expected-res))))))
2677
2678 (test " <BR/>" '() '(*TOP* (BR)))
2679 (test "<BR></BR>" '() '(*TOP* (BR)))
2680 (test " <BR CLEAR='ALL'%nCLASS='Class1'/>" '()
2681 '(*TOP* (BR (@ (CLEAR "ALL") (CLASS "Class1")))))
2682 (test " <A HREF='URL'> link <I>itlink </I> &amp;amp;</A>" '()
2683 '(*TOP* (A (@ (HREF "URL")) " link " (I "itlink ") " &amp;")))
2684 (test " <A HREF='URL' xml:space='preserve'> link <I>itlink </I> &amp;amp;</A>" '()
2685 '(*TOP* (A (@ (xml:space "preserve") (HREF "URL"))
2686 " link " (I "itlink ") " &amp;")))
2687 (test " <A HREF='URL' xml:space='preserve'> link <I xml:space='default'>itlink </I> &amp;amp;</A>" '()
2688 '(*TOP* (A (@ (xml:space "preserve") (HREF "URL"))
2689 " link " (I (@ (xml:space "default"))
2690 "itlink ") " &amp;")))
2691 (test " <A HREF='URL' xml:space='preserve'> </A>" '()
2692 '(*TOP* (A (@ (xml:space "preserve") (HREF "URL")) " ")))
2693 (test " <A HREF='URL' xml:space='default'> </A>" '()
2694 '(*TOP* (A (@ (xml:space "default") (HREF "URL")))))
2695 (test " <P><?pi1 p1 content ?>?<?pi2 pi2? content? ??></P>" '()
2696 '(*TOP* (P (*PI* pi1 "p1 content ") "?"
2697 (*PI* pi2 "pi2? content? ?"))))
2698 (test " <P>some text <![CDATA[<]]>1%n&quot;<B>strong</B>&quot;%r</P>"
2699 '()
2700 `(*TOP* (P ,(unesc-string "some text <1%n\"")
2701 (B "strong") ,(unesc-string "\"%n"))))
2702 (test " <P><![CDATA[<BR>%n<![CDATA[<BR>]]&gt;]]></P>" '()
2703 `(*TOP* (P ,(unesc-string "<BR>%n<![CDATA[<BR>]]>"))))
2704 ; (test "<T1><T2>it&apos;s%r%nand that%n</T2>%r%n%r%n%n</T1>" '()
2705 ; '(*TOP* (T1 (T2 "it's%nand that%n") "%n%n%n")))
2706 (test "<T1><T2>it&apos;s%r%nand that%n</T2>%r%n%r%n%n</T1>" '()
2707 `(*TOP* (T1 (T2 ,(unesc-string "it's%nand that%n")))))
2708 (test "<T1><T2>it&apos;s%rand that%n</T2>%r%n%r%n%n</T1>" '()
2709 `(*TOP* (T1 (T2 ,(unesc-string "it's%nand that%n")))))
2710 (test "<!DOCTYPE T SYSTEM 'system1' ><!-- comment -->%n<T/>" '()
2711 '(*TOP* (T)))
2712 (test "<?xml version='1.0'?>%n<WEIGHT unit=\"pound\">%n<NET certified='certified'> 67 </NET>%n<GROSS> 95 </GROSS>%n</WEIGHT>" '()
2713 '(*TOP* (*PI* xml "version='1.0'") (WEIGHT (@ (unit "pound"))
2714 (NET (@ (certified "certified")) " 67 ")
2715 (GROSS " 95 "))
2716 ))
2717 ; (test "<?xml version='1.0'?>%n<WEIGHT unit=\"pound\">%n<NET certified='certified'> 67 </NET>%n<GROSS> 95 </GROSS>%n</WEIGHT>" '()
2718 ; '(*TOP* (*PI* xml "version='1.0'") (WEIGHT (@ (unit "pound"))
2719 ; "%n" (NET (@ (certified "certified")) " 67 ")
2720 ; "%n" (GROSS " 95 ") "%n")
2721 ; ))
2722 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" '()
2723 '(*TOP* (URI1:DIV (@ (URI1:B "A") (B "B")) (URI1:P (BR)))))
2724 (test "<DIV A:B='A' B='B' xmlns:A='URI1' xmlns='URI1'><A:P xmlns=''><BR/></A:P></DIV>" '((UA . "URI1"))
2725 '(*TOP* (@@ (*NAMESPACES* (UA "URI1")))
2726 (UA:DIV (@ (UA:B "A") (B "B")) (UA:P (BR)))))
2727
2728 ; A few tests from XML Namespaces Recommendation
2729 (test (string-append
2730 "<x xmlns:edi='http://ecommerce.org/schema'>"
2731 "<!-- the 'taxClass' attribute's ns http://ecommerce.org/schema -->"
2732 "<lineItem edi:taxClass='exempt'>Baby food</lineItem>" nl
2733 "</x>") '()
2734 '(*TOP*
2735 (x (lineItem
2736 (@ (http://ecommerce.org/schema:taxClass "exempt"))
2737 "Baby food"))))
2738 (test (string-append
2739 "<x xmlns:edi='http://ecommerce.org/schema'>"
2740 "<!-- the 'taxClass' attribute's ns http://ecommerce.org/schema -->"
2741 "<lineItem edi:taxClass='exempt'>Baby food</lineItem>"
2742 "</x>") '((EDI . "http://ecommerce.org/schema"))
2743 '(*TOP*
2744 (@@ (*NAMESPACES* (EDI "http://ecommerce.org/schema")))
2745 (x (lineItem
2746 (@ (EDI:taxClass "exempt"))
2747 "Baby food"))))
2748
2749 (test (string-append
2750 "<bk:book xmlns:bk='urn:loc.gov:books' "
2751 "xmlns:isbn='urn:ISBN:0-395-36341-6'>"
2752 "<bk:title>Cheaper by the Dozen</bk:title>"
2753 "<isbn:number>1568491379</isbn:number></bk:book>")
2754 '()
2755 '(*TOP* (urn:loc.gov:books:book
2756 (urn:loc.gov:books:title "Cheaper by the Dozen")
2757 (urn:ISBN:0-395-36341-6:number "1568491379"))))
2758
2759 (test (string-append
2760 "<!-- initially, the default namespace is 'books' -->"
2761 "<book xmlns='urn:loc.gov:books' "
2762 "xmlns:isbn='urn:ISBN:0-395-36341-6'>"
2763 "<title>Cheaper by the Dozen</title>"
2764 "<isbn:number>1568491379</isbn:number>"
2765 "<notes>"
2766 "<!-- make HTML the default namespace for some commentary -->"
2767 "<p xmlns='urn:w3-org-ns:HTML'>"
2768 "This is a <i>funny</i> book!"
2769 "</p>"
2770 "</notes>"
2771 "</book>") '()
2772 '(*TOP* (urn:loc.gov:books:book
2773 (urn:loc.gov:books:title "Cheaper by the Dozen")
2774 (urn:ISBN:0-395-36341-6:number "1568491379")
2775 (urn:loc.gov:books:notes
2776 (urn:w3-org-ns:HTML:p
2777 "This is a " (urn:w3-org-ns:HTML:i "funny")
2778 " book!")))))
2779
2780 (test (string-append
2781 "<Beers>"
2782 "<!-- the default namespace is now that of HTML -->"
2783 "<table xmlns='http://www.w3.org/TR/REC-html40'>"
2784 "<th><td>Name</td><td>Origin</td><td>Description</td></th>"
2785 "<tr>"
2786 "<!-- no default namespace inside table cells -->"
2787 "<td><brandName xmlns=\"\">Huntsman</brandName></td>"
2788 "<td><origin xmlns=''>Bath, UK</origin></td>"
2789 "<td>"
2790 "<details xmlns=''><class>Bitter</class><hop>Fuggles</hop>"
2791 "<pro>Wonderful hop, light alcohol, good summer beer</pro>"
2792 "<con>Fragile; excessive variance pub to pub</con>"
2793 "</details>"
2794 "</td>"
2795 "</tr>"
2796 "</table>"
2797 "</Beers>")
2798 '((html . "http://www.w3.org/TR/REC-html40"))
2799 '(*TOP*
2800 (@@ (*NAMESPACES* (html "http://www.w3.org/TR/REC-html40")))
2801 (Beers (html:table
2802 (html:th (html:td "Name")
2803 (html:td "Origin")
2804 (html:td "Description"))
2805 (html:tr (html:td (brandName "Huntsman"))
2806 (html:td (origin "Bath, UK"))
2807 (html:td
2808 (details
2809 (class "Bitter")
2810 (hop "Fuggles")
2811 (pro "Wonderful hop, light alcohol, good summer beer")
2812 (con "Fragile; excessive variance pub to pub"))))))))
2813
2814 (test (string-append
2815 "<!-- 1 --><RESERVATION xmlns:HTML='http://www.w3.org/TR/REC-html40'>"
2816 "<!-- 2 --><NAME HTML:CLASS=\"largeSansSerif\">Layman, A</NAME>"
2817 "<!-- 3 --><SEAT CLASS='Y' HTML:CLASS=\"largeMonotype\">33B</SEAT>"
2818 "<!-- 4 --><HTML:A HREF='/cgi-bin/ResStatus'>Check Status</HTML:A>"
2819 "<!-- 5 --><DEPARTURE>1997-05-24T07:55:00+1</DEPARTURE></RESERVATION>")
2820 '((HTML . "http://www.w3.org/TR/REC-html40"))
2821 '(*TOP*
2822 (@@ (*NAMESPACES* (HTML "http://www.w3.org/TR/REC-html40")))
2823 (RESERVATION
2824 (NAME (@ (HTML:CLASS "largeSansSerif")) "Layman, A")
2825 (SEAT (@ (HTML:CLASS "largeMonotype") (CLASS "Y")) "33B")
2826 (HTML:A (@ (HREF "/cgi-bin/ResStatus")) "Check Status")
2827 (DEPARTURE "1997-05-24T07:55:00+1"))))
2828 ; Part of RDF from the XML Infoset
2829 (test (string-concatenate/shared (list-intersperse '(
2830 "<?xml version='1.0' encoding='utf-8' standalone='yes'?>"
2831 "<!-- this can be decoded as US-ASCII or iso-8859-1 as well,"
2832 " since it contains no characters outside the US-ASCII repertoire -->"
2833 "<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'"
2834 " xmlns:rdfs='http://www.w3.org/2000/01/rdf-schema#'"
2835 " xmlns='http://www.w3.org/2001/02/infoset#'>"
2836 "<rdfs:Class ID='Boolean'/>"
2837 "<Boolean ID='Boolean.true'/>"
2838 "<Boolean ID='Boolean.false'/>"
2839 "<!--Info item classes-->"
2840 "<rdfs:Class ID='InfoItem'/>"
2841 "<rdfs:Class ID='Document' rdfs:subClassOf='#InfoItem'/>"
2842 "<rdfs:Class ID='Element' rdfs:subClassOf='#InfoItem'/>"
2843 "<rdfs:Class ID='Attribute' rdfs:subClassOf='#InfoItem'/>"
2844 "<rdfs:Class ID='InfoItemSet'
2845 rdfs:subClassOf='http://www.w3.org/1999/02/22-rdf-syntax-ns#Bag'/>"
2846 "<rdfs:Class ID='AttributeSet' rdfs:subClassOf='#InfoItemSet'/>"
2847 "<!--Info item properties-->"
2848 "<rdfs:Property ID='allDeclarationsProcessed'>"
2849 "<rdfs:domain resource='#Document'/>"
2850 "<rdfs:range resource='#Boolean'/></rdfs:Property>"
2851 "<rdfs:Property ID='attributes'>"
2852 "<rdfs:domain resource='#Element'/>"
2853 "<rdfs:range resource='#AttributeSet'/>"
2854 "</rdfs:Property>"
2855 "</rdf:RDF>")
2856 (string #\newline)))
2857 '((RDF . "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
2858 (RDFS . "http://www.w3.org/2000/01/rdf-schema#")
2859 (ISET . "http://www.w3.org/2001/02/infoset#"))
2860 '(*TOP* (@@ (*NAMESPACES*
2861 (RDF "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
2862 (RDFS "http://www.w3.org/2000/01/rdf-schema#")
2863 (ISET "http://www.w3.org/2001/02/infoset#")))
2864 (*PI* xml "version='1.0' encoding='utf-8' standalone='yes'")
2865 (RDF:RDF
2866 (RDFS:Class (@ (ID "Boolean")))
2867 (ISET:Boolean (@ (ID "Boolean.true")))
2868 (ISET:Boolean (@ (ID "Boolean.false")))
2869 (RDFS:Class (@ (ID "InfoItem")))
2870 (RDFS:Class (@ (RDFS:subClassOf "#InfoItem") (ID "Document")))
2871 (RDFS:Class (@ (RDFS:subClassOf "#InfoItem") (ID "Element")))
2872 (RDFS:Class (@ (RDFS:subClassOf "#InfoItem") (ID "Attribute")))
2873 (RDFS:Class
2874 (@ (RDFS:subClassOf
2875 "http://www.w3.org/1999/02/22-rdf-syntax-ns#Bag")
2876 (ID "InfoItemSet")))
2877 (RDFS:Class
2878 (@ (RDFS:subClassOf "#InfoItemSet") (ID "AttributeSet")))
2879 (RDFS:Property
2880 (@ (ID "allDeclarationsProcessed"))
2881 (RDFS:domain (@ (resource "#Document")))
2882 (RDFS:range (@ (resource "#Boolean"))))
2883 (RDFS:Property
2884 (@ (ID "attributes"))
2885 (RDFS:domain (@ (resource "#Element")))
2886 (RDFS:range (@ (resource "#AttributeSet")))))))
2887
2888 ; Part of RDF from RSS of the Daemon News Mall
2889 (test (string-concatenate/shared (list-intersperse '(
2890 "<?xml version='1.0'?><rdf:RDF "
2891 "xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' "
2892 "xmlns='http://my.netscape.com/rdf/simple/0.9/'>"
2893 "<channel>"
2894 "<title>Daemon News Mall</title>"
2895 "<link>http://mall.daemonnews.org/</link>"
2896 "<description>Central source for all your BSD needs</description>"
2897 "</channel>"
2898 "<item>"
2899 "<title>Daemon News Jan/Feb Issue NOW Available! Subscribe $24.95</title>"
2900 "<link>http://mall.daemonnews.org/?page=shop/flypage&amp;product_id=880</link>"
2901 "</item>"
2902 "<item>"
2903 "<title>The Design and Implementation of the 4.4BSD Operating System $54.95</title>"
2904 "<link>http://mall.daemonnews.org/?page=shop/flypage&amp;product_id=912&amp;category_id=1761</link>"
2905 "</item>"
2906 "</rdf:RDF>")
2907 (string #\newline)
2908 ))
2909 '((RDF . "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
2910 (RSS . "http://my.netscape.com/rdf/simple/0.9/")
2911 (ISET . "http://www.w3.org/2001/02/infoset#"))
2912 '(*TOP* (@@ (*NAMESPACES*
2913 (RDF "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
2914 (RSS "http://my.netscape.com/rdf/simple/0.9/")
2915 (ISET "http://www.w3.org/2001/02/infoset#")))
2916 (*PI* xml "version='1.0'")
2917 (RDF:RDF (RSS:channel
2918 (RSS:title "Daemon News Mall")
2919 (RSS:link "http://mall.daemonnews.org/")
2920 (RSS:description "Central source for all your BSD needs"))
2921 (RSS:item
2922 (RSS:title
2923 "Daemon News Jan/Feb Issue NOW Available! Subscribe $24.95")
2924 (RSS:link
2925 "http://mall.daemonnews.org/?page=shop/flypage&product_id=880"))
2926 (RSS:item
2927 (RSS:title
2928 "The Design and Implementation of the 4.4BSD Operating System $54.95")
2929 (RSS:link
2930 "http://mall.daemonnews.org/?page=shop/flypage&product_id=912&category_id=1761")))))
2931
2932 (test (string-concatenate/shared (list-intersperse
2933 '("<Forecasts TStamp='958082142'>"
2934 "<TAF TStamp='958066200' LatLon='36.583, -121.850' BId='724915'"
2935 " SName='KMRY, MONTEREY PENINSULA'>"
2936 "<VALID TRange='958068000, 958154400'>111730Z 111818</VALID>"
2937 "<PERIOD TRange='958068000, 958078800'>"
2938 "<PREVAILING>31010KT P6SM FEW030</PREVAILING>"
2939 "</PERIOD>"
2940 "<PERIOD TRange='958078800, 958104000' Title='FM2100'>"
2941 "<PREVAILING>29016KT P6SM FEW040</PREVAILING>"
2942 "</PERIOD>"
2943 "<PERIOD TRange='958104000, 958154400' Title='FM0400'>"
2944 "<PREVAILING>29010KT P6SM SCT200</PREVAILING>"
2945 "<VAR Title='BECMG 0708' TRange='958114800, 958118400'>VRB05KT</VAR>"
2946 "</PERIOD></TAF>"
2947 "</Forecasts>")
2948 (string #\newline)
2949 ))
2950 '()
2951 '(*TOP* (Forecasts
2952 (@ (TStamp "958082142"))
2953 (TAF (@ (TStamp "958066200")
2954 (SName "KMRY, MONTEREY PENINSULA")
2955 (LatLon "36.583, -121.850")
2956 (BId "724915"))
2957 (VALID (@ (TRange "958068000, 958154400")) "111730Z 111818")
2958 (PERIOD (@ (TRange "958068000, 958078800"))
2959 (PREVAILING "31010KT P6SM FEW030"))
2960 (PERIOD (@ (Title "FM2100") (TRange "958078800, 958104000"))
2961 (PREVAILING "29016KT P6SM FEW040"))
2962 (PERIOD (@ (Title "FM0400") (TRange "958104000, 958154400"))
2963 (PREVAILING "29010KT P6SM SCT200")
2964 (VAR (@ (Title "BECMG 0708")
2965 (TRange "958114800, 958118400"))
2966 "VRB05KT"))))))
2967 ))
2968
2969 (run-test
2970 (newline)
2971 (display "All tests passed")
2972 (newline)
2973 )