| 1 |
:mod:`!xml.parsers.expat` --- Fast XML parsing using Expat |
| 2 |
========================================================== |
| 3 |
|
| 4 |
.. module:: xml.parsers.expat |
| 5 |
:synopsis: An interface to the Expat non-validating XML parser. |
| 6 |
|
| 7 |
.. moduleauthor:: Paul Prescod <[email protected]> |
| 8 |
|
| 9 |
-------------- |
| 10 |
|
| 11 |
.. Markup notes: |
| 12 |
|
| 13 |
Many of the attributes of the XMLParser objects are callbacks. Since |
| 14 |
signature information must be presented, these are described using the method |
| 15 |
directive. Since they are attributes which are set by client code, in-text |
| 16 |
references to these attributes should be marked using the :member: role. |
| 17 |
|
| 18 |
|
| 19 |
.. note:: |
| 20 |
|
| 21 |
If you need to parse untrusted or unauthenticated data, see |
| 22 |
:ref:`xml-security`. |
| 23 |
|
| 24 |
|
| 25 |
.. index:: single: Expat |
| 26 |
|
| 27 |
The :mod:`!xml.parsers.expat` module is a Python interface to the Expat |
| 28 |
non-validating XML parser. The module provides a single extension type, |
| 29 |
:class:`xmlparser`, that represents the current state of an XML parser. After |
| 30 |
an :class:`xmlparser` object has been created, various attributes of the object |
| 31 |
can be set to handler functions. When an XML document is then fed to the |
| 32 |
parser, the handler functions are called for the character data and markup in |
| 33 |
the XML document. |
| 34 |
|
| 35 |
.. index:: pair: module; pyexpat |
| 36 |
|
| 37 |
This module uses the :mod:`pyexpat` module to provide access to the Expat |
| 38 |
parser. Direct use of the :mod:`pyexpat` module is deprecated. |
| 39 |
|
| 40 |
This module provides one exception and one type object: |
| 41 |
|
| 42 |
|
| 43 |
.. exception:: ExpatError |
| 44 |
|
| 45 |
The exception raised when Expat reports an error. See section |
| 46 |
:ref:`expaterror-objects` for more information on interpreting Expat errors. |
| 47 |
|
| 48 |
|
| 49 |
.. exception:: error |
| 50 |
|
| 51 |
Alias for :exc:`ExpatError`. |
| 52 |
|
| 53 |
|
| 54 |
.. data:: XMLParserType |
| 55 |
|
| 56 |
The type of the return values from the :func:`ParserCreate` function. |
| 57 |
|
| 58 |
The :mod:`!xml.parsers.expat` module contains two functions: |
| 59 |
|
| 60 |
|
| 61 |
.. function:: ErrorString(errno) |
| 62 |
|
| 63 |
Returns an explanatory string for a given error number *errno*. |
| 64 |
|
| 65 |
|
| 66 |
.. function:: ParserCreate(encoding=None, namespace_separator=None) |
| 67 |
|
| 68 |
Creates and returns a new :class:`xmlparser` object. *encoding*, if specified, |
| 69 |
must be a string naming the encoding used by the XML data. Expat doesn't |
| 70 |
support as many encodings as Python does, and its repertoire of encodings can't |
| 71 |
be extended; it supports UTF-8, UTF-16, ISO-8859-1 (Latin1), and ASCII. If |
| 72 |
*encoding* [1]_ is given it will override the implicit or explicit encoding of the |
| 73 |
document. |
| 74 |
|
| 75 |
.. _xmlparser-non-root: |
| 76 |
|
| 77 |
Parsers created through :func:`!ParserCreate` are called "root" parsers, |
| 78 |
in the sense that they do not have any parent parser attached. Non-root |
| 79 |
parsers are created by :meth:`parser.ExternalEntityParserCreate |
| 80 |
<xmlparser.ExternalEntityParserCreate>`. |
| 81 |
|
| 82 |
Expat can optionally do XML namespace processing for you, enabled by providing a |
| 83 |
value for *namespace_separator*. The value must be a one-character string; a |
| 84 |
:exc:`ValueError` will be raised if the string has an illegal length (``None`` |
| 85 |
is considered the same as omission). When namespace processing is enabled, |
| 86 |
element type names and attribute names that belong to a namespace will be |
| 87 |
expanded. The element name passed to the element handlers |
| 88 |
:attr:`StartElementHandler` and :attr:`EndElementHandler` will be the |
| 89 |
concatenation of the namespace URI, the namespace separator character, and the |
| 90 |
local part of the name. If the namespace separator is a zero byte (``chr(0)``) |
| 91 |
then the namespace URI and the local part will be concatenated without any |
| 92 |
separator. |
| 93 |
|
| 94 |
For example, if *namespace_separator* is set to a space character (``' '``) and |
| 95 |
the following document is parsed: |
| 96 |
|
| 97 |
.. code-block:: xml |
| 98 |
|
| 99 |
<?xml version="1.0"?> |
| 100 |
<root xmlns = "http://default-namespace.org/" |
| 101 |
xmlns:py = "http://www.python.org/ns/"> |
| 102 |
<py:elem1 /> |
| 103 |
<elem2 xmlns="" /> |
| 104 |
</root> |
| 105 |
|
| 106 |
:attr:`StartElementHandler` will receive the following strings for each |
| 107 |
element:: |
| 108 |
|
| 109 |
http://default-namespace.org/ root |
| 110 |
http://www.python.org/ns/ elem1 |
| 111 |
elem2 |
| 112 |
|
| 113 |
Due to limitations in the ``Expat`` library used by :mod:`pyexpat`, |
| 114 |
the :class:`xmlparser` instance returned can only be used to parse a single |
| 115 |
XML document. Call ``ParserCreate`` for each document to provide unique |
| 116 |
parser instances. |
| 117 |
|
| 118 |
|
| 119 |
.. seealso:: |
| 120 |
|
| 121 |
`The Expat XML Parser <http://www.libexpat.org/>`_ |
| 122 |
Home page of the Expat project. |
| 123 |
|
| 124 |
|
| 125 |
.. _xmlparser-objects: |
| 126 |
|
| 127 |
XMLParser Objects |
| 128 |
----------------- |
| 129 |
|
| 130 |
:class:`xmlparser` objects have the following methods: |
| 131 |
|
| 132 |
|
| 133 |
.. method:: xmlparser.Parse(data[, isfinal]) |
| 134 |
|
| 135 |
Parses the contents of the string *data*, calling the appropriate handler |
| 136 |
functions to process the parsed data. *isfinal* must be true on the final call |
| 137 |
to this method; it allows the parsing of a single file in fragments, |
| 138 |
not the submission of multiple files. |
| 139 |
*data* can be the empty string at any time. |
| 140 |
|
| 141 |
|
| 142 |
.. method:: xmlparser.ParseFile(file) |
| 143 |
|
| 144 |
Parse XML data reading from the object *file*. *file* only needs to provide |
| 145 |
the ``read(nbytes)`` method, returning the empty string when there's no more |
| 146 |
data. |
| 147 |
|
| 148 |
|
| 149 |
.. method:: xmlparser.SetBase(base) |
| 150 |
|
| 151 |
Sets the base to be used for resolving relative URIs in system identifiers in |
| 152 |
declarations. Resolving relative identifiers is left to the application: this |
| 153 |
value will be passed through as the *base* argument to the |
| 154 |
:func:`ExternalEntityRefHandler`, :func:`NotationDeclHandler`, and |
| 155 |
:func:`UnparsedEntityDeclHandler` functions. |
| 156 |
|
| 157 |
|
| 158 |
.. method:: xmlparser.GetBase() |
| 159 |
|
| 160 |
Returns a string containing the base set by a previous call to :meth:`SetBase`, |
| 161 |
or ``None`` if :meth:`SetBase` hasn't been called. |
| 162 |
|
| 163 |
|
| 164 |
.. method:: xmlparser.GetInputContext() |
| 165 |
|
| 166 |
Returns the input data that generated the current event as a string. The data is |
| 167 |
in the encoding of the entity which contains the text. When called while an |
| 168 |
event handler is not active, the return value is ``None``. |
| 169 |
|
| 170 |
|
| 171 |
.. method:: xmlparser.ExternalEntityParserCreate(context[, encoding]) |
| 172 |
|
| 173 |
Create a "child" parser which can be used to parse an external parsed entity |
| 174 |
referred to by content parsed by the parent parser. The *context* parameter |
| 175 |
should be the string passed to the :meth:`ExternalEntityRefHandler` handler |
| 176 |
function, described below. The child parser is created with the |
| 177 |
:attr:`ordered_attributes` and :attr:`specified_attributes` set to the values of |
| 178 |
this parser. |
| 179 |
|
| 180 |
.. method:: xmlparser.SetParamEntityParsing(flag) |
| 181 |
|
| 182 |
Control parsing of parameter entities (including the external DTD subset). |
| 183 |
Possible *flag* values are :const:`XML_PARAM_ENTITY_PARSING_NEVER`, |
| 184 |
:const:`XML_PARAM_ENTITY_PARSING_UNLESS_STANDALONE` and |
| 185 |
:const:`XML_PARAM_ENTITY_PARSING_ALWAYS`. Return true if setting the flag |
| 186 |
was successful. |
| 187 |
|
| 188 |
.. method:: xmlparser.UseForeignDTD([flag]) |
| 189 |
|
| 190 |
Calling this with a true value for *flag* (the default) will cause Expat to call |
| 191 |
the :attr:`ExternalEntityRefHandler` with :const:`None` for all arguments to |
| 192 |
allow an alternate DTD to be loaded. If the document does not contain a |
| 193 |
document type declaration, the :attr:`ExternalEntityRefHandler` will still be |
| 194 |
called, but the :attr:`StartDoctypeDeclHandler` and |
| 195 |
:attr:`EndDoctypeDeclHandler` will not be called. |
| 196 |
|
| 197 |
Passing a false value for *flag* will cancel a previous call that passed a true |
| 198 |
value, but otherwise has no effect. |
| 199 |
|
| 200 |
This method can only be called before the :meth:`Parse` or :meth:`ParseFile` |
| 201 |
methods are called; calling it after either of those have been called causes |
| 202 |
:exc:`ExpatError` to be raised with the :attr:`code` attribute set to |
| 203 |
``errors.codes[errors.XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING]``. |
| 204 |
|
| 205 |
.. method:: xmlparser.SetReparseDeferralEnabled(enabled) |
| 206 |
|
| 207 |
.. warning:: |
| 208 |
|
| 209 |
Calling ``SetReparseDeferralEnabled(False)`` has security implications, |
| 210 |
as detailed below; please make sure to understand these consequences |
| 211 |
prior to using the ``SetReparseDeferralEnabled`` method. |
| 212 |
|
| 213 |
Expat 2.6.0 introduced a security mechanism called "reparse deferral" |
| 214 |
where instead of causing denial of service through quadratic runtime |
| 215 |
from reparsing large tokens, reparsing of unfinished tokens is now delayed |
| 216 |
by default until a sufficient amount of input is reached. |
| 217 |
Due to this delay, registered handlers may — depending of the sizing of |
| 218 |
input chunks pushed to Expat — no longer be called right after pushing new |
| 219 |
input to the parser. Where immediate feedback and taking over responsibility |
| 220 |
of protecting against denial of service from large tokens are both wanted, |
| 221 |
calling ``SetReparseDeferralEnabled(False)`` disables reparse deferral |
| 222 |
for the current Expat parser instance, temporarily or altogether. |
| 223 |
Calling ``SetReparseDeferralEnabled(True)`` allows re-enabling reparse |
| 224 |
deferral. |
| 225 |
|
| 226 |
Note that :meth:`SetReparseDeferralEnabled` has been backported to some |
| 227 |
prior releases of CPython as a security fix. Check for availability of |
| 228 |
:meth:`SetReparseDeferralEnabled` using :func:`hasattr` if used in code |
| 229 |
running across a variety of Python versions. |
| 230 |
|
| 231 |
.. versionadded:: 3.13 |
| 232 |
|
| 233 |
.. method:: xmlparser.GetReparseDeferralEnabled() |
| 234 |
|
| 235 |
Returns whether reparse deferral is currently enabled for the given |
| 236 |
Expat parser instance. |
| 237 |
|
| 238 |
.. versionadded:: 3.13 |
| 239 |
|
| 240 |
|
| 241 |
:class:`!xmlparser` objects have the following methods to mitigate some |
| 242 |
common XML vulnerabilities. |
| 243 |
|
| 244 |
.. method:: xmlparser.SetAllocTrackerActivationThreshold(threshold, /) |
| 245 |
|
| 246 |
Sets the number of allocated bytes of dynamic memory needed to activate |
| 247 |
protection against disproportionate use of RAM. |
| 248 |
|
| 249 |
By default, parser objects have an allocation activation threshold of 64 MiB, |
| 250 |
or equivalently 67,108,864 bytes. |
| 251 |
|
| 252 |
An :exc:`ExpatError` is raised if this method is called on a |
| 253 |
|xml-non-root-parser| parser. |
| 254 |
The corresponding :attr:`~ExpatError.lineno` and :attr:`~ExpatError.offset` |
| 255 |
should not be used as they may have no special meaning. |
| 256 |
|
| 257 |
.. versionadded:: 3.14.1 |
| 258 |
|
| 259 |
.. method:: xmlparser.SetAllocTrackerMaximumAmplification(max_factor, /) |
| 260 |
|
| 261 |
Sets the maximum amplification factor between direct input and bytes |
| 262 |
of dynamic memory allocated. |
| 263 |
|
| 264 |
The amplification factor is calculated as ``allocated / direct`` |
| 265 |
while parsing, where ``direct`` is the number of bytes read from |
| 266 |
the primary document in parsing and ``allocated`` is the number |
| 267 |
of bytes of dynamic memory allocated in the parser hierarchy. |
| 268 |
|
| 269 |
The *max_factor* value must be a non-NaN :class:`float` value greater than |
| 270 |
or equal to 1.0. Amplification factors greater than 100.0 can be observed |
| 271 |
near the start of parsing even with benign files in practice. In particular, |
| 272 |
the activation threshold should be carefully chosen to avoid false positives. |
| 273 |
|
| 274 |
By default, parser objects have a maximum amplification factor of 100.0. |
| 275 |
|
| 276 |
An :exc:`ExpatError` is raised if this method is called on a |
| 277 |
|xml-non-root-parser| parser or if *max_factor* is outside the valid range. |
| 278 |
The corresponding :attr:`~ExpatError.lineno` and :attr:`~ExpatError.offset` |
| 279 |
should not be used as they may have no special meaning. |
| 280 |
|
| 281 |
.. note:: |
| 282 |
|
| 283 |
The maximum amplification factor is only considered if the threshold |
| 284 |
that can be adjusted by :meth:`.SetAllocTrackerActivationThreshold` |
| 285 |
is exceeded. |
| 286 |
|
| 287 |
.. versionadded:: 3.14.1 |
| 288 |
|
| 289 |
|
| 290 |
:class:`xmlparser` objects have the following attributes: |
| 291 |
|
| 292 |
|
| 293 |
.. attribute:: xmlparser.buffer_size |
| 294 |
|
| 295 |
The size of the buffer used when :attr:`buffer_text` is true. |
| 296 |
A new buffer size can be set by assigning a new integer value |
| 297 |
to this attribute. |
| 298 |
When the size is changed, the buffer will be flushed. |
| 299 |
|
| 300 |
|
| 301 |
.. attribute:: xmlparser.buffer_text |
| 302 |
|
| 303 |
Setting this to true causes the :class:`xmlparser` object to buffer textual |
| 304 |
content returned by Expat to avoid multiple calls to the |
| 305 |
:meth:`CharacterDataHandler` callback whenever possible. This can improve |
| 306 |
performance substantially since Expat normally breaks character data into chunks |
| 307 |
at every line ending. This attribute is false by default, and may be changed at |
| 308 |
any time. Note that when it is false, data that does not contain newlines |
| 309 |
may be chunked too. |
| 310 |
|
| 311 |
|
| 312 |
.. attribute:: xmlparser.buffer_used |
| 313 |
|
| 314 |
If :attr:`buffer_text` is enabled, the number of bytes stored in the buffer. |
| 315 |
These bytes represent UTF-8 encoded text. This attribute has no meaningful |
| 316 |
interpretation when :attr:`buffer_text` is false. |
| 317 |
|
| 318 |
|
| 319 |
.. attribute:: xmlparser.ordered_attributes |
| 320 |
|
| 321 |
Setting this attribute to a non-zero integer causes the attributes to be |
| 322 |
reported as a list rather than a dictionary. The attributes are presented in |
| 323 |
the order found in the document text. For each attribute, two list entries are |
| 324 |
presented: the attribute name and the attribute value. (Older versions of this |
| 325 |
module also used this format.) By default, this attribute is false; it may be |
| 326 |
changed at any time. |
| 327 |
|
| 328 |
|
| 329 |
.. attribute:: xmlparser.specified_attributes |
| 330 |
|
| 331 |
If set to a non-zero integer, the parser will report only those attributes which |
| 332 |
were specified in the document instance and not those which were derived from |
| 333 |
attribute declarations. Applications which set this need to be especially |
| 334 |
careful to use what additional information is available from the declarations as |
| 335 |
needed to comply with the standards for the behavior of XML processors. By |
| 336 |
default, this attribute is false; it may be changed at any time. |
| 337 |
|
| 338 |
|
| 339 |
The following attributes contain values relating to the most recent error |
| 340 |
encountered by an :class:`xmlparser` object, and will only have correct values |
| 341 |
once a call to :meth:`Parse` or :meth:`ParseFile` has raised an |
| 342 |
:exc:`xml.parsers.expat.ExpatError` exception. |
| 343 |
|
| 344 |
|
| 345 |
.. attribute:: xmlparser.ErrorByteIndex |
| 346 |
|
| 347 |
Byte index at which an error occurred. |
| 348 |
|
| 349 |
|
| 350 |
.. attribute:: xmlparser.ErrorCode |
| 351 |
|
| 352 |
Numeric code specifying the problem. This value can be passed to the |
| 353 |
:func:`ErrorString` function, or compared to one of the constants defined in the |
| 354 |
``errors`` object. |
| 355 |
|
| 356 |
|
| 357 |
.. attribute:: xmlparser.ErrorColumnNumber |
| 358 |
|
| 359 |
Column number at which an error occurred. |
| 360 |
|
| 361 |
|
| 362 |
.. attribute:: xmlparser.ErrorLineNumber |
| 363 |
|
| 364 |
Line number at which an error occurred. |
| 365 |
|
| 366 |
The following attributes contain values relating to the current parse location |
| 367 |
in an :class:`xmlparser` object. During a callback reporting a parse event they |
| 368 |
indicate the location of the first of the sequence of characters that generated |
| 369 |
the event. When called outside of a callback, the position indicated will be |
| 370 |
just past the last parse event (regardless of whether there was an associated |
| 371 |
callback). |
| 372 |
|
| 373 |
|
| 374 |
.. attribute:: xmlparser.CurrentByteIndex |
| 375 |
|
| 376 |
Current byte index in the parser input. |
| 377 |
|
| 378 |
|
| 379 |
.. attribute:: xmlparser.CurrentColumnNumber |
| 380 |
|
| 381 |
Current column number in the parser input. |
| 382 |
|
| 383 |
|
| 384 |
.. attribute:: xmlparser.CurrentLineNumber |
| 385 |
|
| 386 |
Current line number in the parser input. |
| 387 |
|
| 388 |
Here is the list of handlers that can be set. To set a handler on an |
| 389 |
:class:`xmlparser` object *o*, use ``o.handlername = func``. *handlername* must |
| 390 |
be taken from the following list, and *func* must be a callable object accepting |
| 391 |
the correct number of arguments. The arguments are all strings, unless |
| 392 |
otherwise stated. |
| 393 |
|
| 394 |
|
| 395 |
.. method:: xmlparser.XmlDeclHandler(version, encoding, standalone) |
| 396 |
|
| 397 |
Called when the XML declaration is parsed. The XML declaration is the |
| 398 |
(optional) declaration of the applicable version of the XML recommendation, the |
| 399 |
encoding of the document text, and an optional "standalone" declaration. |
| 400 |
*version* and *encoding* will be strings, and *standalone* will be ``1`` if the |
| 401 |
document is declared standalone, ``0`` if it is declared not to be standalone, |
| 402 |
or ``-1`` if the standalone clause was omitted. This is only available with |
| 403 |
Expat version 1.95.0 or newer. |
| 404 |
|
| 405 |
|
| 406 |
.. method:: xmlparser.StartDoctypeDeclHandler(doctypeName, systemId, publicId, has_internal_subset) |
| 407 |
|
| 408 |
Called when Expat begins parsing the document type declaration (``<!DOCTYPE |
| 409 |
...``). The *doctypeName* is provided exactly as presented. The *systemId* and |
| 410 |
*publicId* parameters give the system and public identifiers if specified, or |
| 411 |
``None`` if omitted. *has_internal_subset* will be true if the document |
| 412 |
contains an internal document declaration subset. This requires Expat version |
| 413 |
1.2 or newer. |
| 414 |
|
| 415 |
|
| 416 |
.. method:: xmlparser.EndDoctypeDeclHandler() |
| 417 |
|
| 418 |
Called when Expat is done parsing the document type declaration. This requires |
| 419 |
Expat version 1.2 or newer. |
| 420 |
|
| 421 |
|
| 422 |
.. method:: xmlparser.ElementDeclHandler(name, model) |
| 423 |
|
| 424 |
Called once for each element type declaration. *name* is the name of the |
| 425 |
element type, and *model* is a representation of the content model. |
| 426 |
|
| 427 |
|
| 428 |
.. method:: xmlparser.AttlistDeclHandler(elname, attname, type, default, required) |
| 429 |
|
| 430 |
Called for each declared attribute for an element type. If an attribute list |
| 431 |
declaration declares three attributes, this handler is called three times, once |
| 432 |
for each attribute. *elname* is the name of the element to which the |
| 433 |
declaration applies and *attname* is the name of the attribute declared. The |
| 434 |
attribute type is a string passed as *type*; the possible values are |
| 435 |
``'CDATA'``, ``'ID'``, ``'IDREF'``, ... *default* gives the default value for |
| 436 |
the attribute used when the attribute is not specified by the document instance, |
| 437 |
or ``None`` if there is no default value (``#IMPLIED`` values). If the |
| 438 |
attribute is required to be given in the document instance, *required* will be |
| 439 |
true. This requires Expat version 1.95.0 or newer. |
| 440 |
|
| 441 |
|
| 442 |
.. method:: xmlparser.StartElementHandler(name, attributes) |
| 443 |
|
| 444 |
Called for the start of every element. *name* is a string containing the |
| 445 |
element name, and *attributes* is the element attributes. If |
| 446 |
:attr:`ordered_attributes` is true, this is a list (see |
| 447 |
:attr:`ordered_attributes` for a full description). Otherwise it's a |
| 448 |
dictionary mapping names to values. |
| 449 |
|
| 450 |
|
| 451 |
.. method:: xmlparser.EndElementHandler(name) |
| 452 |
|
| 453 |
Called for the end of every element. |
| 454 |
|
| 455 |
|
| 456 |
.. method:: xmlparser.ProcessingInstructionHandler(target, data) |
| 457 |
|
| 458 |
Called for every processing instruction. |
| 459 |
|
| 460 |
|
| 461 |
.. method:: xmlparser.CharacterDataHandler(data) |
| 462 |
|
| 463 |
Called for character data. This will be called for normal character data, CDATA |
| 464 |
marked content, and ignorable whitespace. Applications which must distinguish |
| 465 |
these cases can use the :attr:`StartCdataSectionHandler`, |
| 466 |
:attr:`EndCdataSectionHandler`, and :attr:`ElementDeclHandler` callbacks to |
| 467 |
collect the required information. Note that the character data may be |
| 468 |
chunked even if it is short and so you may receive more than one call to |
| 469 |
:meth:`CharacterDataHandler`. Set the :attr:`buffer_text` instance attribute |
| 470 |
to ``True`` to avoid that. |
| 471 |
|
| 472 |
|
| 473 |
.. method:: xmlparser.UnparsedEntityDeclHandler(entityName, base, systemId, publicId, notationName) |
| 474 |
|
| 475 |
Called for unparsed (NDATA) entity declarations. This is only present for |
| 476 |
version 1.2 of the Expat library; for more recent versions, use |
| 477 |
:attr:`EntityDeclHandler` instead. (The underlying function in the Expat |
| 478 |
library has been declared obsolete.) |
| 479 |
|
| 480 |
|
| 481 |
.. method:: xmlparser.EntityDeclHandler(entityName, is_parameter_entity, value, base, systemId, publicId, notationName) |
| 482 |
|
| 483 |
Called for all entity declarations. For parameter and internal entities, |
| 484 |
*value* will be a string giving the declared contents of the entity; this will |
| 485 |
be ``None`` for external entities. The *notationName* parameter will be |
| 486 |
``None`` for parsed entities, and the name of the notation for unparsed |
| 487 |
entities. *is_parameter_entity* will be true if the entity is a parameter entity |
| 488 |
or false for general entities (most applications only need to be concerned with |
| 489 |
general entities). This is only available starting with version 1.95.0 of the |
| 490 |
Expat library. |
| 491 |
|
| 492 |
|
| 493 |
.. method:: xmlparser.NotationDeclHandler(notationName, base, systemId, publicId) |
| 494 |
|
| 495 |
Called for notation declarations. *notationName*, *base*, and *systemId*, and |
| 496 |
*publicId* are strings if given. If the public identifier is omitted, |
| 497 |
*publicId* will be ``None``. |
| 498 |
|
| 499 |
|
| 500 |
.. method:: xmlparser.StartNamespaceDeclHandler(prefix, uri) |
| 501 |
|
| 502 |
Called when an element contains a namespace declaration. Namespace declarations |
| 503 |
are processed before the :attr:`StartElementHandler` is called for the element |
| 504 |
on which declarations are placed. |
| 505 |
|
| 506 |
|
| 507 |
.. method:: xmlparser.EndNamespaceDeclHandler(prefix) |
| 508 |
|
| 509 |
Called when the closing tag is reached for an element that contained a |
| 510 |
namespace declaration. This is called once for each namespace declaration on |
| 511 |
the element in the reverse of the order for which the |
| 512 |
:attr:`StartNamespaceDeclHandler` was called to indicate the start of each |
| 513 |
namespace declaration's scope. Calls to this handler are made after the |
| 514 |
corresponding :attr:`EndElementHandler` for the end of the element. |
| 515 |
|
| 516 |
|
| 517 |
.. method:: xmlparser.CommentHandler(data) |
| 518 |
|
| 519 |
Called for comments. *data* is the text of the comment, excluding the leading |
| 520 |
``'<!-``\ ``-'`` and trailing ``'-``\ ``->'``. |
| 521 |
|
| 522 |
|
| 523 |
.. method:: xmlparser.StartCdataSectionHandler() |
| 524 |
|
| 525 |
Called at the start of a CDATA section. This and :attr:`EndCdataSectionHandler` |
| 526 |
are needed to be able to identify the syntactical start and end for CDATA |
| 527 |
sections. |
| 528 |
|
| 529 |
|
| 530 |
.. method:: xmlparser.EndCdataSectionHandler() |
| 531 |
|
| 532 |
Called at the end of a CDATA section. |
| 533 |
|
| 534 |
|
| 535 |
.. method:: xmlparser.DefaultHandler(data) |
| 536 |
|
| 537 |
Called for any characters in the XML document for which no applicable handler |
| 538 |
has been specified. This means characters that are part of a construct which |
| 539 |
could be reported, but for which no handler has been supplied. |
| 540 |
|
| 541 |
|
| 542 |
.. method:: xmlparser.DefaultHandlerExpand(data) |
| 543 |
|
| 544 |
This is the same as the :func:`DefaultHandler`, but doesn't inhibit expansion |
| 545 |
of internal entities. The entity reference will not be passed to the default |
| 546 |
handler. |
| 547 |
|
| 548 |
|
| 549 |
.. method:: xmlparser.NotStandaloneHandler() |
| 550 |
|
| 551 |
Called if the XML document hasn't been declared as being a standalone document. |
| 552 |
This happens when there is an external subset or a reference to a parameter |
| 553 |
entity, but the XML declaration does not set standalone to ``yes`` in an XML |
| 554 |
declaration. If this handler returns ``0``, then the parser will raise an |
| 555 |
:const:`XML_ERROR_NOT_STANDALONE` error. If this handler is not set, no |
| 556 |
exception is raised by the parser for this condition. |
| 557 |
|
| 558 |
|
| 559 |
.. method:: xmlparser.ExternalEntityRefHandler(context, base, systemId, publicId) |
| 560 |
|
| 561 |
.. warning:: |
| 562 |
|
| 563 |
Implementing a handler that accesses local files and/or the network |
| 564 |
may create a vulnerability to |
| 565 |
`external entity attacks <https://en.wikipedia.org/wiki/XML_external_entity_attack>`_ |
| 566 |
if :class:`xmlparser` is used with user-provided XML content. |
| 567 |
Please reflect on your `threat model <https://en.wikipedia.org/wiki/Threat_model>`_ |
| 568 |
before implementing this handler. |
| 569 |
|
| 570 |
Called for references to external entities. *base* is the current base, as set |
| 571 |
by a previous call to :meth:`SetBase`. The public and system identifiers, |
| 572 |
*systemId* and *publicId*, are strings if given; if the public identifier is not |
| 573 |
given, *publicId* will be ``None``. The *context* value is opaque and should |
| 574 |
only be used as described below. |
| 575 |
|
| 576 |
For external entities to be parsed, this handler must be implemented. It is |
| 577 |
responsible for creating the sub-parser using |
| 578 |
``ExternalEntityParserCreate(context)``, initializing it with the appropriate |
| 579 |
callbacks, and parsing the entity. This handler should return an integer; if it |
| 580 |
returns ``0``, the parser will raise an |
| 581 |
:const:`XML_ERROR_EXTERNAL_ENTITY_HANDLING` error, otherwise parsing will |
| 582 |
continue. |
| 583 |
|
| 584 |
If this handler is not provided, external entities are reported by the |
| 585 |
:attr:`DefaultHandler` callback, if provided. |
| 586 |
|
| 587 |
|
| 588 |
.. _expaterror-objects: |
| 589 |
|
| 590 |
ExpatError Exceptions |
| 591 |
--------------------- |
| 592 |
|
| 593 |
.. sectionauthor:: Fred L. Drake, Jr. <[email protected]> |
| 594 |
|
| 595 |
|
| 596 |
:exc:`ExpatError` exceptions have a number of interesting attributes: |
| 597 |
|
| 598 |
|
| 599 |
.. attribute:: ExpatError.code |
| 600 |
|
| 601 |
Expat's internal error number for the specific error. The |
| 602 |
:data:`errors.messages <xml.parsers.expat.errors.messages>` dictionary maps |
| 603 |
these error numbers to Expat's error messages. For example:: |
| 604 |
|
| 605 |
from xml.parsers.expat import ParserCreate, ExpatError, errors |
| 606 |
|
| 607 |
p = ParserCreate() |
| 608 |
try: |
| 609 |
p.Parse(some_xml_document) |
| 610 |
except ExpatError as err: |
| 611 |
print("Error:", errors.messages[err.code]) |
| 612 |
|
| 613 |
The :mod:`~xml.parsers.expat.errors` module also provides error message |
| 614 |
constants and a dictionary :data:`~xml.parsers.expat.errors.codes` mapping |
| 615 |
these messages back to the error codes, see below. |
| 616 |
|
| 617 |
|
| 618 |
.. attribute:: ExpatError.lineno |
| 619 |
|
| 620 |
Line number on which the error was detected. The first line is numbered ``1``. |
| 621 |
|
| 622 |
|
| 623 |
.. attribute:: ExpatError.offset |
| 624 |
|
| 625 |
Character offset into the line where the error occurred. The first column is |
| 626 |
numbered ``0``. |
| 627 |
|
| 628 |
|
| 629 |
.. _expat-example: |
| 630 |
|
| 631 |
Example |
| 632 |
------- |
| 633 |
|
| 634 |
The following program defines three handlers that just print out their |
| 635 |
arguments. :: |
| 636 |
|
| 637 |
import xml.parsers.expat |
| 638 |
|
| 639 |
# 3 handler functions |
| 640 |
def start_element(name, attrs): |
| 641 |
print('Start element:', name, attrs) |
| 642 |
def end_element(name): |
| 643 |
print('End element:', name) |
| 644 |
def char_data(data): |
| 645 |
print('Character data:', repr(data)) |
| 646 |
|
| 647 |
p = xml.parsers.expat.ParserCreate() |
| 648 |
|
| 649 |
p.StartElementHandler = start_element |
| 650 |
p.EndElementHandler = end_element |
| 651 |
p.CharacterDataHandler = char_data |
| 652 |
|
| 653 |
p.Parse("""<?xml version="1.0"?> |
| 654 |
<parent id="top"><child1 name="paul">Text goes here</child1> |
| 655 |
<child2 name="fred">More text</child2> |
| 656 |
</parent>""", 1) |
| 657 |
|
| 658 |
The output from this program is:: |
| 659 |
|
| 660 |
Start element: parent {'id': 'top'} |
| 661 |
Start element: child1 {'name': 'paul'} |
| 662 |
Character data: 'Text goes here' |
| 663 |
End element: child1 |
| 664 |
Character data: '\n' |
| 665 |
Start element: child2 {'name': 'fred'} |
| 666 |
Character data: 'More text' |
| 667 |
End element: child2 |
| 668 |
Character data: '\n' |
| 669 |
End element: parent |
| 670 |
|
| 671 |
|
| 672 |
.. _expat-content-models: |
| 673 |
|
| 674 |
Content Model Descriptions |
| 675 |
-------------------------- |
| 676 |
|
| 677 |
.. module:: xml.parsers.expat.model |
| 678 |
|
| 679 |
.. sectionauthor:: Fred L. Drake, Jr. <[email protected]> |
| 680 |
|
| 681 |
Content models are described using nested tuples. Each tuple contains four |
| 682 |
values: the type, the quantifier, the name, and a tuple of children. Children |
| 683 |
are simply additional content model descriptions. |
| 684 |
|
| 685 |
The values of the first two fields are constants defined in the |
| 686 |
:mod:`!xml.parsers.expat.model` module. These constants can be collected in two |
| 687 |
groups: the model type group and the quantifier group. |
| 688 |
|
| 689 |
The constants in the model type group are: |
| 690 |
|
| 691 |
|
| 692 |
.. data:: XML_CTYPE_ANY |
| 693 |
:noindex: |
| 694 |
|
| 695 |
The element named by the model name was declared to have a content model of |
| 696 |
``ANY``. |
| 697 |
|
| 698 |
|
| 699 |
.. data:: XML_CTYPE_CHOICE |
| 700 |
:noindex: |
| 701 |
|
| 702 |
The named element allows a choice from a number of options; this is used for |
| 703 |
content models such as ``(A | B | C)``. |
| 704 |
|
| 705 |
|
| 706 |
.. data:: XML_CTYPE_EMPTY |
| 707 |
:noindex: |
| 708 |
|
| 709 |
Elements which are declared to be ``EMPTY`` have this model type. |
| 710 |
|
| 711 |
|
| 712 |
.. data:: XML_CTYPE_MIXED |
| 713 |
:noindex: |
| 714 |
|
| 715 |
|
| 716 |
.. data:: XML_CTYPE_NAME |
| 717 |
:noindex: |
| 718 |
|
| 719 |
|
| 720 |
.. data:: XML_CTYPE_SEQ |
| 721 |
:noindex: |
| 722 |
|
| 723 |
Models which represent a series of models which follow one after the other are |
| 724 |
indicated with this model type. This is used for models such as ``(A, B, C)``. |
| 725 |
|
| 726 |
The constants in the quantifier group are: |
| 727 |
|
| 728 |
|
| 729 |
.. data:: XML_CQUANT_NONE |
| 730 |
:noindex: |
| 731 |
|
| 732 |
No modifier is given, so it can appear exactly once, as for ``A``. |
| 733 |
|
| 734 |
|
| 735 |
.. data:: XML_CQUANT_OPT |
| 736 |
:noindex: |
| 737 |
|
| 738 |
The model is optional: it can appear once or not at all, as for ``A?``. |
| 739 |
|
| 740 |
|
| 741 |
.. data:: XML_CQUANT_PLUS |
| 742 |
:noindex: |
| 743 |
|
| 744 |
The model must occur one or more times (like ``A+``). |
| 745 |
|
| 746 |
|
| 747 |
.. data:: XML_CQUANT_REP |
| 748 |
:noindex: |
| 749 |
|
| 750 |
The model must occur zero or more times, as for ``A*``. |
| 751 |
|
| 752 |
|
| 753 |
.. _expat-errors: |
| 754 |
|
| 755 |
Expat error constants |
| 756 |
--------------------- |
| 757 |
|
| 758 |
.. module:: xml.parsers.expat.errors |
| 759 |
|
| 760 |
The following constants are provided in the :mod:`!xml.parsers.expat.errors` |
| 761 |
module. These constants are useful in interpreting some of the attributes of |
| 762 |
the :exc:`ExpatError` exception objects raised when an error has occurred. |
| 763 |
Since for backwards compatibility reasons, the constants' value is the error |
| 764 |
*message* and not the numeric error *code*, you do this by comparing its |
| 765 |
:attr:`code` attribute with |
| 766 |
:samp:`errors.codes[errors.XML_ERROR_{CONSTANT_NAME}]`. |
| 767 |
|
| 768 |
The ``errors`` module has the following attributes: |
| 769 |
|
| 770 |
.. data:: codes |
| 771 |
|
| 772 |
A dictionary mapping string descriptions to their error codes. |
| 773 |
|
| 774 |
.. versionadded:: 3.2 |
| 775 |
|
| 776 |
|
| 777 |
.. data:: messages |
| 778 |
|
| 779 |
A dictionary mapping numeric error codes to their string descriptions. |
| 780 |
|
| 781 |
.. versionadded:: 3.2 |
| 782 |
|
| 783 |
|
| 784 |
.. data:: XML_ERROR_ASYNC_ENTITY |
| 785 |
|
| 786 |
|
| 787 |
.. data:: XML_ERROR_ATTRIBUTE_EXTERNAL_ENTITY_REF |
| 788 |
|
| 789 |
An entity reference in an attribute value referred to an external entity instead |
| 790 |
of an internal entity. |
| 791 |
|
| 792 |
|
| 793 |
.. data:: XML_ERROR_BAD_CHAR_REF |
| 794 |
|
| 795 |
A character reference referred to a character which is illegal in XML (for |
| 796 |
example, character ``0``, or '``�``'). |
| 797 |
|
| 798 |
|
| 799 |
.. data:: XML_ERROR_BINARY_ENTITY_REF |
| 800 |
|
| 801 |
An entity reference referred to an entity which was declared with a notation, so |
| 802 |
cannot be parsed. |
| 803 |
|
| 804 |
|
| 805 |
.. data:: XML_ERROR_DUPLICATE_ATTRIBUTE |
| 806 |
|
| 807 |
An attribute was used more than once in a start tag. |
| 808 |
|
| 809 |
|
| 810 |
.. data:: XML_ERROR_INCORRECT_ENCODING |
| 811 |
|
| 812 |
|
| 813 |
.. data:: XML_ERROR_INVALID_TOKEN |
| 814 |
|
| 815 |
Raised when an input byte could not properly be assigned to a character; for |
| 816 |
example, a NUL byte (value ``0``) in a UTF-8 input stream. |
| 817 |
|
| 818 |
|
| 819 |
.. data:: XML_ERROR_JUNK_AFTER_DOC_ELEMENT |
| 820 |
|
| 821 |
Something other than whitespace occurred after the document element. |
| 822 |
|
| 823 |
|
| 824 |
.. data:: XML_ERROR_MISPLACED_XML_PI |
| 825 |
|
| 826 |
An XML declaration was found somewhere other than the start of the input data. |
| 827 |
|
| 828 |
|
| 829 |
.. data:: XML_ERROR_NO_ELEMENTS |
| 830 |
|
| 831 |
The document contains no elements (XML requires all documents to contain exactly |
| 832 |
one top-level element).. |
| 833 |
|
| 834 |
|
| 835 |
.. data:: XML_ERROR_NO_MEMORY |
| 836 |
|
| 837 |
Expat was not able to allocate memory internally. |
| 838 |
|
| 839 |
|
| 840 |
.. data:: XML_ERROR_PARAM_ENTITY_REF |
| 841 |
|
| 842 |
A parameter entity reference was found where it was not allowed. |
| 843 |
|
| 844 |
|
| 845 |
.. data:: XML_ERROR_PARTIAL_CHAR |
| 846 |
|
| 847 |
An incomplete character was found in the input. |
| 848 |
|
| 849 |
|
| 850 |
.. data:: XML_ERROR_RECURSIVE_ENTITY_REF |
| 851 |
|
| 852 |
An entity reference contained another reference to the same entity; possibly via |
| 853 |
a different name, and possibly indirectly. |
| 854 |
|
| 855 |
|
| 856 |
.. data:: XML_ERROR_SYNTAX |
| 857 |
|
| 858 |
Some unspecified syntax error was encountered. |
| 859 |
|
| 860 |
|
| 861 |
.. data:: XML_ERROR_TAG_MISMATCH |
| 862 |
|
| 863 |
An end tag did not match the innermost open start tag. |
| 864 |
|
| 865 |
|
| 866 |
.. data:: XML_ERROR_UNCLOSED_TOKEN |
| 867 |
|
| 868 |
Some token (such as a start tag) was not closed before the end of the stream or |
| 869 |
the next token was encountered. |
| 870 |
|
| 871 |
|
| 872 |
.. data:: XML_ERROR_UNDEFINED_ENTITY |
| 873 |
|
| 874 |
A reference was made to an entity which was not defined. |
| 875 |
|
| 876 |
|
| 877 |
.. data:: XML_ERROR_UNKNOWN_ENCODING |
| 878 |
|
| 879 |
The document encoding is not supported by Expat. |
| 880 |
|
| 881 |
|
| 882 |
.. data:: XML_ERROR_UNCLOSED_CDATA_SECTION |
| 883 |
|
| 884 |
A CDATA marked section was not closed. |
| 885 |
|
| 886 |
|
| 887 |
.. data:: XML_ERROR_EXTERNAL_ENTITY_HANDLING |
| 888 |
|
| 889 |
|
| 890 |
.. data:: XML_ERROR_NOT_STANDALONE |
| 891 |
|
| 892 |
The parser determined that the document was not "standalone" though it declared |
| 893 |
itself to be in the XML declaration, and the :attr:`NotStandaloneHandler` was |
| 894 |
set and returned ``0``. |
| 895 |
|
| 896 |
|
| 897 |
.. data:: XML_ERROR_UNEXPECTED_STATE |
| 898 |
|
| 899 |
|
| 900 |
.. data:: XML_ERROR_ENTITY_DECLARED_IN_PE |
| 901 |
|
| 902 |
|
| 903 |
.. data:: XML_ERROR_FEATURE_REQUIRES_XML_DTD |
| 904 |
|
| 905 |
An operation was requested that requires DTD support to be compiled in, but |
| 906 |
Expat was configured without DTD support. This should never be reported by a |
| 907 |
standard build of the :mod:`!xml.parsers.expat` module. |
| 908 |
|
| 909 |
|
| 910 |
.. data:: XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING |
| 911 |
|
| 912 |
A behavioral change was requested after parsing started that can only be changed |
| 913 |
before parsing has started. This is (currently) only raised by |
| 914 |
:meth:`UseForeignDTD`. |
| 915 |
|
| 916 |
|
| 917 |
.. data:: XML_ERROR_UNBOUND_PREFIX |
| 918 |
|
| 919 |
An undeclared prefix was found when namespace processing was enabled. |
| 920 |
|
| 921 |
|
| 922 |
.. data:: XML_ERROR_UNDECLARING_PREFIX |
| 923 |
|
| 924 |
The document attempted to remove the namespace declaration associated with a |
| 925 |
prefix. |
| 926 |
|
| 927 |
|
| 928 |
.. data:: XML_ERROR_INCOMPLETE_PE |
| 929 |
|
| 930 |
A parameter entity contained incomplete markup. |
| 931 |
|
| 932 |
|
| 933 |
.. data:: XML_ERROR_XML_DECL |
| 934 |
|
| 935 |
The document contained no document element at all. |
| 936 |
|
| 937 |
|
| 938 |
.. data:: XML_ERROR_TEXT_DECL |
| 939 |
|
| 940 |
There was an error parsing a text declaration in an external entity. |
| 941 |
|
| 942 |
|
| 943 |
.. data:: XML_ERROR_PUBLICID |
| 944 |
|
| 945 |
Characters were found in the public id that are not allowed. |
| 946 |
|
| 947 |
|
| 948 |
.. data:: XML_ERROR_SUSPENDED |
| 949 |
|
| 950 |
The requested operation was made on a suspended parser, but isn't allowed. This |
| 951 |
includes attempts to provide additional input or to stop the parser. |
| 952 |
|
| 953 |
|
| 954 |
.. data:: XML_ERROR_NOT_SUSPENDED |
| 955 |
|
| 956 |
An attempt to resume the parser was made when the parser had not been suspended. |
| 957 |
|
| 958 |
|
| 959 |
.. data:: XML_ERROR_ABORTED |
| 960 |
|
| 961 |
This should not be reported to Python applications. |
| 962 |
|
| 963 |
|
| 964 |
.. data:: XML_ERROR_FINISHED |
| 965 |
|
| 966 |
The requested operation was made on a parser which was finished parsing input, |
| 967 |
but isn't allowed. This includes attempts to provide additional input or to |
| 968 |
stop the parser. |
| 969 |
|
| 970 |
|
| 971 |
.. data:: XML_ERROR_SUSPEND_PE |
| 972 |
|
| 973 |
|
| 974 |
.. data:: XML_ERROR_RESERVED_PREFIX_XML |
| 975 |
|
| 976 |
An attempt was made to |
| 977 |
undeclare reserved namespace prefix ``xml`` |
| 978 |
or to bind it to another namespace URI. |
| 979 |
|
| 980 |
|
| 981 |
.. data:: XML_ERROR_RESERVED_PREFIX_XMLNS |
| 982 |
|
| 983 |
An attempt was made to declare or undeclare reserved namespace prefix ``xmlns``. |
| 984 |
|
| 985 |
|
| 986 |
.. data:: XML_ERROR_RESERVED_NAMESPACE_URI |
| 987 |
|
| 988 |
An attempt was made to bind the URI of one the reserved namespace |
| 989 |
prefixes ``xml`` and ``xmlns`` to another namespace prefix. |
| 990 |
|
| 991 |
|
| 992 |
.. data:: XML_ERROR_INVALID_ARGUMENT |
| 993 |
|
| 994 |
This should not be reported to Python applications. |
| 995 |
|
| 996 |
|
| 997 |
.. data:: XML_ERROR_NO_BUFFER |
| 998 |
|
| 999 |
This should not be reported to Python applications. |
| 1000 |
|
| 1001 |
|
| 1002 |
.. data:: XML_ERROR_AMPLIFICATION_LIMIT_BREACH |
| 1003 |
|
| 1004 |
The limit on input amplification factor (from DTD and entities) |
| 1005 |
has been breached. |
| 1006 |
|
| 1007 |
|
| 1008 |
.. data:: XML_ERROR_NOT_STARTED |
| 1009 |
|
| 1010 |
The parser was tried to be stopped or suspended before it started. |
| 1011 |
|
| 1012 |
.. versionadded:: 3.14 |
| 1013 |
|
| 1014 |
|
| 1015 |
.. rubric:: Footnotes |
| 1016 |
|
| 1017 |
.. [1] The encoding string included in XML output should conform to the |
| 1018 |
appropriate standards. For example, "UTF-8" is valid, but "UTF8" is |
| 1019 |
not. See https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl |
| 1020 |
and https://www.iana.org/assignments/character-sets/character-sets.xhtml. |
| 1021 |
|
| 1022 |
.. |xml-non-root-parser| replace:: :ref:`non-root <xmlparser-non-root>` |