| 1 |
============================= |
| 2 |
What's New in Pyparsing 3.2.x |
| 3 |
============================= |
| 4 |
|
| 5 |
:author: Paul McGuire |
| 6 |
|
| 7 |
:date: September, 2025 |
| 8 |
|
| 9 |
:abstract: This document summarizes the changes made |
| 10 |
in the 3.2.x releases of pyparsing. |
| 11 |
|
| 12 |
.. contents:: :depth: 4 |
| 13 |
|
| 14 |
|
| 15 |
Supported Python versions |
| 16 |
========================= |
| 17 |
|
| 18 |
- Added support for Python 3.13 and 3.14. |
| 19 |
|
| 20 |
- Python versions before 3.9 are no longer supported. |
| 21 |
Removed legacy Py2.x support and other deprecated features. Pyparsing |
| 22 |
now requires Python 3.9 or later. If you are using an earlier 3.x |
| 23 |
version of Python, use pyparsing 3.1; for Python 2.x, use Pyparsing |
| 24 |
2.4.7. |
| 25 |
|
| 26 |
|
| 27 |
New Features |
| 28 |
============ |
| 29 |
|
| 30 |
- Added type annotations to remainder of ``pyparsing`` package, and added ``mypy`` |
| 31 |
run to ``tox.ini``, so that type annotations are now run as part of pyparsing's CI. |
| 32 |
|
| 33 |
- Exception message format can now be customized, by overriding |
| 34 |
``ParseBaseException.format_message``:: |
| 35 |
|
| 36 |
def custom_exception_message(exc) -> str: |
| 37 |
found_phrase = f", found {exc.found}" if exc.found else "" |
| 38 |
return f"{exc.lineno}:{exc.column} {exc.msg}{found_phrase}" |
| 39 |
|
| 40 |
ParseBaseException.formatted_message = custom_exception_message |
| 41 |
|
| 42 |
- ``run_tests`` now detects if an exception is raised in a parse action, and will |
| 43 |
report it with an enhanced error message, with the exception type, string, |
| 44 |
and parse action name. |
| 45 |
|
| 46 |
- ``QuotedString`` now handles translation of escaped integer, hex, octal, and |
| 47 |
Unicode sequences to their corresponding characters. |
| 48 |
|
| 49 |
- Defined a more performant regular expression used internally by ``common_html_entity``. |
| 50 |
|
| 51 |
- ``Regex`` instances can now be created using a callable that takes no arguments |
| 52 |
and just returns a string or a compiled regular expression, so that creating complex |
| 53 |
regular expression patterns can be deferred until they are actually used for the first |
| 54 |
time in the parser. |
| 55 |
|
| 56 |
- Fixed the displayed output of ``Regex`` terms to deduplicate repeated backslashes, |
| 57 |
for easier reading in debugging, printing, and railroad diagrams. |
| 58 |
|
| 59 |
- Railroad diagramming improvements |
| 60 |
|
| 61 |
- Updated generated railroad diagrams to make non-terminal elements links to their related |
| 62 |
sub-diagrams. This *greatly* improves navigation of the diagram, especially for |
| 63 |
large, complex parsers. |
| 64 |
|
| 65 |
- Fixed railroad diagrams that get generated with a parser containing a `Regex` element |
| 66 |
defined using a verbose pattern - the pattern gets flattened and comments removed |
| 67 |
before creating the corresponding diagram element. |
| 68 |
|
| 69 |
- Added optional argument `show_hidden` to `ParserElement.create_diagram()` to show |
| 70 |
elements that are used internally by pyparsing, but are not part of the actual |
| 71 |
parser grammar. For instance, the `Tag` class can insert values into the parsed |
| 72 |
results but it does not actually parse any input, so by default it is not included |
| 73 |
in a railroad diagram. By calling `create_diagram()` with `show_hidden = True`, |
| 74 |
these internal elements will be included. (You can see this in the tag_metadata.py |
| 75 |
script in the examples directory.) |
| 76 |
|
| 77 |
- Fixed the displayed output of `Regex` terms to deduplicate repeated backslashes, |
| 78 |
for easier reading in debugging, printing, and railroad diagrams. |
| 79 |
|
| 80 |
- Simplified railroad diagrams emitted for parsers using `infix_notation()`, by hiding |
| 81 |
lookahead terms. Renamed internally generated expressions for clarity, and improved |
| 82 |
diagramming. |
| 83 |
|
| 84 |
|
| 85 |
API Changes |
| 86 |
=========== |
| 87 |
|
| 88 |
Possible breaking changes |
| 89 |
------------------------- |
| 90 |
- Fixed code in ``ParseElementEnhance`` subclasses that |
| 91 |
replaced detailed exception messages raised in contained expressions with a |
| 92 |
less-specific and less-informative generic exception message and location. |
| 93 |
|
| 94 |
If your code has conditional logic based on the message content in raised |
| 95 |
``ParseExceptions``, this bugfix may require changes in your code. |
| 96 |
|
| 97 |
- Fixed bug in ``transform_string()`` where whitespace |
| 98 |
in the input string was not properly preserved in the output string. |
| 99 |
|
| 100 |
If your code uses ``transform_string``, this bugfix may require changes in |
| 101 |
your code. |
| 102 |
|
| 103 |
- Fixed bug where an ``IndexError`` raised in a parse action was |
| 104 |
incorrectly handled as an ``IndexError`` raised as part of the ``ParserElement`` |
| 105 |
parsing methods, and reraised as a ``ParseException``. Now an ``IndexError`` |
| 106 |
that raises inside a parse action will properly propagate out as an ``IndexError``. |
| 107 |
|
| 108 |
If your code raises ``IndexError`` in parse actions, this bugfix may require |
| 109 |
changes in your code. |
| 110 |
|
| 111 |
|
| 112 |
Additional API changes |
| 113 |
---------------------- |
| 114 |
- Added optional ``flatten`` Boolean argument to ``ParseResults.as_list()``, to |
| 115 |
return the parsed values in a flattened list. |
| 116 |
|
| 117 |
- Added ``indent`` and ``base_1`` arguments to ``pyparsing.testing.with_line_numbers``. When |
| 118 |
using ``with_line_numbers`` inside a parse action, set ``base_1`` =False, since the |
| 119 |
reported ``loc`` value is 0-based. ``indent`` can be a leading string (typically of |
| 120 |
spaces or tabs) to indent the numbered string passed to ``with_line_numbers``. |
| 121 |
|
| 122 |
|
| 123 |
New / Enhanced Examples |
| 124 |
======================= |
| 125 |
- Added query syntax to ``mongodb_query_expression.py`` with: |
| 126 |
|
| 127 |
- better support for array fields ("contains all", |
| 128 |
"contains any", and "contains none") |
| 129 |
- "like" and "not like" operators to support SQL "%" wildcard matching |
| 130 |
and "=~" operator to support regex matching |
| 131 |
- text search using "search for" |
| 132 |
- dates and datetimes as query values |
| 133 |
- ``a[0]`` style array referencing |
| 134 |
|
| 135 |
- Added ``lox_parser.py`` example, a parser for the Lox language used as a tutorial in |
| 136 |
Robert Nystrom's "Crafting Interpreters" (http://craftinginterpreters.com/). |
| 137 |
|
| 138 |
- Added ``complex_chemical_formulas.py`` example, to add parsing capability for |
| 139 |
formulas such as "Ba(BrO₃)₂·H₂O". |
| 140 |
|
| 141 |
- Updated ``tag_emitter.py`` to use new ``Tag`` class, introduced in pyparsing |
| 142 |
3.1.3. |
| 143 |
|
| 144 |
|
| 145 |
Acknowledgments |
| 146 |
=============== |
| 147 |
Again, thanks to the many contributors who submitted issues, questions, suggestions, |
| 148 |
and PRs. |