Debian Codegraph @

docs/whats_new_in_3_1.rst

raw
1 =============================
2 What's New in Pyparsing 3.1.0
3 =============================
4
5 :author: Paul McGuire
6
7 :date: October, 2024
8
9 :abstract: This document summarizes the changes made
10 in the 3.1.x releases of pyparsing.
11
12 .. contents:: :depth: 4
13
14
15 Supported Python versions
16 =========================
17 - Added support for Python 3.12.
18
19 - All internal string expressions using '%' string interpolation and ``str.format()``
20 converted to f-strings.
21
22
23 New Features
24 ============
25 - Added new ``Tag`` ParserElement, for inserting metadata into the parsed results.
26 This allows a parser to add metadata or annotations to the parsed tokens.
27 The ``Tag`` element also accepts an optional ``value`` parameter, defaulting to ``True``.
28 See the new ``tag_metadata.py`` example in the ``examples`` directory.
29
30 Example:
31
32 .. doctest::
33
34 >>> # add tag indicating mood
35 >>> end_punc = "." | ("!" + Tag("enthusiastic"))
36 >>> greeting = "Hello" + Word(alphas) + end_punc
37
38 >>> result = greeting.parse_string("Hello World.")
39 >>> print(result.dump())
40 ['Hello', 'World', '.']
41
42 >>> result = greeting.parse_string("Hello World!")
43 >>> print(result.dump())
44 ['Hello', 'World', '!']
45 - enthusiastic: True
46
47 - Extended ``expr[]`` notation for repetition of ``expr`` to accept a
48 slice, where the slice's stop value indicates a ``stop_on``
49 expression:
50
51 .. testcode::
52
53 test = "BEGIN aaa bbb ccc END"
54 BEGIN, END = Keyword.using_each("BEGIN END".split())
55 body_word = Word(alphas)
56
57 # new slice syntax support
58 expr = BEGIN + Group(body_word[...:END]) + END
59 # equivalent to
60 # BEGIN + Group(ZeroOrMore(body_word, stop_on=END)) + END
61
62 print(expr.parse_string(test))
63
64 Prints:
65
66 .. testoutput::
67
68 ['BEGIN', ['aaa', 'bbb', 'ccc'], 'END']
69
70 - Added new class method ``ParserElement.using_each``, to simplify code
71 that creates a sequence of ``Literals``, ``Keywords``, or other ``ParserElement``
72 subclasses.
73
74 For instance, to define suppressible punctuation, you would previously
75 write::
76
77 LPAR, RPAR, LBRACE, RBRACE, SEMI = map(Suppress, "(){};")
78
79 You can now write::
80
81 LPAR, RPAR, LBRACE, RBRACE, SEMI = Suppress.using_each("(){};")
82
83 ``using_each`` will also accept optional keyword args, which it will
84 pass through to the class initializer. Here is an expression for
85 single-letter variable names that might be used in an algebraic
86 expression:
87
88 .. testcode::
89
90 algebra_var = MatchFirst(
91 Char.using_each(string.ascii_lowercase, as_keyword=True)
92 )
93
94 - Added new builtin ``python_quoted_string``, which will match any form
95 of single-line or multiline quoted strings defined in Python.
96
97 - ``Word`` arguments are now validated if ``min`` and ``max`` are both
98 given, that ``min`` <= ``max``; raises ``ValueError`` if values are invalid.
99
100 - Added '·' (Unicode MIDDLE DOT) to the set of ``Latin1.identbodychars``.
101
102 - Added ``ieee_float`` expression to ``pyparsing.common``, which parses float values,
103 plus "NaN", "Inf", "Infinity".
104
105 - Minor performance speedup in ``trim_arity``, to benefit any parsers using parse actions.
106
107
108 API Changes
109 ===========
110 - ``Optional(expr)`` may now be written as ``expr | ""``
111
112 This will make this code:
113
114 .. testcode::
115
116 "{" + Optional(Literal("A") | Literal("a")) + "}"
117
118 writable as:
119
120 .. testcode::
121
122 "{" + (Literal("A") | Literal("a") | "") + "}"
123
124 Some related changes implemented as part of this work:
125 - ``Literal("")`` now internally generates an ``Empty()`` (and no longer raises an exception)
126 - ``Empty`` is now a subclass of ``Literal``
127
128 - Added new class property ``identifier`` to all Unicode set classes in ``pyparsing.unicode``,
129 using the class's values for ``cls.identchars`` and ``cls.identbodychars``. Now Unicode-aware
130 parsers that formerly wrote:
131
132 .. testcode::
133
134 ppu = pyparsing.unicode
135 ident = Word(ppu.Greek.identchars, ppu.Greek.identbodychars)
136
137 can now write:
138
139 .. testcode::
140
141 ident = ppu.Greek.identifier
142 # or
143 ident = ppu.Ελληνικά.identifier
144
145 - Added bool ``embed`` argument to ``ParserElement.create_diagram()``.
146 When passed as True, the resulting diagram will omit the ``<DOCTYPE>``,
147 ``<HEAD>``, and ``<BODY>`` tags so that it can be embedded in other
148 HTML source. (Useful when embedding a call to ``create_diagram()`` in
149 a PyScript HTML page.)
150
151 - Added ``recurse`` argument to ``ParserElement.set_debug`` to set the
152 debug flag on an expression and all of its sub-expressions.
153
154 - Reworked ``delimited_list`` function into the new ``DelimitedList`` class.
155 ``DelimitedList`` has the same constructor interface as ``delimited_list``, and
156 in this release, ``delimited_list`` changes from a function to a synonym for
157 ``DelimitedList``. ``delimited_list`` and the older ``delimitedList`` method will be
158 deprecated in a future release, in favor of ``DelimitedList``.
159
160 - ``ParseResults`` now has a new method ``deepcopy()``, in addition to the current
161 ``copy()`` method. ``copy()`` only makes a shallow copy - any contained ``ParseResults``
162 are copied as references - changes in the copy will be seen as changes in the original.
163 In many cases, a shallow copy is sufficient, but some applications require a deep copy.
164 ``deepcopy()`` makes a deeper copy: any contained ``ParseResults`` or other mappings or
165 containers are built with copies from the original, and do not get changed if the
166 original is later changed.
167
168 - Added named field "url" to ``pyparsing.common.url``, returning the entire
169 parsed URL string.
170
171 - Added exception type to ``trace_parse_action`` exception output.
172
173 - Added ``<META>`` tag to HTML generated for railroad diagrams to force UTF-8 encoding
174 with older browsers, to better display Unicode parser characters.
175
176 - To address a compatibility issue in RDFLib, added a property setter for the
177 ``ParserElement.name`` property, to call ``ParserElement.set_name``.
178
179 - Modified ``ParserElement.set_name()`` to accept a None value, to clear the defined
180 name and corresponding error message for a ``ParserElement``.
181
182 - Updated railroad diagram generation for ``ZeroOrMore`` and ``OneOrMore`` expressions with
183 ``stop_on`` expressions.
184
185
186 Discontinued Features
187 =====================
188
189 Python 2.x no longer supported
190 ------------------------------
191 Removed Py2.x support and other deprecated features. Pyparsing
192 now requires Python 3.6.8 or later. If you are using an earlier
193 version of Python, you must use a Pyparsing 2.4.x version.
194
195 Other discontinued / deprecated features
196 ----------------------------------------
197 - ``ParserElement.validate()`` is deprecated. It predates the support for left-recursive
198 parsers, and was prone to false positives (warning that a grammar was invalid when
199 it was in fact valid). It will be removed in a future pyparsing release. In its
200 place, developers should use debugging and analytical tools, such as ``ParserElement.set_debug()``
201 and ``ParserElement.create_diagram()``.
202
203
204 Fixed Bugs
205 ==========
206 - Updated ``ci.yml`` permissions to limit default access to source.
207
208 - Updated ``create_diagram()`` code to be compatible with railroad-diagrams package
209 version 3.0.
210
211 - Fixed bug in ``pyparsing.common.url``, when input URL is not alone
212 on an input line.
213
214 - Fixed bug in srange, when parsing escaped '/' and '\' inside a
215 range set.
216
217 - Fixed exception messages for some ``ParserElements`` with custom names,
218 which instead showed their contained expression names.
219
220 - Fixed bug in ``Word`` when ``max=2``. Also added performance enhancement
221 when specifying ``exact`` argument.
222
223 - Fixed bug when parse actions returned an empty string for an expression that
224 had a results name, that the results name was not saved. That is:
225
226 .. doctest::
227
228 >>> expr = Literal("X").add_parse_action(lambda tokens: "")("value")
229 >>> result = expr.parse_string("X")
230 >>> result["value"]
231 ''
232
233 would raise a ``KeyError``. Now empty strings will be saved with the associated
234 results name.
235
236 - Fixed bug in ``SkipTo`` where ignore expressions were not properly handled while
237 scanning for the target expression.
238
239 - Fixed bug in ``NotAny``, where parse actions on the negated expr were not being run.
240 This could cause ``NotAny`` to incorrectly fail if the expr would normally match,
241 but would fail to match if a condition used as a parse action returned False.
242
243 - Fixed ``create_diagram()`` to accept keyword args, to be passed through to the
244 ``template.render()`` method to generate the output HTML.
245
246 - Fixed bug in ``python_quoted_string`` regex.
247
248 - Fixed regression in Word(min).
249
250 - Fixed bug in bad exception messages raised by Forward expressions.
251
252 - Fixed regression in SkipTo, where ignored expressions were not checked when looking
253 for the target expression.
254
255 - Updated pep8 synonym wrappers for better type checking compatibility.
256
257 - Fixed empty error message bug. This _should_ return
258 pyparsing's exception messages to a former, more helpful form. If you have code that
259 parses the exception messages returned by pyparsing, this may require some code
260 changes.
261
262 - Fixed issue where PEP8 compatibility names for ``ParserElement`` static methods were
263 not themselves defined as ``staticmethods``. When called using a ``ParserElement`` instance,
264 this resulted in a ``TypeError`` exception.
265
266 - Fixed some cosmetics/bugs in railroad diagrams:
267
268 - fixed groups being shown even when ``show_groups`` = False
269
270 - show results names as quoted strings when ``show_results_names`` = True
271
272 - only use integer loop counter if repetition > 2
273
274
275 New / Enhanced Examples
276 =======================
277 - Added example ``mongodb_query_expression.py``, to convert human-readable infix query
278 expressions, such as::
279
280 a==100 and b>=200
281
282 and transform them into an equivalent query argument for the pymongo package::
283
284 {'$and': [{'a': 100}, {'b': {'$gte': 200}}]}
285
286 Supports many equality and inequality operators - see the docstring for the
287 ``transform_query`` function for many more examples.
288
289 - ``invRegex.py`` example renamed to ``inv_regex.py`` and updated to PEP-8
290 variable and method naming.
291
292 - Removed examples ``sparser.py`` and ``pymicko.py``, since each included its
293 own GPL license in the header. Since this conflicts with pyparsing's
294 MIT license, they were removed from the distribution to avoid
295 confusion among those making use of them in their own projects.
296
297 - Updated the ``lucene_grammar.py`` example (better support for '*' and '?' wildcards)
298 and corrected the test cases!
299
300 - Added ``bf.py`` Brainf*ck parser/executor example. Illustrates using
301 a pyparsing grammar to parse language syntax, and attach executable AST nodes to
302 the parsed results.
303
304 - Added ``tag_emitter.py`` to examples. This example demonstrates how to insert
305 tags into your parsed results that are not part of the original parsed text.
306
307 - Updated example ``select_parser.py`` to use PEP8 names and added Groups for better retrieval
308 of parsed values from multiple SELECT clauses.
309
310 - Added example ``email_address_parser.py``.
311
312 - Added example ``directx_x_file_parser.py`` to parse DirectX template definitions, and
313 generate a Pyparsing parser from a template to parse .x files.
314
315 - ``delta_time``, ``lua_parser``, ``decaf_parser``, and ``roman_numerals`` examples cleaned up
316 to use latest PEP8 names and add minor enhancements.
317
318 - Fixed bug (and corresponding test code) in ``delta_time`` example that did not handle
319 weekday references in time expressions (like "Monday at 4pm") when the weekday was
320 the same as the current weekday.
321
322
323 Acknowledgments
324 ===============
325 Again, thanks to the many contributors who submitted issues, questions, suggestions,
326 and PRs.