Debian Codegraph @

PKG-INFO

raw
1 Metadata-Version: 2.1
2 Name: funcparserlib
3 Version: 2.0.0a0
4 Summary: Recursive descent parsing library based on functional combinators
5 Home-page: https://funcparserlib.pirx.ru
6 License: MIT
7 Author: Andrey Vlasovskikh
8 Author-email: [email protected]
9 Requires-Python: >=3.8,<4.0
10 Classifier: Development Status :: 5 - Production/Stable
11 Classifier: Intended Audience :: Developers
12 Classifier: License :: OSI Approved :: MIT License
13 Classifier: Operating System :: OS Independent
14 Classifier: Programming Language :: Python :: 3
15 Classifier: Programming Language :: Python :: 3.8
16 Classifier: Programming Language :: Python :: 3.9
17 Classifier: Programming Language :: Python :: 3.10
18 Classifier: Programming Language :: Python :: 3.11
19 Classifier: Programming Language :: Python :: 3.12
20 Project-URL: Repository, https://github.com/vlasovskikh/funcparserlib
21 Description-Content-Type: text/markdown
22
23 Funcparserlib
24 =============
25
26 Recursive descent parsing library for Python based on functional combinators.
27
28 [![PyPI](https://img.shields.io/pypi/v/funcparserlib)](https://pypi.org/project/funcparserlib/)
29 [![PyPI - Downloads](https://img.shields.io/pypi/dm/funcparserlib)](https://pypi.org/project/funcparserlib/)
30
31
32 Description
33 -----------
34
35 The primary focus of `funcparserlib` is **parsing little languages** or **external DSLs** (domain specific languages).
36
37 Parsers made with `funcparserlib` are pure-Python LL(\*) parsers. It means that it's **very easy to write parsers** without thinking about lookaheads and other hardcore parsing stuff. However, recursive descent parsing is a rather slow method compared to LL(k) or LR(k) algorithms. Still, parsing with `funcparserlib` is **at least twice faster than PyParsing**, a very popular library for Python.
38
39 The source code of `funcparserlib` is only 1.2K lines of code, with lots of comments. Its API is fully type hinted. It features the longest parsed prefix error reporting, as well as a tiny lexer generator for token position tracking.
40
41 The idea of parser combinators used in `funcparserlib` comes from the [Introduction to Functional Programming](https://www.cl.cam.ac.uk/teaching/Lectures/funprog-jrh-1996/) course. We have converted it from ML into Python.
42
43
44 Installation
45 ------------
46
47 You can install `funcparserlib` from [PyPI](https://pypi.org/project/funcparserlib/):
48
49 ```shell
50 $ pip install funcparserlib
51 ```
52
53 There are no dependencies on other libraries.
54
55
56 Documentation
57 -------------
58
59 * [Getting Started](https://funcparserlib.pirx.ru/getting-started/)
60 * Your **starting point** with `funcparserlib`
61 * [API Reference](https://funcparserlib.pirx.ru/api/)
62 * Learn the details of the API
63
64 There are several examples available in the `tests/` directory:
65
66 * [GraphViz DOT parser](https://github.com/vlasovskikh/funcparserlib/blob/master/tests/dot.py)
67 * [JSON parser](https://github.com/vlasovskikh/funcparserlib/blob/master/tests/json.py)
68
69 See also [the changelog](https://funcparserlib.pirx.ru/changes/).
70
71
72 Example
73 -------
74
75 Let's consider a little language of **numeric expressions** with a syntax similar to Python expressions. Here are some expression strings in this language:
76
77 ```
78 0
79 1 + 2 + 3
80 -1 + 2 ** 32
81 3.1415926 * (2 + 7.18281828e-1) * 42
82 ```
83
84
85 Here is **the complete source code** of the tokenizer and the parser for this language written using `funcparserlib`:
86
87 ```python
88 from typing import List, Tuple, Union
89 from dataclasses import dataclass
90
91 from funcparserlib.lexer import make_tokenizer, TokenSpec, Token
92 from funcparserlib.parser import tok, Parser, many, forward_decl, finished
93
94
95 @dataclass
96 class BinaryExpr:
97 op: str
98 left: "Expr"
99 right: "Expr"
100
101
102 Expr = Union[BinaryExpr, int, float]
103
104
105 def tokenize(s: str) -> List[Token]:
106 specs = [
107 TokenSpec("whitespace", r"\s+"),
108 TokenSpec("float", r"[+\-]?\d+\.\d*([Ee][+\-]?\d+)*"),
109 TokenSpec("int", r"[+\-]?\d+"),
110 TokenSpec("op", r"(\*\*)|[+\-*/()]"),
111 ]
112 tokenizer = make_tokenizer(specs)
113 return [t for t in tokenizer(s) if t.type != "whitespace"]
114
115
116 def parse(tokens: List[Token]) -> Expr:
117 int_num = tok("int") >> int
118 float_num = tok("float") >> float
119 number = int_num | float_num
120
121 expr: Parser[Token, Expr] = forward_decl()
122 parenthesized = -op("(") + expr + -op(")")
123 primary = number | parenthesized
124 power = primary + many(op("**") + primary) >> to_expr
125 term = power + many((op("*") | op("/")) + power) >> to_expr
126 sum = term + many((op("+") | op("-")) + term) >> to_expr
127 expr.define(sum)
128
129 document = expr + -finished
130
131 return document.parse(tokens)
132
133
134 def op(name: str) -> Parser[Token, str]:
135 return tok("op", name)
136
137
138 def to_expr(args: Tuple[Expr, List[Tuple[str, Expr]]]) -> Expr:
139 first, rest = args
140 result = first
141 for op, expr in rest:
142 result = BinaryExpr(op, result, expr)
143 return result
144 ```
145
146 Now, consider this numeric expression: `3.1415926 * (2 + 7.18281828e-1) * 42`.
147
148 Let's `tokenize()` it using the tokenizer we've created with `funcparserlib.lexer`:
149
150 ```
151 [
152 Token('float', '3.1415926'),
153 Token('op', '*'),
154 Token('op', '('),
155 Token('int', '2'),
156 Token('op', '+'),
157 Token('float', '7.18281828e-1'),
158 Token('op', ')'),
159 Token('op', '*'),
160 Token('int', '42'),
161 ]
162 ```
163
164 Let's `parse()` these tokens into an expression tree using our parser created with `funcparserlib.parser`:
165
166 ```
167 BinaryExpr(
168 op='*',
169 left=BinaryExpr(
170 op='*',
171 left=3.1415926,
172 right=BinaryExpr(op='+', left=2, right=0.718281828),
173 ),
174 right=42,
175 )
176 ```
177
178 Learn how to write this parser using `funcparserlib` in the [Getting Started](https://funcparserlib.pirx.ru/getting-started/) guide!
179
180
181 Used By
182 -------
183
184 Some open-source projects that use `funcparserlib` as an explicit dependency:
185
186 * [Hy](https://github.com/hylang/hy), a Lisp dialect that's embedded in Python
187 * 4.7K stars, version `~=1.0`, Python 3.8+
188 * [Splash](https://github.com/scrapinghub/splash), a JavaScript rendering service with HTTP API, by Scrapinghub
189 * 3.9K stars, version `*`. Python 3 in Docker
190 * [graphite-beacon](https://github.com/klen/graphite-beacon), a simple alerting system for Graphite metrics
191 * 453 stars, version `==0.3.6`, Python 2 and 3
192 * [blockdiag](https://github.com/blockdiag/blockdiag), generates block-diagram image file from spec-text file
193 * 194 stars, version `>= 1.0.0a0`, Python 3.7+
194 * [kll](https://github.com/kiibohd/kll), Keyboard Layout Language (KLL) compiler
195 * 113 stars, copied source code, Python 3.5+
196
197
198 Next
199 ----
200
201 Read the [Getting Started](https://funcparserlib.pirx.ru/getting-started/) guide to start learning `funcparserlib`.
202