| 1 |
Metadata-Version: 2.1 |
| 2 |
Name: funcparserlib |
| 3 |
Version: 2.0.0a0 |
| 4 |
Summary: Recursive descent parsing library based on functional combinators |
| 5 |
Home-page: https://funcparserlib.pirx.ru |
| 6 |
License: MIT |
| 7 |
Author: Andrey Vlasovskikh |
| 8 |
Author-email: [email protected] |
| 9 |
Requires-Python: >=3.8,<4.0 |
| 10 |
Classifier: Development Status :: 5 - Production/Stable |
| 11 |
Classifier: Intended Audience :: Developers |
| 12 |
Classifier: License :: OSI Approved :: MIT License |
| 13 |
Classifier: Operating System :: OS Independent |
| 14 |
Classifier: Programming Language :: Python :: 3 |
| 15 |
Classifier: Programming Language :: Python :: 3.8 |
| 16 |
Classifier: Programming Language :: Python :: 3.9 |
| 17 |
Classifier: Programming Language :: Python :: 3.10 |
| 18 |
Classifier: Programming Language :: Python :: 3.11 |
| 19 |
Classifier: Programming Language :: Python :: 3.12 |
| 20 |
Project-URL: Repository, https://github.com/vlasovskikh/funcparserlib |
| 21 |
Description-Content-Type: text/markdown |
| 22 |
|
| 23 |
Funcparserlib |
| 24 |
============= |
| 25 |
|
| 26 |
Recursive descent parsing library for Python based on functional combinators. |
| 27 |
|
| 28 |
[](https://pypi.org/project/funcparserlib/) |
| 29 |
[](https://pypi.org/project/funcparserlib/) |
| 30 |
|
| 31 |
|
| 32 |
Description |
| 33 |
----------- |
| 34 |
|
| 35 |
The primary focus of `funcparserlib` is **parsing little languages** or **external DSLs** (domain specific languages). |
| 36 |
|
| 37 |
Parsers made with `funcparserlib` are pure-Python LL(\*) parsers. It means that it's **very easy to write parsers** without thinking about lookaheads and other hardcore parsing stuff. However, recursive descent parsing is a rather slow method compared to LL(k) or LR(k) algorithms. Still, parsing with `funcparserlib` is **at least twice faster than PyParsing**, a very popular library for Python. |
| 38 |
|
| 39 |
The source code of `funcparserlib` is only 1.2K lines of code, with lots of comments. Its API is fully type hinted. It features the longest parsed prefix error reporting, as well as a tiny lexer generator for token position tracking. |
| 40 |
|
| 41 |
The idea of parser combinators used in `funcparserlib` comes from the [Introduction to Functional Programming](https://www.cl.cam.ac.uk/teaching/Lectures/funprog-jrh-1996/) course. We have converted it from ML into Python. |
| 42 |
|
| 43 |
|
| 44 |
Installation |
| 45 |
------------ |
| 46 |
|
| 47 |
You can install `funcparserlib` from [PyPI](https://pypi.org/project/funcparserlib/): |
| 48 |
|
| 49 |
```shell |
| 50 |
$ pip install funcparserlib |
| 51 |
``` |
| 52 |
|
| 53 |
There are no dependencies on other libraries. |
| 54 |
|
| 55 |
|
| 56 |
Documentation |
| 57 |
------------- |
| 58 |
|
| 59 |
* [Getting Started](https://funcparserlib.pirx.ru/getting-started/) |
| 60 |
* Your **starting point** with `funcparserlib` |
| 61 |
* [API Reference](https://funcparserlib.pirx.ru/api/) |
| 62 |
* Learn the details of the API |
| 63 |
|
| 64 |
There are several examples available in the `tests/` directory: |
| 65 |
|
| 66 |
* [GraphViz DOT parser](https://github.com/vlasovskikh/funcparserlib/blob/master/tests/dot.py) |
| 67 |
* [JSON parser](https://github.com/vlasovskikh/funcparserlib/blob/master/tests/json.py) |
| 68 |
|
| 69 |
See also [the changelog](https://funcparserlib.pirx.ru/changes/). |
| 70 |
|
| 71 |
|
| 72 |
Example |
| 73 |
------- |
| 74 |
|
| 75 |
Let's consider a little language of **numeric expressions** with a syntax similar to Python expressions. Here are some expression strings in this language: |
| 76 |
|
| 77 |
``` |
| 78 |
0 |
| 79 |
1 + 2 + 3 |
| 80 |
-1 + 2 ** 32 |
| 81 |
3.1415926 * (2 + 7.18281828e-1) * 42 |
| 82 |
``` |
| 83 |
|
| 84 |
|
| 85 |
Here is **the complete source code** of the tokenizer and the parser for this language written using `funcparserlib`: |
| 86 |
|
| 87 |
```python |
| 88 |
from typing import List, Tuple, Union |
| 89 |
from dataclasses import dataclass |
| 90 |
|
| 91 |
from funcparserlib.lexer import make_tokenizer, TokenSpec, Token |
| 92 |
from funcparserlib.parser import tok, Parser, many, forward_decl, finished |
| 93 |
|
| 94 |
|
| 95 |
@dataclass |
| 96 |
class BinaryExpr: |
| 97 |
op: str |
| 98 |
left: "Expr" |
| 99 |
right: "Expr" |
| 100 |
|
| 101 |
|
| 102 |
Expr = Union[BinaryExpr, int, float] |
| 103 |
|
| 104 |
|
| 105 |
def tokenize(s: str) -> List[Token]: |
| 106 |
specs = [ |
| 107 |
TokenSpec("whitespace", r"\s+"), |
| 108 |
TokenSpec("float", r"[+\-]?\d+\.\d*([Ee][+\-]?\d+)*"), |
| 109 |
TokenSpec("int", r"[+\-]?\d+"), |
| 110 |
TokenSpec("op", r"(\*\*)|[+\-*/()]"), |
| 111 |
] |
| 112 |
tokenizer = make_tokenizer(specs) |
| 113 |
return [t for t in tokenizer(s) if t.type != "whitespace"] |
| 114 |
|
| 115 |
|
| 116 |
def parse(tokens: List[Token]) -> Expr: |
| 117 |
int_num = tok("int") >> int |
| 118 |
float_num = tok("float") >> float |
| 119 |
number = int_num | float_num |
| 120 |
|
| 121 |
expr: Parser[Token, Expr] = forward_decl() |
| 122 |
parenthesized = -op("(") + expr + -op(")") |
| 123 |
primary = number | parenthesized |
| 124 |
power = primary + many(op("**") + primary) >> to_expr |
| 125 |
term = power + many((op("*") | op("/")) + power) >> to_expr |
| 126 |
sum = term + many((op("+") | op("-")) + term) >> to_expr |
| 127 |
expr.define(sum) |
| 128 |
|
| 129 |
document = expr + -finished |
| 130 |
|
| 131 |
return document.parse(tokens) |
| 132 |
|
| 133 |
|
| 134 |
def op(name: str) -> Parser[Token, str]: |
| 135 |
return tok("op", name) |
| 136 |
|
| 137 |
|
| 138 |
def to_expr(args: Tuple[Expr, List[Tuple[str, Expr]]]) -> Expr: |
| 139 |
first, rest = args |
| 140 |
result = first |
| 141 |
for op, expr in rest: |
| 142 |
result = BinaryExpr(op, result, expr) |
| 143 |
return result |
| 144 |
``` |
| 145 |
|
| 146 |
Now, consider this numeric expression: `3.1415926 * (2 + 7.18281828e-1) * 42`. |
| 147 |
|
| 148 |
Let's `tokenize()` it using the tokenizer we've created with `funcparserlib.lexer`: |
| 149 |
|
| 150 |
``` |
| 151 |
[ |
| 152 |
Token('float', '3.1415926'), |
| 153 |
Token('op', '*'), |
| 154 |
Token('op', '('), |
| 155 |
Token('int', '2'), |
| 156 |
Token('op', '+'), |
| 157 |
Token('float', '7.18281828e-1'), |
| 158 |
Token('op', ')'), |
| 159 |
Token('op', '*'), |
| 160 |
Token('int', '42'), |
| 161 |
] |
| 162 |
``` |
| 163 |
|
| 164 |
Let's `parse()` these tokens into an expression tree using our parser created with `funcparserlib.parser`: |
| 165 |
|
| 166 |
``` |
| 167 |
BinaryExpr( |
| 168 |
op='*', |
| 169 |
left=BinaryExpr( |
| 170 |
op='*', |
| 171 |
left=3.1415926, |
| 172 |
right=BinaryExpr(op='+', left=2, right=0.718281828), |
| 173 |
), |
| 174 |
right=42, |
| 175 |
) |
| 176 |
``` |
| 177 |
|
| 178 |
Learn how to write this parser using `funcparserlib` in the [Getting Started](https://funcparserlib.pirx.ru/getting-started/) guide! |
| 179 |
|
| 180 |
|
| 181 |
Used By |
| 182 |
------- |
| 183 |
|
| 184 |
Some open-source projects that use `funcparserlib` as an explicit dependency: |
| 185 |
|
| 186 |
* [Hy](https://github.com/hylang/hy), a Lisp dialect that's embedded in Python |
| 187 |
* 4.7K stars, version `~=1.0`, Python 3.8+ |
| 188 |
* [Splash](https://github.com/scrapinghub/splash), a JavaScript rendering service with HTTP API, by Scrapinghub |
| 189 |
* 3.9K stars, version `*`. Python 3 in Docker |
| 190 |
* [graphite-beacon](https://github.com/klen/graphite-beacon), a simple alerting system for Graphite metrics |
| 191 |
* 453 stars, version `==0.3.6`, Python 2 and 3 |
| 192 |
* [blockdiag](https://github.com/blockdiag/blockdiag), generates block-diagram image file from spec-text file |
| 193 |
* 194 stars, version `>= 1.0.0a0`, Python 3.7+ |
| 194 |
* [kll](https://github.com/kiibohd/kll), Keyboard Layout Language (KLL) compiler |
| 195 |
* 113 stars, copied source code, Python 3.5+ |
| 196 |
|
| 197 |
|
| 198 |
Next |
| 199 |
---- |
| 200 |
|
| 201 |
Read the [Getting Started](https://funcparserlib.pirx.ru/getting-started/) guide to start learning `funcparserlib`. |
| 202 |
|