Debian Codegraph @

.cargo/vendor/nom/doc/nom_recipes.md

raw
1 # Common recipes to build nom parsers
2
3 These are short recipes for accomplishing common tasks with nom.
4
5 * [Whitespace](#whitespace)
6 + [Wrapper combinators that eat whitespace before and after a parser](#wrapper-combinators-that-eat-whitespace-before-and-after-a-parser)
7 * [Comments](#comments)
8 + [`// C++/EOL-style comments`](#-ceol-style-comments)
9 + [`/* C-style comments */`](#-c-style-comments-)
10 * [Identifiers](#identifiers)
11 + [`Rust-Style Identifiers`](#rust-style-identifiers)
12 * [Literal Values](#literal-values)
13 + [Escaped Strings](#escaped-strings)
14 + [Integers](#integers)
15 - [Hexadecimal](#hexadecimal)
16 - [Octal](#octal)
17 - [Binary](#binary)
18 - [Decimal](#decimal)
19 + [Floating Point Numbers](#floating-point-numbers)
20
21 ## Whitespace
22
23
24
25 ### Wrapper combinators that eat whitespace before and after a parser
26
27 ```rust
28 use nom::{
29 IResult,
30 Parser,
31 error::ParseError,
32 sequence::delimited,
33 character::complete::multispace0,
34 };
35
36 /// A combinator that takes a parser `inner` and produces a parser that also consumes both leading and
37 /// trailing whitespace, returning the output of `inner`.
38 pub fn ws<'a, O, E: ParseError<&'a str>, F>(
39 inner: F,
40 ) -> impl Parser<&'a str, Output = O, Error = E>
41 where
42 F: Parser<&'a str, Output = O, Error = E>,
43 {
44 delimited(multispace0, inner, multispace0)
45 }
46 ```
47
48 To eat only trailing whitespace, replace `delimited(...)` with `terminated(&inner, multispace0)`.
49 Likewise, the eat only leading whitespace, replace `delimited(...)` with `preceded(multispace0,
50 &inner)`. You can use your own parser instead of `multispace0` if you want to skip a different set
51 of lexemes.
52
53 ## Comments
54
55 ### `// C++/EOL-style comments`
56
57 This version uses `%` to start a comment, does not consume the newline character, and returns an
58 output of `()`.
59
60 ```rust
61 use nom::{
62 IResult,
63 Parser,
64 error::ParseError,
65 combinator::value,
66 sequence::pair,
67 bytes::complete::is_not,
68 character::complete::char,
69 };
70
71 pub fn peol_comment<'a, E: ParseError<&'a str>>(i: &'a str) -> IResult<&'a str, (), E>
72 {
73 value(
74 (), // Output is thrown away.
75 pair(char('%'), is_not("\n\r"))
76 ).parse(i)
77 }
78 ```
79
80 ### `/* C-style comments */`
81
82 Inline comments surrounded with sentinel tags `(*` and `*)`. This version returns an output of `()`
83 and does not handle nested comments.
84
85 ```rust
86 use nom::{
87 IResult,
88 Parser,
89 error::ParseError,
90 combinator::value,
91 bytes::complete::{tag, take_until},
92 };
93
94 pub fn pinline_comment<'a, E: ParseError<&'a str>>(i: &'a str) -> IResult<&'a str, (), E> {
95 value(
96 (), // Output is thrown away.
97 (
98 tag("(*"),
99 take_until("*)"),
100 tag("*)")
101 )
102 ).parse(i)
103 }
104 ```
105
106 ## Identifiers
107
108 ### `Rust-Style Identifiers`
109
110 Parsing identifiers that may start with a letter (or underscore) and may contain underscores,
111 letters and numbers may be parsed like this:
112
113 ```rust
114 use nom::{
115 IResult,
116 Parser,
117 branch::alt,
118 multi::many0_count,
119 combinator::recognize,
120 sequence::pair,
121 character::complete::{alpha1, alphanumeric1},
122 bytes::complete::tag,
123 };
124
125 pub fn identifier(input: &str) -> IResult<&str, &str> {
126 recognize(
127 pair(
128 alt((alpha1, tag("_"))),
129 many0_count(alt((alphanumeric1, tag("_"))))
130 )
131 ).parse(input)
132 }
133 ```
134
135 Let's say we apply this to the identifier `hello_world123abc`. The first `alt` parser would
136 recognize `h`. The `pair` combinator ensures that `ello_world123abc` will be piped to the next
137 `alphanumeric0` parser, which recognizes every remaining character. However, the `pair` combinator
138 returns a tuple of the results of its sub-parsers. The `recognize` parser produces a `&str` of the
139 input text that was parsed, which in this case is the entire `&str` `hello_world123abc`.
140
141 ## Literal Values
142
143 ### Escaped Strings
144
145 This is [one of the examples](https://github.com/rust-bakery/nom/blob/main/examples/string.rs) in
146 the examples directory.
147
148 ### Integers
149
150 The following recipes all return string slices rather than integer values. How to obtain an
151 integer value instead is demonstrated for hexadecimal integers. The others are similar.
152
153 The parsers allow the grouping character `_`, which allows one to group the digits by byte, for
154 example: `0xA4_3F_11_28`. If you prefer to exclude the `_` character, the lambda to convert from a
155 string slice to an integer value is slightly simpler. You can also strip the `_` from the string
156 slice that is returned, which is demonstrated in the second hexadecimal number parser.
157
158 If you wish to limit the number of digits in a valid integer literal, replace `many1` with
159 `many_m_n` in the recipes.
160
161 #### Hexadecimal
162
163 The parser outputs the string slice of the digits without the leading `0x`/`0X`.
164
165 ```rust
166 use nom::{
167 IResult,
168 Parser,
169 branch::alt,
170 multi::{many0, many1},
171 combinator::recognize,
172 sequence::{preceded, terminated},
173 character::complete::{char, one_of},
174 bytes::complete::tag,
175 };
176
177 fn hexadecimal(input: &str) -> IResult<&str, &str> { // <'a, E: ParseError<&'a str>>
178 preceded(
179 alt((tag("0x"), tag("0X"))),
180 recognize(
181 many1(
182 terminated(one_of("0123456789abcdefABCDEF"), many0(char('_')))
183 )
184 )
185 ).parse(input)
186 }
187 ```
188
189 If you want it to return the integer value instead, use map:
190
191 ```rust
192 use nom::{
193 IResult,
194 Parser,
195 branch::alt,
196 multi::{many0, many1},
197 combinator::{map_res, recognize},
198 sequence::{preceded, terminated},
199 character::complete::{char, one_of},
200 bytes::complete::tag,
201 };
202
203 fn hexadecimal_value(input: &str) -> IResult<&str, i64> {
204 map_res(
205 preceded(
206 alt((tag("0x"), tag("0X"))),
207 recognize(
208 many1(
209 terminated(one_of("0123456789abcdefABCDEF"), many0(char('_')))
210 )
211 )
212 ),
213 |out: &str| i64::from_str_radix(&str::replace(&out, "_", ""), 16)
214 ).parse(input)
215 }
216 ```
217
218 #### Octal
219
220 ```rust
221 use nom::{
222 IResult,
223 Parser,
224 branch::alt,
225 multi::{many0, many1},
226 combinator::recognize,
227 sequence::{preceded, terminated},
228 character::complete::{char, one_of},
229 bytes::complete::tag,
230 };
231
232 fn octal(input: &str) -> IResult<&str, &str> {
233 preceded(
234 alt((tag("0o"), tag("0O"))),
235 recognize(
236 many1(
237 terminated(one_of("01234567"), many0(char('_')))
238 )
239 )
240 ).parse(input)
241 }
242 ```
243
244 #### Binary
245
246 ```rust
247 use nom::{
248 IResult,
249 Parser,
250 branch::alt,
251 multi::{many0, many1},
252 combinator::recognize,
253 sequence::{preceded, terminated},
254 character::complete::{char, one_of},
255 bytes::complete::tag,
256 };
257
258 fn binary(input: &str) -> IResult<&str, &str> {
259 preceded(
260 alt((tag("0b"), tag("0B"))),
261 recognize(
262 many1(
263 terminated(one_of("01"), many0(char('_')))
264 )
265 )
266 ).parse(input)
267 }
268 ```
269
270 #### Decimal
271
272 ```rust
273 use nom::{
274 IResult,
275 Parser,
276 multi::{many0, many1},
277 combinator::recognize,
278 sequence::terminated,
279 character::complete::{char, one_of},
280 };
281
282 fn decimal(input: &str) -> IResult<&str, &str> {
283 recognize(
284 many1(
285 terminated(one_of("0123456789"), many0(char('_')))
286 )
287 ).parse(input)
288 }
289 ```
290
291 ### Floating Point Numbers
292
293 The following is adapted from [the Python parser by Valentin Lorentz (ProgVal)](https://github.com/ProgVal/rust-python-parser/blob/master/src/numbers.rs).
294
295 ```rust
296 use nom::{
297 IResult,
298 Parser,
299 branch::alt,
300 multi::{many0, many1},
301 combinator::{opt, recognize},
302 sequence::{preceded, terminated},
303 character::complete::{char, one_of},
304 };
305
306 fn float(input: &str) -> IResult<&str, &str> {
307 alt((
308 // Case one: .42
309 recognize(
310 (
311 char('.'),
312 decimal,
313 opt((
314 one_of("eE"),
315 opt(one_of("+-")),
316 decimal
317 ))
318 )
319 )
320 , // Case two: 42e42 and 42.42e42
321 recognize(
322 (
323 decimal,
324 opt(preceded(
325 char('.'),
326 decimal,
327 )),
328 one_of("eE"),
329 opt(one_of("+-")),
330 decimal
331 )
332 )
333 , // Case three: 42. and 42.42
334 recognize(
335 (
336 decimal,
337 char('.'),
338 opt(decimal)
339 )
340 )
341 )).parse(input)
342 }
343
344 fn decimal(input: &str) -> IResult<&str, &str> {
345 recognize(
346 many1(
347 terminated(one_of("0123456789"), many0(char('_')))
348 )
349 ).parse(input)
350 }
351 ```
352
353 # implementing FromStr
354
355 The [FromStr trait](https://doc.rust-lang.org/std/str/trait.FromStr.html) provides
356 a common interface to parse from a string.
357
358 ```rust
359 use nom::{
360 IResult, Parser, Finish, error::Error,
361 bytes::complete::{tag, take_while},
362 };
363 use std::str::FromStr;
364
365 // will recognize the name in "Hello, name!"
366 fn parse_name(input: &str) -> IResult<&str, &str> {
367 let (i, _) = tag("Hello, ").parse(input)?;
368 let (i, name) = take_while(|c:char| c.is_alphabetic())(i)?;
369 let (i, _) = tag("!")(i)?;
370
371 Ok((i, name))
372 }
373
374 // with FromStr, the result cannot be a reference to the input, it must be owned
375 #[derive(Debug)]
376 pub struct Name(pub String);
377
378 impl FromStr for Name {
379 // the error must be owned as well
380 type Err = Error<String>;
381
382 fn from_str(s: &str) -> Result<Self, Self::Err> {
383 match parse_name(s).finish() {
384 Ok((_remaining, name)) => Ok(Name(name.to_string())),
385 Err(Error { input, code }) => Err(Error {
386 input: input.to_string(),
387 code,
388 })
389 }
390 }
391 }
392
393 fn main() {
394 // parsed: Ok(Name("nom"))
395 println!("parsed: {:?}", "Hello, nom!".parse::<Name>());
396
397 // parsed: Err(Error { input: "123!", code: Tag })
398 println!("parsed: {:?}", "Hello, 123!".parse::<Name>());
399 }
400 ```
401