| 1 |
/*! |
| 2 |
This crate provides routines for searching strings for matches of a [regular |
| 3 |
expression] (aka "regex"). The regex syntax supported by this crate is similar |
| 4 |
to other regex engines, but it lacks several features that are not known how to |
| 5 |
implement efficiently. This includes, but is not limited to, look-around and |
| 6 |
backreferences. In exchange, all regex searches in this crate have worst case |
| 7 |
`O(m * n)` time complexity, where `m` is proportional to the size of the regex |
| 8 |
and `n` is proportional to the size of the string being searched. |
| 9 |
|
| 10 |
[regular expression]: https://en.wikipedia.org/wiki/Regular_expression |
| 11 |
|
| 12 |
If you just want API documentation, then skip to the [`Regex`] type. Otherwise, |
| 13 |
here's a quick example showing one way of parsing the output of a grep-like |
| 14 |
program: |
| 15 |
|
| 16 |
```rust |
| 17 |
use regex::Regex; |
| 18 |
|
| 19 |
let re = Regex::new(r"(?m)^([^:]+):([0-9]+):(.+)$").unwrap(); |
| 20 |
let hay = "\ |
| 21 |
path/to/foo:54:Blue Harvest |
| 22 |
path/to/bar:90:Something, Something, Something, Dark Side |
| 23 |
path/to/baz:3:It's a Trap! |
| 24 |
"; |
| 25 |
|
| 26 |
let mut results = vec![]; |
| 27 |
for (_, [path, lineno, line]) in re.captures_iter(hay).map(|c| c.extract()) { |
| 28 |
results.push((path, lineno.parse::<u64>()?, line)); |
| 29 |
} |
| 30 |
assert_eq!(results, vec![ |
| 31 |
("path/to/foo", 54, "Blue Harvest"), |
| 32 |
("path/to/bar", 90, "Something, Something, Something, Dark Side"), |
| 33 |
("path/to/baz", 3, "It's a Trap!"), |
| 34 |
]); |
| 35 |
# Ok::<(), Box<dyn std::error::Error>>(()) |
| 36 |
``` |
| 37 |
|
| 38 |
# Overview |
| 39 |
|
| 40 |
The primary type in this crate is a [`Regex`]. Its most important methods are |
| 41 |
as follows: |
| 42 |
|
| 43 |
* [`Regex::new`] compiles a regex using the default configuration. A |
| 44 |
[`RegexBuilder`] permits setting a non-default configuration. (For example, |
| 45 |
case insensitive matching, verbose mode and others.) |
| 46 |
* [`Regex::is_match`] reports whether a match exists in a particular haystack. |
| 47 |
* [`Regex::find`] reports the byte offsets of a match in a haystack, if one |
| 48 |
exists. [`Regex::find_iter`] returns an iterator over all such matches. |
| 49 |
* [`Regex::captures`] returns a [`Captures`], which reports both the byte |
| 50 |
offsets of a match in a haystack and the byte offsets of each matching capture |
| 51 |
group from the regex in the haystack. |
| 52 |
[`Regex::captures_iter`] returns an iterator over all such matches. |
| 53 |
|
| 54 |
There is also a [`RegexSet`], which permits searching for multiple regex |
| 55 |
patterns simultaneously in a single search. However, it currently only reports |
| 56 |
which patterns match and *not* the byte offsets of a match. |
| 57 |
|
| 58 |
Otherwise, this top-level crate documentation is organized as follows: |
| 59 |
|
| 60 |
* [Usage](#usage) shows how to add the `regex` crate to your Rust project. |
| 61 |
* [Examples](#examples) provides a limited selection of regex search examples. |
| 62 |
* [Performance](#performance) provides a brief summary of how to optimize regex |
| 63 |
searching speed. |
| 64 |
* [Unicode](#unicode) discusses support for non-ASCII patterns. |
| 65 |
* [Syntax](#syntax) enumerates the specific regex syntax supported by this |
| 66 |
crate. |
| 67 |
* [Untrusted input](#untrusted-input) discusses how this crate deals with regex |
| 68 |
patterns or haystacks that are untrusted. |
| 69 |
* [Crate features](#crate-features) documents the Cargo features that can be |
| 70 |
enabled or disabled for this crate. |
| 71 |
* [Other crates](#other-crates) links to other crates in the `regex` family. |
| 72 |
|
| 73 |
# Usage |
| 74 |
|
| 75 |
The `regex` crate is [on crates.io](https://crates.io/crates/regex) and can be |
| 76 |
used by adding `regex` to your dependencies in your project's `Cargo.toml`. |
| 77 |
Or more simply, just run `cargo add regex`. |
| 78 |
|
| 79 |
Here is a complete example that creates a new Rust project, adds a dependency |
| 80 |
on `regex`, creates the source code for a regex search and then runs the |
| 81 |
program. |
| 82 |
|
| 83 |
First, create the project in a new directory: |
| 84 |
|
| 85 |
```text |
| 86 |
$ mkdir regex-example |
| 87 |
$ cd regex-example |
| 88 |
$ cargo init |
| 89 |
``` |
| 90 |
|
| 91 |
Second, add a dependency on `regex`: |
| 92 |
|
| 93 |
```text |
| 94 |
$ cargo add regex |
| 95 |
``` |
| 96 |
|
| 97 |
Third, edit `src/main.rs`. Delete what's there and replace it with this: |
| 98 |
|
| 99 |
``` |
| 100 |
use regex::Regex; |
| 101 |
|
| 102 |
fn main() { |
| 103 |
let re = Regex::new(r"Hello (?<name>\w+)!").unwrap(); |
| 104 |
let Some(caps) = re.captures("Hello Murphy!") else { |
| 105 |
println!("no match!"); |
| 106 |
return; |
| 107 |
}; |
| 108 |
println!("The name is: {}", &caps["name"]); |
| 109 |
} |
| 110 |
``` |
| 111 |
|
| 112 |
Fourth, run it with `cargo run`: |
| 113 |
|
| 114 |
```text |
| 115 |
$ cargo run |
| 116 |
Compiling memchr v2.5.0 |
| 117 |
Compiling regex-syntax v0.7.1 |
| 118 |
Compiling aho-corasick v1.0.1 |
| 119 |
Compiling regex v1.8.1 |
| 120 |
Compiling regex-example v0.1.0 (/tmp/regex-example) |
| 121 |
Finished dev [unoptimized + debuginfo] target(s) in 4.22s |
| 122 |
Running `target/debug/regex-example` |
| 123 |
The name is: Murphy |
| 124 |
``` |
| 125 |
|
| 126 |
The first time you run the program will show more output like above. But |
| 127 |
subsequent runs shouldn't have to re-compile the dependencies. |
| 128 |
|
| 129 |
# Examples |
| 130 |
|
| 131 |
This section provides a few examples, in tutorial style, showing how to |
| 132 |
search a haystack with a regex. There are more examples throughout the API |
| 133 |
documentation. |
| 134 |
|
| 135 |
Before starting though, it's worth defining a few terms: |
| 136 |
|
| 137 |
* A **regex** is a Rust value whose type is `Regex`. We use `re` as a |
| 138 |
variable name for a regex. |
| 139 |
* A **pattern** is the string that is used to build a regex. We use `pat` as |
| 140 |
a variable name for a pattern. |
| 141 |
* A **haystack** is the string that is searched by a regex. We use `hay` as a |
| 142 |
variable name for a haystack. |
| 143 |
|
| 144 |
Sometimes the words "regex" and "pattern" are used interchangeably. |
| 145 |
|
| 146 |
General use of regular expressions in this crate proceeds by compiling a |
| 147 |
**pattern** into a **regex**, and then using that regex to search, split or |
| 148 |
replace parts of a **haystack**. |
| 149 |
|
| 150 |
### Example: find a middle initial |
| 151 |
|
| 152 |
We'll start off with a very simple example: a regex that looks for a specific |
| 153 |
name but uses a wildcard to match a middle initial. Our pattern serves as |
| 154 |
something like a template that will match a particular name with *any* middle |
| 155 |
initial. |
| 156 |
|
| 157 |
```rust |
| 158 |
use regex::Regex; |
| 159 |
|
| 160 |
// We use 'unwrap()' here because it would be a bug in our program if the |
| 161 |
// pattern failed to compile to a regex. Panicking in the presence of a bug |
| 162 |
// is okay. |
| 163 |
let re = Regex::new(r"Homer (.)\. Simpson").unwrap(); |
| 164 |
let hay = "Homer J. Simpson"; |
| 165 |
let Some(caps) = re.captures(hay) else { return }; |
| 166 |
assert_eq!("J", &caps[1]); |
| 167 |
``` |
| 168 |
|
| 169 |
There are a few things worth noticing here in our first example: |
| 170 |
|
| 171 |
* The `.` is a special pattern meta character that means "match any single |
| 172 |
character except for new lines." (More precisely, in this crate, it means |
| 173 |
"match any UTF-8 encoding of any Unicode scalar value other than `\n`.") |
| 174 |
* We can match an actual `.` literally by escaping it, i.e., `\.`. |
| 175 |
* We use Rust's [raw strings] to avoid needing to deal with escape sequences in |
| 176 |
both the regex pattern syntax and in Rust's string literal syntax. If we didn't |
| 177 |
use raw strings here, we would have had to use `\\.` to match a literal `.` |
| 178 |
character. That is, `r"\."` and `"\\."` are equivalent patterns. |
| 179 |
* We put our wildcard `.` instruction in parentheses. These parentheses have a |
| 180 |
special meaning that says, "make whatever part of the haystack matches within |
| 181 |
these parentheses available as a capturing group." After finding a match, we |
| 182 |
access this capture group with `&caps[1]`. |
| 183 |
|
| 184 |
[raw strings]: https://doc.rust-lang.org/stable/reference/tokens.html#raw-string-literals |
| 185 |
|
| 186 |
Otherwise, we execute a search using `re.captures(hay)` and return from our |
| 187 |
function if no match occurred. We then reference the middle initial by asking |
| 188 |
for the part of the haystack that matched the capture group indexed at `1`. |
| 189 |
(The capture group at index 0 is implicit and always corresponds to the entire |
| 190 |
match. In this case, that's `Homer J. Simpson`.) |
| 191 |
|
| 192 |
### Example: named capture groups |
| 193 |
|
| 194 |
Continuing from our middle initial example above, we can tweak the pattern |
| 195 |
slightly to give a name to the group that matches the middle initial: |
| 196 |
|
| 197 |
```rust |
| 198 |
use regex::Regex; |
| 199 |
|
| 200 |
// Note that (?P<middle>.) is a different way to spell the same thing. |
| 201 |
let re = Regex::new(r"Homer (?<middle>.)\. Simpson").unwrap(); |
| 202 |
let hay = "Homer J. Simpson"; |
| 203 |
let Some(caps) = re.captures(hay) else { return }; |
| 204 |
assert_eq!("J", &caps["middle"]); |
| 205 |
``` |
| 206 |
|
| 207 |
Giving a name to a group can be useful when there are multiple groups in |
| 208 |
a pattern. It makes the code referring to those groups a bit easier to |
| 209 |
understand. |
| 210 |
|
| 211 |
### Example: validating a particular date format |
| 212 |
|
| 213 |
This examples shows how to confirm whether a haystack, in its entirety, matches |
| 214 |
a particular date format: |
| 215 |
|
| 216 |
```rust |
| 217 |
use regex::Regex; |
| 218 |
|
| 219 |
let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap(); |
| 220 |
assert!(re.is_match("2010-03-14")); |
| 221 |
``` |
| 222 |
|
| 223 |
Notice the use of the `^` and `$` anchors. In this crate, every regex search is |
| 224 |
run with an implicit `(?s:.)*?` at the beginning of its pattern, which allows |
| 225 |
the regex to match anywhere in a haystack. Anchors, as above, can be used to |
| 226 |
ensure that the full haystack matches a pattern. |
| 227 |
|
| 228 |
This crate is also Unicode aware by default, which means that `\d` might match |
| 229 |
more than you might expect it to. For example: |
| 230 |
|
| 231 |
```rust |
| 232 |
use regex::Regex; |
| 233 |
|
| 234 |
let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap(); |
| 235 |
assert!(re.is_match("𝟚𝟘𝟙𝟘-𝟘𝟛-𝟙𝟜")); |
| 236 |
``` |
| 237 |
|
| 238 |
To only match an ASCII decimal digit, all of the following are equivalent: |
| 239 |
|
| 240 |
* `[0-9]` |
| 241 |
* `(?-u:\d)` |
| 242 |
* `[[:digit:]]` |
| 243 |
* `[\d&&\p{ascii}]` |
| 244 |
|
| 245 |
### Example: finding dates in a haystack |
| 246 |
|
| 247 |
In the previous example, we showed how one might validate that a haystack, |
| 248 |
in its entirety, corresponded to a particular date format. But what if we wanted |
| 249 |
to extract all things that look like dates in a specific format from a haystack? |
| 250 |
To do this, we can use an iterator API to find all matches (notice that we've |
| 251 |
removed the anchors and switched to looking for ASCII-only digits): |
| 252 |
|
| 253 |
```rust |
| 254 |
use regex::Regex; |
| 255 |
|
| 256 |
let re = Regex::new(r"[0-9]{4}-[0-9]{2}-[0-9]{2}").unwrap(); |
| 257 |
let hay = "What do 1865-04-14, 1881-07-02, 1901-09-06 and 1963-11-22 have in common?"; |
| 258 |
// 'm' is a 'Match', and 'as_str()' returns the matching part of the haystack. |
| 259 |
let dates: Vec<&str> = re.find_iter(hay).map(|m| m.as_str()).collect(); |
| 260 |
assert_eq!(dates, vec![ |
| 261 |
"1865-04-14", |
| 262 |
"1881-07-02", |
| 263 |
"1901-09-06", |
| 264 |
"1963-11-22", |
| 265 |
]); |
| 266 |
``` |
| 267 |
|
| 268 |
We can also iterate over [`Captures`] values instead of [`Match`] values, and |
| 269 |
that in turn permits accessing each component of the date via capturing groups: |
| 270 |
|
| 271 |
```rust |
| 272 |
use regex::Regex; |
| 273 |
|
| 274 |
let re = Regex::new(r"(?<y>[0-9]{4})-(?<m>[0-9]{2})-(?<d>[0-9]{2})").unwrap(); |
| 275 |
let hay = "What do 1865-04-14, 1881-07-02, 1901-09-06 and 1963-11-22 have in common?"; |
| 276 |
// 'm' is a 'Match', and 'as_str()' returns the matching part of the haystack. |
| 277 |
let dates: Vec<(&str, &str, &str)> = re.captures_iter(hay).map(|caps| { |
| 278 |
// The unwraps are okay because every capture group must match if the whole |
| 279 |
// regex matches, and in this context, we know we have a match. |
| 280 |
// |
| 281 |
// Note that we use `caps.name("y").unwrap().as_str()` instead of |
| 282 |
// `&caps["y"]` because the lifetime of the former is the same as the |
| 283 |
// lifetime of `hay` above, but the lifetime of the latter is tied to the |
| 284 |
// lifetime of `caps` due to how the `Index` trait is defined. |
| 285 |
let year = caps.name("y").unwrap().as_str(); |
| 286 |
let month = caps.name("m").unwrap().as_str(); |
| 287 |
let day = caps.name("d").unwrap().as_str(); |
| 288 |
(year, month, day) |
| 289 |
}).collect(); |
| 290 |
assert_eq!(dates, vec![ |
| 291 |
("1865", "04", "14"), |
| 292 |
("1881", "07", "02"), |
| 293 |
("1901", "09", "06"), |
| 294 |
("1963", "11", "22"), |
| 295 |
]); |
| 296 |
``` |
| 297 |
|
| 298 |
### Example: simpler capture group extraction |
| 299 |
|
| 300 |
One can use [`Captures::extract`] to make the code from the previous example a |
| 301 |
bit simpler in this case: |
| 302 |
|
| 303 |
```rust |
| 304 |
use regex::Regex; |
| 305 |
|
| 306 |
let re = Regex::new(r"([0-9]{4})-([0-9]{2})-([0-9]{2})").unwrap(); |
| 307 |
let hay = "What do 1865-04-14, 1881-07-02, 1901-09-06 and 1963-11-22 have in common?"; |
| 308 |
let dates: Vec<(&str, &str, &str)> = re.captures_iter(hay).map(|caps| { |
| 309 |
let (_, [year, month, day]) = caps.extract(); |
| 310 |
(year, month, day) |
| 311 |
}).collect(); |
| 312 |
assert_eq!(dates, vec![ |
| 313 |
("1865", "04", "14"), |
| 314 |
("1881", "07", "02"), |
| 315 |
("1901", "09", "06"), |
| 316 |
("1963", "11", "22"), |
| 317 |
]); |
| 318 |
``` |
| 319 |
|
| 320 |
`Captures::extract` works by ensuring that the number of matching groups match |
| 321 |
the number of groups requested via the `[year, month, day]` syntax. If they do, |
| 322 |
then the substrings for each corresponding capture group are automatically |
| 323 |
returned in an appropriately sized array. Rust's syntax for pattern matching |
| 324 |
arrays does the rest. |
| 325 |
|
| 326 |
### Example: replacement with named capture groups |
| 327 |
|
| 328 |
Building on the previous example, perhaps we'd like to rearrange the date |
| 329 |
formats. This can be done by finding each match and replacing it with |
| 330 |
something different. The [`Regex::replace_all`] routine provides a convenient |
| 331 |
way to do this, including by supporting references to named groups in the |
| 332 |
replacement string: |
| 333 |
|
| 334 |
```rust |
| 335 |
use regex::Regex; |
| 336 |
|
| 337 |
let re = Regex::new(r"(?<y>\d{4})-(?<m>\d{2})-(?<d>\d{2})").unwrap(); |
| 338 |
let before = "1973-01-05, 1975-08-25 and 1980-10-18"; |
| 339 |
let after = re.replace_all(before, "$m/$d/$y"); |
| 340 |
assert_eq!(after, "01/05/1973, 08/25/1975 and 10/18/1980"); |
| 341 |
``` |
| 342 |
|
| 343 |
The replace methods are actually polymorphic in the replacement, which |
| 344 |
provides more flexibility than is seen here. (See the documentation for |
| 345 |
[`Regex::replace`] for more details.) |
| 346 |
|
| 347 |
### Example: verbose mode |
| 348 |
|
| 349 |
When your regex gets complicated, you might consider using something other |
| 350 |
than regex. But if you stick with regex, you can use the `x` flag to enable |
| 351 |
insignificant whitespace mode or "verbose mode." In this mode, whitespace |
| 352 |
is treated as insignificant and one may write comments. This may make your |
| 353 |
patterns easier to comprehend. |
| 354 |
|
| 355 |
```rust |
| 356 |
use regex::Regex; |
| 357 |
|
| 358 |
let re = Regex::new(r"(?x) |
| 359 |
(?P<y>\d{4}) # the year, including all Unicode digits |
| 360 |
- |
| 361 |
(?P<m>\d{2}) # the month, including all Unicode digits |
| 362 |
- |
| 363 |
(?P<d>\d{2}) # the day, including all Unicode digits |
| 364 |
").unwrap(); |
| 365 |
|
| 366 |
let before = "1973-01-05, 1975-08-25 and 1980-10-18"; |
| 367 |
let after = re.replace_all(before, "$m/$d/$y"); |
| 368 |
assert_eq!(after, "01/05/1973, 08/25/1975 and 10/18/1980"); |
| 369 |
``` |
| 370 |
|
| 371 |
If you wish to match against whitespace in this mode, you can still use `\s`, |
| 372 |
`\n`, `\t`, etc. For escaping a single space character, you can escape it |
| 373 |
directly with `\ `, use its hex character code `\x20` or temporarily disable |
| 374 |
the `x` flag, e.g., `(?-x: )`. |
| 375 |
|
| 376 |
### Example: match multiple regular expressions simultaneously |
| 377 |
|
| 378 |
This demonstrates how to use a [`RegexSet`] to match multiple (possibly |
| 379 |
overlapping) regexes in a single scan of a haystack: |
| 380 |
|
| 381 |
```rust |
| 382 |
use regex::RegexSet; |
| 383 |
|
| 384 |
let set = RegexSet::new(&[ |
| 385 |
r"\w+", |
| 386 |
r"\d+", |
| 387 |
r"\pL+", |
| 388 |
r"foo", |
| 389 |
r"bar", |
| 390 |
r"barfoo", |
| 391 |
r"foobar", |
| 392 |
]).unwrap(); |
| 393 |
|
| 394 |
// Iterate over and collect all of the matches. Each match corresponds to the |
| 395 |
// ID of the matching pattern. |
| 396 |
let matches: Vec<_> = set.matches("foobar").into_iter().collect(); |
| 397 |
assert_eq!(matches, vec![0, 2, 3, 4, 6]); |
| 398 |
|
| 399 |
// You can also test whether a particular regex matched: |
| 400 |
let matches = set.matches("foobar"); |
| 401 |
assert!(!matches.matched(5)); |
| 402 |
assert!(matches.matched(6)); |
| 403 |
``` |
| 404 |
|
| 405 |
# Performance |
| 406 |
|
| 407 |
This section briefly discusses a few concerns regarding the speed and resource |
| 408 |
usage of regexes. |
| 409 |
|
| 410 |
### Only ask for what you need |
| 411 |
|
| 412 |
When running a search with a regex, there are generally three different types |
| 413 |
of information one can ask for: |
| 414 |
|
| 415 |
1. Does a regex match in a haystack? |
| 416 |
2. Where does a regex match in a haystack? |
| 417 |
3. Where do each of the capturing groups match in a haystack? |
| 418 |
|
| 419 |
Generally speaking, this crate could provide a function to answer only #3, |
| 420 |
which would subsume #1 and #2 automatically. However, it can be significantly |
| 421 |
more expensive to compute the location of capturing group matches, so it's best |
| 422 |
not to do it if you don't need to. |
| 423 |
|
| 424 |
Therefore, only ask for what you need. For example, don't use [`Regex::find`] |
| 425 |
if you only need to test if a regex matches a haystack. Use [`Regex::is_match`] |
| 426 |
instead. |
| 427 |
|
| 428 |
### Unicode can impact memory usage and search speed |
| 429 |
|
| 430 |
This crate has first class support for Unicode and it is **enabled by default**. |
| 431 |
In many cases, the extra memory required to support it will be negligible and |
| 432 |
it typically won't impact search speed. But it can in some cases. |
| 433 |
|
| 434 |
With respect to memory usage, the impact of Unicode principally manifests |
| 435 |
through the use of Unicode character classes. Unicode character classes |
| 436 |
tend to be quite large. For example, `\w` by default matches around 140,000 |
| 437 |
distinct codepoints. This requires additional memory, and tends to slow down |
| 438 |
regex compilation. While a `\w` here and there is unlikely to be noticed, |
| 439 |
writing `\w{100}` will for example result in quite a large regex by default. |
| 440 |
Indeed, `\w` is considerably larger than its ASCII-only version, so if your |
| 441 |
requirements are satisfied by ASCII, it's probably a good idea to stick to |
| 442 |
ASCII classes. The ASCII-only version of `\w` can be spelled in a number of |
| 443 |
ways. All of the following are equivalent: |
| 444 |
|
| 445 |
* `[0-9A-Za-z_]` |
| 446 |
* `(?-u:\w)` |
| 447 |
* `[[:word:]]` |
| 448 |
* `[\w&&\p{ascii}]` |
| 449 |
|
| 450 |
With respect to search speed, Unicode tends to be handled pretty well, even when |
| 451 |
using large Unicode character classes. However, some of the faster internal |
| 452 |
regex engines cannot handle a Unicode aware word boundary assertion. So if you |
| 453 |
don't need Unicode-aware word boundary assertions, you might consider using |
| 454 |
`(?-u:\b)` instead of `\b`, where the former uses an ASCII-only definition of |
| 455 |
a word character. |
| 456 |
|
| 457 |
### Literals might accelerate searches |
| 458 |
|
| 459 |
This crate tends to be quite good at recognizing literals in a regex pattern |
| 460 |
and using them to accelerate a search. If it is at all possible to include |
| 461 |
some kind of literal in your pattern, then it might make search substantially |
| 462 |
faster. For example, in the regex `\w+@\w+`, the engine will look for |
| 463 |
occurrences of `@` and then try a reverse match for `\w+` to find the start |
| 464 |
position. |
| 465 |
|
| 466 |
### Avoid re-compiling regexes, especially in a loop |
| 467 |
|
| 468 |
It is an anti-pattern to compile the same pattern in a loop since regex |
| 469 |
compilation is typically expensive. (It takes anywhere from a few microseconds |
| 470 |
to a few **milliseconds** depending on the size of the pattern.) Not only is |
| 471 |
compilation itself expensive, but this also prevents optimizations that reuse |
| 472 |
allocations internally to the regex engine. |
| 473 |
|
| 474 |
In Rust, it can sometimes be a pain to pass regexes around if they're used from |
| 475 |
inside a helper function. Instead, we recommend using crates like [`once_cell`] |
| 476 |
and [`lazy_static`] to ensure that patterns are compiled exactly once. |
| 477 |
|
| 478 |
[`once_cell`]: https://crates.io/crates/once_cell |
| 479 |
[`lazy_static`]: https://crates.io/crates/lazy_static |
| 480 |
|
| 481 |
This example shows how to use `once_cell`: |
| 482 |
|
| 483 |
```rust |
| 484 |
use { |
| 485 |
once_cell::sync::Lazy, |
| 486 |
regex::Regex, |
| 487 |
}; |
| 488 |
|
| 489 |
fn some_helper_function(haystack: &str) -> bool { |
| 490 |
static RE: Lazy<Regex> = Lazy::new(|| Regex::new(r"...").unwrap()); |
| 491 |
RE.is_match(haystack) |
| 492 |
} |
| 493 |
|
| 494 |
fn main() { |
| 495 |
assert!(some_helper_function("abc")); |
| 496 |
assert!(!some_helper_function("ac")); |
| 497 |
} |
| 498 |
``` |
| 499 |
|
| 500 |
Specifically, in this example, the regex will be compiled when it is used for |
| 501 |
the first time. On subsequent uses, it will reuse the previously built `Regex`. |
| 502 |
Notice how one can define the `Regex` locally to a specific function. |
| 503 |
|
| 504 |
### Sharing a regex across threads can result in contention |
| 505 |
|
| 506 |
While a single `Regex` can be freely used from multiple threads simultaneously, |
| 507 |
there is a small synchronization cost that must be paid. Generally speaking, |
| 508 |
one shouldn't expect to observe this unless the principal task in each thread |
| 509 |
is searching with the regex *and* most searches are on short haystacks. In this |
| 510 |
case, internal contention on shared resources can spike and increase latency, |
| 511 |
which in turn may slow down each individual search. |
| 512 |
|
| 513 |
One can work around this by cloning each `Regex` before sending it to another |
| 514 |
thread. The cloned regexes will still share the same internal read-only portion |
| 515 |
of its compiled state (it's reference counted), but each thread will get |
| 516 |
optimized access to the mutable space that is used to run a search. In general, |
| 517 |
there is no additional cost in memory to doing this. The only cost is the added |
| 518 |
code complexity required to explicitly clone the regex. (If you share the same |
| 519 |
`Regex` across multiple threads, each thread still gets its own mutable space, |
| 520 |
but accessing that space is slower.) |
| 521 |
|
| 522 |
# Unicode |
| 523 |
|
| 524 |
This section discusses what kind of Unicode support this regex library has. |
| 525 |
Before showing some examples, we'll summarize the relevant points: |
| 526 |
|
| 527 |
* This crate almost fully implements "Basic Unicode Support" (Level 1) as |
| 528 |
specified by the [Unicode Technical Standard #18][UTS18]. The full details |
| 529 |
of what is supported are documented in [UNICODE.md] in the root of the regex |
| 530 |
crate repository. There is virtually no support for "Extended Unicode Support" |
| 531 |
(Level 2) from UTS#18. |
| 532 |
* The top-level [`Regex`] runs searches *as if* iterating over each of the |
| 533 |
codepoints in the haystack. That is, the fundamental atom of matching is a |
| 534 |
single codepoint. |
| 535 |
* [`bytes::Regex`], in contrast, permits disabling Unicode mode for part of all |
| 536 |
of your pattern in all cases. When Unicode mode is disabled, then a search is |
| 537 |
run *as if* iterating over each byte in the haystack. That is, the fundamental |
| 538 |
atom of matching is a single byte. (A top-level `Regex` also permits disabling |
| 539 |
Unicode and thus matching *as if* it were one byte at a time, but only when |
| 540 |
doing so wouldn't permit matching invalid UTF-8.) |
| 541 |
* When Unicode mode is enabled (the default), `.` will match an entire Unicode |
| 542 |
scalar value, even when it is encoded using multiple bytes. When Unicode mode |
| 543 |
is disabled (e.g., `(?-u:.)`), then `.` will match a single byte in all cases. |
| 544 |
* The character classes `\w`, `\d` and `\s` are all Unicode-aware by default. |
| 545 |
Use `(?-u:\w)`, `(?-u:\d)` and `(?-u:\s)` to get their ASCII-only definitions. |
| 546 |
* Similarly, `\b` and `\B` use a Unicode definition of a "word" character. |
| 547 |
To get ASCII-only word boundaries, use `(?-u:\b)` and `(?-u:\B)`. This also |
| 548 |
applies to the special word boundary assertions. (That is, `\b{start}`, |
| 549 |
`\b{end}`, `\b{start-half}`, `\b{end-half}`.) |
| 550 |
* `^` and `$` are **not** Unicode-aware in multi-line mode. Namely, they only |
| 551 |
recognize `\n` (assuming CRLF mode is not enabled) and not any of the other |
| 552 |
forms of line terminators defined by Unicode. |
| 553 |
* Case insensitive searching is Unicode-aware and uses simple case folding. |
| 554 |
* Unicode general categories, scripts and many boolean properties are available |
| 555 |
by default via the `\p{property name}` syntax. |
| 556 |
* In all cases, matches are reported using byte offsets. Or more precisely, |
| 557 |
UTF-8 code unit offsets. This permits constant time indexing and slicing of the |
| 558 |
haystack. |
| 559 |
|
| 560 |
[UTS18]: https://unicode.org/reports/tr18/ |
| 561 |
[UNICODE.md]: https://github.com/rust-lang/regex/blob/master/UNICODE.md |
| 562 |
|
| 563 |
Patterns themselves are **only** interpreted as a sequence of Unicode scalar |
| 564 |
values. This means you can use Unicode characters directly in your pattern: |
| 565 |
|
| 566 |
```rust |
| 567 |
use regex::Regex; |
| 568 |
|
| 569 |
let re = Regex::new(r"(?i)Δ+").unwrap(); |
| 570 |
let m = re.find("ΔδΔ").unwrap(); |
| 571 |
assert_eq!((0, 6), (m.start(), m.end())); |
| 572 |
// alternatively: |
| 573 |
assert_eq!(0..6, m.range()); |
| 574 |
``` |
| 575 |
|
| 576 |
As noted above, Unicode general categories, scripts, script extensions, ages |
| 577 |
and a smattering of boolean properties are available as character classes. For |
| 578 |
example, you can match a sequence of numerals, Greek or Cherokee letters: |
| 579 |
|
| 580 |
```rust |
| 581 |
use regex::Regex; |
| 582 |
|
| 583 |
let re = Regex::new(r"[\pN\p{Greek}\p{Cherokee}]+").unwrap(); |
| 584 |
let m = re.find("abcΔᎠβⅠᏴγδⅡxyz").unwrap(); |
| 585 |
assert_eq!(3..23, m.range()); |
| 586 |
``` |
| 587 |
|
| 588 |
While not specific to Unicode, this library also supports character class set |
| 589 |
operations. Namely, one can nest character classes arbitrarily and perform set |
| 590 |
operations on them. Those set operations are union (the default), intersection, |
| 591 |
difference and symmetric difference. These set operations tend to be most |
| 592 |
useful with Unicode character classes. For example, to match any codepoint |
| 593 |
that is both in the `Greek` script and in the `Letter` general category: |
| 594 |
|
| 595 |
```rust |
| 596 |
use regex::Regex; |
| 597 |
|
| 598 |
let re = Regex::new(r"[\p{Greek}&&\pL]+").unwrap(); |
| 599 |
let subs: Vec<&str> = re.find_iter("ΔδΔ𐅌ΔδΔ").map(|m| m.as_str()).collect(); |
| 600 |
assert_eq!(subs, vec!["ΔδΔ", "ΔδΔ"]); |
| 601 |
|
| 602 |
// If we just matches on Greek, then all codepoints would match! |
| 603 |
let re = Regex::new(r"\p{Greek}+").unwrap(); |
| 604 |
let subs: Vec<&str> = re.find_iter("ΔδΔ𐅌ΔδΔ").map(|m| m.as_str()).collect(); |
| 605 |
assert_eq!(subs, vec!["ΔδΔ𐅌ΔδΔ"]); |
| 606 |
``` |
| 607 |
|
| 608 |
### Opt out of Unicode support |
| 609 |
|
| 610 |
The [`bytes::Regex`] type that can be used to search `&[u8]` haystacks. By |
| 611 |
default, haystacks are conventionally treated as UTF-8 just like it is with the |
| 612 |
main `Regex` type. However, this behavior can be disabled by turning off the |
| 613 |
`u` flag, even if doing so could result in matching invalid UTF-8. For example, |
| 614 |
when the `u` flag is disabled, `.` will match any byte instead of any Unicode |
| 615 |
scalar value. |
| 616 |
|
| 617 |
Disabling the `u` flag is also possible with the standard `&str`-based `Regex` |
| 618 |
type, but it is only allowed where the UTF-8 invariant is maintained. For |
| 619 |
example, `(?-u:\w)` is an ASCII-only `\w` character class and is legal in an |
| 620 |
`&str`-based `Regex`, but `(?-u:\W)` will attempt to match *any byte* that |
| 621 |
isn't in `(?-u:\w)`, which in turn includes bytes that are invalid UTF-8. |
| 622 |
Similarly, `(?-u:\xFF)` will attempt to match the raw byte `\xFF` (instead of |
| 623 |
`U+00FF`), which is invalid UTF-8 and therefore is illegal in `&str`-based |
| 624 |
regexes. |
| 625 |
|
| 626 |
Finally, since Unicode support requires bundling large Unicode data |
| 627 |
tables, this crate exposes knobs to disable the compilation of those |
| 628 |
data tables, which can be useful for shrinking binary size and reducing |
| 629 |
compilation times. For details on how to do that, see the section on [crate |
| 630 |
features](#crate-features). |
| 631 |
|
| 632 |
# Syntax |
| 633 |
|
| 634 |
The syntax supported in this crate is documented below. |
| 635 |
|
| 636 |
Note that the regular expression parser and abstract syntax are exposed in |
| 637 |
a separate crate, [`regex-syntax`](https://docs.rs/regex-syntax). |
| 638 |
|
| 639 |
### Matching one character |
| 640 |
|
| 641 |
<pre class="rust"> |
| 642 |
. any character except new line (includes new line with s flag) |
| 643 |
[0-9] any ASCII digit |
| 644 |
\d digit (\p{Nd}) |
| 645 |
\D not digit |
| 646 |
\pX Unicode character class identified by a one-letter name |
| 647 |
\p{Greek} Unicode character class (general category or script) |
| 648 |
\PX Negated Unicode character class identified by a one-letter name |
| 649 |
\P{Greek} negated Unicode character class (general category or script) |
| 650 |
</pre> |
| 651 |
|
| 652 |
### Character classes |
| 653 |
|
| 654 |
<pre class="rust"> |
| 655 |
[xyz] A character class matching either x, y or z (union). |
| 656 |
[^xyz] A character class matching any character except x, y and z. |
| 657 |
[a-z] A character class matching any character in range a-z. |
| 658 |
[[:alpha:]] ASCII character class ([A-Za-z]) |
| 659 |
[[:^alpha:]] Negated ASCII character class ([^A-Za-z]) |
| 660 |
[x[^xyz]] Nested/grouping character class (matching any character except y and z) |
| 661 |
[a-y&&xyz] Intersection (matching x or y) |
| 662 |
[0-9&&[^4]] Subtraction using intersection and negation (matching 0-9 except 4) |
| 663 |
[0-9--4] Direct subtraction (matching 0-9 except 4) |
| 664 |
[a-g~~b-h] Symmetric difference (matching `a` and `h` only) |
| 665 |
[\[\]] Escaping in character classes (matching [ or ]) |
| 666 |
[a&&b] An empty character class matching nothing |
| 667 |
</pre> |
| 668 |
|
| 669 |
Any named character class may appear inside a bracketed `[...]` character |
| 670 |
class. For example, `[\p{Greek}[:digit:]]` matches any ASCII digit or any |
| 671 |
codepoint in the `Greek` script. `[\p{Greek}&&\pL]` matches Greek letters. |
| 672 |
|
| 673 |
Precedence in character classes, from most binding to least: |
| 674 |
|
| 675 |
1. Ranges: `[a-cd]` == `[[a-c]d]` |
| 676 |
2. Union: `[ab&&bc]` == `[[ab]&&[bc]]` |
| 677 |
3. Intersection, difference, symmetric difference. All three have equivalent |
| 678 |
precedence, and are evaluated in left-to-right order. For example, |
| 679 |
`[\pL--\p{Greek}&&\p{Uppercase}]` == `[[\pL--\p{Greek}]&&\p{Uppercase}]`. |
| 680 |
4. Negation: `[^a-z&&b]` == `[^[a-z&&b]]`. |
| 681 |
|
| 682 |
### Composites |
| 683 |
|
| 684 |
<pre class="rust"> |
| 685 |
xy concatenation (x followed by y) |
| 686 |
x|y alternation (x or y, prefer x) |
| 687 |
</pre> |
| 688 |
|
| 689 |
This example shows how an alternation works, and what it means to prefer a |
| 690 |
branch in the alternation over subsequent branches. |
| 691 |
|
| 692 |
``` |
| 693 |
use regex::Regex; |
| 694 |
|
| 695 |
let haystack = "samwise"; |
| 696 |
// If 'samwise' comes first in our alternation, then it is |
| 697 |
// preferred as a match, even if the regex engine could |
| 698 |
// technically detect that 'sam' led to a match earlier. |
| 699 |
let re = Regex::new(r"samwise|sam").unwrap(); |
| 700 |
assert_eq!("samwise", re.find(haystack).unwrap().as_str()); |
| 701 |
// But if 'sam' comes first, then it will match instead. |
| 702 |
// In this case, it is impossible for 'samwise' to match |
| 703 |
// because 'sam' is a prefix of it. |
| 704 |
let re = Regex::new(r"sam|samwise").unwrap(); |
| 705 |
assert_eq!("sam", re.find(haystack).unwrap().as_str()); |
| 706 |
``` |
| 707 |
|
| 708 |
### Repetitions |
| 709 |
|
| 710 |
<pre class="rust"> |
| 711 |
x* zero or more of x (greedy) |
| 712 |
x+ one or more of x (greedy) |
| 713 |
x? zero or one of x (greedy) |
| 714 |
x*? zero or more of x (ungreedy/lazy) |
| 715 |
x+? one or more of x (ungreedy/lazy) |
| 716 |
x?? zero or one of x (ungreedy/lazy) |
| 717 |
x{n,m} at least n x and at most m x (greedy) |
| 718 |
x{n,} at least n x (greedy) |
| 719 |
x{n} exactly n x |
| 720 |
x{n,m}? at least n x and at most m x (ungreedy/lazy) |
| 721 |
x{n,}? at least n x (ungreedy/lazy) |
| 722 |
x{n}? exactly n x |
| 723 |
</pre> |
| 724 |
|
| 725 |
### Empty matches |
| 726 |
|
| 727 |
<pre class="rust"> |
| 728 |
^ the beginning of a haystack (or start-of-line with multi-line mode) |
| 729 |
$ the end of a haystack (or end-of-line with multi-line mode) |
| 730 |
\A only the beginning of a haystack (even with multi-line mode enabled) |
| 731 |
\z only the end of a haystack (even with multi-line mode enabled) |
| 732 |
\b a Unicode word boundary (\w on one side and \W, \A, or \z on other) |
| 733 |
\B not a Unicode word boundary |
| 734 |
\b{start}, \< a Unicode start-of-word boundary (\W|\A on the left, \w on the right) |
| 735 |
\b{end}, \> a Unicode end-of-word boundary (\w on the left, \W|\z on the right)) |
| 736 |
\b{start-half} half of a Unicode start-of-word boundary (\W|\A on the left) |
| 737 |
\b{end-half} half of a Unicode end-of-word boundary (\W|\z on the right) |
| 738 |
</pre> |
| 739 |
|
| 740 |
The empty regex is valid and matches the empty string. For example, the |
| 741 |
empty regex matches `abc` at positions `0`, `1`, `2` and `3`. When using the |
| 742 |
top-level [`Regex`] on `&str` haystacks, an empty match that splits a codepoint |
| 743 |
is guaranteed to never be returned. However, such matches are permitted when |
| 744 |
using a [`bytes::Regex`]. For example: |
| 745 |
|
| 746 |
```rust |
| 747 |
let re = regex::Regex::new(r"").unwrap(); |
| 748 |
let ranges: Vec<_> = re.find_iter("💩").map(|m| m.range()).collect(); |
| 749 |
assert_eq!(ranges, vec![0..0, 4..4]); |
| 750 |
|
| 751 |
let re = regex::bytes::Regex::new(r"").unwrap(); |
| 752 |
let ranges: Vec<_> = re.find_iter("💩".as_bytes()).map(|m| m.range()).collect(); |
| 753 |
assert_eq!(ranges, vec![0..0, 1..1, 2..2, 3..3, 4..4]); |
| 754 |
``` |
| 755 |
|
| 756 |
Note that an empty regex is distinct from a regex that can never match. |
| 757 |
For example, the regex `[a&&b]` is a character class that represents the |
| 758 |
intersection of `a` and `b`. That intersection is empty, which means the |
| 759 |
character class is empty. Since nothing is in the empty set, `[a&&b]` matches |
| 760 |
nothing, not even the empty string. |
| 761 |
|
| 762 |
### Grouping and flags |
| 763 |
|
| 764 |
<pre class="rust"> |
| 765 |
(exp) numbered capture group (indexed by opening parenthesis) |
| 766 |
(?P<name>exp) named (also numbered) capture group (names must be alpha-numeric) |
| 767 |
(?<name>exp) named (also numbered) capture group (names must be alpha-numeric) |
| 768 |
(?:exp) non-capturing group |
| 769 |
(?flags) set flags within current group |
| 770 |
(?flags:exp) set flags for exp (non-capturing) |
| 771 |
</pre> |
| 772 |
|
| 773 |
Capture group names must be any sequence of alpha-numeric Unicode codepoints, |
| 774 |
in addition to `.`, `_`, `[` and `]`. Names must start with either an `_` or |
| 775 |
an alphabetic codepoint. Alphabetic codepoints correspond to the `Alphabetic` |
| 776 |
Unicode property, while numeric codepoints correspond to the union of the |
| 777 |
`Decimal_Number`, `Letter_Number` and `Other_Number` general categories. |
| 778 |
|
| 779 |
Flags are each a single character. For example, `(?x)` sets the flag `x` |
| 780 |
and `(?-x)` clears the flag `x`. Multiple flags can be set or cleared at |
| 781 |
the same time: `(?xy)` sets both the `x` and `y` flags and `(?x-y)` sets |
| 782 |
the `x` flag and clears the `y` flag. |
| 783 |
|
| 784 |
All flags are by default disabled unless stated otherwise. They are: |
| 785 |
|
| 786 |
<pre class="rust"> |
| 787 |
i case-insensitive: letters match both upper and lower case |
| 788 |
m multi-line mode: ^ and $ match begin/end of line |
| 789 |
s allow . to match \n |
| 790 |
R enables CRLF mode: when multi-line mode is enabled, \r\n is used |
| 791 |
U swap the meaning of x* and x*? |
| 792 |
u Unicode support (enabled by default) |
| 793 |
x verbose mode, ignores whitespace and allow line comments (starting with `#`) |
| 794 |
</pre> |
| 795 |
|
| 796 |
Note that in verbose mode, whitespace is ignored everywhere, including within |
| 797 |
character classes. To insert whitespace, use its escaped form or a hex literal. |
| 798 |
For example, `\ ` or `\x20` for an ASCII space. |
| 799 |
|
| 800 |
Flags can be toggled within a pattern. Here's an example that matches |
| 801 |
case-insensitively for the first part but case-sensitively for the second part: |
| 802 |
|
| 803 |
```rust |
| 804 |
use regex::Regex; |
| 805 |
|
| 806 |
let re = Regex::new(r"(?i)a+(?-i)b+").unwrap(); |
| 807 |
let m = re.find("AaAaAbbBBBb").unwrap(); |
| 808 |
assert_eq!(m.as_str(), "AaAaAbb"); |
| 809 |
``` |
| 810 |
|
| 811 |
Notice that the `a+` matches either `a` or `A`, but the `b+` only matches |
| 812 |
`b`. |
| 813 |
|
| 814 |
Multi-line mode means `^` and `$` no longer match just at the beginning/end of |
| 815 |
the input, but also at the beginning/end of lines: |
| 816 |
|
| 817 |
``` |
| 818 |
use regex::Regex; |
| 819 |
|
| 820 |
let re = Regex::new(r"(?m)^line \d+").unwrap(); |
| 821 |
let m = re.find("line one\nline 2\n").unwrap(); |
| 822 |
assert_eq!(m.as_str(), "line 2"); |
| 823 |
``` |
| 824 |
|
| 825 |
Note that `^` matches after new lines, even at the end of input: |
| 826 |
|
| 827 |
``` |
| 828 |
use regex::Regex; |
| 829 |
|
| 830 |
let re = Regex::new(r"(?m)^").unwrap(); |
| 831 |
let m = re.find_iter("test\n").last().unwrap(); |
| 832 |
assert_eq!((m.start(), m.end()), (5, 5)); |
| 833 |
``` |
| 834 |
|
| 835 |
When both CRLF mode and multi-line mode are enabled, then `^` and `$` will |
| 836 |
match either `\r` and `\n`, but never in the middle of a `\r\n`: |
| 837 |
|
| 838 |
``` |
| 839 |
use regex::Regex; |
| 840 |
|
| 841 |
let re = Regex::new(r"(?mR)^foo$").unwrap(); |
| 842 |
let m = re.find("\r\nfoo\r\n").unwrap(); |
| 843 |
assert_eq!(m.as_str(), "foo"); |
| 844 |
``` |
| 845 |
|
| 846 |
Unicode mode can also be selectively disabled, although only when the result |
| 847 |
*would not* match invalid UTF-8. One good example of this is using an ASCII |
| 848 |
word boundary instead of a Unicode word boundary, which might make some regex |
| 849 |
searches run faster: |
| 850 |
|
| 851 |
```rust |
| 852 |
use regex::Regex; |
| 853 |
|
| 854 |
let re = Regex::new(r"(?-u:\b).+(?-u:\b)").unwrap(); |
| 855 |
let m = re.find("$$abc$$").unwrap(); |
| 856 |
assert_eq!(m.as_str(), "abc"); |
| 857 |
``` |
| 858 |
|
| 859 |
### Escape sequences |
| 860 |
|
| 861 |
Note that this includes all possible escape sequences, even ones that are |
| 862 |
documented elsewhere. |
| 863 |
|
| 864 |
<pre class="rust"> |
| 865 |
\* literal *, applies to all ASCII except [0-9A-Za-z<>] |
| 866 |
\a bell (\x07) |
| 867 |
\f form feed (\x0C) |
| 868 |
\t horizontal tab |
| 869 |
\n new line |
| 870 |
\r carriage return |
| 871 |
\v vertical tab (\x0B) |
| 872 |
\A matches at the beginning of a haystack |
| 873 |
\z matches at the end of a haystack |
| 874 |
\b word boundary assertion |
| 875 |
\B negated word boundary assertion |
| 876 |
\b{start}, \< start-of-word boundary assertion |
| 877 |
\b{end}, \> end-of-word boundary assertion |
| 878 |
\b{start-half} half of a start-of-word boundary assertion |
| 879 |
\b{end-half} half of a end-of-word boundary assertion |
| 880 |
\123 octal character code, up to three digits (when enabled) |
| 881 |
\x7F hex character code (exactly two digits) |
| 882 |
\x{10FFFF} any hex character code corresponding to a Unicode code point |
| 883 |
\u007F hex character code (exactly four digits) |
| 884 |
\u{7F} any hex character code corresponding to a Unicode code point |
| 885 |
\U0000007F hex character code (exactly eight digits) |
| 886 |
\U{7F} any hex character code corresponding to a Unicode code point |
| 887 |
\p{Letter} Unicode character class |
| 888 |
\P{Letter} negated Unicode character class |
| 889 |
\d, \s, \w Perl character class |
| 890 |
\D, \S, \W negated Perl character class |
| 891 |
</pre> |
| 892 |
|
| 893 |
### Perl character classes (Unicode friendly) |
| 894 |
|
| 895 |
These classes are based on the definitions provided in |
| 896 |
[UTS#18](https://www.unicode.org/reports/tr18/#Compatibility_Properties): |
| 897 |
|
| 898 |
<pre class="rust"> |
| 899 |
\d digit (\p{Nd}) |
| 900 |
\D not digit |
| 901 |
\s whitespace (\p{White_Space}) |
| 902 |
\S not whitespace |
| 903 |
\w word character (\p{Alphabetic} + \p{M} + \d + \p{Pc} + \p{Join_Control}) |
| 904 |
\W not word character |
| 905 |
</pre> |
| 906 |
|
| 907 |
### ASCII character classes |
| 908 |
|
| 909 |
These classes are based on the definitions provided in |
| 910 |
[UTS#18](https://www.unicode.org/reports/tr18/#Compatibility_Properties): |
| 911 |
|
| 912 |
<pre class="rust"> |
| 913 |
[[:alnum:]] alphanumeric ([0-9A-Za-z]) |
| 914 |
[[:alpha:]] alphabetic ([A-Za-z]) |
| 915 |
[[:ascii:]] ASCII ([\x00-\x7F]) |
| 916 |
[[:blank:]] blank ([\t ]) |
| 917 |
[[:cntrl:]] control ([\x00-\x1F\x7F]) |
| 918 |
[[:digit:]] digits ([0-9]) |
| 919 |
[[:graph:]] graphical ([!-~]) |
| 920 |
[[:lower:]] lower case ([a-z]) |
| 921 |
[[:print:]] printable ([ -~]) |
| 922 |
[[:punct:]] punctuation ([!-/:-@\[-`{-~]) |
| 923 |
[[:space:]] whitespace ([\t\n\v\f\r ]) |
| 924 |
[[:upper:]] upper case ([A-Z]) |
| 925 |
[[:word:]] word characters ([0-9A-Za-z_]) |
| 926 |
[[:xdigit:]] hex digit ([0-9A-Fa-f]) |
| 927 |
</pre> |
| 928 |
|
| 929 |
# Untrusted input |
| 930 |
|
| 931 |
This crate is meant to be able to run regex searches on untrusted haystacks |
| 932 |
without fear of [ReDoS]. This crate also, to a certain extent, supports |
| 933 |
untrusted patterns. |
| 934 |
|
| 935 |
[ReDoS]: https://en.wikipedia.org/wiki/ReDoS |
| 936 |
|
| 937 |
This crate differs from most (but not all) other regex engines in that it |
| 938 |
doesn't use unbounded backtracking to run a regex search. In those cases, |
| 939 |
one generally cannot use untrusted patterns *or* untrusted haystacks because |
| 940 |
it can be very difficult to know whether a particular pattern will result in |
| 941 |
catastrophic backtracking or not. |
| 942 |
|
| 943 |
We'll first discuss how this crate deals with untrusted inputs and then wrap |
| 944 |
it up with a realistic discussion about what practice really looks like. |
| 945 |
|
| 946 |
### Panics |
| 947 |
|
| 948 |
Outside of clearly documented cases, most APIs in this crate are intended to |
| 949 |
never panic regardless of the inputs given to them. For example, `Regex::new`, |
| 950 |
`Regex::is_match`, `Regex::find` and `Regex::captures` should never panic. That |
| 951 |
is, it is an API promise that those APIs will never panic no matter what inputs |
| 952 |
are given to them. With that said, regex engines are complicated beasts, and |
| 953 |
providing a rock solid guarantee that these APIs literally never panic is |
| 954 |
essentially equivalent to saying, "there are no bugs in this library." That is |
| 955 |
a bold claim, and not really one that can be feasibly made with a straight |
| 956 |
face. |
| 957 |
|
| 958 |
Don't get the wrong impression here. This crate is extensively tested, not just |
| 959 |
with unit and integration tests, but also via fuzz testing. For example, this |
| 960 |
crate is part of the [OSS-fuzz project]. Panics should be incredibly rare, but |
| 961 |
it is possible for bugs to exist, and thus possible for a panic to occur. If |
| 962 |
you need a rock solid guarantee against panics, then you should wrap calls into |
| 963 |
this library with [`std::panic::catch_unwind`]. |
| 964 |
|
| 965 |
It's also worth pointing out that this library will *generally* panic when |
| 966 |
other regex engines would commit undefined behavior. When undefined behavior |
| 967 |
occurs, your program might continue as if nothing bad has happened, but it also |
| 968 |
might mean your program is open to the worst kinds of exploits. In contrast, |
| 969 |
the worst thing a panic can do is a denial of service. |
| 970 |
|
| 971 |
[OSS-fuzz project]: https://android.googlesource.com/platform/external/oss-fuzz/+/refs/tags/android-t-preview-1/projects/rust-regex/ |
| 972 |
[`std::panic::catch_unwind`]: https://doc.rust-lang.org/std/panic/fn.catch_unwind.html |
| 973 |
|
| 974 |
### Untrusted patterns |
| 975 |
|
| 976 |
The principal way this crate deals with them is by limiting their size by |
| 977 |
default. The size limit can be configured via [`RegexBuilder::size_limit`]. The |
| 978 |
idea of a size limit is that compiling a pattern into a `Regex` will fail if it |
| 979 |
becomes "too big." Namely, while *most* resources consumed by compiling a regex |
| 980 |
are approximately proportional (albeit with some high constant factors in some |
| 981 |
cases, such as with Unicode character classes) to the length of the pattern |
| 982 |
itself, there is one particular exception to this: counted repetitions. Namely, |
| 983 |
this pattern: |
| 984 |
|
| 985 |
```text |
| 986 |
a{5}{5}{5}{5}{5}{5} |
| 987 |
``` |
| 988 |
|
| 989 |
Is equivalent to this pattern: |
| 990 |
|
| 991 |
```text |
| 992 |
a{15625} |
| 993 |
``` |
| 994 |
|
| 995 |
In both of these cases, the actual pattern string is quite small, but the |
| 996 |
resulting `Regex` value is quite large. Indeed, as the first pattern shows, |
| 997 |
it isn't enough to locally limit the size of each repetition because they can |
| 998 |
be stacked in a way that results in exponential growth. |
| 999 |
|
| 1000 |
To provide a bit more context, a simplified view of regex compilation looks |
| 1001 |
like this: |
| 1002 |
|
| 1003 |
* The pattern string is parsed into a structured representation called an AST. |
| 1004 |
Counted repetitions are not expanded and Unicode character classes are not |
| 1005 |
looked up in this stage. That is, the size of the AST is proportional to the |
| 1006 |
size of the pattern with "reasonable" constant factors. In other words, one |
| 1007 |
can reasonably limit the memory used by an AST by limiting the length of the |
| 1008 |
pattern string. |
| 1009 |
* The AST is translated into an HIR. Counted repetitions are still *not* |
| 1010 |
expanded at this stage, but Unicode character classes are embedded into the |
| 1011 |
HIR. The memory usage of a HIR is still proportional to the length of the |
| 1012 |
original pattern string, but the constant factors---mostly as a result of |
| 1013 |
Unicode character classes---can be quite high. Still though, the memory used by |
| 1014 |
an HIR can be reasonably limited by limiting the length of the pattern string. |
| 1015 |
* The HIR is compiled into a [Thompson NFA]. This is the stage at which |
| 1016 |
something like `\w{5}` is rewritten to `\w\w\w\w\w`. Thus, this is the stage |
| 1017 |
at which [`RegexBuilder::size_limit`] is enforced. If the NFA exceeds the |
| 1018 |
configured size, then this stage will fail. |
| 1019 |
|
| 1020 |
[Thompson NFA]: https://en.wikipedia.org/wiki/Thompson%27s_construction |
| 1021 |
|
| 1022 |
The size limit helps avoid two different kinds of exorbitant resource usage: |
| 1023 |
|
| 1024 |
* It avoids permitting exponential memory usage based on the size of the |
| 1025 |
pattern string. |
| 1026 |
* It avoids long search times. This will be discussed in more detail in the |
| 1027 |
next section, but worst case search time *is* dependent on the size of the |
| 1028 |
regex. So keeping regexes limited to a reasonable size is also a way of keeping |
| 1029 |
search times reasonable. |
| 1030 |
|
| 1031 |
Finally, it's worth pointing out that regex compilation is guaranteed to take |
| 1032 |
worst case `O(m)` time, where `m` is proportional to the size of regex. The |
| 1033 |
size of the regex here is *after* the counted repetitions have been expanded. |
| 1034 |
|
| 1035 |
**Advice for those using untrusted regexes**: limit the pattern length to |
| 1036 |
something small and expand it as needed. Configure [`RegexBuilder::size_limit`] |
| 1037 |
to something small and then expand it as needed. |
| 1038 |
|
| 1039 |
### Untrusted haystacks |
| 1040 |
|
| 1041 |
The main way this crate guards against searches from taking a long time is by |
| 1042 |
using algorithms that guarantee a `O(m * n)` worst case time and space bound. |
| 1043 |
Namely: |
| 1044 |
|
| 1045 |
* `m` is proportional to the size of the regex, where the size of the regex |
| 1046 |
includes the expansion of all counted repetitions. (See the previous section on |
| 1047 |
untrusted patterns.) |
| 1048 |
* `n` is proportional to the length, in bytes, of the haystack. |
| 1049 |
|
| 1050 |
In other words, if you consider `m` to be a constant (for example, the regex |
| 1051 |
pattern is a literal in the source code), then the search can be said to run |
| 1052 |
in "linear time." Or equivalently, "linear time with respect to the size of the |
| 1053 |
haystack." |
| 1054 |
|
| 1055 |
But the `m` factor here is important not to ignore. If a regex is |
| 1056 |
particularly big, the search times can get quite slow. This is why, in part, |
| 1057 |
[`RegexBuilder::size_limit`] exists. |
| 1058 |
|
| 1059 |
**Advice for those searching untrusted haystacks**: As long as your regexes |
| 1060 |
are not enormous, you should expect to be able to search untrusted haystacks |
| 1061 |
without fear. If you aren't sure, you should benchmark it. Unlike backtracking |
| 1062 |
engines, if your regex is so big that it's likely to result in slow searches, |
| 1063 |
this is probably something you'll be able to observe regardless of what the |
| 1064 |
haystack is made up of. |
| 1065 |
|
| 1066 |
### Iterating over matches |
| 1067 |
|
| 1068 |
One thing that is perhaps easy to miss is that the worst case time |
| 1069 |
complexity bound of `O(m * n)` applies to methods like [`Regex::is_match`], |
| 1070 |
[`Regex::find`] and [`Regex::captures`]. It does **not** apply to |
| 1071 |
[`Regex::find_iter`] or [`Regex::captures_iter`]. Namely, since iterating over |
| 1072 |
all matches can execute many searches, and each search can scan the entire |
| 1073 |
haystack, the worst case time complexity for iterators is `O(m * n^2)`. |
| 1074 |
|
| 1075 |
One example of where this occurs is when a pattern consists of an alternation, |
| 1076 |
where an earlier branch of the alternation requires scanning the entire |
| 1077 |
haystack only to discover that there is no match. It also requires a later |
| 1078 |
branch of the alternation to have matched at the beginning of the search. For |
| 1079 |
example, consider the pattern `.*[^A-Z]|[A-Z]` and the haystack `AAAAA`. The |
| 1080 |
first search will scan to the end looking for matches of `.*[^A-Z]` even though |
| 1081 |
a finite automata engine (as in this crate) knows that `[A-Z]` has already |
| 1082 |
matched the first character of the haystack. This is due to the greedy nature |
| 1083 |
of regex searching. That first search will report a match at the first `A` only |
| 1084 |
after scanning to the end to discover that no other match exists. The next |
| 1085 |
search then begins at the second `A` and the behavior repeats. |
| 1086 |
|
| 1087 |
There is no way to avoid this. This means that if both patterns and haystacks |
| 1088 |
are untrusted and you're iterating over all matches, you're susceptible to |
| 1089 |
worst case quadratic time complexity. One possible way to mitigate this |
| 1090 |
is to drop down to the lower level `regex-automata` crate and use its |
| 1091 |
`meta::Regex` iterator APIs. There, you can configure the search to operate |
| 1092 |
in "earliest" mode by passing a `Input::new(haystack).earliest(true)` to |
| 1093 |
`meta::Regex::find_iter` (for example). By enabling this mode, you give up |
| 1094 |
the normal greedy match semantics of regex searches and instead ask the regex |
| 1095 |
engine to immediately stop as soon as a match has been found. Enabling this |
| 1096 |
mode will thus restore the worst case `O(m * n)` time complexity bound, but at |
| 1097 |
the cost of different semantics. |
| 1098 |
|
| 1099 |
### Untrusted inputs in practice |
| 1100 |
|
| 1101 |
While providing a `O(m * n)` worst case time bound on all searches goes a long |
| 1102 |
way toward preventing [ReDoS], that doesn't mean every search you can possibly |
| 1103 |
run will complete without burning CPU time. In general, there are a few ways |
| 1104 |
for the `m * n` time bound to still bite you: |
| 1105 |
|
| 1106 |
* You are searching an exceptionally long haystack. No matter how you slice |
| 1107 |
it, a longer haystack will take more time to search. This crate may often make |
| 1108 |
very quick work of even long haystacks because of its literal optimizations, |
| 1109 |
but those aren't available for all regexes. |
| 1110 |
* Unicode character classes can cause searches to be quite slow in some cases. |
| 1111 |
This is especially true when they are combined with counted repetitions. While |
| 1112 |
the regex size limit above will protect you from the most egregious cases, |
| 1113 |
the default size limit still permits pretty big regexes that can execute more |
| 1114 |
slowly than one might expect. |
| 1115 |
* While routines like [`Regex::find`] and [`Regex::captures`] guarantee |
| 1116 |
worst case `O(m * n)` search time, routines like [`Regex::find_iter`] and |
| 1117 |
[`Regex::captures_iter`] actually have worst case `O(m * n^2)` search time. |
| 1118 |
This is because `find_iter` runs many searches, and each search takes worst |
| 1119 |
case `O(m * n)` time. Thus, iteration of all matches in a haystack has |
| 1120 |
worst case `O(m * n^2)`. A good example of a pattern that exhibits this is |
| 1121 |
`(?:A+){1000}|` or even `.*[^A-Z]|[A-Z]`. |
| 1122 |
|
| 1123 |
In general, unstrusted haystacks are easier to stomach than untrusted patterns. |
| 1124 |
Untrusted patterns give a lot more control to the caller to impact the |
| 1125 |
performance of a search. In many cases, a regex search will actually execute in |
| 1126 |
average case `O(n)` time (i.e., not dependent on the size of the regex), but |
| 1127 |
this can't be guaranteed in general. Therefore, permitting untrusted patterns |
| 1128 |
means that your only line of defense is to put a limit on how big `m` (and |
| 1129 |
perhaps also `n`) can be in `O(m * n)`. `n` is limited by simply inspecting |
| 1130 |
the length of the haystack while `m` is limited by *both* applying a limit to |
| 1131 |
the length of the pattern *and* a limit on the compiled size of the regex via |
| 1132 |
[`RegexBuilder::size_limit`]. |
| 1133 |
|
| 1134 |
It bears repeating: if you're accepting untrusted patterns, it would be a good |
| 1135 |
idea to start with conservative limits on `m` and `n`, and then carefully |
| 1136 |
increase them as needed. |
| 1137 |
|
| 1138 |
# Crate features |
| 1139 |
|
| 1140 |
By default, this crate tries pretty hard to make regex matching both as fast |
| 1141 |
as possible and as correct as it can be. This means that there is a lot of |
| 1142 |
code dedicated to performance, the handling of Unicode data and the Unicode |
| 1143 |
data itself. Overall, this leads to more dependencies, larger binaries and |
| 1144 |
longer compile times. This trade off may not be appropriate in all cases, and |
| 1145 |
indeed, even when all Unicode and performance features are disabled, one is |
| 1146 |
still left with a perfectly serviceable regex engine that will work well in |
| 1147 |
many cases. (Note that code is not arbitrarily reducible, and for this reason, |
| 1148 |
the [`regex-lite`](https://docs.rs/regex-lite) crate exists to provide an even |
| 1149 |
more minimal experience by cutting out Unicode and performance, but still |
| 1150 |
maintaining the linear search time bound.) |
| 1151 |
|
| 1152 |
This crate exposes a number of features for controlling that trade off. Some |
| 1153 |
of these features are strictly performance oriented, such that disabling them |
| 1154 |
won't result in a loss of functionality, but may result in worse performance. |
| 1155 |
Other features, such as the ones controlling the presence or absence of Unicode |
| 1156 |
data, can result in a loss of functionality. For example, if one disables the |
| 1157 |
`unicode-case` feature (described below), then compiling the regex `(?i)a` |
| 1158 |
will fail since Unicode case insensitivity is enabled by default. Instead, |
| 1159 |
callers must use `(?i-u)a` to disable Unicode case folding. Stated differently, |
| 1160 |
enabling or disabling any of the features below can only add or subtract from |
| 1161 |
the total set of valid regular expressions. Enabling or disabling a feature |
| 1162 |
will never modify the match semantics of a regular expression. |
| 1163 |
|
| 1164 |
Most features below are enabled by default. Features that aren't enabled by |
| 1165 |
default are noted. |
| 1166 |
|
| 1167 |
### Ecosystem features |
| 1168 |
|
| 1169 |
* **std** - |
| 1170 |
When enabled, this will cause `regex` to use the standard library. In terms |
| 1171 |
of APIs, `std` causes error types to implement the `std::error::Error` |
| 1172 |
trait. Enabling `std` will also result in performance optimizations, |
| 1173 |
including SIMD and faster synchronization primitives. Notably, **disabling |
| 1174 |
the `std` feature will result in the use of spin locks**. To use a regex |
| 1175 |
engine without `std` and without spin locks, you'll need to drop down to |
| 1176 |
the [`regex-automata`](https://docs.rs/regex-automata) crate. |
| 1177 |
* **logging** - |
| 1178 |
When enabled, the `log` crate is used to emit messages about regex |
| 1179 |
compilation and search strategies. This is **disabled by default**. This is |
| 1180 |
typically only useful to someone working on this crate's internals, but might |
| 1181 |
be useful if you're doing some rabbit hole performance hacking. Or if you're |
| 1182 |
just interested in the kinds of decisions being made by the regex engine. |
| 1183 |
|
| 1184 |
### Performance features |
| 1185 |
|
| 1186 |
* **perf** - |
| 1187 |
Enables all performance related features except for `perf-dfa-full`. This |
| 1188 |
feature is enabled by default is intended to cover all reasonable features |
| 1189 |
that improve performance, even if more are added in the future. |
| 1190 |
* **perf-dfa** - |
| 1191 |
Enables the use of a lazy DFA for matching. The lazy DFA is used to compile |
| 1192 |
portions of a regex to a very fast DFA on an as-needed basis. This can |
| 1193 |
result in substantial speedups, usually by an order of magnitude on large |
| 1194 |
haystacks. The lazy DFA does not bring in any new dependencies, but it can |
| 1195 |
make compile times longer. |
| 1196 |
* **perf-dfa-full** - |
| 1197 |
Enables the use of a full DFA for matching. Full DFAs are problematic because |
| 1198 |
they have worst case `O(2^n)` construction time. For this reason, when this |
| 1199 |
feature is enabled, full DFAs are only used for very small regexes and a |
| 1200 |
very small space bound is used during determinization to avoid the DFA |
| 1201 |
from blowing up. This feature is not enabled by default, even as part of |
| 1202 |
`perf`, because it results in fairly sizeable increases in binary size and |
| 1203 |
compilation time. It can result in faster search times, but they tend to be |
| 1204 |
more modest and limited to non-Unicode regexes. |
| 1205 |
* **perf-onepass** - |
| 1206 |
Enables the use of a one-pass DFA for extracting the positions of capture |
| 1207 |
groups. This optimization applies to a subset of certain types of NFAs and |
| 1208 |
represents the fastest engine in this crate for dealing with capture groups. |
| 1209 |
* **perf-backtrack** - |
| 1210 |
Enables the use of a bounded backtracking algorithm for extracting the |
| 1211 |
positions of capture groups. This usually sits between the slowest engine |
| 1212 |
(the PikeVM) and the fastest engine (one-pass DFA) for extracting capture |
| 1213 |
groups. It's used whenever the regex is not one-pass and is small enough. |
| 1214 |
* **perf-inline** - |
| 1215 |
Enables the use of aggressive inlining inside match routines. This reduces |
| 1216 |
the overhead of each match. The aggressive inlining, however, increases |
| 1217 |
compile times and binary size. |
| 1218 |
* **perf-literal** - |
| 1219 |
Enables the use of literal optimizations for speeding up matches. In some |
| 1220 |
cases, literal optimizations can result in speedups of _several_ orders of |
| 1221 |
magnitude. Disabling this drops the `aho-corasick` and `memchr` dependencies. |
| 1222 |
* **perf-cache** - |
| 1223 |
This feature used to enable a faster internal cache at the cost of using |
| 1224 |
additional dependencies, but this is no longer an option. A fast internal |
| 1225 |
cache is now used unconditionally with no additional dependencies. This may |
| 1226 |
change in the future. |
| 1227 |
|
| 1228 |
### Unicode features |
| 1229 |
|
| 1230 |
* **unicode** - |
| 1231 |
Enables all Unicode features. This feature is enabled by default, and will |
| 1232 |
always cover all Unicode features, even if more are added in the future. |
| 1233 |
* **unicode-age** - |
| 1234 |
Provide the data for the |
| 1235 |
[Unicode `Age` property](https://www.unicode.org/reports/tr44/tr44-24.html#Character_Age). |
| 1236 |
This makes it possible to use classes like `\p{Age:6.0}` to refer to all |
| 1237 |
codepoints first introduced in Unicode 6.0 |
| 1238 |
* **unicode-bool** - |
| 1239 |
Provide the data for numerous Unicode boolean properties. The full list |
| 1240 |
is not included here, but contains properties like `Alphabetic`, `Emoji`, |
| 1241 |
`Lowercase`, `Math`, `Uppercase` and `White_Space`. |
| 1242 |
* **unicode-case** - |
| 1243 |
Provide the data for case insensitive matching using |
| 1244 |
[Unicode's "simple loose matches" specification](https://www.unicode.org/reports/tr18/#Simple_Loose_Matches). |
| 1245 |
* **unicode-gencat** - |
| 1246 |
Provide the data for |
| 1247 |
[Unicode general categories](https://www.unicode.org/reports/tr44/tr44-24.html#General_Category_Values). |
| 1248 |
This includes, but is not limited to, `Decimal_Number`, `Letter`, |
| 1249 |
`Math_Symbol`, `Number` and `Punctuation`. |
| 1250 |
* **unicode-perl** - |
| 1251 |
Provide the data for supporting the Unicode-aware Perl character classes, |
| 1252 |
corresponding to `\w`, `\s` and `\d`. This is also necessary for using |
| 1253 |
Unicode-aware word boundary assertions. Note that if this feature is |
| 1254 |
disabled, the `\s` and `\d` character classes are still available if the |
| 1255 |
`unicode-bool` and `unicode-gencat` features are enabled, respectively. |
| 1256 |
* **unicode-script** - |
| 1257 |
Provide the data for |
| 1258 |
[Unicode scripts and script extensions](https://www.unicode.org/reports/tr24/). |
| 1259 |
This includes, but is not limited to, `Arabic`, `Cyrillic`, `Hebrew`, |
| 1260 |
`Latin` and `Thai`. |
| 1261 |
* **unicode-segment** - |
| 1262 |
Provide the data necessary to provide the properties used to implement the |
| 1263 |
[Unicode text segmentation algorithms](https://www.unicode.org/reports/tr29/). |
| 1264 |
This enables using classes like `\p{gcb=Extend}`, `\p{wb=Katakana}` and |
| 1265 |
`\p{sb=ATerm}`. |
| 1266 |
|
| 1267 |
# Other crates |
| 1268 |
|
| 1269 |
This crate has two required dependencies and several optional dependencies. |
| 1270 |
This section briefly describes them with the goal of raising awareness of how |
| 1271 |
different components of this crate may be used independently. |
| 1272 |
|
| 1273 |
It is somewhat unusual for a regex engine to have dependencies, as most regex |
| 1274 |
libraries are self contained units with no dependencies other than a particular |
| 1275 |
environment's standard library. Indeed, for other similarly optimized regex |
| 1276 |
engines, most or all of the code in the dependencies of this crate would |
| 1277 |
normally just be unseparable or coupled parts of the crate itself. But since |
| 1278 |
Rust and its tooling ecosystem make the use of dependencies so easy, it made |
| 1279 |
sense to spend some effort de-coupling parts of this crate and making them |
| 1280 |
independently useful. |
| 1281 |
|
| 1282 |
We only briefly describe each crate here. |
| 1283 |
|
| 1284 |
* [`regex-lite`](https://docs.rs/regex-lite) is not a dependency of `regex`, |
| 1285 |
but rather, a standalone zero-dependency simpler version of `regex` that |
| 1286 |
prioritizes compile times and binary size. In exchange, it eschews Unicode |
| 1287 |
support and performance. Its match semantics are as identical as possible to |
| 1288 |
the `regex` crate, and for the things it supports, its APIs are identical to |
| 1289 |
the APIs in this crate. In other words, for a lot of use cases, it is a drop-in |
| 1290 |
replacement. |
| 1291 |
* [`regex-syntax`](https://docs.rs/regex-syntax) provides a regular expression |
| 1292 |
parser via `Ast` and `Hir` types. It also provides routines for extracting |
| 1293 |
literals from a pattern. Folks can use this crate to do analysis, or even to |
| 1294 |
build their own regex engine without having to worry about writing a parser. |
| 1295 |
* [`regex-automata`](https://docs.rs/regex-automata) provides the regex engines |
| 1296 |
themselves. One of the downsides of finite automata based regex engines is that |
| 1297 |
they often need multiple internal engines in order to have similar or better |
| 1298 |
performance than an unbounded backtracking engine in practice. `regex-automata` |
| 1299 |
in particular provides public APIs for a PikeVM, a bounded backtracker, a |
| 1300 |
one-pass DFA, a lazy DFA, a fully compiled DFA and a meta regex engine that |
| 1301 |
combines all them together. It also has native multi-pattern support and |
| 1302 |
provides a way to compile and serialize full DFAs such that they can be loaded |
| 1303 |
and searched in a no-std no-alloc environment. `regex-automata` itself doesn't |
| 1304 |
even have a required dependency on `regex-syntax`! |
| 1305 |
* [`memchr`](https://docs.rs/memchr) provides low level SIMD vectorized |
| 1306 |
routines for quickly finding the location of single bytes or even substrings |
| 1307 |
in a haystack. In other words, it provides fast `memchr` and `memmem` routines. |
| 1308 |
These are used by this crate in literal optimizations. |
| 1309 |
* [`aho-corasick`](https://docs.rs/aho-corasick) provides multi-substring |
| 1310 |
search. It also provides SIMD vectorized routines in the case where the number |
| 1311 |
of substrings to search for is relatively small. The `regex` crate also uses |
| 1312 |
this for literal optimizations. |
| 1313 |
*/ |
| 1314 |
|
| 1315 |
#![no_std] |
| 1316 |
#![deny(missing_docs)] |
| 1317 |
#![cfg_attr(feature = "pattern", feature(pattern))] |
| 1318 |
#![warn(missing_debug_implementations)] |
| 1319 |
|
| 1320 |
#[cfg(doctest)] |
| 1321 |
doc_comment::doctest!("../README.md"); |
| 1322 |
|
| 1323 |
extern crate alloc; |
| 1324 |
#[cfg(any(test, feature = "std"))] |
| 1325 |
extern crate std; |
| 1326 |
|
| 1327 |
pub use crate::error::Error; |
| 1328 |
|
| 1329 |
pub use crate::{builders::string::*, regex::string::*, regexset::string::*}; |
| 1330 |
|
| 1331 |
mod builders; |
| 1332 |
pub mod bytes; |
| 1333 |
mod error; |
| 1334 |
mod find_byte; |
| 1335 |
#[cfg(feature = "pattern")] |
| 1336 |
mod pattern; |
| 1337 |
mod regex; |
| 1338 |
mod regexset; |
| 1339 |
|
| 1340 |
/// Escapes all regular expression meta characters in `pattern`. |
| 1341 |
/// |
| 1342 |
/// The string returned may be safely used as a literal in a regular |
| 1343 |
/// expression. |
| 1344 |
pub fn escape(pattern: &str) -> alloc::string::String { |
| 1345 |
regex_syntax::escape(pattern) |
| 1346 |
} |