| 1 |
Rust Quasi-Quoting |
| 2 |
================== |
| 3 |
|
| 4 |
[<img alt="github" src="https://img.shields.io/badge/github-dtolnay/quote-8da0cb?style=for-the-badge&labelColor=555555&logo=github" height="20">](https://github.com/dtolnay/quote) |
| 5 |
[<img alt="crates.io" src="https://img.shields.io/crates/v/quote.svg?style=for-the-badge&color=fc8d62&logo=rust" height="20">](https://crates.io/crates/quote) |
| 6 |
[<img alt="docs.rs" src="https://img.shields.io/badge/docs.rs-quote-66c2a5?style=for-the-badge&labelColor=555555&logo=docs.rs" height="20">](https://docs.rs/quote) |
| 7 |
[<img alt="build status" src="https://img.shields.io/github/actions/workflow/status/dtolnay/quote/ci.yml?branch=master&style=for-the-badge" height="20">](https://github.com/dtolnay/quote/actions?query=branch%3Amaster) |
| 8 |
|
| 9 |
This crate provides the [`quote!`] macro for turning Rust syntax tree data |
| 10 |
structures into tokens of source code. |
| 11 |
|
| 12 |
[`quote!`]: https://docs.rs/quote/1.0/quote/macro.quote.html |
| 13 |
|
| 14 |
Procedural macros in Rust receive a stream of tokens as input, execute arbitrary |
| 15 |
Rust code to determine how to manipulate those tokens, and produce a stream of |
| 16 |
tokens to hand back to the compiler to compile into the caller's crate. |
| 17 |
Quasi-quoting is a solution to one piece of that — producing tokens to |
| 18 |
return to the compiler. |
| 19 |
|
| 20 |
The idea of quasi-quoting is that we write *code* that we treat as *data*. |
| 21 |
Within the `quote!` macro, we can write what looks like code to our text editor |
| 22 |
or IDE. We get all the benefits of the editor's brace matching, syntax |
| 23 |
highlighting, indentation, and maybe autocompletion. But rather than compiling |
| 24 |
that as code into the current crate, we can treat it as data, pass it around, |
| 25 |
mutate it, and eventually hand it back to the compiler as tokens to compile into |
| 26 |
the macro caller's crate. |
| 27 |
|
| 28 |
This crate is motivated by the procedural macro use case, but is a |
| 29 |
general-purpose Rust quasi-quoting library and is not specific to procedural |
| 30 |
macros. |
| 31 |
|
| 32 |
```toml |
| 33 |
[dependencies] |
| 34 |
quote = "1.0" |
| 35 |
``` |
| 36 |
|
| 37 |
*Version requirement: Quote supports rustc 1.56 and up.*<br> |
| 38 |
[*Release notes*](https://github.com/dtolnay/quote/releases) |
| 39 |
|
| 40 |
<br> |
| 41 |
|
| 42 |
## Syntax |
| 43 |
|
| 44 |
The quote crate provides a [`quote!`] macro within which you can write Rust code |
| 45 |
that gets packaged into a [`TokenStream`] and can be treated as data. You should |
| 46 |
think of `TokenStream` as representing a fragment of Rust source code. |
| 47 |
|
| 48 |
[`TokenStream`]: https://docs.rs/proc-macro2/1.0/proc_macro2/struct.TokenStream.html |
| 49 |
|
| 50 |
Within the `quote!` macro, interpolation is done with `#var`. Any type |
| 51 |
implementing the [`quote::ToTokens`] trait can be interpolated. This includes |
| 52 |
most Rust primitive types as well as most of the syntax tree types from [`syn`]. |
| 53 |
|
| 54 |
[`quote::ToTokens`]: https://docs.rs/quote/1.0/quote/trait.ToTokens.html |
| 55 |
[`syn`]: https://github.com/dtolnay/syn |
| 56 |
|
| 57 |
```rust |
| 58 |
let tokens = quote! { |
| 59 |
struct SerializeWith #generics #where_clause { |
| 60 |
value: &'a #field_ty, |
| 61 |
phantom: core::marker::PhantomData<#item_ty>, |
| 62 |
} |
| 63 |
|
| 64 |
impl #generics serde::Serialize for SerializeWith #generics #where_clause { |
| 65 |
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error> |
| 66 |
where |
| 67 |
S: serde::Serializer, |
| 68 |
{ |
| 69 |
#path(self.value, serializer) |
| 70 |
} |
| 71 |
} |
| 72 |
|
| 73 |
SerializeWith { |
| 74 |
value: #value, |
| 75 |
phantom: core::marker::PhantomData::<#item_ty>, |
| 76 |
} |
| 77 |
}; |
| 78 |
``` |
| 79 |
|
| 80 |
<br> |
| 81 |
|
| 82 |
## Repetition |
| 83 |
|
| 84 |
Repetition is done using `#(...)*` or `#(...),*` similar to `macro_rules!`. This |
| 85 |
iterates through the elements of any variable interpolated within the repetition |
| 86 |
and inserts a copy of the repetition body for each one. The variables in an |
| 87 |
interpolation may be anything that implements `IntoIterator`, including `Vec` or |
| 88 |
a pre-existing iterator. |
| 89 |
|
| 90 |
- `#(#var)*` — no separators |
| 91 |
- `#(#var),*` — the character before the asterisk is used as a separator |
| 92 |
- `#( struct #var; )*` — the repetition can contain other things |
| 93 |
- `#( #k => println!("{}", #v), )*` — even multiple interpolations |
| 94 |
|
| 95 |
Note that there is a difference between `#(#var ,)*` and `#(#var),*`—the latter |
| 96 |
does not produce a trailing comma. This matches the behavior of delimiters in |
| 97 |
`macro_rules!`. |
| 98 |
|
| 99 |
<br> |
| 100 |
|
| 101 |
## Returning tokens to the compiler |
| 102 |
|
| 103 |
The `quote!` macro evaluates to an expression of type |
| 104 |
`proc_macro2::TokenStream`. Meanwhile Rust procedural macros are expected to |
| 105 |
return the type `proc_macro::TokenStream`. |
| 106 |
|
| 107 |
The difference between the two types is that `proc_macro` types are entirely |
| 108 |
specific to procedural macros and cannot ever exist in code outside of a |
| 109 |
procedural macro, while `proc_macro2` types may exist anywhere including tests |
| 110 |
and non-macro code like main.rs and build.rs. This is why even the procedural |
| 111 |
macro ecosystem is largely built around `proc_macro2`, because that ensures the |
| 112 |
libraries are unit testable and accessible in non-macro contexts. |
| 113 |
|
| 114 |
There is a [`From`]-conversion in both directions so returning the output of |
| 115 |
`quote!` from a procedural macro usually looks like `tokens.into()` or |
| 116 |
`proc_macro::TokenStream::from(tokens)`. |
| 117 |
|
| 118 |
[`From`]: https://doc.rust-lang.org/std/convert/trait.From.html |
| 119 |
|
| 120 |
<br> |
| 121 |
|
| 122 |
## Examples |
| 123 |
|
| 124 |
### Combining quoted fragments |
| 125 |
|
| 126 |
Usually you don't end up constructing an entire final `TokenStream` in one |
| 127 |
piece. Different parts may come from different helper functions. The tokens |
| 128 |
produced by `quote!` themselves implement `ToTokens` and so can be interpolated |
| 129 |
into later `quote!` invocations to build up a final result. |
| 130 |
|
| 131 |
```rust |
| 132 |
let type_definition = quote! {...}; |
| 133 |
let methods = quote! {...}; |
| 134 |
|
| 135 |
let tokens = quote! { |
| 136 |
#type_definition |
| 137 |
#methods |
| 138 |
}; |
| 139 |
``` |
| 140 |
|
| 141 |
### Constructing identifiers |
| 142 |
|
| 143 |
Suppose we have an identifier `ident` which came from somewhere in a macro |
| 144 |
input and we need to modify it in some way for the macro output. Let's consider |
| 145 |
prepending the identifier with an underscore. |
| 146 |
|
| 147 |
Simply interpolating the identifier next to an underscore will not have the |
| 148 |
behavior of concatenating them. The underscore and the identifier will continue |
| 149 |
to be two separate tokens as if you had written `_ x`. |
| 150 |
|
| 151 |
```rust |
| 152 |
// incorrect |
| 153 |
quote! { |
| 154 |
let mut _#ident = 0; |
| 155 |
} |
| 156 |
``` |
| 157 |
|
| 158 |
The solution is to build a new identifier token with the correct value. As this |
| 159 |
is such a common case, the `format_ident!` macro provides a convenient utility |
| 160 |
for doing so correctly. |
| 161 |
|
| 162 |
```rust |
| 163 |
let varname = format_ident!("_{}", ident); |
| 164 |
quote! { |
| 165 |
let mut #varname = 0; |
| 166 |
} |
| 167 |
``` |
| 168 |
|
| 169 |
Alternatively, the APIs provided by Syn and proc-macro2 can be used to directly |
| 170 |
build the identifier. This is roughly equivalent to the above, but will not |
| 171 |
handle `ident` being a raw identifier. |
| 172 |
|
| 173 |
```rust |
| 174 |
let concatenated = format!("_{}", ident); |
| 175 |
let varname = syn::Ident::new(&concatenated, ident.span()); |
| 176 |
quote! { |
| 177 |
let mut #varname = 0; |
| 178 |
} |
| 179 |
``` |
| 180 |
|
| 181 |
### Making method calls |
| 182 |
|
| 183 |
Let's say our macro requires some type specified in the macro input to have a |
| 184 |
constructor called `new`. We have the type in a variable called `field_type` of |
| 185 |
type `syn::Type` and want to invoke the constructor. |
| 186 |
|
| 187 |
```rust |
| 188 |
// incorrect |
| 189 |
quote! { |
| 190 |
let value = #field_type::new(); |
| 191 |
} |
| 192 |
``` |
| 193 |
|
| 194 |
This works only sometimes. If `field_type` is `String`, the expanded code |
| 195 |
contains `String::new()` which is fine. But if `field_type` is something like |
| 196 |
`Vec<i32>` then the expanded code is `Vec<i32>::new()` which is invalid syntax. |
| 197 |
Ordinarily in handwritten Rust we would write `Vec::<i32>::new()` but for macros |
| 198 |
often the following is more convenient. |
| 199 |
|
| 200 |
```rust |
| 201 |
quote! { |
| 202 |
let value = <#field_type>::new(); |
| 203 |
} |
| 204 |
``` |
| 205 |
|
| 206 |
This expands to `<Vec<i32>>::new()` which behaves correctly. |
| 207 |
|
| 208 |
A similar pattern is appropriate for trait methods. |
| 209 |
|
| 210 |
```rust |
| 211 |
quote! { |
| 212 |
let value = <#field_type as core::default::Default>::default(); |
| 213 |
} |
| 214 |
``` |
| 215 |
|
| 216 |
<br> |
| 217 |
|
| 218 |
## Hygiene |
| 219 |
|
| 220 |
Any interpolated tokens preserve the `Span` information provided by their |
| 221 |
`ToTokens` implementation. Tokens that originate within a `quote!` invocation |
| 222 |
are spanned with [`Span::call_site()`]. |
| 223 |
|
| 224 |
[`Span::call_site()`]: https://docs.rs/proc-macro2/1.0/proc_macro2/struct.Span.html#method.call_site |
| 225 |
|
| 226 |
A different span can be provided explicitly through the [`quote_spanned!`] |
| 227 |
macro. |
| 228 |
|
| 229 |
[`quote_spanned!`]: https://docs.rs/quote/1.0/quote/macro.quote_spanned.html |
| 230 |
|
| 231 |
<br> |
| 232 |
|
| 233 |
## Non-macro code generators |
| 234 |
|
| 235 |
When using `quote` in a build.rs or main.rs and writing the output out to a |
| 236 |
file, consider having the code generator pass the tokens through [prettyplease] |
| 237 |
before writing. This way if an error occurs in the generated code it is |
| 238 |
convenient for a human to read and debug. |
| 239 |
|
| 240 |
Be aware that no kind of hygiene or span information is retained when tokens are |
| 241 |
written to a file; the conversion from tokens to source code is lossy. |
| 242 |
|
| 243 |
Example usage in build.rs: |
| 244 |
|
| 245 |
```rust |
| 246 |
let output = quote! { ... }; |
| 247 |
let syntax_tree = syn::parse2(output).unwrap(); |
| 248 |
let formatted = prettyplease::unparse(&syntax_tree); |
| 249 |
|
| 250 |
let out_dir = env::var_os("OUT_DIR").unwrap(); |
| 251 |
let dest_path = Path::new(&out_dir).join("out.rs"); |
| 252 |
fs::write(dest_path, formatted).unwrap(); |
| 253 |
``` |
| 254 |
|
| 255 |
[prettyplease]: https://github.com/dtolnay/prettyplease |
| 256 |
|
| 257 |
<br> |
| 258 |
|
| 259 |
#### License |
| 260 |
|
| 261 |
<sup> |
| 262 |
Licensed under either of <a href="LICENSE-APACHE">Apache License, Version |
| 263 |
2.0</a> or <a href="LICENSE-MIT">MIT license</a> at your option. |
| 264 |
</sup> |
| 265 |
|
| 266 |
<br> |
| 267 |
|
| 268 |
<sub> |
| 269 |
Unless you explicitly state otherwise, any contribution intentionally submitted |
| 270 |
for inclusion in this crate by you, as defined in the Apache-2.0 license, shall |
| 271 |
be dual licensed as above, without any additional terms or conditions. |
| 272 |
</sub> |