| 1 |
# Polyfill for "JS objects in wasm" |
| 2 |
|
| 3 |
One of the main goals of `wasm-bindgen` is to allow working with and passing |
| 4 |
around JS objects in wasm, but that's not allowed today! While indeed true, |
| 5 |
that's where the polyfill comes in. |
| 6 |
|
| 7 |
The question here is how we shoehorn JS objects into a `u32` for wasm to use. |
| 8 |
The current strategy for this approach is to maintain a module-local variable |
| 9 |
in the generated `foo.js` file: a `heap`. |
| 10 |
|
| 11 |
### Temporary JS objects on the "stack" |
| 12 |
|
| 13 |
The first slots in the `heap` in `foo.js` are considered a stack. This stack, |
| 14 |
like typical program execution stacks, grows down. JS objects are pushed on the |
| 15 |
bottom of the stack, and their index in the stack is the identifier that's passed |
| 16 |
to wasm. A stack pointer is maintained to figure out where the next item is |
| 17 |
pushed. |
| 18 |
|
| 19 |
JS objects are then only removed from the bottom of the stack as well. Removal |
| 20 |
is simply storing null then incrementing a counter. Because of the "stack-y" |
| 21 |
nature of this scheme it only works for when wasm doesn't hold onto a JS object |
| 22 |
(aka it only gets a "reference" in Rust parlance). |
| 23 |
|
| 24 |
Let's take a look at an example. |
| 25 |
|
| 26 |
```rust |
| 27 |
// foo.rs |
| 28 |
#[wasm_bindgen] |
| 29 |
pub fn foo(a: &JsValue) { |
| 30 |
// ... |
| 31 |
} |
| 32 |
``` |
| 33 |
|
| 34 |
Here we're using the special `JsValue` type from the `wasm-bindgen` library |
| 35 |
itself. Our exported function, `foo`, takes a *reference* to an object. This |
| 36 |
notably means that it can't persist the object past the lifetime of this |
| 37 |
function call. |
| 38 |
|
| 39 |
Now what we actually want to generate is a JS module that looks like (in |
| 40 |
TypeScript parlance) |
| 41 |
|
| 42 |
```ts |
| 43 |
// foo.d.ts |
| 44 |
export function foo(a: any); |
| 45 |
``` |
| 46 |
|
| 47 |
and what we actually generate looks something like: |
| 48 |
|
| 49 |
```js |
| 50 |
// foo.js |
| 51 |
import * as wasm from './foo_bg'; |
| 52 |
|
| 53 |
const heap = new Array(32); |
| 54 |
heap.push(undefined, null, true, false); |
| 55 |
let stack_pointer = 32; |
| 56 |
|
| 57 |
function addBorrowedObject(obj) { |
| 58 |
stack_pointer -= 1; |
| 59 |
heap[stack_pointer] = obj; |
| 60 |
return stack_pointer; |
| 61 |
} |
| 62 |
|
| 63 |
export function foo(arg0) { |
| 64 |
const idx0 = addBorrowedObject(arg0); |
| 65 |
try { |
| 66 |
wasm.foo(idx0); |
| 67 |
} finally { |
| 68 |
heap[stack_pointer++] = undefined; |
| 69 |
} |
| 70 |
} |
| 71 |
``` |
| 72 |
|
| 73 |
Here we can see a few notable points of action: |
| 74 |
|
| 75 |
* The wasm file was renamed to `foo_bg.wasm`, and we can see how the JS module |
| 76 |
generated here is importing from the wasm file. |
| 77 |
* Next we can see our `heap` module variable which is to store all JS values |
| 78 |
reference-able from wasm. |
| 79 |
* Our exported function `foo`, takes an arbitrary argument, `arg0`, which is |
| 80 |
converted to an index with the `addBorrowedObject` object function. The index |
| 81 |
is then passed to wasm so wasm can operate with it. |
| 82 |
* Finally, we have a `finally` which frees the stack slot as it's no longer |
| 83 |
used, popping the value that was pushed at the start of the function. |
| 84 |
|
| 85 |
It's also helpful to dig into the Rust side of things to see what's going on |
| 86 |
there! Let's take a look at the code that `#[wasm_bindgen]` generates in Rust: |
| 87 |
|
| 88 |
```rust |
| 89 |
// what the user wrote |
| 90 |
pub fn foo(a: &JsValue) { |
| 91 |
// ... |
| 92 |
} |
| 93 |
|
| 94 |
#[export_name = "foo"] |
| 95 |
pub extern "C" fn __wasm_bindgen_generated_foo(arg0: u32) { |
| 96 |
let arg0 = unsafe { |
| 97 |
ManuallyDrop::new(JsValue::__from_idx(arg0)) |
| 98 |
}; |
| 99 |
let arg0 = &*arg0; |
| 100 |
foo(arg0); |
| 101 |
} |
| 102 |
``` |
| 103 |
|
| 104 |
And as with the JS, the notable points here are: |
| 105 |
|
| 106 |
* The original function, `foo`, is unmodified in the output |
| 107 |
* A generated function here (with a unique name) is the one that's actually |
| 108 |
exported from the wasm module |
| 109 |
* Our generated function takes an integer argument (our index) and then wraps it |
| 110 |
in a `JsValue`. There's some trickery here that's not worth going into just |
| 111 |
yet, but we'll see in a bit what's happening under the hood. |
| 112 |
|
| 113 |
### Long-lived JS objects |
| 114 |
|
| 115 |
The above strategy is useful when JS objects are only temporarily used in Rust, |
| 116 |
for example only during one function call. Sometimes, though, objects may have a |
| 117 |
dynamic lifetime or otherwise need to be stored on Rust's heap. To cope with |
| 118 |
this there's a second half of management of JS objects, naturally corresponding |
| 119 |
to the other side of the JS `heap` array. |
| 120 |
|
| 121 |
JS Objects passed to wasm that are not references are assumed to have a dynamic |
| 122 |
lifetime inside of the wasm module. As a result the strict push/pop of the stack |
| 123 |
won't work and we need more permanent storage for the JS objects. To cope with |
| 124 |
this we build our own "slab allocator" of sorts. |
| 125 |
|
| 126 |
A picture (or code) is worth a thousand words so let's show what happens with an |
| 127 |
example. |
| 128 |
|
| 129 |
```rust |
| 130 |
// foo.rs |
| 131 |
#[wasm_bindgen] |
| 132 |
pub fn foo(a: JsValue) { |
| 133 |
// ... |
| 134 |
} |
| 135 |
``` |
| 136 |
|
| 137 |
Note that the `&` is missing in front of the `JsValue` we had before, and in |
| 138 |
Rust parlance this means it's taking ownership of the JS value. The exported ES |
| 139 |
module interface is the same as before, but the ownership mechanics are slightly |
| 140 |
different. Let's see the generated JS's slab in action: |
| 141 |
|
| 142 |
```js |
| 143 |
import * as wasm from './foo_bg'; // imports from wasm file |
| 144 |
|
| 145 |
const heap = new Array(32); |
| 146 |
heap.push(undefined, null, true, false); |
| 147 |
let heap_next = 36; |
| 148 |
|
| 149 |
function addHeapObject(obj) { |
| 150 |
if (heap_next === heap.length) |
| 151 |
heap.push(heap.length + 1); |
| 152 |
const idx = heap_next; |
| 153 |
heap_next = heap[idx]; |
| 154 |
heap[idx] = obj; |
| 155 |
return idx; |
| 156 |
} |
| 157 |
|
| 158 |
export function foo(arg0) { |
| 159 |
const idx0 = addHeapObject(arg0); |
| 160 |
wasm.foo(idx0); |
| 161 |
} |
| 162 |
|
| 163 |
export function __wbindgen_object_drop_ref(idx) { |
| 164 |
heap[idx ] = heap_next; |
| 165 |
heap_next = idx; |
| 166 |
} |
| 167 |
``` |
| 168 |
|
| 169 |
Unlike before we're now calling `addHeapObject` on the argument to `foo` rather |
| 170 |
than `addBorrowedObject`. This function will use `heap` and `heap_next` as a |
| 171 |
slab allocator to acquire a slot to store the object, placing a structure there |
| 172 |
once it's found. Note that this is going on the right-half of the array, unlike |
| 173 |
the stack which resides on the left half. This discipline mirrors the stack/heap |
| 174 |
in normal programs, roughly. |
| 175 |
|
| 176 |
Another curious aspect of this generated module is the |
| 177 |
`__wbindgen_object_drop_ref` function. This is one that's actually imported to |
| 178 |
wasm rather than used in this module! This function is used to signal the end of |
| 179 |
the lifetime of a `JsValue` in Rust, or in other words when it goes out of |
| 180 |
scope. Otherwise though this function is largely just a general "slab free" |
| 181 |
implementation. |
| 182 |
|
| 183 |
And finally, let's take a look at the Rust generated again too: |
| 184 |
|
| 185 |
```rust |
| 186 |
// what the user wrote |
| 187 |
pub fn foo(a: JsValue) { |
| 188 |
// ... |
| 189 |
} |
| 190 |
|
| 191 |
#[export_name = "foo"] |
| 192 |
pub extern "C" fn __wasm_bindgen_generated_foo(arg0: u32) { |
| 193 |
let arg0 = unsafe { |
| 194 |
JsValue::__from_idx(arg0) |
| 195 |
}; |
| 196 |
foo(arg0); |
| 197 |
} |
| 198 |
``` |
| 199 |
|
| 200 |
Ah that looks much more familiar! Not much interesting is happening here, so |
| 201 |
let's move on to... |
| 202 |
|
| 203 |
### Anatomy of `JsValue` |
| 204 |
|
| 205 |
Currently the `JsValue` struct is actually quite simple in Rust, it's: |
| 206 |
|
| 207 |
```rust |
| 208 |
pub struct JsValue { |
| 209 |
idx: u32, |
| 210 |
} |
| 211 |
|
| 212 |
// "private" constructors |
| 213 |
|
| 214 |
impl Drop for JsValue { |
| 215 |
fn drop(&mut self) { |
| 216 |
unsafe { |
| 217 |
__wbindgen_object_drop_ref(self.idx); |
| 218 |
} |
| 219 |
} |
| 220 |
} |
| 221 |
``` |
| 222 |
|
| 223 |
Or in other words it's a newtype wrapper around a `u32`, the index that we're |
| 224 |
passed from wasm. The destructor here is where the `__wbindgen_object_drop_ref` |
| 225 |
function is called to relinquish our reference count of the JS object, freeing |
| 226 |
up our slot in the `slab` that we saw above. |
| 227 |
|
| 228 |
If you'll recall as well, when we took `&JsValue` above we generated a wrapper |
| 229 |
of `ManuallyDrop` around the local binding, and that's because we wanted to |
| 230 |
avoid invoking this destructor when the object comes from the stack. |
| 231 |
|
| 232 |
### Working with `heap` in reality |
| 233 |
|
| 234 |
The above explanations are pretty close to what happens today, but in reality |
| 235 |
there's a few differences especially around handling constant values like |
| 236 |
`undefined`, `null`, etc. Be sure to check out the actual generated JS and the |
| 237 |
generation code for the full details! |