| 1 |
# Automatic dispatch |
| 2 |
|
| 3 |
The primary function of [`autoray`](autoray) is to enable writing high level |
| 4 |
array / tensor code that is agnostic to the backend arrays being supplied. |
| 5 |
It does this via ***'automatic dispatch'***, which has a few notable |
| 6 |
differences to other approaches: |
| 7 |
|
| 8 |
* It is automatic - generally neither you or the backend array library needs |
| 9 |
to implement any dispatch logic, instead [`autoray`](autoray) finds, if |
| 10 |
neccesary 'translates', and then caches the relevant functions when they are |
| 11 |
first called. |
| 12 |
|
| 13 |
* It is specialized for array functions and treats [`numpy`](numpy) as the |
| 14 |
reference interface for call signatures of 'equivalent' functions, although |
| 15 |
it doesn't rely or numpy or require it to be installed. |
| 16 |
|
| 17 |
* Despite this, there is no fixed API as such - if a backend can be |
| 18 |
inferred, and the relevant function imported, a [`do`](autoray.do) call is |
| 19 |
valid. |
| 20 |
|
| 21 |
|
| 22 |
## Basics |
| 23 |
|
| 24 |
The main function of [`autoray`](autoray) is [`do`](autoray.do), which takes a |
| 25 |
function name followed by `*args` and `**kwargs`, and automatically looks up |
| 26 |
(and caches) the correct backend function. There are four main ways that the |
| 27 |
backend is inferred: |
| 28 |
|
| 29 |
***1. Automatic backend:*** |
| 30 |
|
| 31 |
```python |
| 32 |
do('sqrt', x) |
| 33 |
``` |
| 34 |
|
| 35 |
Here the backend is inferred from ``x``. By default dispatch happens on the |
| 36 |
first argument, but various functions (such as ``'stack'`` and ``'einsum'``) |
| 37 |
know to dispatch on other arguments. |
| 38 |
|
| 39 |
***2. Backend 'like' another array:*** |
| 40 |
|
| 41 |
```python |
| 42 |
do('random.normal', size=(2, 3, 4), like=x) |
| 43 |
``` |
| 44 |
|
| 45 |
Here the backend is inferred from another array and can thus be implicitly |
| 46 |
propagated, even when functions take no array arguments. Some creation routines |
| 47 |
such as ``"eye"`` and ``"zeros"`` will also set the default ``dtype`` and / or |
| 48 |
device to match ``like`` in this case. |
| 49 |
|
| 50 |
***3. Explicit backend:*** |
| 51 |
|
| 52 |
```python |
| 53 |
do('einsum', eq, x, y, like='customlib') |
| 54 |
``` |
| 55 |
|
| 56 |
Here one simply supplies the desired function backend explicitly. |
| 57 |
|
| 58 |
***4. Context manager*** |
| 59 |
|
| 60 |
```python |
| 61 |
with backend_like('autoray.lazy'): |
| 62 |
xy = do('tensordot', x, y, 1) |
| 63 |
z = do('trace', xy) |
| 64 |
``` |
| 65 |
|
| 66 |
Here you set a default backend for a whole block of code. This default |
| 67 |
overrides method 1. above but 2. and 3. still take precedence. The argument to |
| 68 |
[`backend_like`](autoray.backend_like) can be a backend string or an example |
| 69 |
array. |
| 70 |
|
| 71 |
|
| 72 |
## Namespace API |
| 73 |
|
| 74 |
An alternative way to call functions is to get a namespace object using |
| 75 |
[`get_namespace`](autoray.get_namespace). There are two main ways to use this. |
| 76 |
|
| 77 |
**1. Late dispatch -- alternative syntax to `do`:** |
| 78 |
|
| 79 |
Here dispatch still occurs according to above 4 patterns, but with a syntax |
| 80 |
similar to a standard array library. |
| 81 |
|
| 82 |
```python |
| 83 |
xp = ar.get_namespace() |
| 84 |
x = xp.ones((3, 4), like="torch") |
| 85 |
xp.sqrt(x) |
| 86 |
``` |
| 87 |
|
| 88 |
````{hint} |
| 89 |
`autoray` also provides this 'default namespace' as an importable object like |
| 90 |
so: |
| 91 |
```python |
| 92 |
from autoray import numpy as np |
| 93 |
|
| 94 |
np.fn_name(*args, like=like, **kwargs) |
| 95 |
``` |
| 96 |
```` |
| 97 |
|
| 98 |
**2. Early dispatch -- specialized to a particular `backend` (and optionally |
| 99 |
`dtype` and `device`)** |
| 100 |
|
| 101 |
Here [`get_namespace`](autoray.get_namespace) dispatches on a `like` argument: |
| 102 |
|
| 103 |
```python |
| 104 |
# if `like` is an array, dtype and device will be picked up from it |
| 105 |
xp = ar.get_namespace(like="torch", device="cuda", dtype="float32") |
| 106 |
x = xp.ones((3, 4)) |
| 107 |
xp.sqrt(x) |
| 108 |
``` |
| 109 |
|
| 110 |
Function calls then lookup and cache the specific backend function, removing |
| 111 |
any dispatch overhead. This is a concise and fast way to use `autoray` within a |
| 112 |
function where you are happy to lock in the backend. It is also similar how the |
| 113 |
[*python array api*](https://data-apis.org/array-api) works. |
| 114 |
|
| 115 |
|
| 116 |
### Manual dispatch functions |
| 117 |
|
| 118 |
You can manually break the process into two steps with the following functions: |
| 119 |
|
| 120 |
* [`autoray.infer_backend`](autoray.infer_backend) - return the backend name |
| 121 |
for a single array. |
| 122 |
* [`autoray.infer_backend_multi`](autoray.infer_backend_multi) - return the |
| 123 |
backend name based on multiple arrays. |
| 124 |
* [`autoray.get_lib_fn`](autoray.get_lib_fn) - return the actual function for a |
| 125 |
given backend and function name. |
| 126 |
|
| 127 |
If you know you are going to use a function repeatedly, you can thus avoid the |
| 128 |
(albeit minor) overhead of dispatching each call separately, for instance: |
| 129 |
|
| 130 |
```python |
| 131 |
def matmul_chain(*arrays): |
| 132 |
# if the arrays might be a mix of backends, use infer_backend_multi, |
| 133 |
# but here we just dispatch on the first array |
| 134 |
backend = infer_backend(arrays[0]) |
| 135 |
fn = get_lib_fn(backend, 'matmul') |
| 136 |
return functools.reduce(fn, arrays) |
| 137 |
``` |
| 138 |
|
| 139 |
### Other special functions |
| 140 |
|
| 141 |
There are a few high level functions that might be preferred to attribute |
| 142 |
access, for reasons of consitency: |
| 143 |
|
| 144 |
* [`autoray.shape`](autoray.shape) - return the shape of an array. In most |
| 145 |
cases `x.shape` is fine, but this ensures the output is `tuple[int]` |
| 146 |
and also works for builtins without calling `numpy`. |
| 147 |
* [`autoray.ndim`](autoray.ndim) - return the number of dimensions of an array. |
| 148 |
* [`autoray.size`](autoray.size) - return the total number of elements in an |
| 149 |
array |
| 150 |
* [`autoray.dag`](autoray.dag) - return the adjoint of an array, i.e. the |
| 151 |
transpose with complex conjugation. |
| 152 |
|
| 153 |
Functions for dealing with dtypes: |
| 154 |
|
| 155 |
* [`autoray.get_dtype_name`](autoray.get_dtype_name) - return the name of the |
| 156 |
dtype of an array as a string |
| 157 |
* [`autoray.to_backend_dtype`](autoray.to_backend_dtype) - turn a string |
| 158 |
specified dtype into the equivalent dtype for a given backend |
| 159 |
* [`autoray.astype`](autoray.astype) - cast an array to a given dtype, |
| 160 |
specified as a string. |
| 161 |
|
| 162 |
And for converting any array to a numpy array: |
| 163 |
|
| 164 |
* [`autoray.to_numpy`](autoray.to_numpy) |
| 165 |
|
| 166 |
```{hint} |
| 167 |
All of these can be called via [`do`](autoray.do) as well, e.g. |
| 168 |
`do('shape', x)`. |
| 169 |
``` |
| 170 |
|
| 171 |
|
| 172 |
## Backends |
| 173 |
|
| 174 |
In [`autoray`](autoray) a backend internally is simply specified by a string. |
| 175 |
By default, the `backend` of an array is name of the library that the class is |
| 176 |
defined in, and the relevant functions are assumed to be in the namespace of |
| 177 |
`backend`. If that is the case (e.g. `cupy`), then that library is already |
| 178 |
compatible with `autoray`. Note all backend lookups are cached on |
| 179 |
`obj.__class__` for speed. |
| 180 |
|
| 181 |
`autoray` also handles common cases where the functions are in a different |
| 182 |
library or sub-module (such as `jax -> jax.numpy`). This requires a simple |
| 183 |
mapping to be specified, which `autoray` does for various libraries. |
| 184 |
|
| 185 |
You can explicitly register a backend name (and thus default location) for a |
| 186 |
specific class with the function |
| 187 |
[`register_backend`](autoray.register_backend): |
| 188 |
|
| 189 |
```python |
| 190 |
register_backend(mylib.myobjs.MyClass, 'mylib.myfuncs') |
| 191 |
``` |
| 192 |
Now when `autoray` encounters an instance of `MyClass` it will look for |
| 193 |
functions in `mylib.myfuncs` instead of `mylib`. You could also use an |
| 194 |
arbitrary name for the backend, and then alias it to the correct location |
| 195 |
separately. |
| 196 |
|
| 197 |
|
| 198 |
````{note} |
| 199 |
`autoray` is aware of the `scipy` namespace and relevant submodules for |
| 200 |
`numpy`, `cupy`, `jax`, for example: |
| 201 |
|
| 202 |
```python |
| 203 |
do('scipy.linalg.exp', x) |
| 204 |
``` |
| 205 |
```` |
| 206 |
|
| 207 |
## Functions |
| 208 |
|
| 209 |
Once a `backend` is inferred and the location of the relevant functions is |
| 210 |
known, `autoray` tries to import and cache the relevant function from that |
| 211 |
namespace. Many libraries (e.g. `cupy`, `dask`, `jax`, `autograd`, `sparse`, |
| 212 |
...) actively mirror the `numpy` API, so there is little else to be done. |
| 213 |
|
| 214 |
Some other libraries (e.g. `tensorflow`, `pytorch`, ...) diverge from the |
| 215 |
`numpy` API more, and yet have largely equivalent functions, simply defined in |
| 216 |
slight different places with different names and / or signatures. `autoray` |
| 217 |
has a simple translation mechanism for: |
| 218 |
|
| 219 |
* when functions are in a different module (e.g. |
| 220 |
`'trace' -> tensorflow.linalg.trace`) |
| 221 |
* when functions have a different name (e.g. `'sum' -> tensorflow.reduce_sum`) |
| 222 |
* when functions have a different signature (e.g. |
| 223 |
`tensordot(a, b, axes) -> torch.tensordot(a, b, dims)`) |
| 224 |
|
| 225 |
If you want to directly provide a missing or *alternative* implementation of |
| 226 |
some function for a particular backend you can swap one in with |
| 227 |
[`register_function`](autoray.register_function): |
| 228 |
|
| 229 |
```python |
| 230 |
def my_custom_torch_svd(x): |
| 231 |
import torch |
| 232 |
|
| 233 |
print('Hello SVD!') |
| 234 |
u, s, v = torch.svd(x) |
| 235 |
|
| 236 |
return u, s, v.T |
| 237 |
|
| 238 |
ar.register_function('torch', 'linalg.svd', my_custom_torch_svd) |
| 239 |
|
| 240 |
x = ar.do('random.uniform', size=(3, 4), like='torch') |
| 241 |
|
| 242 |
ar.do('linalg.svd', x) |
| 243 |
# Hello SVD! |
| 244 |
# (tensor([[-0.5832, 0.6188, -0.5262], |
| 245 |
# [-0.5787, -0.7711, -0.2655], |
| 246 |
# [-0.5701, 0.1497, 0.8078]]), |
| 247 |
# tensor([2.0336, 0.8518, 0.4572]), |
| 248 |
# tensor([[-0.4568, -0.3166, -0.6835, -0.4732], |
| 249 |
# [-0.5477, 0.2825, -0.2756, 0.7377], |
| 250 |
# [ 0.2468, -0.8423, -0.0993, 0.4687]])) |
| 251 |
``` |
| 252 |
|
| 253 |
If you want to make use of the existing function you can supply ``wrap=True`` |
| 254 |
in which case the custom function supplied should act like a decorator: |
| 255 |
|
| 256 |
```python |
| 257 |
def my_custom_sum_wrapper(old_fn): |
| 258 |
|
| 259 |
def new_fn(*args, **kwargs): |
| 260 |
print('Hello sum!') |
| 261 |
return old_fn(*args **kwargs) |
| 262 |
|
| 263 |
return new_fn |
| 264 |
|
| 265 |
ar.register_function('torch', 'sum', my_custom_sum_wrapper, wrap=True) |
| 266 |
|
| 267 |
ar.do('sum', x) |
| 268 |
# Hello sum! |
| 269 |
# tensor(5.4099) |
| 270 |
``` |
| 271 |
|
| 272 |
Though be careful, if you call [`register_function`](autoray.register_function) |
| 273 |
again it will now wrap the |
| 274 |
*new* function! Note you can combine |
| 275 |
[`register_backend`](autoray.register_backend) and |
| 276 |
[`register_function`](autoray.register_function) to dynamically define array |
| 277 |
types and functions from anywhere. See also |
| 278 |
[`register_dispatch`](autoray.register_dispatch) for |
| 279 |
controlling which arguments are used to infer the backend for any function. |
| 280 |
|
| 281 |
|
| 282 |
### Composing new functions |
| 283 |
|
| 284 |
Sometimes you want to define a function that is composed of many array |
| 285 |
functions, but you want to dispatch at the level of the whole block, not each |
| 286 |
individual call, or indeed use a completely different implementation. For |
| 287 |
instance, you might want to use a [`numba`](https://numba.pydata.org/) or |
| 288 |
[`pythran`](https://pythran.readthedocs.io/en/latest/) compiled version for |
| 289 |
`numpy`. |
| 290 |
|
| 291 |
The [`autoray.compose`](autoray.compose) function allows you to do this. You |
| 292 |
decorate a function, that forms the default implementation, then you can |
| 293 |
register alternative implementations for specific backends. For instance: |
| 294 |
|
| 295 |
```python |
| 296 |
from autoray import compose |
| 297 |
from numba import njit |
| 298 |
|
| 299 |
@compose |
| 300 |
def my_func(x): |
| 301 |
# get how many elements are needed to sum to 20 |
| 302 |
return ar.do('sum', ar.do('cumsum', x, 0) < 20) |
| 303 |
|
| 304 |
# register a numba implementation |
| 305 |
@my_func.register('numpy') |
| 306 |
@njit |
| 307 |
def my_func_numba(x): |
| 308 |
s = 0.0 |
| 309 |
i = 0 |
| 310 |
while s < 20: |
| 311 |
s += x[i] |
| 312 |
i += 1 |
| 313 |
return i - 1 |
| 314 |
|
| 315 |
# any calls like this now dispatch to my_func_numba |
| 316 |
do('my_func', x_numpy) |
| 317 |
``` |
| 318 |
|
| 319 |
|
| 320 |
### Deviations from `numpy` |
| 321 |
|
| 322 |
As stated above, `autoray` does not have an explicit API, but where there exist |
| 323 |
equivalent functions, `autoray` uses the call signature of `numpy` as a |
| 324 |
reference. The following are deviations from this: |
| 325 |
|
| 326 |
* `do('linalg.svd', x)` - `autoray` defaults to `full_matrices=False`, since |
| 327 |
this is generally always desired, and many libraries do not even support |
| 328 |
`full_matrices=True`. |
| 329 |
|
| 330 |
|
| 331 |
------------------------------------------------------------------------------- |
| 332 |
|
| 333 |
## Comparison to alternatives |
| 334 |
|
| 335 |
* The [Python Array API](https://data-apis.org/array-api) |
| 336 |
(`x.__array_namespace__()`) is a proposed standard for array libraries to |
| 337 |
implement that many are converging to. It hopefully will be the de-facto |
| 338 |
approach for writing backend array agnostic code, and |
| 339 |
[autoray.get_namespace](autoray.get_namespace) provides a similar mechanism |
| 340 |
with this in mind. The current benefit of `autoray` is that it does not |
| 341 |
require the backend library to explicitly support it, and it can dynamically |
| 342 |
compose and replace functions etc. |
| 343 |
|
| 344 |
* The ``__array_function__`` protocol has been |
| 345 |
[suggested](https://www.numpy.org/neps/nep-0018-array-function-protocol.html) |
| 346 |
and now implemented in ``numpy``. This is more orientated around array |
| 347 |
classes that explicitly subclass `numpy.ndarray`. |
| 348 |
|
| 349 |
* The [uarray](https://github.com/Quansight-Labs/uarray) project appears to |
| 350 |
have similar goals but is still being developed. |
| 351 |
|
| 352 |
* [`functools.singledispatch`](https://docs.python.org/3/library/functools.html#functools.singledispatch) |
| 353 |
is a general *single* dispatch mechanism, but it is slower and requires the |
| 354 |
user to explicitly register each function they want to dispatch on. |
| 355 |
|
| 356 |
* [`plum`](https://github.com/beartype/plum) is a general *multiple* dispatch |
| 357 |
mechanism, but again it would require registering every function for every |
| 358 |
backend explicitly. |