Debian Codegraph @

docs/automatic_dispatch.md

raw
1 # Automatic dispatch
2
3 The primary function of [`autoray`](autoray) is to enable writing high level
4 array / tensor code that is agnostic to the backend arrays being supplied.
5 It does this via ***'automatic dispatch'***, which has a few notable
6 differences to other approaches:
7
8 * It is automatic - generally neither you or the backend array library needs
9 to implement any dispatch logic, instead [`autoray`](autoray) finds, if
10 neccesary 'translates', and then caches the relevant functions when they are
11 first called.
12
13 * It is specialized for array functions and treats [`numpy`](numpy) as the
14 reference interface for call signatures of 'equivalent' functions, although
15 it doesn't rely or numpy or require it to be installed.
16
17 * Despite this, there is no fixed API as such - if a backend can be
18 inferred, and the relevant function imported, a [`do`](autoray.do) call is
19 valid.
20
21
22 ## Basics
23
24 The main function of [`autoray`](autoray) is [`do`](autoray.do), which takes a
25 function name followed by `*args` and `**kwargs`, and automatically looks up
26 (and caches) the correct backend function. There are four main ways that the
27 backend is inferred:
28
29 ***1. Automatic backend:***
30
31 ```python
32 do('sqrt', x)
33 ```
34
35 Here the backend is inferred from ``x``. By default dispatch happens on the
36 first argument, but various functions (such as ``'stack'`` and ``'einsum'``)
37 know to dispatch on other arguments.
38
39 ***2. Backend 'like' another array:***
40
41 ```python
42 do('random.normal', size=(2, 3, 4), like=x)
43 ```
44
45 Here the backend is inferred from another array and can thus be implicitly
46 propagated, even when functions take no array arguments. Some creation routines
47 such as ``"eye"`` and ``"zeros"`` will also set the default ``dtype`` and / or
48 device to match ``like`` in this case.
49
50 ***3. Explicit backend:***
51
52 ```python
53 do('einsum', eq, x, y, like='customlib')
54 ```
55
56 Here one simply supplies the desired function backend explicitly.
57
58 ***4. Context manager***
59
60 ```python
61 with backend_like('autoray.lazy'):
62 xy = do('tensordot', x, y, 1)
63 z = do('trace', xy)
64 ```
65
66 Here you set a default backend for a whole block of code. This default
67 overrides method 1. above but 2. and 3. still take precedence. The argument to
68 [`backend_like`](autoray.backend_like) can be a backend string or an example
69 array.
70
71
72 ## Namespace API
73
74 An alternative way to call functions is to get a namespace object using
75 [`get_namespace`](autoray.get_namespace). There are two main ways to use this.
76
77 **1. Late dispatch -- alternative syntax to `do`:**
78
79 Here dispatch still occurs according to above 4 patterns, but with a syntax
80 similar to a standard array library.
81
82 ```python
83 xp = ar.get_namespace()
84 x = xp.ones((3, 4), like="torch")
85 xp.sqrt(x)
86 ```
87
88 ````{hint}
89 `autoray` also provides this 'default namespace' as an importable object like
90 so:
91 ```python
92 from autoray import numpy as np
93
94 np.fn_name(*args, like=like, **kwargs)
95 ```
96 ````
97
98 **2. Early dispatch -- specialized to a particular `backend` (and optionally
99 `dtype` and `device`)**
100
101 Here [`get_namespace`](autoray.get_namespace) dispatches on a `like` argument:
102
103 ```python
104 # if `like` is an array, dtype and device will be picked up from it
105 xp = ar.get_namespace(like="torch", device="cuda", dtype="float32")
106 x = xp.ones((3, 4))
107 xp.sqrt(x)
108 ```
109
110 Function calls then lookup and cache the specific backend function, removing
111 any dispatch overhead. This is a concise and fast way to use `autoray` within a
112 function where you are happy to lock in the backend. It is also similar how the
113 [*python array api*](https://data-apis.org/array-api) works.
114
115
116 ### Manual dispatch functions
117
118 You can manually break the process into two steps with the following functions:
119
120 * [`autoray.infer_backend`](autoray.infer_backend) - return the backend name
121 for a single array.
122 * [`autoray.infer_backend_multi`](autoray.infer_backend_multi) - return the
123 backend name based on multiple arrays.
124 * [`autoray.get_lib_fn`](autoray.get_lib_fn) - return the actual function for a
125 given backend and function name.
126
127 If you know you are going to use a function repeatedly, you can thus avoid the
128 (albeit minor) overhead of dispatching each call separately, for instance:
129
130 ```python
131 def matmul_chain(*arrays):
132 # if the arrays might be a mix of backends, use infer_backend_multi,
133 # but here we just dispatch on the first array
134 backend = infer_backend(arrays[0])
135 fn = get_lib_fn(backend, 'matmul')
136 return functools.reduce(fn, arrays)
137 ```
138
139 ### Other special functions
140
141 There are a few high level functions that might be preferred to attribute
142 access, for reasons of consitency:
143
144 * [`autoray.shape`](autoray.shape) - return the shape of an array. In most
145 cases `x.shape` is fine, but this ensures the output is `tuple[int]`
146 and also works for builtins without calling `numpy`.
147 * [`autoray.ndim`](autoray.ndim) - return the number of dimensions of an array.
148 * [`autoray.size`](autoray.size) - return the total number of elements in an
149 array
150 * [`autoray.dag`](autoray.dag) - return the adjoint of an array, i.e. the
151 transpose with complex conjugation.
152
153 Functions for dealing with dtypes:
154
155 * [`autoray.get_dtype_name`](autoray.get_dtype_name) - return the name of the
156 dtype of an array as a string
157 * [`autoray.to_backend_dtype`](autoray.to_backend_dtype) - turn a string
158 specified dtype into the equivalent dtype for a given backend
159 * [`autoray.astype`](autoray.astype) - cast an array to a given dtype,
160 specified as a string.
161
162 And for converting any array to a numpy array:
163
164 * [`autoray.to_numpy`](autoray.to_numpy)
165
166 ```{hint}
167 All of these can be called via [`do`](autoray.do) as well, e.g.
168 `do('shape', x)`.
169 ```
170
171
172 ## Backends
173
174 In [`autoray`](autoray) a backend internally is simply specified by a string.
175 By default, the `backend` of an array is name of the library that the class is
176 defined in, and the relevant functions are assumed to be in the namespace of
177 `backend`. If that is the case (e.g. `cupy`), then that library is already
178 compatible with `autoray`. Note all backend lookups are cached on
179 `obj.__class__` for speed.
180
181 `autoray` also handles common cases where the functions are in a different
182 library or sub-module (such as `jax -> jax.numpy`). This requires a simple
183 mapping to be specified, which `autoray` does for various libraries.
184
185 You can explicitly register a backend name (and thus default location) for a
186 specific class with the function
187 [`register_backend`](autoray.register_backend):
188
189 ```python
190 register_backend(mylib.myobjs.MyClass, 'mylib.myfuncs')
191 ```
192 Now when `autoray` encounters an instance of `MyClass` it will look for
193 functions in `mylib.myfuncs` instead of `mylib`. You could also use an
194 arbitrary name for the backend, and then alias it to the correct location
195 separately.
196
197
198 ````{note}
199 `autoray` is aware of the `scipy` namespace and relevant submodules for
200 `numpy`, `cupy`, `jax`, for example:
201
202 ```python
203 do('scipy.linalg.exp', x)
204 ```
205 ````
206
207 ## Functions
208
209 Once a `backend` is inferred and the location of the relevant functions is
210 known, `autoray` tries to import and cache the relevant function from that
211 namespace. Many libraries (e.g. `cupy`, `dask`, `jax`, `autograd`, `sparse`,
212 ...) actively mirror the `numpy` API, so there is little else to be done.
213
214 Some other libraries (e.g. `tensorflow`, `pytorch`, ...) diverge from the
215 `numpy` API more, and yet have largely equivalent functions, simply defined in
216 slight different places with different names and / or signatures. `autoray`
217 has a simple translation mechanism for:
218
219 * when functions are in a different module (e.g.
220 `'trace' -> tensorflow.linalg.trace`)
221 * when functions have a different name (e.g. `'sum' -> tensorflow.reduce_sum`)
222 * when functions have a different signature (e.g.
223 `tensordot(a, b, axes) -> torch.tensordot(a, b, dims)`)
224
225 If you want to directly provide a missing or *alternative* implementation of
226 some function for a particular backend you can swap one in with
227 [`register_function`](autoray.register_function):
228
229 ```python
230 def my_custom_torch_svd(x):
231 import torch
232
233 print('Hello SVD!')
234 u, s, v = torch.svd(x)
235
236 return u, s, v.T
237
238 ar.register_function('torch', 'linalg.svd', my_custom_torch_svd)
239
240 x = ar.do('random.uniform', size=(3, 4), like='torch')
241
242 ar.do('linalg.svd', x)
243 # Hello SVD!
244 # (tensor([[-0.5832, 0.6188, -0.5262],
245 # [-0.5787, -0.7711, -0.2655],
246 # [-0.5701, 0.1497, 0.8078]]),
247 # tensor([2.0336, 0.8518, 0.4572]),
248 # tensor([[-0.4568, -0.3166, -0.6835, -0.4732],
249 # [-0.5477, 0.2825, -0.2756, 0.7377],
250 # [ 0.2468, -0.8423, -0.0993, 0.4687]]))
251 ```
252
253 If you want to make use of the existing function you can supply ``wrap=True``
254 in which case the custom function supplied should act like a decorator:
255
256 ```python
257 def my_custom_sum_wrapper(old_fn):
258
259 def new_fn(*args, **kwargs):
260 print('Hello sum!')
261 return old_fn(*args **kwargs)
262
263 return new_fn
264
265 ar.register_function('torch', 'sum', my_custom_sum_wrapper, wrap=True)
266
267 ar.do('sum', x)
268 # Hello sum!
269 # tensor(5.4099)
270 ```
271
272 Though be careful, if you call [`register_function`](autoray.register_function)
273 again it will now wrap the
274 *new* function! Note you can combine
275 [`register_backend`](autoray.register_backend) and
276 [`register_function`](autoray.register_function) to dynamically define array
277 types and functions from anywhere. See also
278 [`register_dispatch`](autoray.register_dispatch) for
279 controlling which arguments are used to infer the backend for any function.
280
281
282 ### Composing new functions
283
284 Sometimes you want to define a function that is composed of many array
285 functions, but you want to dispatch at the level of the whole block, not each
286 individual call, or indeed use a completely different implementation. For
287 instance, you might want to use a [`numba`](https://numba.pydata.org/) or
288 [`pythran`](https://pythran.readthedocs.io/en/latest/) compiled version for
289 `numpy`.
290
291 The [`autoray.compose`](autoray.compose) function allows you to do this. You
292 decorate a function, that forms the default implementation, then you can
293 register alternative implementations for specific backends. For instance:
294
295 ```python
296 from autoray import compose
297 from numba import njit
298
299 @compose
300 def my_func(x):
301 # get how many elements are needed to sum to 20
302 return ar.do('sum', ar.do('cumsum', x, 0) < 20)
303
304 # register a numba implementation
305 @my_func.register('numpy')
306 @njit
307 def my_func_numba(x):
308 s = 0.0
309 i = 0
310 while s < 20:
311 s += x[i]
312 i += 1
313 return i - 1
314
315 # any calls like this now dispatch to my_func_numba
316 do('my_func', x_numpy)
317 ```
318
319
320 ### Deviations from `numpy`
321
322 As stated above, `autoray` does not have an explicit API, but where there exist
323 equivalent functions, `autoray` uses the call signature of `numpy` as a
324 reference. The following are deviations from this:
325
326 * `do('linalg.svd', x)` - `autoray` defaults to `full_matrices=False`, since
327 this is generally always desired, and many libraries do not even support
328 `full_matrices=True`.
329
330
331 -------------------------------------------------------------------------------
332
333 ## Comparison to alternatives
334
335 * The [Python Array API](https://data-apis.org/array-api)
336 (`x.__array_namespace__()`) is a proposed standard for array libraries to
337 implement that many are converging to. It hopefully will be the de-facto
338 approach for writing backend array agnostic code, and
339 [autoray.get_namespace](autoray.get_namespace) provides a similar mechanism
340 with this in mind. The current benefit of `autoray` is that it does not
341 require the backend library to explicitly support it, and it can dynamically
342 compose and replace functions etc.
343
344 * The ``__array_function__`` protocol has been
345 [suggested](https://www.numpy.org/neps/nep-0018-array-function-protocol.html)
346 and now implemented in ``numpy``. This is more orientated around array
347 classes that explicitly subclass `numpy.ndarray`.
348
349 * The [uarray](https://github.com/Quansight-Labs/uarray) project appears to
350 have similar goals but is still being developed.
351
352 * [`functools.singledispatch`](https://docs.python.org/3/library/functools.html#functools.singledispatch)
353 is a general *single* dispatch mechanism, but it is slower and requires the
354 user to explicitly register each function they want to dispatch on.
355
356 * [`plum`](https://github.com/beartype/plum) is a general *multiple* dispatch
357 mechanism, but again it would require registering every function for every
358 backend explicitly.