Debian Codegraph @

docs/compilation.ipynb

raw
1 {
2 "cells": [
3 {
4 "cell_type": "markdown",
5 "metadata": {},
6 "source": [
7 "# Compilation\n",
8 "\n",
9 "Various libraries provide tools for tracing numeric functions and turning the resulting computation into a more efficient, compiled function. Notably:\n",
10 "\n",
11 "* [``jax.jit``](https://github.com/google/jax)\n",
12 "* [``tensorflow.function``](https://www.tensorflow.org/api_docs/python/tf/function)\n",
13 "* [``torch.jit.trace``](https://pytorch.org/docs/stable/jit.html)\n",
14 "\n",
15 " ``autoray`` is obviously very well suited to these since it just dispatches functions to whichever library is doing the tracing - functions written using autoray should be immediately compatible with all of them.\n",
16 "\n",
17 "**The `autojit` wrapper**\n",
18 "\n",
19 "Moreover, ``autoray`` also provides a *unified interface* for compiling functions so that the compilation backend can be easily switched or automatically identified:\n",
20 "\n",
21 "```python\n",
22 "from autoray import autojit\n",
23 "\n",
24 "mgs = autojit(modified_gram_schmidt)\n",
25 "```\n",
26 "\n",
27 "Currently ``autojit`` supports functions with the signature ``fn(*args, **kwargs) -> array`` where both ``args`` and ``kwargs`` can be any nested combination of ``tuple``, ``list`` and ``dict`` objects containings arrays.\n",
28 "We can compare different compiled versions of this simply by changing the ``backend`` option:\n",
29 "\n",
30 "```python\n",
31 "x = do(\"random.normal\", size=(50, 50), like='numpy')\n",
32 "\n",
33 "# first the uncompiled version\n",
34 "%%timeit\n",
35 "modified_gram_schmidt(x)\n",
36 "# 23.5 ms ± 241 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n",
37 "\n",
38 "# 'python' mode unravels computation into source then uses compile+exec\n",
39 "%%timeit\n",
40 "mgs(x) # backend='python'\n",
41 "# 17.8 ms ± 191 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n",
42 "\n",
43 "%%timeit\n",
44 "mgs(x, backend='torch')\n",
45 "# 11.9 ms ± 80.5 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)\n",
46 "\n",
47 "%%timeit\n",
48 "mgs(x, backend='tensorflow')\n",
49 "# 1.87 ms ± 441 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)\n",
50 "\n",
51 "# need to config jax to run on same footing\n",
52 "from jax.config import config\n",
53 "config.update(\"jax_enable_x64\", True)\n",
54 "config.update('jax_platform_name', 'cpu')\n",
55 "\n",
56 "%%timeit\n",
57 "mgs(x, backend='jax')\n",
58 "# 226 µs ± 14.8 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)\n",
59 "\n",
60 "%%timeit\n",
61 "do('linalg.qr', x, like='numpy')[0] # appriximately the 'C' version\n",
62 "# 156 µs ± 32.1 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n",
63 "```\n",
64 "\n",
65 "Here you see *(with this very for-loop heavy function)*, that there are significant gains to be made for all the compilations options. Whilst ``jax`` for example achieves fantastic performance, it should be noted the compilation step takes a lot of time and scales badly (super-linearly) with the number of computational nodes."
66 ]
67 }
68 ],
69 "metadata": {
70 "kernelspec": {
71 "display_name": "numpy",
72 "language": "python",
73 "name": "python3"
74 },
75 "language_info": {
76 "codemirror_mode": {
77 "name": "ipython",
78 "version": 3
79 },
80 "file_extension": ".py",
81 "mimetype": "text/x-python",
82 "name": "python",
83 "nbconvert_exporter": "python",
84 "pygments_lexer": "ipython3",
85 "version": "3.10.9"
86 },
87 "orig_nbformat": 4
88 },
89 "nbformat": 4,
90 "nbformat_minor": 2
91 }