| 1 |
Using Compyle |
| 2 |
============== |
| 3 |
|
| 4 |
In this section we provide more details on the compyle package and how it can be |
| 5 |
used. |
| 6 |
|
| 7 |
An overview of functionality |
| 8 |
----------------------------- |
| 9 |
|
| 10 |
The functionality provided falls into two broad categories, |
| 11 |
|
| 12 |
- Common parallel algorithms that will work across backends. This includes, |
| 13 |
elementwise operations, reductions, and prefix-sums/scans. |
| 14 |
|
| 15 |
- Specific support to run code on a particular backend. This is for code that |
| 16 |
will only work on one backend by definition. This is necessary in order to |
| 17 |
best use different hardware and also use differences in the particular |
| 18 |
backend implementations. For example, the notion of local (or shared) memory |
| 19 |
only has meaning on a GPGPU. In this category we provide support to compile |
| 20 |
and execute Cython code, and also create and execute a GPU kernel. |
| 21 |
|
| 22 |
In addition there is common functionality to perform type annotations. At a |
| 23 |
lower level, there are code translators (transpilers) that handle generation |
| 24 |
of Cython and C code from annotated Python code. Technically these transpilers |
| 25 |
can be reused by users to do other things but we only go over the higher level |
| 26 |
tools in this documentation. All the code is fairly extensively tested and |
| 27 |
developed using a test-driven approach. In fact, a good place to see examples |
| 28 |
are the tests. |
| 29 |
|
| 30 |
We now go into the details of each of these so as to provide a high-level |
| 31 |
overview of what can be done with Compyle. |
| 32 |
|
| 33 |
Annotating functions |
| 34 |
--------------------- |
| 35 |
|
| 36 |
The first step in getting started using Compyle is to annotate your functions and |
| 37 |
also declare variables in code. |
| 38 |
|
| 39 |
Annotation is provided by a simple decorator, called ``annotate``. One can |
| 40 |
declare local variables inside these functions using ``declare``. A simple |
| 41 |
example serves to illustrate these:: |
| 42 |
|
| 43 |
|
| 44 |
@annotate(i='int', x='floatp', return_='float') |
| 45 |
def f(i, x): |
| 46 |
return x[i]*2.0 |
| 47 |
|
| 48 |
@annotate(i='int', floatp='x, y', return_='float') |
| 49 |
def g(i, x, y): |
| 50 |
return f(i, x)*y[i] |
| 51 |
|
| 52 |
|
| 53 |
Note that for convenience ``annotate``, accepts types and variable names in |
| 54 |
two different ways, which you can use interchangeably. |
| 55 |
|
| 56 |
1. You can simply use ``var_name=type_str``, or ``var_name=type`` where the |
| 57 |
type is from the ``compyle.types`` module. |
| 58 |
|
| 59 |
2. You can instead use ``type_name='x, y, z'``, which is often very |
| 60 |
convenient. The order of the variables is not important and need not match |
| 61 |
with the order in which they are declared. |
| 62 |
|
| 63 |
You can use ``return_=type``, where ``type`` is an appropriate type or |
| 64 |
standard string representing one of the types. If the return type is not |
| 65 |
specified it assumes a ``void`` return. |
| 66 |
|
| 67 |
|
| 68 |
The definitions of the various standard types is in ``compyle.types.TYPES``. Some |
| 69 |
are listed below: |
| 70 |
|
| 71 |
- ``'float', 'double', 'int', 'long', 'uint', 'ulong'``: etc. are exactly as |
| 72 |
you would expect. |
| 73 |
|
| 74 |
- ``'doublep'`` would refer to a double pointer, i.e. ``double*`` and |
| 75 |
similarly for anything with a ``p`` at the end. |
| 76 |
|
| 77 |
- ``gdoublep`` would be a ``global doublep``, which makes sense with OpenCL |
| 78 |
where you would have ``__global double* xx``. The global address space |
| 79 |
specification is ignored when Cython code is generated, so this is safe to |
| 80 |
use with Cython code too. |
| 81 |
|
| 82 |
- ``ldoublep`` would be equivalent to ``__local double*`` in OpenCL, for local |
| 83 |
memory. Again this address space qualifier is ignored in Cython. |
| 84 |
|
| 85 |
All these types are available in the ``compyle.types`` module namespace also for |
| 86 |
your convenience. The ``int, float, long`` types are accessible as ``int_, |
| 87 |
float_, long_`` so as not to override the default Python types. For example |
| 88 |
the function ``f`` in the above could also have been declared like so:: |
| 89 |
|
| 90 |
from compyle.types import floatp, float_, int_ |
| 91 |
|
| 92 |
@annotate(i=int_, x=floatp, return_=float_) |
| 93 |
def f(i, x): |
| 94 |
return x[i]*2.0 |
| 95 |
|
| 96 |
|
| 97 |
One can also use custom types (albeit with care) by using the |
| 98 |
``compyle.typs.KnownType`` class. This is convenient in other scenarios where you |
| 99 |
could potentially pass instances/structs to a function. We will discuss this |
| 100 |
later but all of the basic types discussed above are all instances of |
| 101 |
``KnownType``. |
| 102 |
|
| 103 |
Compyle actually supports Python3 style annotations but only for the function |
| 104 |
arguments and NOT for the local variables. The only caveat is you must use the |
| 105 |
types in ``compyle.types``, i.e. you must use ``KnownType`` instances as the |
| 106 |
types for things to work. |
| 107 |
|
| 108 |
JIT transpilation |
| 109 |
----------------- |
| 110 |
|
| 111 |
Compyle also support just-in-time transpilation when annotations of a function |
| 112 |
are not provided. These functions are annotated at runtime when the |
| 113 |
call arguments are passed. The generated kernel and annotated functions |
| 114 |
are then cached with the types of the call arguments as key. Thus, |
| 115 |
the function ``f`` defined in the previous section can also be defined |
| 116 |
as follows:: |
| 117 |
|
| 118 |
@annotate |
| 119 |
def f(i, x): |
| 120 |
return x[i]*2.0 |
| 121 |
|
| 122 |
While using in-built functions such as ``sin``, ``cos``, ``abs`` etc. it is |
| 123 |
recommended that you store the value in a variable or appropriate type |
| 124 |
before returning it. If not the return type will default to ``double``. |
| 125 |
For example,:: |
| 126 |
|
| 127 |
@annotate |
| 128 |
def f(i, x): |
| 129 |
return abs(x[i]) |
| 130 |
|
| 131 |
This will set the return type of function ``f`` to the default |
| 132 |
type, ``double`` even when ``x`` is an array of integers. |
| 133 |
To avoid this problem, one could define ``f`` instead as,:: |
| 134 |
|
| 135 |
@annotate |
| 136 |
def f(i, x): |
| 137 |
y = declare('int') |
| 138 |
y = abs(x[i]) |
| 139 |
return y |
| 140 |
|
| 141 |
Currently JIT support is only limited to the common parallel algorithms explained |
| 142 |
in a later section. |
| 143 |
|
| 144 |
Declaring variables |
| 145 |
------------------- |
| 146 |
|
| 147 |
In addition to annotating the function arguments and return types, it is |
| 148 |
important to be able to declare the local variables. We provide a simple |
| 149 |
``declare`` function that lets us do this. One again, a few examples serve to |
| 150 |
illustrate this:: |
| 151 |
|
| 152 |
i = declare('int') |
| 153 |
x = declare('float') |
| 154 |
u, v = declare('double', 2) |
| 155 |
|
| 156 |
Notice the last one where we passed an additional argument of the number of |
| 157 |
types we want. This is really done to keep this functional in pure Python so |
| 158 |
that your code executes on Python also. In Cython these would produce:: |
| 159 |
|
| 160 |
cdef int i |
| 161 |
cdef float x |
| 162 |
cdef double u, v |
| 163 |
|
| 164 |
On OpenCL this would produce the equivalent:: |
| 165 |
|
| 166 |
int i; |
| 167 |
float x; |
| 168 |
double u, v; |
| 169 |
|
| 170 |
Technically one could also write:: |
| 171 |
|
| 172 |
f = declare('float4') |
| 173 |
|
| 174 |
but clearly this would only work on OpenCL, however, you can definitely |
| 175 |
declare other variables too! |
| 176 |
|
| 177 |
Note that in OpenCL/Cython code if you do not declare a variable, it is |
| 178 |
automatically declared as a ``double`` to prevent compilation errors. |
| 179 |
|
| 180 |
We often also require arrays, ``declare`` also supports this, for example |
| 181 |
consider these examples:: |
| 182 |
|
| 183 |
r = declare('matrix(3)') |
| 184 |
a = declare('matrix((2, 2))') |
| 185 |
u, v = declare('matrix(2)', 2) |
| 186 |
|
| 187 |
This reduces to the following on OpenCL:: |
| 188 |
|
| 189 |
double r[3]; |
| 190 |
double a[3][3]; |
| 191 |
double u[2], v[2]; |
| 192 |
|
| 193 |
Note that this will only work with fixed sizes, and not with dynamic sizes. As |
| 194 |
we pointed out earlier, dynamic memory allocation is not allowed. Of course |
| 195 |
you could easily do this with Cython code but the declare syntax does not |
| 196 |
allow this. |
| 197 |
|
| 198 |
If you want non-double matrices, you can simply pass a type as in:: |
| 199 |
|
| 200 |
a = declare('matrix((2, 2), "int")') |
| 201 |
|
| 202 |
Which would result in:: |
| 203 |
|
| 204 |
int a[2][2]; |
| 205 |
|
| 206 |
As you can imagine, just being able to do this opens up quite a few |
| 207 |
possibilities. You could also do things like this:: |
| 208 |
|
| 209 |
xloc = declare('LOCAL_MEM matrix(128)') |
| 210 |
|
| 211 |
which will become in OpenCL:: |
| 212 |
|
| 213 |
LOCAL_MEM double xloc[128]; |
| 214 |
|
| 215 |
The ``LOCAL_MEM`` is a special variable that expands to the appropriate flag |
| 216 |
on OpenCL or CUDA to allow you to write kernels once and have them run on |
| 217 |
either OpenCL or CUDA. These special variables are discussed later below. |
| 218 |
|
| 219 |
Writing the functions |
| 220 |
---------------------- |
| 221 |
|
| 222 |
All of basic Python is supported. As you may have seen in the examples, you |
| 223 |
can write code that uses the following: |
| 224 |
|
| 225 |
- Indexing (only positive indices please). |
| 226 |
- Conditionals with if/elif/else. |
| 227 |
- While loops. |
| 228 |
- For loops with the ``for var in range(...)`` construction. |
| 229 |
- Nested fors. |
| 230 |
- Ternary operators. |
| 231 |
|
| 232 |
This allows us to write most numerical code. Fancy slicing etc. are not |
| 233 |
supported, numpy based slicing and striding are not supported. You are |
| 234 |
supposed to write these out elementwise. The idea is to keep things simple. |
| 235 |
Yes, this may make things verbose but it does keep our life simple and |
| 236 |
eventually yours too. |
| 237 |
|
| 238 |
Do not create any Python data structures in the code unless you do not want to |
| 239 |
run the code on a GPU. No numpy arrays can be created, also avoid calling any |
| 240 |
numpy functions as these will NOT translate to any GPU code. You have to write |
| 241 |
what you need by hand. Having said that, all the basic math functions and |
| 242 |
symbols are automatically available. Essentially all of ``math`` is available. |
| 243 |
All of the ``math.h`` constants are also available for use. |
| 244 |
|
| 245 |
If you declare a global constant it will be automatically defined in the |
| 246 |
generated code. For example:: |
| 247 |
|
| 248 |
MY_CONST = 42 |
| 249 |
|
| 250 |
@annotate(x='double', return_='double') |
| 251 |
def f(x): |
| 252 |
return x + MY_CONST |
| 253 |
|
| 254 |
|
| 255 |
The ``MY_CONST`` will be automatically injected in your generated code. |
| 256 |
|
| 257 |
Now you may wonder about how you can call an external library that is not in |
| 258 |
``math.h``. Lets say you have an external CUDA library, how do you call that? |
| 259 |
We have a simple approach for this which we discuss later. We call this an |
| 260 |
``Extern`` and discuss it later. |
| 261 |
|
| 262 |
|
| 263 |
.. _PyOpenCL: https://documen.tician.de/pyopencl/ |
| 264 |
.. _Cython: http://www.cython.org |
| 265 |
.. _PySPH: http://pysph.readthedocs.io |
| 266 |
|
| 267 |
Common parallel algorithms |
| 268 |
--------------------------- |
| 269 |
|
| 270 |
Compyle provides a few very powerful parallel algorithms. These are all directly |
| 271 |
motivated by Andreas Kloeckner's PyOpenCL_ package. On the GPU they are |
| 272 |
wrappers on top of the functionality provided there. These algorithms make it |
| 273 |
possible to implement scalable algorithms for a variety of common numerical |
| 274 |
problems. In PySPH_ for example all of the GPU based nearest neighbor finding |
| 275 |
algorithms are written with these fundamental primitives and scale very well. |
| 276 |
|
| 277 |
All of the following parallel algorithms allow choice of a suitable backend |
| 278 |
and take a keyword argument to specify this backend. If no backend is provided |
| 279 |
a default is chosen from the ``compyle.config`` module. You can get the global |
| 280 |
config using:: |
| 281 |
|
| 282 |
from compyle.config import get_config |
| 283 |
|
| 284 |
cfg = get_config() |
| 285 |
cfg.use_openmp = True |
| 286 |
cfg.use_opencl = True |
| 287 |
|
| 288 |
etc. The following are the parallel algorithms available from the |
| 289 |
``compyle.parallel`` module. |
| 290 |
|
| 291 |
``Elementwise`` |
| 292 |
~~~~~~~~~~~~~~~ |
| 293 |
|
| 294 |
This is also available as a decorator ``elementwise``. One can pass it an |
| 295 |
annotated function and an optional backend. The elementwise processes every |
| 296 |
element in the second argument to the function. The elementwise basically |
| 297 |
passes the function an index of the element it is processing and parallelizes |
| 298 |
the calls to this automatically. If you are familiar with writing GPU kernels, |
| 299 |
this is the same thing except the index is passed along to you. |
| 300 |
|
| 301 |
Here is a very simple example that shows how this works for a case where we |
| 302 |
compute ``y = a*sin(x) + b`` where ``y, a, x, b`` are all numpy arrays but let |
| 303 |
us say we want to do this in parallel:: |
| 304 |
|
| 305 |
import numpy as np |
| 306 |
from compyle.api import annotate, Elementwise, get_config |
| 307 |
|
| 308 |
@annotate(i='int', doublep='x, y, a, b') |
| 309 |
def axpb(i, x, y, a, b): |
| 310 |
y[i] = a[i]*sin(x[i]) + b[i] |
| 311 |
|
| 312 |
# Setup the input data |
| 313 |
n = 1000000 |
| 314 |
x = np.linspace(0, 1, n) |
| 315 |
y = np.zeros_like(x) |
| 316 |
a = np.random.random(n) |
| 317 |
b = np.random.random(n) |
| 318 |
|
| 319 |
# Use OpenMP |
| 320 |
get_config().use_openmp = True |
| 321 |
|
| 322 |
# Now run this in parallel with Cython. |
| 323 |
backend = 'cython' |
| 324 |
e = Elementwise(axpb, backend=backend) |
| 325 |
e(x, y, a, b) |
| 326 |
|
| 327 |
This will call the ``axpb`` function in parallel and if your problem is large |
| 328 |
enough will effectively scale on all your cores. Its as simple as that. |
| 329 |
|
| 330 |
Now let us say we want to run this with OpenCL. The only issue with OpenCL is |
| 331 |
that the data needs to be sent to the GPU. This is transparently handled by a |
| 332 |
simple ``Array`` wrapper that handles this for us automatically. Here is a |
| 333 |
simple example building on the above:: |
| 334 |
|
| 335 |
from compyle.api import wrap |
| 336 |
|
| 337 |
backend = 'opencl' |
| 338 |
x, y, a, b = wrap(x, y, a, b, backend=backend) |
| 339 |
|
| 340 |
What this does is to wrap each of the arrays and also sends the data to the |
| 341 |
device. ``x`` is now an instance of ``compyle.array.Array``, this simple |
| 342 |
class has two attributes, ``data`` and ``dev``. The first is the original data |
| 343 |
and the second is a suitable device array from PyOpenCL/PyCUDA depending on |
| 344 |
the backend. To get data from the device to the host you can call ``x.pull()`` |
| 345 |
to push data to the device you can call ``x.push()``. |
| 346 |
|
| 347 |
Now that we have this wrapped we can simply do:: |
| 348 |
|
| 349 |
e = Elementwise(axpb, backend=backend) |
| 350 |
e(x, y, a, b) |
| 351 |
|
| 352 |
We do not need to change any of our other code. As you can see this is very convenient. |
| 353 |
|
| 354 |
Here is all the code put together:: |
| 355 |
|
| 356 |
import numpy as np |
| 357 |
from compyle.api import annotate, Elementwise, get_config, wrap |
| 358 |
|
| 359 |
@annotate(i='int', doublep='x, y, a, b') |
| 360 |
def axpb(i, x, y, a, b): |
| 361 |
y[i] = a[i]*sin(x[i]) + b[i] |
| 362 |
|
| 363 |
# Setup the input data |
| 364 |
n = 1000000 |
| 365 |
x = np.linspace(0, 1, n) |
| 366 |
y = np.zeros_like(x) |
| 367 |
a = np.random.random(n) |
| 368 |
b = np.random.random(n) |
| 369 |
|
| 370 |
# Turn on OpenMP for Cython. |
| 371 |
get_config().use_openmp = True |
| 372 |
|
| 373 |
for backend in ('cython', 'opencl'): |
| 374 |
xa, ya, aa, ba = wrap(x, y, a, b, backend=backend) |
| 375 |
e = Elementwise(axpb, backend=backend) |
| 376 |
e(xa, ya, aa, ba) |
| 377 |
|
| 378 |
This will run the code on both backends! We use the for loop just to show that |
| 379 |
this will run on all backends! The ``axpb.py`` example shows this for a |
| 380 |
variety of array sizes and plots the performance. |
| 381 |
|
| 382 |
|
| 383 |
``Reduction`` |
| 384 |
~~~~~~~~~~~~~~~ |
| 385 |
|
| 386 |
The ``compyle.parallel`` module also provides a ``Reduction`` class which can be |
| 387 |
used fairly easily. Using it is a bit complex, a good starting point for this |
| 388 |
is the documentation of PyOpenCL_, here |
| 389 |
https://documen.tician.de/pyopencl/algorithm.html#module-pyopencl.reduction |
| 390 |
|
| 391 |
The difference from the PyOpenCL implementation is that the ``map_expr`` is a |
| 392 |
function rather than a string. |
| 393 |
|
| 394 |
We provide a couple of simple examples to illustrate the above. The first |
| 395 |
example is to find the sum of all elements of an array:: |
| 396 |
|
| 397 |
x = np.linspace(0, 1, 1000)/1000 |
| 398 |
x = wrap(x, backend=backend) |
| 399 |
|
| 400 |
r = Reduction('a+b', backend=backend) |
| 401 |
result = r(x) |
| 402 |
|
| 403 |
Here is an example of a function to find the minimum of an array:: |
| 404 |
|
| 405 |
x = np.linspace(0, 1, 1000)/1000 |
| 406 |
x = wrap(x, backend=backend) |
| 407 |
|
| 408 |
r = Reduction('min(a, b)', neutral='INFINITY', backend=backend) |
| 409 |
result = r(x) |
| 410 |
|
| 411 |
Here is a final one with a map expression thrown in:: |
| 412 |
|
| 413 |
from math import cos, sin |
| 414 |
x = np.linspace(0, 1, 1000)/1000 |
| 415 |
y = x.copy() |
| 416 |
x, y = wrap(x, y, backend=backend) |
| 417 |
|
| 418 |
@annotate(i='int', doublep='x, y') |
| 419 |
def map(i=0, x=[0.0], y=[0.0]): |
| 420 |
return cos(x[i])*sin(y[i]) |
| 421 |
|
| 422 |
r = Reduction('a+b', map_func=map, backend=backend) |
| 423 |
result = r(x, y) |
| 424 |
|
| 425 |
As you can see this is faithful to the PyOpenCL implementation with the only |
| 426 |
difference that the ``map_expr`` is actually a nice function. Further, this |
| 427 |
works on all backends, even on Cython. |
| 428 |
|
| 429 |
|
| 430 |
``Scan`` |
| 431 |
~~~~~~~~~~ |
| 432 |
|
| 433 |
Scans are generalizations of prefix sums / cumulative sums and can be used as |
| 434 |
building blocks to construct a number of parallel algorithms. These include |
| 435 |
but not are limited to sorting, polynomial evaluation, and tree operations. |
| 436 |
Blelloch's literature on prefix sums (`Prefix Sums and Their Applications |
| 437 |
<https://www.cs.cmu.edu/~guyb/papers/Ble93.pdf>`_) has many more examples and |
| 438 |
is a recommended read before using scans. The ``compyle.parallel`` module |
| 439 |
provides a ``Scan`` class which can be used to develop and execute such scans. |
| 440 |
The scans can be run on GPUs using the OpenCL or CUDA backend or on CPUs using |
| 441 |
either the OpenCL or Cython backend. |
| 442 |
|
| 443 |
The scan semantics in compyle are similar to those of the GenericScanKernel in |
| 444 |
PyOpenCL |
| 445 |
(https://documen.tician.de/pyopencl/algorithm.html#pyopencl.scan.GenericScanKernel). Similar |
| 446 |
to the case for reduction, the main differences from the PyOpenCL implementation |
| 447 |
are that the expressions (`input_expr`, `segment_expr`, `output_expr`) are all |
| 448 |
functions rather than strings. |
| 449 |
|
| 450 |
The following examples demonstrate how scans can be used in compyle. The first |
| 451 |
example is to find the cumulative sum of all elements of an array:: |
| 452 |
|
| 453 |
ary = np.arange(10000, dtype=np.int32) |
| 454 |
ary = wrap(ary, backend=backend) |
| 455 |
|
| 456 |
@annotate(i='int', ary='intp', return_='int') |
| 457 |
def input_expr(i, ary): |
| 458 |
return ary[i] |
| 459 |
|
| 460 |
@annotate(int='i, item', ary='intp') |
| 461 |
def output_expr(i, item, ary): |
| 462 |
ary[i] = item |
| 463 |
|
| 464 |
scan = Scan(input_expr, output_expr, 'a+b', dtype=np.int32, |
| 465 |
backend=backend) |
| 466 |
scan(ary=ary) |
| 467 |
ary.pull() |
| 468 |
|
| 469 |
# Result = ary.data |
| 470 |
|
| 471 |
Here is a more complex example of a function that finds the unique elements in |
| 472 |
an array:: |
| 473 |
|
| 474 |
ary = np.random.randint(0, 100, 1000, dtype=np.int32) |
| 475 |
unique_ary = np.zeros(len(ary.data), dtype=np.int32) |
| 476 |
unique_ary = wrap(unique_ary, backend=backend) |
| 477 |
unique_count = np.zeros(1, dtype=np.int32) |
| 478 |
unique_count = wrap(unique_count, backend=backend) |
| 479 |
ary = wrap(ary, backend=backend) |
| 480 |
|
| 481 |
@annotate(i='int', ary='intp', return_='int') |
| 482 |
def input_expr(i, ary): |
| 483 |
if i == 0 or ary[i] != ary[i - 1]: |
| 484 |
return 1 |
| 485 |
else: |
| 486 |
return 0 |
| 487 |
|
| 488 |
@annotate(int='i, prev_item, item, N', ary='intp', |
| 489 |
unique='intp', unique_count='intp') |
| 490 |
def output_expr(i, prev_item, item, N, ary, unique, unique_count): |
| 491 |
if item != prev_item: |
| 492 |
unique[item - 1] = ary[i] |
| 493 |
if i == N - 1: |
| 494 |
unique_count[0] = item |
| 495 |
|
| 496 |
scan = Scan(input_expr, output_expr, 'a+b', dtype=np.int32, backend=backend) |
| 497 |
scan(ary=ary, unique=unique_ary, unique_count=unique_count) |
| 498 |
unique_ary.pull() |
| 499 |
unique_count.pull() |
| 500 |
unique_count = unique_count.data[0] |
| 501 |
unique_ary = unique_ary.data[:unique_count] |
| 502 |
|
| 503 |
# Result = unique_ary |
| 504 |
|
| 505 |
The following points highlight some important details and quirks about using |
| 506 |
scans in compyle: |
| 507 |
|
| 508 |
1. The scan call does not return anything. All output must be handled manually. |
| 509 |
Usually this involves writing the results available in ``output_expr`` |
| 510 |
(``prev_item``, ``item`` and ``last_item``) to an array. |
| 511 |
2. ``input_expr`` might be evaluated multiple times. However, it can be assumed |
| 512 |
that ``input_expr`` for an element or index ``i`` is not evaluated again |
| 513 |
after the output expression ``output_expr`` for that element is |
| 514 |
evaluated. Therefore, it is safe to write the output of a scan back to an |
| 515 |
array also used for the input like in the first example. |
| 516 |
3. (For PyOpenCL users) If a segmented scan is used, unlike PyOpenCL where the |
| 517 |
``across_seg_boundary`` is used to handle the segment logic in the scan |
| 518 |
expression, in compyle the logic is handled automatically. More specifically, |
| 519 |
using ``a + b`` as the scan expression in compyle is equivalent to using |
| 520 |
``(across_seg_boundary ? b : a + b)`` in PyOpenCL. |
| 521 |
|
| 522 |
|
| 523 |
Debugging |
| 524 |
---------- |
| 525 |
|
| 526 |
Debugging can be a bit difficult with multiple different architectures and |
| 527 |
backends. One convenience that compyle provides is that the generated sources |
| 528 |
can be inspected. All the parallel algorithms (``Elementwise, Reduction, |
| 529 |
Scan``) provide a ``.source`` or ``.all_source`` attribute that contains the |
| 530 |
source. For example say you have the following:: |
| 531 |
|
| 532 |
e = Elementwise(axpb, backend=backend) |
| 533 |
e(x, y, a, b) |
| 534 |
|
| 535 |
You can examine the source generated for your functions using:: |
| 536 |
|
| 537 |
e.source |
| 538 |
|
| 539 |
This is probably most useful for end users. For those more curious, all of the |
| 540 |
source generated and used for the complete elementwise (or other) parallel |
| 541 |
algorithm can be seen using:: |
| 542 |
|
| 543 |
e.all_source |
| 544 |
|
| 545 |
This code can be rather long and difficult to read so use this only if you |
| 546 |
really need to see the underlying code from PyOpenCL or PyCUDA. On the GPU |
| 547 |
this will often include multiple kernels as well. Note that on CUDA the |
| 548 |
``all_source`` does not show all of the sources as PyCUDA currently does not |
| 549 |
make it easy to inspect the code. |
| 550 |
|
| 551 |
You can define the environment variable ``COMPYLE_DEBUG``, for example on bash |
| 552 |
with ``export COMPYLE_DEBUG=1`` and run your code. This will print out the |
| 553 |
source code generated for every generated function. For CPU execution it will |
| 554 |
also display compiler output if there is no cached extension available. |
| 555 |
Turning on the debug feature can give you a lot of information, but it is very |
| 556 |
useful if you want to get into the gory details and do not want to edit the |
| 557 |
sources just to look at the generated code. |
| 558 |
|
| 559 |
It is important to note that all the Cython code is saved on disk inside the |
| 560 |
``~/.compyle/`` directory in a platform specific directory but no files are |
| 561 |
written by compyle for any OpenCL or CUDA code and in these cases the |
| 562 |
``COMPYLE_DEBUG`` option is very handy. |
| 563 |
|
| 564 |
|
| 565 |
Abstracting out arrays |
| 566 |
----------------------- |
| 567 |
|
| 568 |
As discussed in the section on Elementwise operations, different backends need |
| 569 |
to do different things with arrays. With OpenCL/CUDA one needs to send the |
| 570 |
array to the device. This is transparently managed by the |
| 571 |
``compyle.array.Array`` class. It is easiest to use this transparently with |
| 572 |
the ``wrap`` convenience function as below:: |
| 573 |
|
| 574 |
x = np.linspace(0, 1, 1000)/1000 |
| 575 |
y = x.copy() |
| 576 |
x, y = wrap(x, y, backend=backend) |
| 577 |
|
| 578 |
Thus these, new arrays can be passed to any operation and is handled transparently. |
| 579 |
|
| 580 |
|
| 581 |
Choice of backend and configuration |
| 582 |
------------------------------------ |
| 583 |
|
| 584 |
The ``compyle.config`` module provides a simple ``Configuration`` class that |
| 585 |
is used internally in Compyle to set things like the backend (Cython, |
| 586 |
OpenCL/CUDA), and some common options like profiling, turning on OpenMP, using |
| 587 |
double on the GPU etc. Here is an example of the various options:: |
| 588 |
|
| 589 |
from compyle.config import get_config |
| 590 |
cfg = get_config() |
| 591 |
cfg.use_double |
| 592 |
cfg.profile |
| 593 |
cfg.use_opencl |
| 594 |
cfg.use_openmp |
| 595 |
|
| 596 |
If one wants to temporarily set an option and perform an action, one can do:: |
| 597 |
|
| 598 |
from compyle.config import use_config |
| 599 |
|
| 600 |
with use_config(use_openmp=False): |
| 601 |
... |
| 602 |
|
| 603 |
Here everything within the ``with`` clause will be executed using the |
| 604 |
specified option and once the clause is exited, the previous settings will be |
| 605 |
restored. This can be convenient. |
| 606 |
|
| 607 |
Templates |
| 608 |
---------- |
| 609 |
|
| 610 |
When creating libraries, it is useful to be able to write a function as a |
| 611 |
"template" where the code can be generated depending on various user options. |
| 612 |
Compyle facilitates this by using Mako_ templates. We provide a convenient |
| 613 |
``compyle.template.Template`` class which can be used for this purpose. A |
| 614 |
trivial and contrived example demonstrates its use below. The example sets any |
| 615 |
number of given arrays to a constant value:: |
| 616 |
|
| 617 |
|
| 618 |
from compyle.types import annotate |
| 619 |
from compyle.template import Template |
| 620 |
|
| 621 |
class SetConstant(Template): |
| 622 |
def __init__(self, name, arrays): |
| 623 |
super(SetConstant, self).__init__(name=name) |
| 624 |
self.arrays = arrays |
| 625 |
|
| 626 |
def my_func(self, value): |
| 627 |
'''The contents of this function are directly injected. |
| 628 |
''' |
| 629 |
tmp = sin(value) |
| 630 |
|
| 631 |
def extra_args(self): |
| 632 |
return self.arrays, {'doublep': ','.join(self.arrays)} |
| 633 |
|
| 634 |
@annotate(i='int', value='double') |
| 635 |
def template(self, i, value): |
| 636 |
'''Set the arrays to a constant value.''' |
| 637 |
''' |
| 638 |
${obj.inject(obj.my_func)} |
| 639 |
% for arr in obj.arrays: |
| 640 |
${arr}[i] = tmp |
| 641 |
% endfor |
| 642 |
''' |
| 643 |
|
| 644 |
set_const = SetConstant('set_const', ['x', 'y', 'z']).function |
| 645 |
print(set_const.source) |
| 646 |
|
| 647 |
This will print out this:: |
| 648 |
|
| 649 |
def set_const(i, value, x, y, z): |
| 650 |
"""Set arrays to constant. |
| 651 |
""" |
| 652 |
tmp = sin(value) |
| 653 |
|
| 654 |
x[i] = tmp |
| 655 |
y[i] = tmp |
| 656 |
z[i] = tmp |
| 657 |
|
| 658 |
|
| 659 |
This is obviously a trivial example but the idea is that one can create fairly |
| 660 |
complex templated functions that can be then transpiled and used in different |
| 661 |
cases. The key point here is the ``template`` method which should simply |
| 662 |
create a string which is rendered using Mako_ and then put into a function. |
| 663 |
The ``extra_args`` method allows us to configure the arguments used by the |
| 664 |
function. The mako template can use the name ``obj`` which is ``self``. The |
| 665 |
``obj.inject`` method allows one to literally inject any function into the |
| 666 |
body of the code with a suitable level of indentation. Of course normal mako |
| 667 |
functionality is available to do a variety of things. |
| 668 |
|
| 669 |
|
| 670 |
.. _Mako: https://www.makotemplates.org/ |
| 671 |
|
| 672 |
Low level functionality |
| 673 |
----------------------- |
| 674 |
|
| 675 |
In addition to the above, there are also powerful low-level functionality that |
| 676 |
is provided in ``compyle.low_level``. |
| 677 |
|
| 678 |
|
| 679 |
``Kernel`` |
| 680 |
~~~~~~~~~~~ |
| 681 |
|
| 682 |
The ``Kernel`` class allows one to execute a pure GPU kernel. Unlike the |
| 683 |
Elementwise functionality above, this is specific to OpenCL/CUDA and will not |
| 684 |
execute via Cython. What this class lets one do is write low-level kernels |
| 685 |
which are often required to extract the best performance from your hardware. |
| 686 |
|
| 687 |
Most of the functionality is exactly the same, one declares functions and |
| 688 |
annotates them and then passes a function to the ``Kernel`` which calls this |
| 689 |
just as we would a normal OpenCL kernel for example. The major advantage is |
| 690 |
that all your code is pure Python. Here is a simple example:: |
| 691 |
|
| 692 |
from compyle.api import annotate, wrap, get_config |
| 693 |
from compyle.low_level import Kernel, LID_0, LDIM_0, GID_0 |
| 694 |
import numpy as np |
| 695 |
|
| 696 |
@annotate(x='doublep', y='doublep', double='a,b') |
| 697 |
def axpb(x, y, a, b): |
| 698 |
i = declare('int') |
| 699 |
i = LDIM_0*GID_0 + LID_0 |
| 700 |
y[i] = a*sin(x[i]) + b |
| 701 |
|
| 702 |
x = np.linspace(0, 1, 10000) |
| 703 |
y = np.zeros_like(x) |
| 704 |
a = 2.0 |
| 705 |
b = 3.0 |
| 706 |
|
| 707 |
get_config().use_opencl = True |
| 708 |
x, y = wrap(x, y) |
| 709 |
|
| 710 |
k = Kernel(axpb) |
| 711 |
k(x, y, a, b) |
| 712 |
|
| 713 |
This is the same Elementwise kernel equivalent from the first example at the |
| 714 |
top but written as a raw kernel. Notice that ``i`` is not passed but computed |
| 715 |
using ``LDIM_0, GID_0 and LID_0`` which are automatically made available on |
| 716 |
OpenCL/CUDA. In addition to these the function ``local_barrier`` is also |
| 717 |
available. Internally these are ``#defines`` that are like so on OpenCL:: |
| 718 |
|
| 719 |
#define LID_0 get_local_id(0) |
| 720 |
#define LID_1 get_local_id(1) |
| 721 |
#define LID_2 get_local_id(2) |
| 722 |
|
| 723 |
#define GID_0 get_group_id(0) |
| 724 |
#define GID_1 get_group_id(1) |
| 725 |
#define GID_2 get_group_id(2) |
| 726 |
|
| 727 |
#define LDIM_0 get_local_size(0) |
| 728 |
#define LDIM_1 get_local_size(1) |
| 729 |
#define LDIM_2 get_local_size(2) |
| 730 |
|
| 731 |
#define GDIM_0 get_num_groups(0) |
| 732 |
#define GDIM_1 get_num_groups(1) |
| 733 |
#define GDIM_2 get_num_groups(2) |
| 734 |
|
| 735 |
#define local_barrier() barrier(CLK_LOCAL_MEM_FENCE); |
| 736 |
|
| 737 |
On CUDA, these are mapped to the equivalent :: |
| 738 |
|
| 739 |
#define LID_0 threadIdx.x |
| 740 |
#define LID_1 threadIdx.y |
| 741 |
#define LID_2 threadIdx.z |
| 742 |
|
| 743 |
#define GID_0 blockIdx.x |
| 744 |
#define GID_1 blockIdx.y |
| 745 |
#define GID_2 blockIdx.z |
| 746 |
|
| 747 |
#define LDIM_0 blockDim.x |
| 748 |
#define LDIM_1 blockDim.y |
| 749 |
#define LDIM_2 blockDim.z |
| 750 |
|
| 751 |
#define GDIM_0 gridDim.x |
| 752 |
#define GDIM_1 gridDim.y |
| 753 |
#define GDIM_2 gridDim.z |
| 754 |
|
| 755 |
#define local_barrier() __syncthreads(); |
| 756 |
|
| 757 |
|
| 758 |
In fact these are all provided by the ``_cluda.py`` in PyOpenCL and PyCUDA. |
| 759 |
These allow us to write CUDA/OpenCL agnostic code from Python. |
| 760 |
|
| 761 |
One may also pass local memory to such a kernel, this trivial example |
| 762 |
demonstrates this:: |
| 763 |
|
| 764 |
|
| 765 |
from compyle.api import annotate |
| 766 |
from compyle.low_level import ( |
| 767 |
Kernel, LID_0, LDIM_0, GID_0, LocalMem, local_barrier |
| 768 |
) |
| 769 |
import numpy as np |
| 770 |
|
| 771 |
@annotate(gdoublep='x', ldoublep='xl') |
| 772 |
def f(x, xl): |
| 773 |
i, thread_id = declare('int', 2) |
| 774 |
thread_id = LID_0 |
| 775 |
i = GID_0*LDIM_0 + thread_id |
| 776 |
|
| 777 |
xl[thread_id] = x[i] |
| 778 |
local_barrier() |
| 779 |
|
| 780 |
|
| 781 |
x = np.linspace(0, 1, 10000) |
| 782 |
|
| 783 |
get_config().use_opencl = True |
| 784 |
x = wrap(x) |
| 785 |
xl = LocalMem(1) |
| 786 |
|
| 787 |
k = Kernel(f) |
| 788 |
k(x, xl) |
| 789 |
|
| 790 |
This kernel does nothing useful and is just meant to demonstrate how one can |
| 791 |
allocate and use local memory. Note that here we "allocated" the local memory |
| 792 |
on the host and are passing it in to the Kernel. The local memory is allocated |
| 793 |
as ``LocalMem(1)``, this implicitly means allocate the required size in |
| 794 |
multiples of the size of the type and the work group size. Thus the allocated |
| 795 |
memory is ``work_group_size * sizeof(double) * 1``. This is convenient as very |
| 796 |
often the exact work group size is not known. |
| 797 |
|
| 798 |
A more complex and meaningful example is the ``vm_kernel.py`` example that is |
| 799 |
included with Compyle. |
| 800 |
|
| 801 |
|
| 802 |
``Cython`` |
| 803 |
~~~~~~~~~~~ |
| 804 |
|
| 805 |
Just like the ``Kernel`` we also have a ``Cython`` class to run pure Cython |
| 806 |
code. Here is an example of its usage:: |
| 807 |
|
| 808 |
from compyle.config import use_config |
| 809 |
from compyle.types import annotate |
| 810 |
from compyle.low_level import Cython, nogil, parallel, prange |
| 811 |
|
| 812 |
import numpy as np |
| 813 |
|
| 814 |
@annotate(n='int', doublep='x, y', a='double') |
| 815 |
def cy_ex(x, y, a, n): |
| 816 |
i = declare('int') |
| 817 |
with nogil, parallel(): |
| 818 |
for i in prange(n): |
| 819 |
y[i] = x[i]*a |
| 820 |
|
| 821 |
n = 1000 |
| 822 |
x = np.linspace(0, 1, n) |
| 823 |
y = np.zeros_like(x) |
| 824 |
a = 2.0 |
| 825 |
|
| 826 |
with use_config(use_openmp=True): |
| 827 |
cy = Cython(cy_ex) |
| 828 |
|
| 829 |
cy(x, y, a, n) |
| 830 |
|
| 831 |
If you look at the above code, we are effectively writing Cython code but |
| 832 |
compiling it and calling it in the last two lines. Note the use of the |
| 833 |
``nogil, parallel`` and ``prange`` functions which are also provided in the |
| 834 |
``low_level`` module. As you can see it is just as easy to write Cython code |
| 835 |
and have it execute in parallel. |
| 836 |
|
| 837 |
|
| 838 |
Externs |
| 839 |
~~~~~~~ |
| 840 |
|
| 841 |
The ``nogil, parallel`` and ``prange`` functions we see in the previous |
| 842 |
section are examples of external functionality. Note that these have no |
| 843 |
straight-forward Python analog or implementation. They are implemented as |
| 844 |
Externs. This functionality allows us to link to external code opening up many |
| 845 |
interesting possibilities. |
| 846 |
|
| 847 |
Note that as far as Compyle is concerned, we need to know if a function needs to |
| 848 |
be wrapped or somehow injected. Externs offer us a way to cleanly inject |
| 849 |
external function definitions and use them. This is useful for example when |
| 850 |
you need to include an external CUDA library. |
| 851 |
|
| 852 |
Let us see how the ``prange`` extern is internally defined:: |
| 853 |
|
| 854 |
from compyle.extern import Extern |
| 855 |
|
| 856 |
class _prange(Extern): |
| 857 |
def link(self, backend): |
| 858 |
# We don't need to link to anything to get prange working. |
| 859 |
return [] |
| 860 |
|
| 861 |
def code(self, backend): |
| 862 |
if backend != 'cython': |
| 863 |
raise NotImplementedError('prange only available with Cython') |
| 864 |
return 'from cython.parallel import prange' |
| 865 |
|
| 866 |
def __call__(self, *args, **kw): |
| 867 |
# Ignore the kwargs. |
| 868 |
return range(*args) |
| 869 |
|
| 870 |
prange = _prange() |
| 871 |
|
| 872 |
|
| 873 |
The Extern object has two important methods, ``link`` and ``code``. The |
| 874 |
``__call__`` interface is provided just so this can be executed with pure |
| 875 |
Python. The link returns a list of link args, these are currently ignored |
| 876 |
until we figure out a good test/example for this. The ``code`` method returns |
| 877 |
a suitable line of code inserted into the generated code. Note that in this |
| 878 |
case it just performs a suitable import. |
| 879 |
|
| 880 |
Thus, with this feature we are able to connect Compyle with other libraries. |
| 881 |
This functionality will probably evolve a little more as we gain more experience |
| 882 |
linking with other libraries. However, we have a clean mechanism for doing so |
| 883 |
already in-place. |