Debian Codegraph @

docs/source/details.rst

raw
1 Using Compyle
2 ==============
3
4 In this section we provide more details on the compyle package and how it can be
5 used.
6
7 An overview of functionality
8 -----------------------------
9
10 The functionality provided falls into two broad categories,
11
12 - Common parallel algorithms that will work across backends. This includes,
13 elementwise operations, reductions, and prefix-sums/scans.
14
15 - Specific support to run code on a particular backend. This is for code that
16 will only work on one backend by definition. This is necessary in order to
17 best use different hardware and also use differences in the particular
18 backend implementations. For example, the notion of local (or shared) memory
19 only has meaning on a GPGPU. In this category we provide support to compile
20 and execute Cython code, and also create and execute a GPU kernel.
21
22 In addition there is common functionality to perform type annotations. At a
23 lower level, there are code translators (transpilers) that handle generation
24 of Cython and C code from annotated Python code. Technically these transpilers
25 can be reused by users to do other things but we only go over the higher level
26 tools in this documentation. All the code is fairly extensively tested and
27 developed using a test-driven approach. In fact, a good place to see examples
28 are the tests.
29
30 We now go into the details of each of these so as to provide a high-level
31 overview of what can be done with Compyle.
32
33 Annotating functions
34 ---------------------
35
36 The first step in getting started using Compyle is to annotate your functions and
37 also declare variables in code.
38
39 Annotation is provided by a simple decorator, called ``annotate``. One can
40 declare local variables inside these functions using ``declare``. A simple
41 example serves to illustrate these::
42
43
44 @annotate(i='int', x='floatp', return_='float')
45 def f(i, x):
46 return x[i]*2.0
47
48 @annotate(i='int', floatp='x, y', return_='float')
49 def g(i, x, y):
50 return f(i, x)*y[i]
51
52
53 Note that for convenience ``annotate``, accepts types and variable names in
54 two different ways, which you can use interchangeably.
55
56 1. You can simply use ``var_name=type_str``, or ``var_name=type`` where the
57 type is from the ``compyle.types`` module.
58
59 2. You can instead use ``type_name='x, y, z'``, which is often very
60 convenient. The order of the variables is not important and need not match
61 with the order in which they are declared.
62
63 You can use ``return_=type``, where ``type`` is an appropriate type or
64 standard string representing one of the types. If the return type is not
65 specified it assumes a ``void`` return.
66
67
68 The definitions of the various standard types is in ``compyle.types.TYPES``. Some
69 are listed below:
70
71 - ``'float', 'double', 'int', 'long', 'uint', 'ulong'``: etc. are exactly as
72 you would expect.
73
74 - ``'doublep'`` would refer to a double pointer, i.e. ``double*`` and
75 similarly for anything with a ``p`` at the end.
76
77 - ``gdoublep`` would be a ``global doublep``, which makes sense with OpenCL
78 where you would have ``__global double* xx``. The global address space
79 specification is ignored when Cython code is generated, so this is safe to
80 use with Cython code too.
81
82 - ``ldoublep`` would be equivalent to ``__local double*`` in OpenCL, for local
83 memory. Again this address space qualifier is ignored in Cython.
84
85 All these types are available in the ``compyle.types`` module namespace also for
86 your convenience. The ``int, float, long`` types are accessible as ``int_,
87 float_, long_`` so as not to override the default Python types. For example
88 the function ``f`` in the above could also have been declared like so::
89
90 from compyle.types import floatp, float_, int_
91
92 @annotate(i=int_, x=floatp, return_=float_)
93 def f(i, x):
94 return x[i]*2.0
95
96
97 One can also use custom types (albeit with care) by using the
98 ``compyle.typs.KnownType`` class. This is convenient in other scenarios where you
99 could potentially pass instances/structs to a function. We will discuss this
100 later but all of the basic types discussed above are all instances of
101 ``KnownType``.
102
103 Compyle actually supports Python3 style annotations but only for the function
104 arguments and NOT for the local variables. The only caveat is you must use the
105 types in ``compyle.types``, i.e. you must use ``KnownType`` instances as the
106 types for things to work.
107
108 JIT transpilation
109 -----------------
110
111 Compyle also support just-in-time transpilation when annotations of a function
112 are not provided. These functions are annotated at runtime when the
113 call arguments are passed. The generated kernel and annotated functions
114 are then cached with the types of the call arguments as key. Thus,
115 the function ``f`` defined in the previous section can also be defined
116 as follows::
117
118 @annotate
119 def f(i, x):
120 return x[i]*2.0
121
122 While using in-built functions such as ``sin``, ``cos``, ``abs`` etc. it is
123 recommended that you store the value in a variable or appropriate type
124 before returning it. If not the return type will default to ``double``.
125 For example,::
126
127 @annotate
128 def f(i, x):
129 return abs(x[i])
130
131 This will set the return type of function ``f`` to the default
132 type, ``double`` even when ``x`` is an array of integers.
133 To avoid this problem, one could define ``f`` instead as,::
134
135 @annotate
136 def f(i, x):
137 y = declare('int')
138 y = abs(x[i])
139 return y
140
141 Currently JIT support is only limited to the common parallel algorithms explained
142 in a later section.
143
144 Declaring variables
145 -------------------
146
147 In addition to annotating the function arguments and return types, it is
148 important to be able to declare the local variables. We provide a simple
149 ``declare`` function that lets us do this. One again, a few examples serve to
150 illustrate this::
151
152 i = declare('int')
153 x = declare('float')
154 u, v = declare('double', 2)
155
156 Notice the last one where we passed an additional argument of the number of
157 types we want. This is really done to keep this functional in pure Python so
158 that your code executes on Python also. In Cython these would produce::
159
160 cdef int i
161 cdef float x
162 cdef double u, v
163
164 On OpenCL this would produce the equivalent::
165
166 int i;
167 float x;
168 double u, v;
169
170 Technically one could also write::
171
172 f = declare('float4')
173
174 but clearly this would only work on OpenCL, however, you can definitely
175 declare other variables too!
176
177 Note that in OpenCL/Cython code if you do not declare a variable, it is
178 automatically declared as a ``double`` to prevent compilation errors.
179
180 We often also require arrays, ``declare`` also supports this, for example
181 consider these examples::
182
183 r = declare('matrix(3)')
184 a = declare('matrix((2, 2))')
185 u, v = declare('matrix(2)', 2)
186
187 This reduces to the following on OpenCL::
188
189 double r[3];
190 double a[3][3];
191 double u[2], v[2];
192
193 Note that this will only work with fixed sizes, and not with dynamic sizes. As
194 we pointed out earlier, dynamic memory allocation is not allowed. Of course
195 you could easily do this with Cython code but the declare syntax does not
196 allow this.
197
198 If you want non-double matrices, you can simply pass a type as in::
199
200 a = declare('matrix((2, 2), "int")')
201
202 Which would result in::
203
204 int a[2][2];
205
206 As you can imagine, just being able to do this opens up quite a few
207 possibilities. You could also do things like this::
208
209 xloc = declare('LOCAL_MEM matrix(128)')
210
211 which will become in OpenCL::
212
213 LOCAL_MEM double xloc[128];
214
215 The ``LOCAL_MEM`` is a special variable that expands to the appropriate flag
216 on OpenCL or CUDA to allow you to write kernels once and have them run on
217 either OpenCL or CUDA. These special variables are discussed later below.
218
219 Writing the functions
220 ----------------------
221
222 All of basic Python is supported. As you may have seen in the examples, you
223 can write code that uses the following:
224
225 - Indexing (only positive indices please).
226 - Conditionals with if/elif/else.
227 - While loops.
228 - For loops with the ``for var in range(...)`` construction.
229 - Nested fors.
230 - Ternary operators.
231
232 This allows us to write most numerical code. Fancy slicing etc. are not
233 supported, numpy based slicing and striding are not supported. You are
234 supposed to write these out elementwise. The idea is to keep things simple.
235 Yes, this may make things verbose but it does keep our life simple and
236 eventually yours too.
237
238 Do not create any Python data structures in the code unless you do not want to
239 run the code on a GPU. No numpy arrays can be created, also avoid calling any
240 numpy functions as these will NOT translate to any GPU code. You have to write
241 what you need by hand. Having said that, all the basic math functions and
242 symbols are automatically available. Essentially all of ``math`` is available.
243 All of the ``math.h`` constants are also available for use.
244
245 If you declare a global constant it will be automatically defined in the
246 generated code. For example::
247
248 MY_CONST = 42
249
250 @annotate(x='double', return_='double')
251 def f(x):
252 return x + MY_CONST
253
254
255 The ``MY_CONST`` will be automatically injected in your generated code.
256
257 Now you may wonder about how you can call an external library that is not in
258 ``math.h``. Lets say you have an external CUDA library, how do you call that?
259 We have a simple approach for this which we discuss later. We call this an
260 ``Extern`` and discuss it later.
261
262
263 .. _PyOpenCL: https://documen.tician.de/pyopencl/
264 .. _Cython: http://www.cython.org
265 .. _PySPH: http://pysph.readthedocs.io
266
267 Common parallel algorithms
268 ---------------------------
269
270 Compyle provides a few very powerful parallel algorithms. These are all directly
271 motivated by Andreas Kloeckner's PyOpenCL_ package. On the GPU they are
272 wrappers on top of the functionality provided there. These algorithms make it
273 possible to implement scalable algorithms for a variety of common numerical
274 problems. In PySPH_ for example all of the GPU based nearest neighbor finding
275 algorithms are written with these fundamental primitives and scale very well.
276
277 All of the following parallel algorithms allow choice of a suitable backend
278 and take a keyword argument to specify this backend. If no backend is provided
279 a default is chosen from the ``compyle.config`` module. You can get the global
280 config using::
281
282 from compyle.config import get_config
283
284 cfg = get_config()
285 cfg.use_openmp = True
286 cfg.use_opencl = True
287
288 etc. The following are the parallel algorithms available from the
289 ``compyle.parallel`` module.
290
291 ``Elementwise``
292 ~~~~~~~~~~~~~~~
293
294 This is also available as a decorator ``elementwise``. One can pass it an
295 annotated function and an optional backend. The elementwise processes every
296 element in the second argument to the function. The elementwise basically
297 passes the function an index of the element it is processing and parallelizes
298 the calls to this automatically. If you are familiar with writing GPU kernels,
299 this is the same thing except the index is passed along to you.
300
301 Here is a very simple example that shows how this works for a case where we
302 compute ``y = a*sin(x) + b`` where ``y, a, x, b`` are all numpy arrays but let
303 us say we want to do this in parallel::
304
305 import numpy as np
306 from compyle.api import annotate, Elementwise, get_config
307
308 @annotate(i='int', doublep='x, y, a, b')
309 def axpb(i, x, y, a, b):
310 y[i] = a[i]*sin(x[i]) + b[i]
311
312 # Setup the input data
313 n = 1000000
314 x = np.linspace(0, 1, n)
315 y = np.zeros_like(x)
316 a = np.random.random(n)
317 b = np.random.random(n)
318
319 # Use OpenMP
320 get_config().use_openmp = True
321
322 # Now run this in parallel with Cython.
323 backend = 'cython'
324 e = Elementwise(axpb, backend=backend)
325 e(x, y, a, b)
326
327 This will call the ``axpb`` function in parallel and if your problem is large
328 enough will effectively scale on all your cores. Its as simple as that.
329
330 Now let us say we want to run this with OpenCL. The only issue with OpenCL is
331 that the data needs to be sent to the GPU. This is transparently handled by a
332 simple ``Array`` wrapper that handles this for us automatically. Here is a
333 simple example building on the above::
334
335 from compyle.api import wrap
336
337 backend = 'opencl'
338 x, y, a, b = wrap(x, y, a, b, backend=backend)
339
340 What this does is to wrap each of the arrays and also sends the data to the
341 device. ``x`` is now an instance of ``compyle.array.Array``, this simple
342 class has two attributes, ``data`` and ``dev``. The first is the original data
343 and the second is a suitable device array from PyOpenCL/PyCUDA depending on
344 the backend. To get data from the device to the host you can call ``x.pull()``
345 to push data to the device you can call ``x.push()``.
346
347 Now that we have this wrapped we can simply do::
348
349 e = Elementwise(axpb, backend=backend)
350 e(x, y, a, b)
351
352 We do not need to change any of our other code. As you can see this is very convenient.
353
354 Here is all the code put together::
355
356 import numpy as np
357 from compyle.api import annotate, Elementwise, get_config, wrap
358
359 @annotate(i='int', doublep='x, y, a, b')
360 def axpb(i, x, y, a, b):
361 y[i] = a[i]*sin(x[i]) + b[i]
362
363 # Setup the input data
364 n = 1000000
365 x = np.linspace(0, 1, n)
366 y = np.zeros_like(x)
367 a = np.random.random(n)
368 b = np.random.random(n)
369
370 # Turn on OpenMP for Cython.
371 get_config().use_openmp = True
372
373 for backend in ('cython', 'opencl'):
374 xa, ya, aa, ba = wrap(x, y, a, b, backend=backend)
375 e = Elementwise(axpb, backend=backend)
376 e(xa, ya, aa, ba)
377
378 This will run the code on both backends! We use the for loop just to show that
379 this will run on all backends! The ``axpb.py`` example shows this for a
380 variety of array sizes and plots the performance.
381
382
383 ``Reduction``
384 ~~~~~~~~~~~~~~~
385
386 The ``compyle.parallel`` module also provides a ``Reduction`` class which can be
387 used fairly easily. Using it is a bit complex, a good starting point for this
388 is the documentation of PyOpenCL_, here
389 https://documen.tician.de/pyopencl/algorithm.html#module-pyopencl.reduction
390
391 The difference from the PyOpenCL implementation is that the ``map_expr`` is a
392 function rather than a string.
393
394 We provide a couple of simple examples to illustrate the above. The first
395 example is to find the sum of all elements of an array::
396
397 x = np.linspace(0, 1, 1000)/1000
398 x = wrap(x, backend=backend)
399
400 r = Reduction('a+b', backend=backend)
401 result = r(x)
402
403 Here is an example of a function to find the minimum of an array::
404
405 x = np.linspace(0, 1, 1000)/1000
406 x = wrap(x, backend=backend)
407
408 r = Reduction('min(a, b)', neutral='INFINITY', backend=backend)
409 result = r(x)
410
411 Here is a final one with a map expression thrown in::
412
413 from math import cos, sin
414 x = np.linspace(0, 1, 1000)/1000
415 y = x.copy()
416 x, y = wrap(x, y, backend=backend)
417
418 @annotate(i='int', doublep='x, y')
419 def map(i=0, x=[0.0], y=[0.0]):
420 return cos(x[i])*sin(y[i])
421
422 r = Reduction('a+b', map_func=map, backend=backend)
423 result = r(x, y)
424
425 As you can see this is faithful to the PyOpenCL implementation with the only
426 difference that the ``map_expr`` is actually a nice function. Further, this
427 works on all backends, even on Cython.
428
429
430 ``Scan``
431 ~~~~~~~~~~
432
433 Scans are generalizations of prefix sums / cumulative sums and can be used as
434 building blocks to construct a number of parallel algorithms. These include
435 but not are limited to sorting, polynomial evaluation, and tree operations.
436 Blelloch's literature on prefix sums (`Prefix Sums and Their Applications
437 <https://www.cs.cmu.edu/~guyb/papers/Ble93.pdf>`_) has many more examples and
438 is a recommended read before using scans. The ``compyle.parallel`` module
439 provides a ``Scan`` class which can be used to develop and execute such scans.
440 The scans can be run on GPUs using the OpenCL or CUDA backend or on CPUs using
441 either the OpenCL or Cython backend.
442
443 The scan semantics in compyle are similar to those of the GenericScanKernel in
444 PyOpenCL
445 (https://documen.tician.de/pyopencl/algorithm.html#pyopencl.scan.GenericScanKernel). Similar
446 to the case for reduction, the main differences from the PyOpenCL implementation
447 are that the expressions (`input_expr`, `segment_expr`, `output_expr`) are all
448 functions rather than strings.
449
450 The following examples demonstrate how scans can be used in compyle. The first
451 example is to find the cumulative sum of all elements of an array::
452
453 ary = np.arange(10000, dtype=np.int32)
454 ary = wrap(ary, backend=backend)
455
456 @annotate(i='int', ary='intp', return_='int')
457 def input_expr(i, ary):
458 return ary[i]
459
460 @annotate(int='i, item', ary='intp')
461 def output_expr(i, item, ary):
462 ary[i] = item
463
464 scan = Scan(input_expr, output_expr, 'a+b', dtype=np.int32,
465 backend=backend)
466 scan(ary=ary)
467 ary.pull()
468
469 # Result = ary.data
470
471 Here is a more complex example of a function that finds the unique elements in
472 an array::
473
474 ary = np.random.randint(0, 100, 1000, dtype=np.int32)
475 unique_ary = np.zeros(len(ary.data), dtype=np.int32)
476 unique_ary = wrap(unique_ary, backend=backend)
477 unique_count = np.zeros(1, dtype=np.int32)
478 unique_count = wrap(unique_count, backend=backend)
479 ary = wrap(ary, backend=backend)
480
481 @annotate(i='int', ary='intp', return_='int')
482 def input_expr(i, ary):
483 if i == 0 or ary[i] != ary[i - 1]:
484 return 1
485 else:
486 return 0
487
488 @annotate(int='i, prev_item, item, N', ary='intp',
489 unique='intp', unique_count='intp')
490 def output_expr(i, prev_item, item, N, ary, unique, unique_count):
491 if item != prev_item:
492 unique[item - 1] = ary[i]
493 if i == N - 1:
494 unique_count[0] = item
495
496 scan = Scan(input_expr, output_expr, 'a+b', dtype=np.int32, backend=backend)
497 scan(ary=ary, unique=unique_ary, unique_count=unique_count)
498 unique_ary.pull()
499 unique_count.pull()
500 unique_count = unique_count.data[0]
501 unique_ary = unique_ary.data[:unique_count]
502
503 # Result = unique_ary
504
505 The following points highlight some important details and quirks about using
506 scans in compyle:
507
508 1. The scan call does not return anything. All output must be handled manually.
509 Usually this involves writing the results available in ``output_expr``
510 (``prev_item``, ``item`` and ``last_item``) to an array.
511 2. ``input_expr`` might be evaluated multiple times. However, it can be assumed
512 that ``input_expr`` for an element or index ``i`` is not evaluated again
513 after the output expression ``output_expr`` for that element is
514 evaluated. Therefore, it is safe to write the output of a scan back to an
515 array also used for the input like in the first example.
516 3. (For PyOpenCL users) If a segmented scan is used, unlike PyOpenCL where the
517 ``across_seg_boundary`` is used to handle the segment logic in the scan
518 expression, in compyle the logic is handled automatically. More specifically,
519 using ``a + b`` as the scan expression in compyle is equivalent to using
520 ``(across_seg_boundary ? b : a + b)`` in PyOpenCL.
521
522
523 Debugging
524 ----------
525
526 Debugging can be a bit difficult with multiple different architectures and
527 backends. One convenience that compyle provides is that the generated sources
528 can be inspected. All the parallel algorithms (``Elementwise, Reduction,
529 Scan``) provide a ``.source`` or ``.all_source`` attribute that contains the
530 source. For example say you have the following::
531
532 e = Elementwise(axpb, backend=backend)
533 e(x, y, a, b)
534
535 You can examine the source generated for your functions using::
536
537 e.source
538
539 This is probably most useful for end users. For those more curious, all of the
540 source generated and used for the complete elementwise (or other) parallel
541 algorithm can be seen using::
542
543 e.all_source
544
545 This code can be rather long and difficult to read so use this only if you
546 really need to see the underlying code from PyOpenCL or PyCUDA. On the GPU
547 this will often include multiple kernels as well. Note that on CUDA the
548 ``all_source`` does not show all of the sources as PyCUDA currently does not
549 make it easy to inspect the code.
550
551 You can define the environment variable ``COMPYLE_DEBUG``, for example on bash
552 with ``export COMPYLE_DEBUG=1`` and run your code. This will print out the
553 source code generated for every generated function. For CPU execution it will
554 also display compiler output if there is no cached extension available.
555 Turning on the debug feature can give you a lot of information, but it is very
556 useful if you want to get into the gory details and do not want to edit the
557 sources just to look at the generated code.
558
559 It is important to note that all the Cython code is saved on disk inside the
560 ``~/.compyle/`` directory in a platform specific directory but no files are
561 written by compyle for any OpenCL or CUDA code and in these cases the
562 ``COMPYLE_DEBUG`` option is very handy.
563
564
565 Abstracting out arrays
566 -----------------------
567
568 As discussed in the section on Elementwise operations, different backends need
569 to do different things with arrays. With OpenCL/CUDA one needs to send the
570 array to the device. This is transparently managed by the
571 ``compyle.array.Array`` class. It is easiest to use this transparently with
572 the ``wrap`` convenience function as below::
573
574 x = np.linspace(0, 1, 1000)/1000
575 y = x.copy()
576 x, y = wrap(x, y, backend=backend)
577
578 Thus these, new arrays can be passed to any operation and is handled transparently.
579
580
581 Choice of backend and configuration
582 ------------------------------------
583
584 The ``compyle.config`` module provides a simple ``Configuration`` class that
585 is used internally in Compyle to set things like the backend (Cython,
586 OpenCL/CUDA), and some common options like profiling, turning on OpenMP, using
587 double on the GPU etc. Here is an example of the various options::
588
589 from compyle.config import get_config
590 cfg = get_config()
591 cfg.use_double
592 cfg.profile
593 cfg.use_opencl
594 cfg.use_openmp
595
596 If one wants to temporarily set an option and perform an action, one can do::
597
598 from compyle.config import use_config
599
600 with use_config(use_openmp=False):
601 ...
602
603 Here everything within the ``with`` clause will be executed using the
604 specified option and once the clause is exited, the previous settings will be
605 restored. This can be convenient.
606
607 Templates
608 ----------
609
610 When creating libraries, it is useful to be able to write a function as a
611 "template" where the code can be generated depending on various user options.
612 Compyle facilitates this by using Mako_ templates. We provide a convenient
613 ``compyle.template.Template`` class which can be used for this purpose. A
614 trivial and contrived example demonstrates its use below. The example sets any
615 number of given arrays to a constant value::
616
617
618 from compyle.types import annotate
619 from compyle.template import Template
620
621 class SetConstant(Template):
622 def __init__(self, name, arrays):
623 super(SetConstant, self).__init__(name=name)
624 self.arrays = arrays
625
626 def my_func(self, value):
627 '''The contents of this function are directly injected.
628 '''
629 tmp = sin(value)
630
631 def extra_args(self):
632 return self.arrays, {'doublep': ','.join(self.arrays)}
633
634 @annotate(i='int', value='double')
635 def template(self, i, value):
636 '''Set the arrays to a constant value.'''
637 '''
638 ${obj.inject(obj.my_func)}
639 % for arr in obj.arrays:
640 ${arr}[i] = tmp
641 % endfor
642 '''
643
644 set_const = SetConstant('set_const', ['x', 'y', 'z']).function
645 print(set_const.source)
646
647 This will print out this::
648
649 def set_const(i, value, x, y, z):
650 """Set arrays to constant.
651 """
652 tmp = sin(value)
653
654 x[i] = tmp
655 y[i] = tmp
656 z[i] = tmp
657
658
659 This is obviously a trivial example but the idea is that one can create fairly
660 complex templated functions that can be then transpiled and used in different
661 cases. The key point here is the ``template`` method which should simply
662 create a string which is rendered using Mako_ and then put into a function.
663 The ``extra_args`` method allows us to configure the arguments used by the
664 function. The mako template can use the name ``obj`` which is ``self``. The
665 ``obj.inject`` method allows one to literally inject any function into the
666 body of the code with a suitable level of indentation. Of course normal mako
667 functionality is available to do a variety of things.
668
669
670 .. _Mako: https://www.makotemplates.org/
671
672 Low level functionality
673 -----------------------
674
675 In addition to the above, there are also powerful low-level functionality that
676 is provided in ``compyle.low_level``.
677
678
679 ``Kernel``
680 ~~~~~~~~~~~
681
682 The ``Kernel`` class allows one to execute a pure GPU kernel. Unlike the
683 Elementwise functionality above, this is specific to OpenCL/CUDA and will not
684 execute via Cython. What this class lets one do is write low-level kernels
685 which are often required to extract the best performance from your hardware.
686
687 Most of the functionality is exactly the same, one declares functions and
688 annotates them and then passes a function to the ``Kernel`` which calls this
689 just as we would a normal OpenCL kernel for example. The major advantage is
690 that all your code is pure Python. Here is a simple example::
691
692 from compyle.api import annotate, wrap, get_config
693 from compyle.low_level import Kernel, LID_0, LDIM_0, GID_0
694 import numpy as np
695
696 @annotate(x='doublep', y='doublep', double='a,b')
697 def axpb(x, y, a, b):
698 i = declare('int')
699 i = LDIM_0*GID_0 + LID_0
700 y[i] = a*sin(x[i]) + b
701
702 x = np.linspace(0, 1, 10000)
703 y = np.zeros_like(x)
704 a = 2.0
705 b = 3.0
706
707 get_config().use_opencl = True
708 x, y = wrap(x, y)
709
710 k = Kernel(axpb)
711 k(x, y, a, b)
712
713 This is the same Elementwise kernel equivalent from the first example at the
714 top but written as a raw kernel. Notice that ``i`` is not passed but computed
715 using ``LDIM_0, GID_0 and LID_0`` which are automatically made available on
716 OpenCL/CUDA. In addition to these the function ``local_barrier`` is also
717 available. Internally these are ``#defines`` that are like so on OpenCL::
718
719 #define LID_0 get_local_id(0)
720 #define LID_1 get_local_id(1)
721 #define LID_2 get_local_id(2)
722
723 #define GID_0 get_group_id(0)
724 #define GID_1 get_group_id(1)
725 #define GID_2 get_group_id(2)
726
727 #define LDIM_0 get_local_size(0)
728 #define LDIM_1 get_local_size(1)
729 #define LDIM_2 get_local_size(2)
730
731 #define GDIM_0 get_num_groups(0)
732 #define GDIM_1 get_num_groups(1)
733 #define GDIM_2 get_num_groups(2)
734
735 #define local_barrier() barrier(CLK_LOCAL_MEM_FENCE);
736
737 On CUDA, these are mapped to the equivalent ::
738
739 #define LID_0 threadIdx.x
740 #define LID_1 threadIdx.y
741 #define LID_2 threadIdx.z
742
743 #define GID_0 blockIdx.x
744 #define GID_1 blockIdx.y
745 #define GID_2 blockIdx.z
746
747 #define LDIM_0 blockDim.x
748 #define LDIM_1 blockDim.y
749 #define LDIM_2 blockDim.z
750
751 #define GDIM_0 gridDim.x
752 #define GDIM_1 gridDim.y
753 #define GDIM_2 gridDim.z
754
755 #define local_barrier() __syncthreads();
756
757
758 In fact these are all provided by the ``_cluda.py`` in PyOpenCL and PyCUDA.
759 These allow us to write CUDA/OpenCL agnostic code from Python.
760
761 One may also pass local memory to such a kernel, this trivial example
762 demonstrates this::
763
764
765 from compyle.api import annotate
766 from compyle.low_level import (
767 Kernel, LID_0, LDIM_0, GID_0, LocalMem, local_barrier
768 )
769 import numpy as np
770
771 @annotate(gdoublep='x', ldoublep='xl')
772 def f(x, xl):
773 i, thread_id = declare('int', 2)
774 thread_id = LID_0
775 i = GID_0*LDIM_0 + thread_id
776
777 xl[thread_id] = x[i]
778 local_barrier()
779
780
781 x = np.linspace(0, 1, 10000)
782
783 get_config().use_opencl = True
784 x = wrap(x)
785 xl = LocalMem(1)
786
787 k = Kernel(f)
788 k(x, xl)
789
790 This kernel does nothing useful and is just meant to demonstrate how one can
791 allocate and use local memory. Note that here we "allocated" the local memory
792 on the host and are passing it in to the Kernel. The local memory is allocated
793 as ``LocalMem(1)``, this implicitly means allocate the required size in
794 multiples of the size of the type and the work group size. Thus the allocated
795 memory is ``work_group_size * sizeof(double) * 1``. This is convenient as very
796 often the exact work group size is not known.
797
798 A more complex and meaningful example is the ``vm_kernel.py`` example that is
799 included with Compyle.
800
801
802 ``Cython``
803 ~~~~~~~~~~~
804
805 Just like the ``Kernel`` we also have a ``Cython`` class to run pure Cython
806 code. Here is an example of its usage::
807
808 from compyle.config import use_config
809 from compyle.types import annotate
810 from compyle.low_level import Cython, nogil, parallel, prange
811
812 import numpy as np
813
814 @annotate(n='int', doublep='x, y', a='double')
815 def cy_ex(x, y, a, n):
816 i = declare('int')
817 with nogil, parallel():
818 for i in prange(n):
819 y[i] = x[i]*a
820
821 n = 1000
822 x = np.linspace(0, 1, n)
823 y = np.zeros_like(x)
824 a = 2.0
825
826 with use_config(use_openmp=True):
827 cy = Cython(cy_ex)
828
829 cy(x, y, a, n)
830
831 If you look at the above code, we are effectively writing Cython code but
832 compiling it and calling it in the last two lines. Note the use of the
833 ``nogil, parallel`` and ``prange`` functions which are also provided in the
834 ``low_level`` module. As you can see it is just as easy to write Cython code
835 and have it execute in parallel.
836
837
838 Externs
839 ~~~~~~~
840
841 The ``nogil, parallel`` and ``prange`` functions we see in the previous
842 section are examples of external functionality. Note that these have no
843 straight-forward Python analog or implementation. They are implemented as
844 Externs. This functionality allows us to link to external code opening up many
845 interesting possibilities.
846
847 Note that as far as Compyle is concerned, we need to know if a function needs to
848 be wrapped or somehow injected. Externs offer us a way to cleanly inject
849 external function definitions and use them. This is useful for example when
850 you need to include an external CUDA library.
851
852 Let us see how the ``prange`` extern is internally defined::
853
854 from compyle.extern import Extern
855
856 class _prange(Extern):
857 def link(self, backend):
858 # We don't need to link to anything to get prange working.
859 return []
860
861 def code(self, backend):
862 if backend != 'cython':
863 raise NotImplementedError('prange only available with Cython')
864 return 'from cython.parallel import prange'
865
866 def __call__(self, *args, **kw):
867 # Ignore the kwargs.
868 return range(*args)
869
870 prange = _prange()
871
872
873 The Extern object has two important methods, ``link`` and ``code``. The
874 ``__call__`` interface is provided just so this can be executed with pure
875 Python. The link returns a list of link args, these are currently ignored
876 until we figure out a good test/example for this. The ``code`` method returns
877 a suitable line of code inserted into the generated code. Note that in this
878 case it just performs a suitable import.
879
880 Thus, with this feature we are able to connect Compyle with other libraries.
881 This functionality will probably evolve a little more as we gain more experience
882 linking with other libraries. However, we have a clean mechanism for doing so
883 already in-place.