| 1 |
Installation |
| 2 |
============== |
| 3 |
|
| 4 |
ComPyle is itself pure Python but depends on numpy_ and requires either Cython_ |
| 5 |
or PyOpenCL_ or PyCUDA_ along with the respective backends of a C/C++ compiler, |
| 6 |
OpenCL and CUDA. If you are only going to execute code on a CPU then all you |
| 7 |
need is Cython_. The full list of requirements is shown in the |
| 8 |
``requirements.txt`` file on the repository. |
| 9 |
|
| 10 |
You should be able to install ComPyle by doing:: |
| 11 |
|
| 12 |
$ pip install compyle |
| 13 |
|
| 14 |
|
| 15 |
Note that when executing code on a CPU, you will need to have a C/C++ compiler |
| 16 |
that is compatible with your Python installation. In addition, if you need to |
| 17 |
use OpenMP you will need to make sure your compiler is compatible with that. |
| 18 |
Some additional information on this is included below. |
| 19 |
|
| 20 |
Installing the bleeding edge |
| 21 |
---------------------------- |
| 22 |
|
| 23 |
Note that if you want the latest bleeding edge of compyle, clone the |
| 24 |
repository and install compyle like so:: |
| 25 |
|
| 26 |
$ git clone https://github.com/pypr/compyle |
| 27 |
$ cd compyle |
| 28 |
$ python setup.py develop |
| 29 |
# Or |
| 30 |
$ pip install -e . |
| 31 |
|
| 32 |
If you just want the latest version and do not want to clone the repository, |
| 33 |
you can also do:: |
| 34 |
|
| 35 |
$ pip install https://github.com/pypr/compyle/zipball/main |
| 36 |
|
| 37 |
|
| 38 |
.. _PyOpenCL: https://documen.tician.de/pyopencl/ |
| 39 |
.. _OpenCL: https://www.khronos.org/opencl/ |
| 40 |
.. _Cython: http://www.cython.org |
| 41 |
.. _numpy: http://www.numpy.org |
| 42 |
.. _PyCUDA: https://documen.tician.de/pycuda |
| 43 |
.. _OpenMP: http://openmp.org/ |
| 44 |
.. _CuPy: https://cupy.chainer.org/ |
| 45 |
|
| 46 |
|
| 47 |
Setting up on GNU/Linux |
| 48 |
------------------------- |
| 49 |
|
| 50 |
This is usually very simple, just installing the standard gcc/g++ packages ought |
| 51 |
to work. OpenMP_ is typically available but if it is not, it can be installed |
| 52 |
with (on apt-compatible systems):: |
| 53 |
|
| 54 |
$ sudo apt-get install libgomp1 |
| 55 |
|
| 56 |
|
| 57 |
Installation with conda on MacOS |
| 58 |
--------------------------------- |
| 59 |
|
| 60 |
Recent conda_ packages make the process of setup very easy on MacOS assuming |
| 61 |
that you have the `XCode command line utilities`_ installed. Please make sure |
| 62 |
you install this. |
| 63 |
|
| 64 |
For example with conda-forge_ the following creates a new Python 3.8 |
| 65 |
environment with compyle installed and working with both OpenMP and OpenCL:: |
| 66 |
|
| 67 |
$ conda create -c conda-forge -n py38 python=3.8 numpy pyopencl |
| 68 |
$ conda activate py38 # or a suitable such invocation |
| 69 |
$ pip install compyle |
| 70 |
|
| 71 |
Note that the above implicitly installs the ``llvm-openmp`` package in the |
| 72 |
environment which works out of the box with clang and provides OpenMP support. |
| 73 |
|
| 74 |
.. _conda: https://docs.conda.io/ |
| 75 |
.. _conda-forge: https://conda-forge.org/ |
| 76 |
.. _XCode command line utilities: http://stackoverflow.com/questions/12228382/after-install-xcode-where-is-clang |
| 77 |
|
| 78 |
|
| 79 |
Possible issues on MacOS |
| 80 |
-------------------------- |
| 81 |
|
| 82 |
Ensure that you have gcc or clang installed by installing XCode. See |
| 83 |
installing `XCode command line utilities`_ if you installed XCode but can't |
| 84 |
find clang or gcc. |
| 85 |
|
| 86 |
If you are getting strange errors of the form:: |
| 87 |
|
| 88 |
lang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated] |
| 89 |
ld: library not found for -lstdc++ |
| 90 |
clang: error: linker command failed with exit code 1 (use -v to see invocation) |
| 91 |
|
| 92 |
Then try this (on a bash shell):: |
| 93 |
|
| 94 |
$ export MACOSX_DEPLOYMENT_TARGET=10.9 |
| 95 |
|
| 96 |
And run your command again (replace the above with a suitable line on other |
| 97 |
shells). This is necessary because your Python was compiled with an older |
| 98 |
deployment target and the current version of XCode that you have installed is |
| 99 |
not compatible with that. By setting the environment variable you allow |
| 100 |
compyle to use a newer version. If this works, it is a good idea to set this |
| 101 |
in your default environment (``.bashrc`` for bash shells) so you do not have |
| 102 |
to do this every time. You may also do this in the compyle configuration file, |
| 103 |
see :ref:`config`. |
| 104 |
|
| 105 |
|
| 106 |
|
| 107 |
OpenMP on MacOS |
| 108 |
~~~~~~~~~~~~~~~~ |
| 109 |
|
| 110 |
These instructions are a bit old and only if you are not using conda as |
| 111 |
discussed above. |
| 112 |
|
| 113 |
│The default clang compiler available on MacOS uses an LLVM backend and does |
| 114 |
not support OpenMP_ out of the box. There are two ways to support OpenMP. The |
| 115 |
first involves installing the OpenMP support for clang. This can be done with |
| 116 |
brew_ using:: |
| 117 |
|
| 118 |
$ brew install libomp |
| 119 |
|
| 120 |
`LLVM <https://formulae.brew.sh/formula/llvm>`_ can also be installed using |
| 121 |
brew_, likewise. Once this is done, you |
| 122 |
can use the following config_:: |
| 123 |
|
| 124 |
import os |
| 125 |
os.environ['CC'] = '/opt/homebrew/opt/llvm@15/bin/clang' |
| 126 |
os.environ['CXX'] = '/opt/homebrew/opt/llvm@15/bin/clang++' |
| 127 |
OMP_CFLAGS=['-I/opt/homebrew/opt/libomp/include', '-I/opt/homebrew/opt/llvm@15/include', '-Xclang', '-fopenmp'] |
| 128 |
OMP_LINK=['-L/opt/homebrew/opt/libomp/lib', '-L/opt/homebrew/opt/llvm@15/lib', '-lomp'] |
| 129 |
|
| 130 |
The above config assumes that you have installed `llvm@15`. You can change the |
| 131 |
config according to the version at the time of installing. Once that is done, |
| 132 |
it should "just work". If you get strange errors, you can also try |
| 133 |
setting the ``MACOSX_DEPLOYMENT_TARGET`` as shown in the previous section. |
| 134 |
|
| 135 |
Another option is to install GCC for MacOS available on brew_ using :: |
| 136 |
|
| 137 |
$ brew install gcc |
| 138 |
|
| 139 |
Once this is done, you need to use this as your default compiler. The ``gcc`` |
| 140 |
formula on brew currently ships with gcc version 9. Therefore, you can |
| 141 |
tell Python to use the GCC installed by brew by setting:: |
| 142 |
|
| 143 |
$ export CC=gcc-9 |
| 144 |
$ export CXX=g++-9 |
| 145 |
|
| 146 |
Note that you still do need to have the command-line-tools for XCode |
| 147 |
installed, otherwise the important header files are not available. See |
| 148 |
`how-to-install-xcode-command-line-tools |
| 149 |
<https://stackoverflow.com/questions/9329243/how-to-install-xcode-command-line-tools>`_ |
| 150 |
for more details. You may also want to set these environment variables in your |
| 151 |
``.bashrc`` so you don't have to do this every time. You may also do this in |
| 152 |
the compyle configuration file, see :ref:`config`. |
| 153 |
|
| 154 |
Once you do this, compyle will automatically use this version of GCC and will |
| 155 |
also work with OpenMP. Note that on some preliminary benchmarks, GCC's OpenMP |
| 156 |
implementation seems about 10% or so faster than the LLVM version. Your |
| 157 |
mileage may vary. |
| 158 |
|
| 159 |
.. _brew: http://brew.sh/ |
| 160 |
|
| 161 |
|
| 162 |
Setting up on Windows |
| 163 |
---------------------- |
| 164 |
|
| 165 |
Windows will work but you need to make sure you have the right compiler |
| 166 |
installed. See this page for the details of what you need installed. |
| 167 |
|
| 168 |
https://wiki.python.org/moin/WindowsCompilers |
| 169 |
|
| 170 |
OpenMP will work if you have this installed. For recent Python versions |
| 171 |
(>=3.5), install the `Microsoft Build Tools for Visual Studio 2019 |
| 172 |
<https://www.visualstudio.com/downloads/#build-tools-for-visual-studio-2019>`_ |
| 173 |
|
| 174 |
|
| 175 |
Setting up OpenCL/CUDA |
| 176 |
----------------------- |
| 177 |
|
| 178 |
This is too involved a topic to discuss here, instead look at the appropriate |
| 179 |
documentation for PyOpenCL_ and PyCUDA_. Once those packages work correctly, |
| 180 |
you should be all set. Note that if you are only using OpenCL/CUDA you do not |
| 181 |
need to have Cython or a C/C++ compiler. Some features on CUDA require the use |
| 182 |
of the CuPy_ library. |
| 183 |
|
| 184 |
If you want to use OpenCL support, you will need to install the ``pyopencl`` |
| 185 |
package (``conda install -c conda-forge pyopencl`` or ``pip install |
| 186 |
pyopencl``). For CUDA Support, you will need to install ``pycuda`` and |
| 187 |
``cupy``. Of course this assumes you have the required hardware for this. |
| 188 |
|
| 189 |
|
| 190 |
.. _config: |
| 191 |
|
| 192 |
Using the configuration file |
| 193 |
----------------------------- |
| 194 |
|
| 195 |
Instead of setting environment variables and build options on the shell you |
| 196 |
can have them setup using a simple configuration file. |
| 197 |
|
| 198 |
The file is located in ``~/.compyle/config.py``. Here ``~`` is your home |
| 199 |
directory which on Linux is ``/home/username``, on MacOS ``/Users/username`` |
| 200 |
and on Windows the location is likely ``\Users\username``. This file is |
| 201 |
executed and certain options may be set there. |
| 202 |
|
| 203 |
For example if you wish to set the environment variables ``CC`` and ``CXX`` |
| 204 |
you could do this in the ``config.py``:: |
| 205 |
|
| 206 |
import os |
| 207 |
|
| 208 |
os.environ['CC'] = 'gcc-9' |
| 209 |
os.environ['CXX'] = 'g++-9' |
| 210 |
|
| 211 |
If you are using an atypical compiler like icc, Cray, or PGI, you can set |
| 212 |
these up here too. You may also setup custom OpenMP related flags. For |
| 213 |
example, on a Cray system you may do the following:: |
| 214 |
|
| 215 |
OMP_CFLAGS = ['-homp'] |
| 216 |
OMP_LINK = ['-homp'] |
| 217 |
|
| 218 |
The ``OMP_CFLAGS`` and ``OMP_LINK`` parameters should be lists. Other packages |
| 219 |
like pyzoltan or pysph may also use this file for customizations. |