| 1 |
======================= |
| 2 |
Extending/Embedding FAQ |
| 3 |
======================= |
| 4 |
|
| 5 |
.. only:: html |
| 6 |
|
| 7 |
.. contents:: |
| 8 |
|
| 9 |
.. highlight:: c |
| 10 |
|
| 11 |
|
| 12 |
.. XXX need review for Python 3. |
| 13 |
|
| 14 |
|
| 15 |
Can I create my own functions in C? |
| 16 |
----------------------------------- |
| 17 |
|
| 18 |
Yes, you can create built-in modules containing functions, variables, exceptions |
| 19 |
and even new types in C. This is explained in the document |
| 20 |
:ref:`extending-index`. |
| 21 |
|
| 22 |
Most intermediate or advanced Python books will also cover this topic. |
| 23 |
|
| 24 |
|
| 25 |
Can I create my own functions in C++? |
| 26 |
------------------------------------- |
| 27 |
|
| 28 |
Yes, using the C compatibility features found in C++. Place ``extern "C" { |
| 29 |
... }`` around the Python include files and put ``extern "C"`` before each |
| 30 |
function that is going to be called by the Python interpreter. Global or static |
| 31 |
C++ objects with constructors are probably not a good idea. |
| 32 |
|
| 33 |
|
| 34 |
.. _c-wrapper-software: |
| 35 |
|
| 36 |
Writing C is hard; are there any alternatives? |
| 37 |
---------------------------------------------- |
| 38 |
|
| 39 |
There are a number of alternatives to writing your own C extensions, depending |
| 40 |
on what you're trying to do. :ref:`Recommended third party tools <c-api-tools>` |
| 41 |
offer both simpler and more sophisticated approaches to creating C and C++ |
| 42 |
extensions for Python. |
| 43 |
|
| 44 |
|
| 45 |
How can I execute arbitrary Python statements from C? |
| 46 |
----------------------------------------------------- |
| 47 |
|
| 48 |
The highest-level function to do this is :c:func:`PyRun_SimpleString` which takes |
| 49 |
a single string argument to be executed in the context of the module |
| 50 |
``__main__`` and returns ``0`` for success and ``-1`` when an exception occurred |
| 51 |
(including :exc:`SyntaxError`). If you want more control, use |
| 52 |
:c:func:`PyRun_String`; see the source for :c:func:`PyRun_SimpleString` in |
| 53 |
``Python/pythonrun.c``. |
| 54 |
|
| 55 |
|
| 56 |
How can I evaluate an arbitrary Python expression from C? |
| 57 |
--------------------------------------------------------- |
| 58 |
|
| 59 |
Call the function :c:func:`PyRun_String` from the previous question with the |
| 60 |
start symbol :c:data:`Py_eval_input`; it parses an expression, evaluates it and |
| 61 |
returns its value. |
| 62 |
|
| 63 |
|
| 64 |
How do I extract C values from a Python object? |
| 65 |
----------------------------------------------- |
| 66 |
|
| 67 |
That depends on the object's type. If it's a tuple, :c:func:`PyTuple_Size` |
| 68 |
returns its length and :c:func:`PyTuple_GetItem` returns the item at a specified |
| 69 |
index. Lists have similar functions, :c:func:`PyList_Size` and |
| 70 |
:c:func:`PyList_GetItem`. |
| 71 |
|
| 72 |
For bytes, :c:func:`PyBytes_Size` returns its length and |
| 73 |
:c:func:`PyBytes_AsStringAndSize` provides a pointer to its value and its |
| 74 |
length. Note that Python bytes objects may contain null bytes so C's |
| 75 |
:c:func:`!strlen` should not be used. |
| 76 |
|
| 77 |
To test the type of an object, first make sure it isn't ``NULL``, and then use |
| 78 |
:c:func:`PyBytes_Check`, :c:func:`PyTuple_Check`, :c:func:`PyList_Check`, etc. |
| 79 |
|
| 80 |
There is also a high-level API to Python objects which is provided by the |
| 81 |
so-called 'abstract' interface -- read ``Include/abstract.h`` for further |
| 82 |
details. It allows interfacing with any kind of Python sequence using calls |
| 83 |
like :c:func:`PySequence_Length`, :c:func:`PySequence_GetItem`, etc. as well |
| 84 |
as many other useful protocols such as numbers (:c:func:`PyNumber_Index` et |
| 85 |
al.) and mappings in the PyMapping APIs. |
| 86 |
|
| 87 |
|
| 88 |
How do I use Py_BuildValue() to create a tuple of arbitrary length? |
| 89 |
------------------------------------------------------------------- |
| 90 |
|
| 91 |
You can't. Use :c:func:`PyTuple_Pack` instead. |
| 92 |
|
| 93 |
|
| 94 |
How do I call an object's method from C? |
| 95 |
---------------------------------------- |
| 96 |
|
| 97 |
The :c:func:`PyObject_CallMethod` function can be used to call an arbitrary |
| 98 |
method of an object. The parameters are the object, the name of the method to |
| 99 |
call, a format string like that used with :c:func:`Py_BuildValue`, and the |
| 100 |
argument values:: |
| 101 |
|
| 102 |
PyObject * |
| 103 |
PyObject_CallMethod(PyObject *object, const char *method_name, |
| 104 |
const char *arg_format, ...); |
| 105 |
|
| 106 |
This works for any object that has methods -- whether built-in or user-defined. |
| 107 |
You are responsible for eventually :c:func:`Py_DECREF`\ 'ing the return value. |
| 108 |
|
| 109 |
To call, e.g., a file object's "seek" method with arguments 10, 0 (assuming the |
| 110 |
file object pointer is "f"):: |
| 111 |
|
| 112 |
res = PyObject_CallMethod(f, "seek", "(ii)", 10, 0); |
| 113 |
if (res == NULL) { |
| 114 |
... an exception occurred ... |
| 115 |
} |
| 116 |
else { |
| 117 |
Py_DECREF(res); |
| 118 |
} |
| 119 |
|
| 120 |
Note that since :c:func:`PyObject_CallObject` *always* wants a tuple for the |
| 121 |
argument list, to call a function without arguments, pass "()" for the format, |
| 122 |
and to call a function with one argument, surround the argument in parentheses, |
| 123 |
e.g. "(i)". |
| 124 |
|
| 125 |
|
| 126 |
How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)? |
| 127 |
---------------------------------------------------------------------------------------- |
| 128 |
|
| 129 |
In Python code, define an object that supports the ``write()`` method. Assign |
| 130 |
this object to :data:`sys.stdout` and :data:`sys.stderr`. Call print_error, or |
| 131 |
just allow the standard traceback mechanism to work. Then, the output will go |
| 132 |
wherever your ``write()`` method sends it. |
| 133 |
|
| 134 |
The easiest way to do this is to use the :class:`io.StringIO` class: |
| 135 |
|
| 136 |
.. code-block:: pycon |
| 137 |
|
| 138 |
>>> import io, sys |
| 139 |
>>> sys.stdout = io.StringIO() |
| 140 |
>>> print('foo') |
| 141 |
>>> print('hello world!') |
| 142 |
>>> sys.stderr.write(sys.stdout.getvalue()) |
| 143 |
foo |
| 144 |
hello world! |
| 145 |
|
| 146 |
A custom object to do the same would look like this: |
| 147 |
|
| 148 |
.. code-block:: pycon |
| 149 |
|
| 150 |
>>> import io, sys |
| 151 |
>>> class StdoutCatcher(io.TextIOBase): |
| 152 |
... def __init__(self): |
| 153 |
... self.data = [] |
| 154 |
... def write(self, stuff): |
| 155 |
... self.data.append(stuff) |
| 156 |
... |
| 157 |
>>> import sys |
| 158 |
>>> sys.stdout = StdoutCatcher() |
| 159 |
>>> print('foo') |
| 160 |
>>> print('hello world!') |
| 161 |
>>> sys.stderr.write(''.join(sys.stdout.data)) |
| 162 |
foo |
| 163 |
hello world! |
| 164 |
|
| 165 |
|
| 166 |
How do I access a module written in Python from C? |
| 167 |
-------------------------------------------------- |
| 168 |
|
| 169 |
You can get a pointer to the module object as follows:: |
| 170 |
|
| 171 |
module = PyImport_ImportModule("<modulename>"); |
| 172 |
|
| 173 |
If the module hasn't been imported yet (i.e. it is not yet present in |
| 174 |
:data:`sys.modules`), this initializes the module; otherwise it simply returns |
| 175 |
the value of ``sys.modules["<modulename>"]``. Note that it doesn't enter the |
| 176 |
module into any namespace -- it only ensures it has been initialized and is |
| 177 |
stored in :data:`sys.modules`. |
| 178 |
|
| 179 |
You can then access the module's attributes (i.e. any name defined in the |
| 180 |
module) as follows:: |
| 181 |
|
| 182 |
attr = PyObject_GetAttrString(module, "<attrname>"); |
| 183 |
|
| 184 |
Calling :c:func:`PyObject_SetAttrString` to assign to variables in the module |
| 185 |
also works. |
| 186 |
|
| 187 |
|
| 188 |
How do I interface to C++ objects from Python? |
| 189 |
---------------------------------------------- |
| 190 |
|
| 191 |
Depending on your requirements, there are many approaches. To do this manually, |
| 192 |
begin by reading :ref:`the "Extending and Embedding" document |
| 193 |
<extending-index>`. Realize that for the Python run-time system, there isn't a |
| 194 |
whole lot of difference between C and C++ -- so the strategy of building a new |
| 195 |
Python type around a C structure (pointer) type will also work for C++ objects. |
| 196 |
|
| 197 |
For C++ libraries, see :ref:`c-wrapper-software`. |
| 198 |
|
| 199 |
|
| 200 |
I added a module using the Setup file and the make fails; why? |
| 201 |
-------------------------------------------------------------- |
| 202 |
|
| 203 |
Setup must end in a newline, if there is no newline there, the build process |
| 204 |
fails. (Fixing this requires some ugly shell script hackery, and this bug is so |
| 205 |
minor that it doesn't seem worth the effort.) |
| 206 |
|
| 207 |
|
| 208 |
How do I debug an extension? |
| 209 |
---------------------------- |
| 210 |
|
| 211 |
When using GDB with dynamically loaded extensions, you can't set a breakpoint in |
| 212 |
your extension until your extension is loaded. |
| 213 |
|
| 214 |
In your ``.gdbinit`` file (or interactively), add the command: |
| 215 |
|
| 216 |
.. code-block:: none |
| 217 |
|
| 218 |
br _PyImport_LoadDynamicModule |
| 219 |
|
| 220 |
Then, when you run GDB: |
| 221 |
|
| 222 |
.. code-block:: shell-session |
| 223 |
|
| 224 |
$ gdb /local/bin/python |
| 225 |
gdb) run myscript.py |
| 226 |
gdb) continue # repeat until your extension is loaded |
| 227 |
gdb) finish # so that your extension is loaded |
| 228 |
gdb) br myfunction.c:50 |
| 229 |
gdb) continue |
| 230 |
|
| 231 |
I want to compile a Python module on my Linux system, but some files are missing. Why? |
| 232 |
-------------------------------------------------------------------------------------- |
| 233 |
|
| 234 |
Most packaged versions of Python omit some files |
| 235 |
required for compiling Python extensions. |
| 236 |
|
| 237 |
For Red Hat, install the python3-devel RPM to get the necessary files. |
| 238 |
|
| 239 |
For Debian, run ``apt-get install python3-dev``. |
| 240 |
|
| 241 |
How do I tell "incomplete input" from "invalid input"? |
| 242 |
------------------------------------------------------ |
| 243 |
|
| 244 |
Sometimes you want to emulate the Python interactive interpreter's behavior, |
| 245 |
where it gives you a continuation prompt when the input is incomplete (e.g. you |
| 246 |
typed the start of an "if" statement or you didn't close your parentheses or |
| 247 |
triple string quotes), but it gives you a syntax error message immediately when |
| 248 |
the input is invalid. |
| 249 |
|
| 250 |
In Python you can use the :mod:`codeop` module, which approximates the parser's |
| 251 |
behavior sufficiently. IDLE uses this, for example. |
| 252 |
|
| 253 |
The easiest way to do it in C is to call :c:func:`PyRun_InteractiveLoop` (perhaps |
| 254 |
in a separate thread) and let the Python interpreter handle the input for |
| 255 |
you. You can also set the :c:func:`PyOS_ReadlineFunctionPointer` to point at your |
| 256 |
custom input function. See ``Modules/readline.c`` and ``Parser/myreadline.c`` |
| 257 |
for more hints. |
| 258 |
|
| 259 |
How do I find undefined g++ symbols __builtin_new or __pure_virtual? |
| 260 |
-------------------------------------------------------------------- |
| 261 |
|
| 262 |
To dynamically load g++ extension modules, you must recompile Python, relink it |
| 263 |
using g++ (change LINKCC in the Python Modules Makefile), and link your |
| 264 |
extension module using g++ (e.g., ``g++ -shared -o mymodule.so mymodule.o``). |
| 265 |
|
| 266 |
|
| 267 |
Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)? |
| 268 |
---------------------------------------------------------------------------------------------------------------- |
| 269 |
|
| 270 |
Yes, you can inherit from built-in classes such as :class:`int`, :class:`list`, |
| 271 |
:class:`dict`, etc. |
| 272 |
|
| 273 |
The Boost Python Library (BPL, https://www.boost.org/libs/python/doc/index.html) |
| 274 |
provides a way of doing this from C++ (i.e. you can inherit from an extension |
| 275 |
class written in C++ using the BPL). |