Debian Codegraph @

Doc/faq/extending.rst

raw
1 =======================
2 Extending/Embedding FAQ
3 =======================
4
5 .. only:: html
6
7 .. contents::
8
9 .. highlight:: c
10
11
12 .. XXX need review for Python 3.
13
14
15 Can I create my own functions in C?
16 -----------------------------------
17
18 Yes, you can create built-in modules containing functions, variables, exceptions
19 and even new types in C. This is explained in the document
20 :ref:`extending-index`.
21
22 Most intermediate or advanced Python books will also cover this topic.
23
24
25 Can I create my own functions in C++?
26 -------------------------------------
27
28 Yes, using the C compatibility features found in C++. Place ``extern "C" {
29 ... }`` around the Python include files and put ``extern "C"`` before each
30 function that is going to be called by the Python interpreter. Global or static
31 C++ objects with constructors are probably not a good idea.
32
33
34 .. _c-wrapper-software:
35
36 Writing C is hard; are there any alternatives?
37 ----------------------------------------------
38
39 There are a number of alternatives to writing your own C extensions, depending
40 on what you're trying to do. :ref:`Recommended third party tools <c-api-tools>`
41 offer both simpler and more sophisticated approaches to creating C and C++
42 extensions for Python.
43
44
45 How can I execute arbitrary Python statements from C?
46 -----------------------------------------------------
47
48 The highest-level function to do this is :c:func:`PyRun_SimpleString` which takes
49 a single string argument to be executed in the context of the module
50 ``__main__`` and returns ``0`` for success and ``-1`` when an exception occurred
51 (including :exc:`SyntaxError`). If you want more control, use
52 :c:func:`PyRun_String`; see the source for :c:func:`PyRun_SimpleString` in
53 ``Python/pythonrun.c``.
54
55
56 How can I evaluate an arbitrary Python expression from C?
57 ---------------------------------------------------------
58
59 Call the function :c:func:`PyRun_String` from the previous question with the
60 start symbol :c:data:`Py_eval_input`; it parses an expression, evaluates it and
61 returns its value.
62
63
64 How do I extract C values from a Python object?
65 -----------------------------------------------
66
67 That depends on the object's type. If it's a tuple, :c:func:`PyTuple_Size`
68 returns its length and :c:func:`PyTuple_GetItem` returns the item at a specified
69 index. Lists have similar functions, :c:func:`PyList_Size` and
70 :c:func:`PyList_GetItem`.
71
72 For bytes, :c:func:`PyBytes_Size` returns its length and
73 :c:func:`PyBytes_AsStringAndSize` provides a pointer to its value and its
74 length. Note that Python bytes objects may contain null bytes so C's
75 :c:func:`!strlen` should not be used.
76
77 To test the type of an object, first make sure it isn't ``NULL``, and then use
78 :c:func:`PyBytes_Check`, :c:func:`PyTuple_Check`, :c:func:`PyList_Check`, etc.
79
80 There is also a high-level API to Python objects which is provided by the
81 so-called 'abstract' interface -- read ``Include/abstract.h`` for further
82 details. It allows interfacing with any kind of Python sequence using calls
83 like :c:func:`PySequence_Length`, :c:func:`PySequence_GetItem`, etc. as well
84 as many other useful protocols such as numbers (:c:func:`PyNumber_Index` et
85 al.) and mappings in the PyMapping APIs.
86
87
88 How do I use Py_BuildValue() to create a tuple of arbitrary length?
89 -------------------------------------------------------------------
90
91 You can't. Use :c:func:`PyTuple_Pack` instead.
92
93
94 How do I call an object's method from C?
95 ----------------------------------------
96
97 The :c:func:`PyObject_CallMethod` function can be used to call an arbitrary
98 method of an object. The parameters are the object, the name of the method to
99 call, a format string like that used with :c:func:`Py_BuildValue`, and the
100 argument values::
101
102 PyObject *
103 PyObject_CallMethod(PyObject *object, const char *method_name,
104 const char *arg_format, ...);
105
106 This works for any object that has methods -- whether built-in or user-defined.
107 You are responsible for eventually :c:func:`Py_DECREF`\ 'ing the return value.
108
109 To call, e.g., a file object's "seek" method with arguments 10, 0 (assuming the
110 file object pointer is "f")::
111
112 res = PyObject_CallMethod(f, "seek", "(ii)", 10, 0);
113 if (res == NULL) {
114 ... an exception occurred ...
115 }
116 else {
117 Py_DECREF(res);
118 }
119
120 Note that since :c:func:`PyObject_CallObject` *always* wants a tuple for the
121 argument list, to call a function without arguments, pass "()" for the format,
122 and to call a function with one argument, surround the argument in parentheses,
123 e.g. "(i)".
124
125
126 How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)?
127 ----------------------------------------------------------------------------------------
128
129 In Python code, define an object that supports the ``write()`` method. Assign
130 this object to :data:`sys.stdout` and :data:`sys.stderr`. Call print_error, or
131 just allow the standard traceback mechanism to work. Then, the output will go
132 wherever your ``write()`` method sends it.
133
134 The easiest way to do this is to use the :class:`io.StringIO` class:
135
136 .. code-block:: pycon
137
138 >>> import io, sys
139 >>> sys.stdout = io.StringIO()
140 >>> print('foo')
141 >>> print('hello world!')
142 >>> sys.stderr.write(sys.stdout.getvalue())
143 foo
144 hello world!
145
146 A custom object to do the same would look like this:
147
148 .. code-block:: pycon
149
150 >>> import io, sys
151 >>> class StdoutCatcher(io.TextIOBase):
152 ... def __init__(self):
153 ... self.data = []
154 ... def write(self, stuff):
155 ... self.data.append(stuff)
156 ...
157 >>> import sys
158 >>> sys.stdout = StdoutCatcher()
159 >>> print('foo')
160 >>> print('hello world!')
161 >>> sys.stderr.write(''.join(sys.stdout.data))
162 foo
163 hello world!
164
165
166 How do I access a module written in Python from C?
167 --------------------------------------------------
168
169 You can get a pointer to the module object as follows::
170
171 module = PyImport_ImportModule("<modulename>");
172
173 If the module hasn't been imported yet (i.e. it is not yet present in
174 :data:`sys.modules`), this initializes the module; otherwise it simply returns
175 the value of ``sys.modules["<modulename>"]``. Note that it doesn't enter the
176 module into any namespace -- it only ensures it has been initialized and is
177 stored in :data:`sys.modules`.
178
179 You can then access the module's attributes (i.e. any name defined in the
180 module) as follows::
181
182 attr = PyObject_GetAttrString(module, "<attrname>");
183
184 Calling :c:func:`PyObject_SetAttrString` to assign to variables in the module
185 also works.
186
187
188 How do I interface to C++ objects from Python?
189 ----------------------------------------------
190
191 Depending on your requirements, there are many approaches. To do this manually,
192 begin by reading :ref:`the "Extending and Embedding" document
193 <extending-index>`. Realize that for the Python run-time system, there isn't a
194 whole lot of difference between C and C++ -- so the strategy of building a new
195 Python type around a C structure (pointer) type will also work for C++ objects.
196
197 For C++ libraries, see :ref:`c-wrapper-software`.
198
199
200 I added a module using the Setup file and the make fails; why?
201 --------------------------------------------------------------
202
203 Setup must end in a newline, if there is no newline there, the build process
204 fails. (Fixing this requires some ugly shell script hackery, and this bug is so
205 minor that it doesn't seem worth the effort.)
206
207
208 How do I debug an extension?
209 ----------------------------
210
211 When using GDB with dynamically loaded extensions, you can't set a breakpoint in
212 your extension until your extension is loaded.
213
214 In your ``.gdbinit`` file (or interactively), add the command:
215
216 .. code-block:: none
217
218 br _PyImport_LoadDynamicModule
219
220 Then, when you run GDB:
221
222 .. code-block:: shell-session
223
224 $ gdb /local/bin/python
225 gdb) run myscript.py
226 gdb) continue # repeat until your extension is loaded
227 gdb) finish # so that your extension is loaded
228 gdb) br myfunction.c:50
229 gdb) continue
230
231 I want to compile a Python module on my Linux system, but some files are missing. Why?
232 --------------------------------------------------------------------------------------
233
234 Most packaged versions of Python omit some files
235 required for compiling Python extensions.
236
237 For Red Hat, install the python3-devel RPM to get the necessary files.
238
239 For Debian, run ``apt-get install python3-dev``.
240
241 How do I tell "incomplete input" from "invalid input"?
242 ------------------------------------------------------
243
244 Sometimes you want to emulate the Python interactive interpreter's behavior,
245 where it gives you a continuation prompt when the input is incomplete (e.g. you
246 typed the start of an "if" statement or you didn't close your parentheses or
247 triple string quotes), but it gives you a syntax error message immediately when
248 the input is invalid.
249
250 In Python you can use the :mod:`codeop` module, which approximates the parser's
251 behavior sufficiently. IDLE uses this, for example.
252
253 The easiest way to do it in C is to call :c:func:`PyRun_InteractiveLoop` (perhaps
254 in a separate thread) and let the Python interpreter handle the input for
255 you. You can also set the :c:func:`PyOS_ReadlineFunctionPointer` to point at your
256 custom input function. See ``Modules/readline.c`` and ``Parser/myreadline.c``
257 for more hints.
258
259 How do I find undefined g++ symbols __builtin_new or __pure_virtual?
260 --------------------------------------------------------------------
261
262 To dynamically load g++ extension modules, you must recompile Python, relink it
263 using g++ (change LINKCC in the Python Modules Makefile), and link your
264 extension module using g++ (e.g., ``g++ -shared -o mymodule.so mymodule.o``).
265
266
267 Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)?
268 ----------------------------------------------------------------------------------------------------------------
269
270 Yes, you can inherit from built-in classes such as :class:`int`, :class:`list`,
271 :class:`dict`, etc.
272
273 The Boost Python Library (BPL, https://www.boost.org/libs/python/doc/index.html)
274 provides a way of doing this from C++ (i.e. you can inherit from an extension
275 class written in C++ using the BPL).