Debian Codegraph @

docs/source/pointer.rst

raw
1 Cppy smart pointer
2 ==================
3
4 CPython relies on reference counting to manage object lifetime. A large pitfall
5 when writing C-extension is to properly handle increfing and decrefing the
6 reference count. Cppy aims at simplifying this process by providing a smart
7 pointer class. Before diving into the details of it helps lets start a CPython
8 reference counting crash-course.
9
10 CPython reference counting crash course
11 ---------------------------------------
12
13 Each object allocated by Python has a reference count, indicating how many
14 times this object is 'used'. When the reference count of an object goes to
15 zero, it is de-allocated. Outside of C extension, one does not need to manage
16 the reference count manually.
17
18 When a function part of Python C-API returns a Python object, it returns a
19 pointer to it. At the time at which the function returns, the referenced
20 object is live and its reference count is above zero. Depending of the
21 function, you do not have the same responsibility with respect to that object
22 reference count:
23
24 - Owned references:
25 Most functions return a ``new`` reference which means that you are responsible
26 for decrefing the object reference count when you are done with it (basically
27 the function increfed the object reference count before returning).
28 In this situation you own a reference.
29
30 - Borrowed reference:
31 Some functions (``PyList_GetItem``, ``PyTuple_GetItem``, ``PyDict_GetItem``, ...)
32 do not incref the object count before returning. In that case, you have only
33 a borrowed reference, you are not responsible for decrefing the object
34 reference count.
35
36 Borrowed references allow to avoid the cost of increfing/decrefing which is
37 nice. However since you do not own the reference, if the object referenced is
38 removed from its owner (list, tuple for the above two mentioned functions) it
39 may just disappear and your reference becomes invalid. This can cause issues.
40 If the object should outlive the container, or the time it will spend in the
41 container you have to incref it manually. Lets now discuss the convention when
42 calling a function.
43
44 When calling a function, the caller is expected to own a reference to each of
45 the arguments passed to the callee. The callee does not own the references, it
46 only borrows them. As a consequence, it should not decref the reference and if
47 it needs to store the object, in for example a C structure, it should incref
48 it. Note that this does not apply in general to Python container since those
49 are manipulated using functions that take care of it. There are however some
50 exceptions that steals a reference, meaning that you are not the owner of the
51 reference after the call. ``PyList_SetItem``, for example, steal references.
52
53 An easy way to get reference count wrong is forgetting to decref some
54 intermediate object before leaving a function. This is particularly true if the
55 function has some early exit point because an exception should be raised. A
56 good practice is to have a single exit point, however it is not always
57 possible/practical and even like this it is possible to miss references, this
58 is typically where cppy can help.
59
60 This is a very brief introduction to reference counting. You can read a bit
61 more in the official `Python documentation`_ and in the `Python API`_
62 documentation.
63
64 .. _Python documentation: https://docs.python.org/3/c-api/intro.html#objects-types-and-reference-counts
65
66 .. _Python API: https://docs.python.org/3/c-api/refcounting.html
67
68 Cppy smart pointer class
69 ------------------------
70
71 Cppy smart pointer (``cppy::ptr``) can be initialized with a pointer to a Python
72 object as follows:
73
74 .. code:: c++
75
76 cppy::ptr obj_ptr( PyUnicode_FromString("test") )
77
78 When created, the class assume that you own the reference, if it is not the
79 case you should incref it first:
80
81 .. code:: c++
82
83 PyObject* function( PyObject* obj )
84 {
85 cppy::ptr obj_ptr( cppy::incref( obj ) );
86 cppy::ptr obj_ptr2( obj, true );
87 }
88
89 .. note::
90
91 Cppy provides convenient inline function for common reference manipulation:
92 - ``cppy::incref``, ``cppy::xincref``, ``cppy::decref``, ``cppy::xdecref`` use the
93 the similarly named Python macros and return the input value.
94 - ``cppy::clear``, ``cppy::replace`` are similar but return void.
95
96 You can also initialize a ``cppy::ptr`` from another ``cppy::ptr`` in which case
97 the reference count will always be incremented.
98
99 The main advantage provided by ``cppy::ptr`` is that it implements a destructor
100 that will be invoked automatically by the c++ runtime when the ``cppy::ptr``
101 goes out of scope. The destructor will decref the reference for you. As a
102 consequence you can be sure that your reference you always be decremented when
103 you leave the function.
104
105 Sometimes, however, that is not what you want, because you want to return the
106 reference the ``cppy::ptr`` manage. You can request the ``cppy::ptr`` to give back
107 the reference using its ``release`` method. Lets illustrate on a tiny example:
108
109 .. code:: c++
110
111 PyObject* function( PyObject* obj )
112 {
113 cppy::ptr repr_ptr( PyObject_Repr( obj ) );
114 return repr_ptr.release();
115 }
116
117 Function which are part of Python C-API are not aware of of ``cppy::ptr`` and
118 when calling them you need to provide the original ``PyObject*``. To access, you
119 simply need to call the ``get`` method of the ``cppy::ptr`` object.
120
121 .. code:: c++
122
123 PyObject* function( PyObject* obj )
124 {
125 cppy::ptr l_ptr( PyList_New() );
126 if( PyList_Append( l_ptr.get(), obj ) != 0 )
127 return 0;
128 return l_ptr.release();
129 }
130
131 Here we see that because we use ``cppy::ptr`` to manage the list, we do not have
132 to worry about decrefing the reference if an exception occurs, the runtime
133 will do it for us. If no exception occurs, we stop managing the reference and
134 we are good.
135
136 Using cppy does not eliminate all the pitfalls of writing C-extensions. For
137 example if you release too early (for example when passing the object to a
138 function that may fail), you can still leak references. However it does
139 alleviate some of the complexity.
140
141 ``cppy::ptr`` methods
142 ---------------------
143
144 All methods that takes a ``PyObject*`` can also accept a ``cppy::ptr``.
145 Most names should be self-explanatory, and apart from the is\_ methods most of
146 them rely on the PyObject\_ functions similarly named:
147
148 .. code:: c++
149
150 bool is_none() const
151 bool is_true() const
152 bool is_false() const
153 bool is_bool() const
154 bool is_int() const
155 bool is_float() const
156 bool is_list() const
157 bool is_dict() const
158 bool is_set() const
159 bool is_bytes() const
160 bool is_str() const
161 bool is_unicode() const
162 bool is_callable() const
163 bool is_iter() const
164 bool is_type( PyTypeObject* cls ) const
165 int is_truthy() const
166 int is_instance( PyObject* cls ) const
167 int is_subclass( PyObject* cls ) const
168 PyObject* iter() const
169 PyObject* next() const
170 PyObject* repr() const
171 PyObject* str() const
172 PyObject* bytes() const
173 PyObject* unicode() const
174 Py_ssize_t length() const
175 PyTypeObject* type() const
176 int richcmp( PyObject* other, int opid ) const
177 long hash() const
178 bool hasattr( PyObject* attr ) const
179 bool hasattr( const char* attr ) const
180 bool hasattr( const std::string& attr ) const
181 PyObject* getattr( PyObject* attr ) const
182 PyObject* getattr( const char* attr ) const
183 PyObject* getattr( const std::string& attr ) const
184 bool setattr( PyObject* attr, PyObject* value ) const
185 bool setattr( const char* attr, PyObject* value ) const
186 bool setattr( const std::string& attr, PyObject* value ) const
187 bool delattr( PyObject* attr ) const
188 bool delattr( const char* attr ) const
189 bool delattr( const std::string& attr ) const
190 PyObject* getitem( PyObject* key ) const
191 bool setitem( PyObject* key, PyObject* value ) const
192 bool delitem( PyObject* key )
193 PyObject* call( PyObject* args, PyObject* kwargs = 0 ) const