| 1 |
Metadata-Version: 2.1 |
| 2 |
Name: dill |
| 3 |
Version: 0.4.1 |
| 4 |
Summary: serialize all of Python |
| 5 |
Home-page: https://github.com/uqfoundation/dill |
| 6 |
Download-URL: https://pypi.org/project/dill/#files |
| 7 |
Author: Mike McKerns |
| 8 |
Author-email: [email protected] |
| 9 |
Maintainer: Mike McKerns |
| 10 |
Maintainer-email: [email protected] |
| 11 |
License: BSD-3-Clause |
| 12 |
Project-URL: Documentation, http://dill.rtfd.io |
| 13 |
Project-URL: Source Code, https://github.com/uqfoundation/dill |
| 14 |
Project-URL: Bug Tracker, https://github.com/uqfoundation/dill/issues |
| 15 |
Platform: Linux |
| 16 |
Platform: Windows |
| 17 |
Platform: Mac |
| 18 |
Classifier: Development Status :: 5 - Production/Stable |
| 19 |
Classifier: Intended Audience :: Developers |
| 20 |
Classifier: Intended Audience :: Science/Research |
| 21 |
Classifier: License :: OSI Approved :: BSD License |
| 22 |
Classifier: Programming Language :: Python :: 3 |
| 23 |
Classifier: Programming Language :: Python :: 3.9 |
| 24 |
Classifier: Programming Language :: Python :: 3.10 |
| 25 |
Classifier: Programming Language :: Python :: 3.11 |
| 26 |
Classifier: Programming Language :: Python :: 3.12 |
| 27 |
Classifier: Programming Language :: Python :: 3.13 |
| 28 |
Classifier: Programming Language :: Python :: 3.14 |
| 29 |
Classifier: Programming Language :: Python :: Implementation :: CPython |
| 30 |
Classifier: Programming Language :: Python :: Implementation :: PyPy |
| 31 |
Classifier: Topic :: Scientific/Engineering |
| 32 |
Classifier: Topic :: Software Development |
| 33 |
Requires-Python: >=3.9 |
| 34 |
License-File: LICENSE |
| 35 |
Provides-Extra: readline |
| 36 |
Provides-Extra: graph |
| 37 |
Requires-Dist: objgraph>=1.7.2; extra == "graph" |
| 38 |
Provides-Extra: profile |
| 39 |
Requires-Dist: gprof2dot>=2022.7.29; extra == "profile" |
| 40 |
|
| 41 |
----------------------------- |
| 42 |
dill: serialize all of Python |
| 43 |
----------------------------- |
| 44 |
|
| 45 |
About Dill |
| 46 |
========== |
| 47 |
|
| 48 |
``dill`` extends Python's ``pickle`` module for serializing and de-serializing |
| 49 |
Python objects to the majority of the built-in Python types. Serialization |
| 50 |
is the process of converting an object to a byte stream, and the inverse |
| 51 |
of which is converting a byte stream back to a Python object hierarchy. |
| 52 |
|
| 53 |
``dill`` provides the user the same interface as the ``pickle`` module, and |
| 54 |
also includes some additional features. In addition to pickling Python |
| 55 |
objects, ``dill`` provides the ability to save the state of an interpreter |
| 56 |
session in a single command. Hence, it would be feasible to save an |
| 57 |
interpreter session, close the interpreter, ship the pickled file to |
| 58 |
another computer, open a new interpreter, unpickle the session and |
| 59 |
thus continue from the 'saved' state of the original interpreter |
| 60 |
session. |
| 61 |
|
| 62 |
``dill`` can be used to store Python objects to a file, but the primary |
| 63 |
usage is to send Python objects across the network as a byte stream. |
| 64 |
``dill`` is quite flexible, and allows arbitrary user defined classes |
| 65 |
and functions to be serialized. Thus ``dill`` is not intended to be |
| 66 |
secure against erroneously or maliciously constructed data. It is |
| 67 |
left to the user to decide whether the data they unpickle is from |
| 68 |
a trustworthy source. |
| 69 |
|
| 70 |
``dill`` is part of ``pathos``, a Python framework for heterogeneous computing. |
| 71 |
``dill`` is in active development, so any user feedback, bug reports, comments, |
| 72 |
or suggestions are highly appreciated. A list of issues is located at |
| 73 |
https://github.com/uqfoundation/dill/issues, with a legacy list maintained at |
| 74 |
https://uqfoundation.github.io/project/pathos/query. |
| 75 |
|
| 76 |
|
| 77 |
Major Features |
| 78 |
============== |
| 79 |
|
| 80 |
``dill`` can pickle the following standard types: |
| 81 |
|
| 82 |
- none, type, bool, int, float, complex, bytes, str, |
| 83 |
- tuple, list, dict, file, buffer, builtin, |
| 84 |
- Python classes, namedtuples, dataclasses, metaclasses, |
| 85 |
- instances of classes, |
| 86 |
- set, frozenset, array, functions, exceptions |
| 87 |
|
| 88 |
``dill`` can also pickle more 'exotic' standard types: |
| 89 |
|
| 90 |
- functions with yields, nested functions, lambdas, |
| 91 |
- cell, method, unboundmethod, module, code, methodwrapper, |
| 92 |
- methoddescriptor, getsetdescriptor, memberdescriptor, wrapperdescriptor, |
| 93 |
- dictproxy, slice, notimplemented, ellipsis, quit |
| 94 |
|
| 95 |
``dill`` cannot yet pickle these standard types: |
| 96 |
|
| 97 |
- frame, generator, traceback |
| 98 |
|
| 99 |
``dill`` also provides the capability to: |
| 100 |
|
| 101 |
- save and load Python interpreter sessions |
| 102 |
- save and extract the source code from functions and classes |
| 103 |
- interactively diagnose pickling errors |
| 104 |
|
| 105 |
|
| 106 |
Current Release |
| 107 |
=============== |
| 108 |
|
| 109 |
The latest released version of ``dill`` is available from: |
| 110 |
|
| 111 |
https://pypi.org/project/dill |
| 112 |
|
| 113 |
``dill`` is distributed under a 3-clause BSD license. |
| 114 |
|
| 115 |
|
| 116 |
Development Version |
| 117 |
=================== |
| 118 |
|
| 119 |
You can get the latest development version with all the shiny new features at: |
| 120 |
|
| 121 |
https://github.com/uqfoundation |
| 122 |
|
| 123 |
If you have a new contribution, please submit a pull request. |
| 124 |
|
| 125 |
|
| 126 |
Installation |
| 127 |
============ |
| 128 |
|
| 129 |
``dill`` can be installed with ``pip``:: |
| 130 |
|
| 131 |
$ pip install dill |
| 132 |
|
| 133 |
To optionally include the ``objgraph`` diagnostic tool in the install:: |
| 134 |
|
| 135 |
$ pip install dill[graph] |
| 136 |
|
| 137 |
To optionally include the ``gprof2dot`` diagnostic tool in the install:: |
| 138 |
|
| 139 |
$ pip install dill[profile] |
| 140 |
|
| 141 |
For windows users, to optionally install session history tools:: |
| 142 |
|
| 143 |
$ pip install dill[readline] |
| 144 |
|
| 145 |
|
| 146 |
Requirements |
| 147 |
============ |
| 148 |
|
| 149 |
``dill`` requires: |
| 150 |
|
| 151 |
- ``python`` (or ``pypy``), **>=3.9** |
| 152 |
- ``setuptools``, **>=42** |
| 153 |
|
| 154 |
Optional requirements: |
| 155 |
|
| 156 |
- ``objgraph``, **>=1.7.2** |
| 157 |
- ``gprof2dot``, **>=2022.7.29** |
| 158 |
- ``pyreadline``, **>=1.7.1** (on windows) |
| 159 |
|
| 160 |
|
| 161 |
Basic Usage |
| 162 |
=========== |
| 163 |
|
| 164 |
``dill`` is a drop-in replacement for ``pickle``. Existing code can be |
| 165 |
updated to allow complete pickling using:: |
| 166 |
|
| 167 |
>>> import dill as pickle |
| 168 |
|
| 169 |
or:: |
| 170 |
|
| 171 |
>>> from dill import dumps, loads |
| 172 |
|
| 173 |
``dumps`` converts the object to a unique byte string, and ``loads`` performs |
| 174 |
the inverse operation:: |
| 175 |
|
| 176 |
>>> squared = lambda x: x**2 |
| 177 |
>>> loads(dumps(squared))(3) |
| 178 |
9 |
| 179 |
|
| 180 |
There are a number of options to control serialization which are provided |
| 181 |
as keyword arguments to several ``dill`` functions: |
| 182 |
|
| 183 |
* with *protocol*, the pickle protocol level can be set. This uses the |
| 184 |
same value as the ``pickle`` module, *DEFAULT_PROTOCOL*. |
| 185 |
* with *byref=True*, ``dill`` to behave a lot more like pickle with |
| 186 |
certain objects (like modules) pickled by reference as opposed to |
| 187 |
attempting to pickle the object itself. |
| 188 |
* with *recurse=True*, objects referred to in the global dictionary are |
| 189 |
recursively traced and pickled, instead of the default behavior of |
| 190 |
attempting to store the entire global dictionary. |
| 191 |
* with *fmode*, the contents of the file can be pickled along with the file |
| 192 |
handle, which is useful if the object is being sent over the wire to a |
| 193 |
remote system which does not have the original file on disk. Options are |
| 194 |
*HANDLE_FMODE* for just the handle, *CONTENTS_FMODE* for the file content |
| 195 |
and *FILE_FMODE* for content and handle. |
| 196 |
* with *ignore=False*, objects reconstructed with types defined in the |
| 197 |
top-level script environment use the existing type in the environment |
| 198 |
rather than a possibly different reconstructed type. |
| 199 |
|
| 200 |
The default serialization can also be set globally in *dill.settings*. |
| 201 |
Thus, we can modify how ``dill`` handles references to the global dictionary |
| 202 |
locally or globally:: |
| 203 |
|
| 204 |
>>> import dill.settings |
| 205 |
>>> dumps(absolute) == dumps(absolute, recurse=True) |
| 206 |
False |
| 207 |
>>> dill.settings['recurse'] = True |
| 208 |
>>> dumps(absolute) == dumps(absolute, recurse=True) |
| 209 |
True |
| 210 |
|
| 211 |
``dill`` also includes source code inspection, as an alternate to pickling:: |
| 212 |
|
| 213 |
>>> import dill.source |
| 214 |
>>> print(dill.source.getsource(squared)) |
| 215 |
squared = lambda x:x**2 |
| 216 |
|
| 217 |
To aid in debugging pickling issues, use *dill.detect* which provides |
| 218 |
tools like pickle tracing:: |
| 219 |
|
| 220 |
>>> import dill.detect |
| 221 |
>>> with dill.detect.trace(): |
| 222 |
>>> dumps(squared) |
| 223 |
┬ F1: <function <lambda> at 0x7fe074f8c280> |
| 224 |
├┬ F2: <function _create_function at 0x7fe074c49c10> |
| 225 |
│└ # F2 [34 B] |
| 226 |
├┬ Co: <code object <lambda> at 0x7fe07501eb30, file "<stdin>", line 1> |
| 227 |
│├┬ F2: <function _create_code at 0x7fe074c49ca0> |
| 228 |
││└ # F2 [19 B] |
| 229 |
│└ # Co [87 B] |
| 230 |
├┬ D1: <dict object at 0x7fe0750d4680> |
| 231 |
│└ # D1 [22 B] |
| 232 |
├┬ D2: <dict object at 0x7fe074c5a1c0> |
| 233 |
│└ # D2 [2 B] |
| 234 |
├┬ D2: <dict object at 0x7fe074f903c0> |
| 235 |
│├┬ D2: <dict object at 0x7fe074f8ebc0> |
| 236 |
││└ # D2 [2 B] |
| 237 |
│└ # D2 [23 B] |
| 238 |
└ # F1 [180 B] |
| 239 |
|
| 240 |
With trace, we see how ``dill`` stored the lambda (``F1``) by first storing |
| 241 |
``_create_function``, the underlying code object (``Co``) and ``_create_code`` |
| 242 |
(which is used to handle code objects), then we handle the reference to |
| 243 |
the global dict (``D2``) plus other dictionaries (``D1`` and ``D2``) that |
| 244 |
save the lambda object's state. A ``#`` marks when the object is actually stored. |
| 245 |
|
| 246 |
|
| 247 |
More Information |
| 248 |
================ |
| 249 |
|
| 250 |
Probably the best way to get started is to look at the documentation at |
| 251 |
http://dill.rtfd.io. Also see ``dill.tests`` for a set of scripts that |
| 252 |
demonstrate how ``dill`` can serialize different Python objects. You can |
| 253 |
run the test suite with ``python -m dill.tests``. The contents of any |
| 254 |
pickle file can be examined with ``undill``. As ``dill`` conforms to |
| 255 |
the ``pickle`` interface, the examples and documentation found at |
| 256 |
http://docs.python.org/library/pickle.html also apply to ``dill`` |
| 257 |
if one will ``import dill as pickle``. The source code is also generally |
| 258 |
well documented, so further questions may be resolved by inspecting the |
| 259 |
code itself. Please feel free to submit a ticket on github, or ask a |
| 260 |
question on stackoverflow (**@Mike McKerns**). |
| 261 |
If you would like to share how you use ``dill`` in your work, please send |
| 262 |
an email (to **mmckerns at uqfoundation dot org**). |
| 263 |
|
| 264 |
|
| 265 |
Citation |
| 266 |
======== |
| 267 |
|
| 268 |
If you use ``dill`` to do research that leads to publication, we ask that you |
| 269 |
acknowledge use of ``dill`` by citing the following in your publication:: |
| 270 |
|
| 271 |
M.M. McKerns, L. Strand, T. Sullivan, A. Fang, M.A.G. Aivazis, |
| 272 |
"Building a framework for predictive science", Proceedings of |
| 273 |
the 10th Python in Science Conference, 2011; |
| 274 |
http://arxiv.org/pdf/1202.1056 |
| 275 |
|
| 276 |
Michael McKerns and Michael Aivazis, |
| 277 |
"pathos: a framework for heterogeneous computing", 2010- ; |
| 278 |
https://uqfoundation.github.io/project/pathos |
| 279 |
|
| 280 |
Please see https://uqfoundation.github.io/project/pathos or |
| 281 |
http://arxiv.org/pdf/1202.1056 for further information. |