| 1 |
========================== |
| 2 |
Serializing Django objects |
| 3 |
========================== |
| 4 |
|
| 5 |
Django's serialization framework provides a mechanism for "translating" Django |
| 6 |
models into other formats. Usually these other formats will be text-based and |
| 7 |
used for sending Django data over a wire, but it's possible for a |
| 8 |
serializer to handle any format (text-based or not). |
| 9 |
|
| 10 |
.. seealso:: |
| 11 |
|
| 12 |
If you just want to get some data from your tables into a serialized |
| 13 |
form, you could use the :djadmin:`dumpdata` management command. |
| 14 |
|
| 15 |
Serializing data |
| 16 |
================ |
| 17 |
|
| 18 |
At the highest level, you can serialize data like this:: |
| 19 |
|
| 20 |
from django.core import serializers |
| 21 |
|
| 22 |
data = serializers.serialize("json", SomeModel.objects.all()) |
| 23 |
|
| 24 |
The arguments to the ``serialize`` function are the format to serialize the data |
| 25 |
to (see `Serialization formats`_) and a |
| 26 |
:class:`~django.db.models.query.QuerySet` to serialize. (Actually, the second |
| 27 |
argument can be any iterator that yields Django model instances, but it'll |
| 28 |
almost always be a QuerySet). |
| 29 |
|
| 30 |
.. function:: django.core.serializers.get_serializer(format) |
| 31 |
|
| 32 |
You can also use a serializer object directly:: |
| 33 |
|
| 34 |
JSONSerializer = serializers.get_serializer("json") |
| 35 |
json_serializer = JSONSerializer() |
| 36 |
json_serializer.serialize(queryset) |
| 37 |
data = json_serializer.getvalue() |
| 38 |
|
| 39 |
This is useful if you want to serialize data directly to a file-like object |
| 40 |
(which includes an :class:`~django.http.HttpResponse`):: |
| 41 |
|
| 42 |
with open("file.json", "w") as out: |
| 43 |
json_serializer.serialize(SomeModel.objects.all(), stream=out) |
| 44 |
|
| 45 |
.. note:: |
| 46 |
|
| 47 |
Calling :func:`~django.core.serializers.get_serializer` with an unknown |
| 48 |
:ref:`format <serialization-formats>` will raise a |
| 49 |
``django.core.serializers.SerializerDoesNotExist`` exception. |
| 50 |
|
| 51 |
.. _subset-of-fields: |
| 52 |
|
| 53 |
Subset of fields |
| 54 |
---------------- |
| 55 |
|
| 56 |
If you only want a subset of fields to be serialized, you can |
| 57 |
specify a ``fields`` argument to the serializer:: |
| 58 |
|
| 59 |
from django.core import serializers |
| 60 |
|
| 61 |
data = serializers.serialize("json", SomeModel.objects.all(), fields=["name", "size"]) |
| 62 |
|
| 63 |
In this example, only the ``name`` and ``size`` attributes of each model will |
| 64 |
be serialized. The primary key is always serialized as the ``pk`` element in the |
| 65 |
resulting output; it never appears in the ``fields`` part. |
| 66 |
|
| 67 |
.. note:: |
| 68 |
|
| 69 |
Depending on your model, you may find that it is not possible to |
| 70 |
deserialize a model that only serializes a subset of its fields. If a |
| 71 |
serialized object doesn't specify all the fields that are required by a |
| 72 |
model, the deserializer will not be able to save deserialized instances. |
| 73 |
|
| 74 |
Inherited models |
| 75 |
---------------- |
| 76 |
|
| 77 |
If you have a model that is defined using an :ref:`abstract base class |
| 78 |
<abstract-base-classes>`, you don't have to do anything special to serialize |
| 79 |
that model. Call the serializer on the object (or objects) that you want to |
| 80 |
serialize, and the output will be a complete representation of the serialized |
| 81 |
object. |
| 82 |
|
| 83 |
However, if you have a model that uses :ref:`multi-table inheritance |
| 84 |
<multi-table-inheritance>`, you also need to serialize all of the base classes |
| 85 |
for the model. This is because only the fields that are locally defined on the |
| 86 |
model will be serialized. For example, consider the following models:: |
| 87 |
|
| 88 |
class Place(models.Model): |
| 89 |
name = models.CharField(max_length=50) |
| 90 |
|
| 91 |
|
| 92 |
class Restaurant(Place): |
| 93 |
serves_hot_dogs = models.BooleanField(default=False) |
| 94 |
|
| 95 |
If you only serialize the Restaurant model:: |
| 96 |
|
| 97 |
data = serializers.serialize("json", Restaurant.objects.all()) |
| 98 |
|
| 99 |
the fields on the serialized output will only contain the ``serves_hot_dogs`` |
| 100 |
attribute. The ``name`` attribute of the base class will be ignored. |
| 101 |
|
| 102 |
In order to fully serialize your ``Restaurant`` instances, you will need to |
| 103 |
serialize the ``Place`` models as well:: |
| 104 |
|
| 105 |
all_objects = [*Restaurant.objects.all(), *Place.objects.all()] |
| 106 |
data = serializers.serialize("json", all_objects) |
| 107 |
|
| 108 |
Deserializing data |
| 109 |
================== |
| 110 |
|
| 111 |
Deserializing data is very similar to serializing it:: |
| 112 |
|
| 113 |
for obj in serializers.deserialize("json", data): |
| 114 |
do_something_with(obj) |
| 115 |
|
| 116 |
As you can see, the ``deserialize`` function takes the same format argument as |
| 117 |
``serialize``, a string or stream of data, and returns an iterator. |
| 118 |
|
| 119 |
However, here it gets slightly complicated. The objects returned by the |
| 120 |
``deserialize`` iterator *aren't* regular Django objects. Instead, they are |
| 121 |
special ``DeserializedObject`` instances that wrap a created -- but unsaved -- |
| 122 |
object and any associated relationship data. |
| 123 |
|
| 124 |
Calling ``DeserializedObject.save()`` saves the object to the database. |
| 125 |
|
| 126 |
.. note:: |
| 127 |
|
| 128 |
If the ``pk`` attribute in the serialized data doesn't exist or is |
| 129 |
null, a new instance will be saved to the database. |
| 130 |
|
| 131 |
This ensures that deserializing is a non-destructive operation even if the |
| 132 |
data in your serialized representation doesn't match what's currently in the |
| 133 |
database. Usually, working with these ``DeserializedObject`` instances looks |
| 134 |
something like:: |
| 135 |
|
| 136 |
for deserialized_object in serializers.deserialize("json", data): |
| 137 |
if object_should_be_saved(deserialized_object): |
| 138 |
deserialized_object.save() |
| 139 |
|
| 140 |
In other words, the usual use is to examine the deserialized objects to make |
| 141 |
sure that they are "appropriate" for saving before doing so. If you trust your |
| 142 |
data source you can instead save the object directly and move on. |
| 143 |
|
| 144 |
The Django object itself can be inspected as ``deserialized_object.object``. |
| 145 |
If fields in the serialized data do not exist on a model, a |
| 146 |
``DeserializationError`` will be raised unless the ``ignorenonexistent`` |
| 147 |
argument is passed in as ``True``:: |
| 148 |
|
| 149 |
serializers.deserialize("json", data, ignorenonexistent=True) |
| 150 |
|
| 151 |
.. _serialization-formats: |
| 152 |
|
| 153 |
Serialization formats |
| 154 |
===================== |
| 155 |
|
| 156 |
Django supports a number of serialization formats, some of which require you |
| 157 |
to install third-party Python modules: |
| 158 |
|
| 159 |
========== ============================================================== |
| 160 |
Identifier Information |
| 161 |
========== ============================================================== |
| 162 |
``xml`` Serializes to and from a simple XML dialect. |
| 163 |
|
| 164 |
``json`` Serializes to and from JSON_. |
| 165 |
|
| 166 |
``jsonl`` Serializes to and from JSONL_. |
| 167 |
|
| 168 |
``yaml`` Serializes to YAML (YAML Ain't a Markup Language). This |
| 169 |
serializer is only available if PyYAML_ is installed. |
| 170 |
========== ============================================================== |
| 171 |
|
| 172 |
.. _json: https://json.org/ |
| 173 |
.. _jsonl: https://jsonlines.org/ |
| 174 |
.. _PyYAML: https://pyyaml.org/ |
| 175 |
|
| 176 |
.. _serialization-formats-xml: |
| 177 |
|
| 178 |
XML |
| 179 |
--- |
| 180 |
|
| 181 |
The basic XML serialization format looks like this: |
| 182 |
|
| 183 |
.. code-block:: xml |
| 184 |
|
| 185 |
<?xml version="1.0" encoding="utf-8"?> |
| 186 |
<django-objects version="1.0"> |
| 187 |
<object pk="123" model="sessions.session"> |
| 188 |
<field type="DateTimeField" name="expire_date">2013-01-16T08:16:59.844560+00:00</field> |
| 189 |
<!-- ... --> |
| 190 |
</object> |
| 191 |
</django-objects> |
| 192 |
|
| 193 |
The whole collection of objects that is either serialized or deserialized is |
| 194 |
represented by a ``<django-objects>``-tag which contains multiple |
| 195 |
``<object>``-elements. Each such object has two attributes: "pk" and "model", |
| 196 |
the latter being represented by the name of the app ("sessions") and the |
| 197 |
lowercase name of the model ("session") separated by a dot. |
| 198 |
|
| 199 |
Each field of the object is serialized as a ``<field>``-element sporting the |
| 200 |
fields "type" and "name". The text content of the element represents the value |
| 201 |
that should be stored. |
| 202 |
|
| 203 |
Foreign keys and other relational fields are treated a little bit differently: |
| 204 |
|
| 205 |
.. code-block:: xml |
| 206 |
|
| 207 |
<object pk="27" model="auth.permission"> |
| 208 |
<!-- ... --> |
| 209 |
<field to="contenttypes.contenttype" name="content_type" rel="ManyToOneRel">9</field> |
| 210 |
<!-- ... --> |
| 211 |
</object> |
| 212 |
|
| 213 |
In this example we specify that the ``auth.Permission`` object with the PK 27 |
| 214 |
has a foreign key to the ``contenttypes.ContentType`` instance with the PK 9. |
| 215 |
|
| 216 |
ManyToMany-relations are exported for the model that binds them. For instance, |
| 217 |
the ``auth.User`` model has such a relation to the ``auth.Permission`` model: |
| 218 |
|
| 219 |
.. code-block:: xml |
| 220 |
|
| 221 |
<object pk="1" model="auth.user"> |
| 222 |
<!-- ... --> |
| 223 |
<field to="auth.permission" name="user_permissions" rel="ManyToManyRel"> |
| 224 |
<object pk="46"></object> |
| 225 |
<object pk="47"></object> |
| 226 |
</field> |
| 227 |
</object> |
| 228 |
|
| 229 |
This example links the given user with the permission models with PKs 46 and 47. |
| 230 |
|
| 231 |
.. admonition:: Control characters |
| 232 |
|
| 233 |
If the content to be serialized contains control characters that are not |
| 234 |
accepted in the XML 1.0 standard, the serialization will fail with a |
| 235 |
:exc:`ValueError` exception. Read also the W3C's explanation of `HTML, |
| 236 |
XHTML, XML and Control Codes |
| 237 |
<https://www.w3.org/International/questions/qa-controls>`_. |
| 238 |
|
| 239 |
.. _serialization-formats-json: |
| 240 |
|
| 241 |
JSON |
| 242 |
---- |
| 243 |
|
| 244 |
When staying with the same example data as before it would be serialized as |
| 245 |
JSON in the following way:: |
| 246 |
|
| 247 |
[ |
| 248 |
{ |
| 249 |
"pk": "4b678b301dfd8a4e0dad910de3ae245b", |
| 250 |
"model": "sessions.session", |
| 251 |
"fields": { |
| 252 |
"expire_date": "2013-01-16T08:16:59.844Z", |
| 253 |
# ... |
| 254 |
}, |
| 255 |
} |
| 256 |
] |
| 257 |
|
| 258 |
The formatting here is a bit simpler than with XML. The whole collection |
| 259 |
is just represented as an array and the objects are represented by JSON objects |
| 260 |
with three properties: "pk", "model" and "fields". "fields" is again an object |
| 261 |
containing each field's name and value as property and property-value |
| 262 |
respectively. |
| 263 |
|
| 264 |
Foreign keys have the PK of the linked object as property value. |
| 265 |
ManyToMany-relations are serialized for the model that defines them and are |
| 266 |
represented as a list of PKs. |
| 267 |
|
| 268 |
Be aware that not all Django output can be passed unmodified to :mod:`json`. |
| 269 |
For example, if you have some custom type in an object to be serialized, you'll |
| 270 |
have to write a custom :mod:`json` encoder for it. Something like this will |
| 271 |
work:: |
| 272 |
|
| 273 |
from django.core.serializers.json import DjangoJSONEncoder |
| 274 |
|
| 275 |
|
| 276 |
class LazyEncoder(DjangoJSONEncoder): |
| 277 |
def default(self, obj): |
| 278 |
if isinstance(obj, YourCustomType): |
| 279 |
return str(obj) |
| 280 |
return super().default(obj) |
| 281 |
|
| 282 |
You can then pass ``cls=LazyEncoder`` to the ``serializers.serialize()`` |
| 283 |
function:: |
| 284 |
|
| 285 |
from django.core.serializers import serialize |
| 286 |
|
| 287 |
serialize("json", SomeModel.objects.all(), cls=LazyEncoder) |
| 288 |
|
| 289 |
Also note that GeoDjango provides a :doc:`customized GeoJSON serializer |
| 290 |
</ref/contrib/gis/serializers>`. |
| 291 |
|
| 292 |
``DjangoJSONEncoder`` |
| 293 |
~~~~~~~~~~~~~~~~~~~~~ |
| 294 |
|
| 295 |
.. class:: django.core.serializers.json.DjangoJSONEncoder |
| 296 |
|
| 297 |
The JSON serializer uses ``DjangoJSONEncoder`` for encoding. A subclass of |
| 298 |
:class:`~json.JSONEncoder`, it handles these additional types: |
| 299 |
|
| 300 |
:class:`~datetime.datetime` |
| 301 |
A string of the form ``YYYY-MM-DDTHH:mm:ss.sssZ`` or |
| 302 |
``YYYY-MM-DDTHH:mm:ss.sss+HH:MM`` as defined in `ECMA-262`_. |
| 303 |
|
| 304 |
:class:`~datetime.date` |
| 305 |
A string of the form ``YYYY-MM-DD`` as defined in `ECMA-262`_. |
| 306 |
|
| 307 |
:class:`~datetime.time` |
| 308 |
A string of the form ``HH:MM:ss.sss`` as defined in `ECMA-262`_. |
| 309 |
|
| 310 |
:class:`~datetime.timedelta` |
| 311 |
A string representing a duration as defined in ISO-8601. For example, |
| 312 |
``timedelta(days=1, hours=2, seconds=3.4)`` is represented as |
| 313 |
``'P1DT02H00M03.400000S'``. |
| 314 |
|
| 315 |
:class:`~decimal.Decimal`, ``Promise`` (``django.utils.functional.lazy()`` objects), :class:`~uuid.UUID` |
| 316 |
A string representation of the object. |
| 317 |
|
| 318 |
.. _ecma-262: https://262.ecma-international.org/5.1/#sec-15.9.1.15 |
| 319 |
|
| 320 |
.. _serialization-formats-jsonl: |
| 321 |
|
| 322 |
JSONL |
| 323 |
----- |
| 324 |
|
| 325 |
*JSONL* stands for *JSON Lines*. With this format, objects are separated by new |
| 326 |
lines, and each line contains a valid JSON object. JSONL serialized data looks |
| 327 |
like this:: |
| 328 |
|
| 329 |
{"pk": "4b678b301dfd8a4e0dad910de3ae245b", "model": "sessions.session", "fields": {...}} |
| 330 |
{"pk": "88bea72c02274f3c9bf1cb2bb8cee4fc", "model": "sessions.session", "fields": {...}} |
| 331 |
{"pk": "9cf0e26691b64147a67e2a9f06ad7a53", "model": "sessions.session", "fields": {...}} |
| 332 |
|
| 333 |
JSONL can be useful for populating large databases, since the data can be |
| 334 |
processed line by line, rather than being loaded into memory all at once. |
| 335 |
|
| 336 |
YAML |
| 337 |
---- |
| 338 |
|
| 339 |
YAML serialization looks quite similar to JSON. The object list is serialized |
| 340 |
as a sequence mappings with the keys "pk", "model" and "fields". Each field is |
| 341 |
again a mapping with the key being name of the field and the value the value: |
| 342 |
|
| 343 |
.. code-block:: yaml |
| 344 |
|
| 345 |
- model: sessions.session |
| 346 |
pk: 4b678b301dfd8a4e0dad910de3ae245b |
| 347 |
fields: |
| 348 |
expire_date: 2013-01-16 08:16:59.844560+00:00 |
| 349 |
|
| 350 |
Referential fields are again represented by the PK or sequence of PKs. |
| 351 |
|
| 352 |
.. _custom-serialization-formats: |
| 353 |
|
| 354 |
Custom serialization formats |
| 355 |
---------------------------- |
| 356 |
|
| 357 |
In addition to the default formats, you can create a custom serialization |
| 358 |
format. |
| 359 |
|
| 360 |
For example, let’s consider a csv serializer and deserializer. First, define a |
| 361 |
``Serializer`` and a ``Deserializer`` class. These can override existing |
| 362 |
serialization format classes: |
| 363 |
|
| 364 |
.. code-block:: python |
| 365 |
:caption: ``path/to/custom_csv_serializer.py`` |
| 366 |
|
| 367 |
import csv |
| 368 |
|
| 369 |
from django.apps import apps |
| 370 |
from django.core import serializers |
| 371 |
from django.core.serializers.base import DeserializationError |
| 372 |
|
| 373 |
|
| 374 |
class Serializer(serializers.python.Serializer): |
| 375 |
def get_dump_object(self, obj): |
| 376 |
dumped_object = super().get_dump_object(obj) |
| 377 |
row = [dumped_object["model"], str(dumped_object["pk"])] |
| 378 |
row += [str(value) for value in dumped_object["fields"].values()] |
| 379 |
return ",".join(row), dumped_object["model"] |
| 380 |
|
| 381 |
def end_object(self, obj): |
| 382 |
dumped_object_str, model = self.get_dump_object(obj) |
| 383 |
if self.first: |
| 384 |
fields = [field.name for field in apps.get_model(model)._meta.fields] |
| 385 |
header = ",".join(fields) |
| 386 |
self.stream.write(f"model,{header}\n") |
| 387 |
self.stream.write(f"{dumped_object_str}\n") |
| 388 |
|
| 389 |
def getvalue(self): |
| 390 |
return super(serializers.python.Serializer, self).getvalue() |
| 391 |
|
| 392 |
|
| 393 |
class Deserializer(serializers.python.Deserializer): |
| 394 |
def __init__(self, stream_or_string, **options): |
| 395 |
if isinstance(stream_or_string, bytes): |
| 396 |
stream_or_string = stream_or_string.decode() |
| 397 |
if isinstance(stream_or_string, str): |
| 398 |
stream_or_string = stream_or_string.splitlines() |
| 399 |
try: |
| 400 |
objects = csv.DictReader(stream_or_string) |
| 401 |
except Exception as exc: |
| 402 |
raise DeserializationError() from exc |
| 403 |
super().__init__(objects, **options) |
| 404 |
|
| 405 |
def _handle_object(self, obj): |
| 406 |
try: |
| 407 |
model_fields = apps.get_model(obj["model"])._meta.fields |
| 408 |
obj["fields"] = { |
| 409 |
field.name: obj[field.name] |
| 410 |
for field in model_fields |
| 411 |
if field.name in obj |
| 412 |
} |
| 413 |
yield from super()._handle_object(obj) |
| 414 |
except (GeneratorExit, DeserializationError): |
| 415 |
raise |
| 416 |
except Exception as exc: |
| 417 |
raise DeserializationError(f"Error deserializing object: {exc}") from exc |
| 418 |
|
| 419 |
Then add the module containing the serializer definitions to your |
| 420 |
:setting:`SERIALIZATION_MODULES` setting:: |
| 421 |
|
| 422 |
SERIALIZATION_MODULES = { |
| 423 |
"csv": "path.to.custom_csv_serializer", |
| 424 |
"json": "django.core.serializers.json", |
| 425 |
} |
| 426 |
|
| 427 |
.. versionchanged:: 5.2 |
| 428 |
|
| 429 |
A ``Deserializer`` class definition was added to each of the provided |
| 430 |
serialization formats. |
| 431 |
|
| 432 |
.. _topics-serialization-natural-keys: |
| 433 |
|
| 434 |
Natural keys |
| 435 |
============ |
| 436 |
|
| 437 |
The default serialization strategy for foreign keys and many-to-many relations |
| 438 |
is to serialize the value of the primary key(s) of the objects in the relation. |
| 439 |
This strategy works well for most objects, but it can cause difficulty in some |
| 440 |
circumstances. |
| 441 |
|
| 442 |
Consider the case of a list of objects that have a foreign key referencing |
| 443 |
:class:`~django.contrib.contenttypes.models.ContentType`. If you're going to |
| 444 |
serialize an object that refers to a content type, then you need to have a way |
| 445 |
to refer to that content type to begin with. Since ``ContentType`` objects are |
| 446 |
automatically created by Django during the database synchronization process, |
| 447 |
the primary key of a given content type isn't easy to predict; it will |
| 448 |
depend on how and when :djadmin:`migrate` was executed. This is true for all |
| 449 |
models which automatically generate objects, notably including |
| 450 |
:class:`~django.contrib.auth.models.Permission`, |
| 451 |
:class:`~django.contrib.auth.models.Group`, and |
| 452 |
:class:`~django.contrib.auth.models.User`. |
| 453 |
|
| 454 |
.. warning:: |
| 455 |
|
| 456 |
You should never include automatically generated objects in a fixture or |
| 457 |
other serialized data. By chance, the primary keys in the fixture |
| 458 |
may match those in the database and loading the fixture will |
| 459 |
have no effect. In the more likely case that they don't match, the fixture |
| 460 |
loading will fail with an :class:`~django.db.IntegrityError`. |
| 461 |
|
| 462 |
There is also the matter of convenience. An integer id isn't always |
| 463 |
the most convenient way to refer to an object; sometimes, a |
| 464 |
more natural reference would be helpful. |
| 465 |
|
| 466 |
It is for these reasons that Django provides *natural keys*. A natural |
| 467 |
key is a tuple of values that can be used to uniquely identify an |
| 468 |
object instance without using the primary key value. |
| 469 |
|
| 470 |
Deserialization of natural keys |
| 471 |
------------------------------- |
| 472 |
|
| 473 |
Consider the following two models:: |
| 474 |
|
| 475 |
from django.db import models |
| 476 |
|
| 477 |
|
| 478 |
class Person(models.Model): |
| 479 |
first_name = models.CharField(max_length=100) |
| 480 |
last_name = models.CharField(max_length=100) |
| 481 |
|
| 482 |
birthdate = models.DateField() |
| 483 |
|
| 484 |
class Meta: |
| 485 |
constraints = [ |
| 486 |
models.UniqueConstraint( |
| 487 |
fields=["first_name", "last_name"], |
| 488 |
name="unique_first_last_name", |
| 489 |
), |
| 490 |
] |
| 491 |
|
| 492 |
|
| 493 |
class Book(models.Model): |
| 494 |
name = models.CharField(max_length=100) |
| 495 |
author = models.ForeignKey(Person, on_delete=models.CASCADE) |
| 496 |
|
| 497 |
Ordinarily, serialized data for ``Book`` would use an integer to refer to |
| 498 |
the author. For example, in JSON, a Book might be serialized as:: |
| 499 |
|
| 500 |
... |
| 501 |
{"pk": 1, "model": "store.book", "fields": {"name": "Mostly Harmless", "author": 42}} |
| 502 |
... |
| 503 |
|
| 504 |
This isn't a particularly natural way to refer to an author. It |
| 505 |
requires that you know the primary key value for the author; it also |
| 506 |
requires that this primary key value is stable and predictable. |
| 507 |
|
| 508 |
However, if we add natural key handling to Person, the fixture becomes |
| 509 |
much more humane. To add natural key handling, you define a default |
| 510 |
Manager for Person with a ``get_by_natural_key()`` method. In the case |
| 511 |
of a Person, a good natural key might be the pair of first and last |
| 512 |
name:: |
| 513 |
|
| 514 |
from django.db import models |
| 515 |
|
| 516 |
|
| 517 |
class PersonManager(models.Manager): |
| 518 |
def get_by_natural_key(self, first_name, last_name): |
| 519 |
return self.get(first_name=first_name, last_name=last_name) |
| 520 |
|
| 521 |
|
| 522 |
class Person(models.Model): |
| 523 |
first_name = models.CharField(max_length=100) |
| 524 |
last_name = models.CharField(max_length=100) |
| 525 |
birthdate = models.DateField() |
| 526 |
|
| 527 |
objects = PersonManager() |
| 528 |
|
| 529 |
class Meta: |
| 530 |
constraints = [ |
| 531 |
models.UniqueConstraint( |
| 532 |
fields=["first_name", "last_name"], |
| 533 |
name="unique_first_last_name", |
| 534 |
), |
| 535 |
] |
| 536 |
|
| 537 |
Now books can use that natural key to refer to ``Person`` objects:: |
| 538 |
|
| 539 |
... |
| 540 |
{ |
| 541 |
"pk": 1, |
| 542 |
"model": "store.book", |
| 543 |
"fields": {"name": "Mostly Harmless", "author": ["Douglas", "Adams"]}, |
| 544 |
} |
| 545 |
... |
| 546 |
|
| 547 |
When you try to load this serialized data, Django will use the |
| 548 |
``get_by_natural_key()`` method to resolve ``["Douglas", "Adams"]`` |
| 549 |
into the primary key of an actual ``Person`` object. |
| 550 |
|
| 551 |
.. note:: |
| 552 |
|
| 553 |
Whatever fields you use for a natural key must be able to uniquely |
| 554 |
identify an object. This will usually mean that your model will |
| 555 |
have a uniqueness clause (either ``unique=True`` on a single field, or a |
| 556 |
``UniqueConstraint`` or ``unique_together`` over multiple fields) for the |
| 557 |
field or fields in your natural key. However, uniqueness doesn't need to be |
| 558 |
enforced at the database level. If you are certain that a set of fields |
| 559 |
will be effectively unique, you can still use those fields as a natural |
| 560 |
key. |
| 561 |
|
| 562 |
Deserialization of objects with no primary key will always check whether the |
| 563 |
model's manager has a ``get_by_natural_key()`` method and if so, use it to |
| 564 |
populate the deserialized object's primary key. |
| 565 |
|
| 566 |
Serialization of natural keys |
| 567 |
----------------------------- |
| 568 |
|
| 569 |
So how do you get Django to emit a natural key when serializing an object? |
| 570 |
Firstly, you need to add another method -- this time to the model itself:: |
| 571 |
|
| 572 |
class Person(models.Model): |
| 573 |
first_name = models.CharField(max_length=100) |
| 574 |
last_name = models.CharField(max_length=100) |
| 575 |
birthdate = models.DateField() |
| 576 |
|
| 577 |
objects = PersonManager() |
| 578 |
|
| 579 |
class Meta: |
| 580 |
constraints = [ |
| 581 |
models.UniqueConstraint( |
| 582 |
fields=["first_name", "last_name"], |
| 583 |
name="unique_first_last_name", |
| 584 |
), |
| 585 |
] |
| 586 |
|
| 587 |
def natural_key(self): |
| 588 |
return (self.first_name, self.last_name) |
| 589 |
|
| 590 |
That method should always return a natural key tuple -- in this |
| 591 |
example, ``(first name, last name)``. Then, when you call |
| 592 |
``serializers.serialize()``, you provide ``use_natural_foreign_keys=True`` |
| 593 |
or ``use_natural_primary_keys=True`` arguments: |
| 594 |
|
| 595 |
.. code-block:: pycon |
| 596 |
|
| 597 |
>>> serializers.serialize( |
| 598 |
... "json", |
| 599 |
... [book1, book2], |
| 600 |
... indent=2, |
| 601 |
... use_natural_foreign_keys=True, |
| 602 |
... use_natural_primary_keys=True, |
| 603 |
... ) |
| 604 |
|
| 605 |
When ``use_natural_foreign_keys=True`` is specified, Django will use the |
| 606 |
``natural_key()`` method to serialize any foreign key reference to objects |
| 607 |
of the type that defines the method. |
| 608 |
|
| 609 |
When ``use_natural_primary_keys=True`` is specified, Django will not provide the |
| 610 |
primary key in the serialized data of this object since it can be calculated |
| 611 |
during deserialization:: |
| 612 |
|
| 613 |
... |
| 614 |
{ |
| 615 |
"model": "store.person", |
| 616 |
"fields": { |
| 617 |
"first_name": "Douglas", |
| 618 |
"last_name": "Adams", |
| 619 |
"birth_date": "1952-03-11", |
| 620 |
}, |
| 621 |
} |
| 622 |
... |
| 623 |
|
| 624 |
This can be useful when you need to load serialized data into an existing |
| 625 |
database and you cannot guarantee that the serialized primary key value is not |
| 626 |
already in use, and do not need to ensure that deserialized objects retain the |
| 627 |
same primary keys. |
| 628 |
|
| 629 |
If you are using :djadmin:`dumpdata` to generate serialized data, use the |
| 630 |
:option:`dumpdata --natural-foreign` and :option:`dumpdata --natural-primary` |
| 631 |
command line flags to generate natural keys. |
| 632 |
|
| 633 |
.. note:: |
| 634 |
|
| 635 |
You don't need to define both ``natural_key()`` and |
| 636 |
``get_by_natural_key()``. If you don't want Django to output |
| 637 |
natural keys during serialization, but you want to retain the |
| 638 |
ability to load natural keys, then you can opt to not implement |
| 639 |
the ``natural_key()`` method. |
| 640 |
|
| 641 |
Conversely, if (for some strange reason) you want Django to output |
| 642 |
natural keys during serialization, but *not* be able to load those |
| 643 |
key values, just don't define the ``get_by_natural_key()`` method. |
| 644 |
|
| 645 |
.. _natural-keys-and-forward-references: |
| 646 |
|
| 647 |
Natural keys and forward references |
| 648 |
----------------------------------- |
| 649 |
|
| 650 |
Sometimes when you use :ref:`natural foreign keys |
| 651 |
<topics-serialization-natural-keys>` you'll need to deserialize data where |
| 652 |
an object has a foreign key referencing another object that hasn't yet been |
| 653 |
deserialized. This is called a "forward reference". |
| 654 |
|
| 655 |
For instance, suppose you have the following objects in your fixture:: |
| 656 |
|
| 657 |
... |
| 658 |
{ |
| 659 |
"model": "store.book", |
| 660 |
"fields": {"name": "Mostly Harmless", "author": ["Douglas", "Adams"]}, |
| 661 |
}, |
| 662 |
... |
| 663 |
{"model": "store.person", "fields": {"first_name": "Douglas", "last_name": "Adams"}}, |
| 664 |
... |
| 665 |
|
| 666 |
In order to handle this situation, you need to pass |
| 667 |
``handle_forward_references=True`` to ``serializers.deserialize()``. This will |
| 668 |
set the ``deferred_fields`` attribute on the ``DeserializedObject`` instances. |
| 669 |
You'll need to keep track of ``DeserializedObject`` instances where this |
| 670 |
attribute isn't ``None`` and later call ``save_deferred_fields()`` on them. |
| 671 |
|
| 672 |
Typical usage looks like this:: |
| 673 |
|
| 674 |
objs_with_deferred_fields = [] |
| 675 |
|
| 676 |
for obj in serializers.deserialize("json", data, handle_forward_references=True): |
| 677 |
obj.save() |
| 678 |
if obj.deferred_fields is not None: |
| 679 |
objs_with_deferred_fields.append(obj) |
| 680 |
|
| 681 |
for obj in objs_with_deferred_fields: |
| 682 |
obj.save_deferred_fields() |
| 683 |
|
| 684 |
For this to work, the ``ForeignKey`` on the referencing model must have |
| 685 |
``null=True``. |
| 686 |
|
| 687 |
Dependencies during serialization |
| 688 |
--------------------------------- |
| 689 |
|
| 690 |
It's often possible to avoid explicitly having to handle forward references by |
| 691 |
taking care with the ordering of objects within a fixture. |
| 692 |
|
| 693 |
To help with this, calls to :djadmin:`dumpdata` that use the :option:`dumpdata |
| 694 |
--natural-foreign` option will serialize any model with a ``natural_key()`` |
| 695 |
method before serializing standard primary key objects. |
| 696 |
|
| 697 |
However, this may not always be enough. If your natural key refers to |
| 698 |
another object (by using a foreign key or natural key to another object |
| 699 |
as part of a natural key), then you need to be able to ensure that |
| 700 |
the objects on which a natural key depends occur in the serialized data |
| 701 |
before the natural key requires them. |
| 702 |
|
| 703 |
To control this ordering, you can define dependencies on your |
| 704 |
``natural_key()`` methods. You do this by setting a ``dependencies`` |
| 705 |
attribute on the ``natural_key()`` method itself. |
| 706 |
|
| 707 |
For example, let's add a natural key to the ``Book`` model from the |
| 708 |
example above:: |
| 709 |
|
| 710 |
class Book(models.Model): |
| 711 |
name = models.CharField(max_length=100) |
| 712 |
author = models.ForeignKey(Person, on_delete=models.CASCADE) |
| 713 |
|
| 714 |
def natural_key(self): |
| 715 |
return (self.name,) + self.author.natural_key() |
| 716 |
|
| 717 |
The natural key for a ``Book`` is a combination of its name and its |
| 718 |
author. This means that ``Person`` must be serialized before ``Book``. |
| 719 |
To define this dependency, we add one extra line:: |
| 720 |
|
| 721 |
def natural_key(self): |
| 722 |
return (self.name,) + self.author.natural_key() |
| 723 |
|
| 724 |
|
| 725 |
natural_key.dependencies = ["example_app.person"] |
| 726 |
|
| 727 |
This definition ensures that all ``Person`` objects are serialized before |
| 728 |
any ``Book`` objects. In turn, any object referencing ``Book`` will be |
| 729 |
serialized after both ``Person`` and ``Book`` have been serialized. |