Debian Codegraph @

docs/howto/writing-migrations.txt

raw
1 =================================
2 How to create database migrations
3 =================================
4
5 This document explains how to structure and write database migrations for
6 different scenarios you might encounter. For introductory material on
7 migrations, see :doc:`the topic guide </topics/migrations>`.
8
9 .. _data-migrations-and-multiple-databases:
10
11 Data migrations and multiple databases
12 ======================================
13
14 When using multiple databases, you may need to figure out whether or not to
15 run a migration against a particular database. For example, you may want to
16 **only** run a migration on a particular database.
17
18 In order to do that you can check the database connection's alias inside a
19 ``RunPython`` operation by looking at the ``schema_editor.connection.alias``
20 attribute::
21
22 from django.db import migrations
23
24
25 def forwards(apps, schema_editor):
26 if schema_editor.connection.alias != "default":
27 return
28 # Your migration code goes here
29
30
31 class Migration(migrations.Migration):
32 dependencies = [
33 # Dependencies to other migrations
34 ]
35
36 operations = [
37 migrations.RunPython(forwards),
38 ]
39
40 You can also provide hints that will be passed to the :meth:`allow_migrate()`
41 method of database routers as ``**hints``:
42
43 .. code-block:: python
44 :caption: ``myapp/dbrouters.py``
45
46 class MyRouter:
47 def allow_migrate(self, db, app_label, model_name=None, **hints):
48 if "target_db" in hints:
49 return db == hints["target_db"]
50 return True
51
52 Then, to leverage this in your migrations, do the following::
53
54 from django.db import migrations
55
56
57 def forwards(apps, schema_editor):
58 # Your migration code goes here
59 ...
60
61
62 class Migration(migrations.Migration):
63 dependencies = [
64 # Dependencies to other migrations
65 ]
66
67 operations = [
68 migrations.RunPython(forwards, hints={"target_db": "default"}),
69 ]
70
71 If your ``RunPython`` or ``RunSQL`` operation only affects one model, it's good
72 practice to pass ``model_name`` as a hint to make it as transparent as possible
73 to the router. This is especially important for reusable and third-party apps.
74
75 Migrations that add unique fields
76 =================================
77
78 Applying a "plain" migration that adds a unique non-nullable field to a table
79 with existing rows will raise an error because the value used to populate
80 existing rows is generated only once, thus breaking the unique constraint.
81
82 Therefore, the following steps should be taken. In this example, we'll add a
83 non-nullable :class:`~django.db.models.UUIDField` with a default value. Modify
84 the respective field according to your needs.
85
86 * Add the field on your model with ``default=uuid.uuid4`` and ``unique=True``
87 arguments (choose an appropriate default for the type of the field you're
88 adding).
89
90 * Run the :djadmin:`makemigrations` command. This should generate a migration
91 with an ``AddField`` operation.
92
93 * Generate two empty migration files for the same app by running
94 ``makemigrations myapp --empty`` twice. We've renamed the migration files to
95 give them meaningful names in the examples below.
96
97 * Copy the ``AddField`` operation from the auto-generated migration (the first
98 of the three new files) to the last migration, change ``AddField`` to
99 ``AlterField``, and add imports of ``uuid`` and ``models``. For example:
100
101 .. code-block:: python
102 :caption: ``0006_remove_uuid_null.py``
103
104 # Generated by Django A.B on YYYY-MM-DD HH:MM
105 from django.db import migrations, models
106 import uuid
107
108
109 class Migration(migrations.Migration):
110 dependencies = [
111 ("myapp", "0005_populate_uuid_values"),
112 ]
113
114 operations = [
115 migrations.AlterField(
116 model_name="mymodel",
117 name="uuid",
118 field=models.UUIDField(default=uuid.uuid4, unique=True),
119 ),
120 ]
121
122 * Edit the first migration file. The generated migration class should look
123 similar to this:
124
125 .. code-block:: python
126 :caption: ``0004_add_uuid_field.py``
127
128 class Migration(migrations.Migration):
129 dependencies = [
130 ("myapp", "0003_auto_20150129_1705"),
131 ]
132
133 operations = [
134 migrations.AddField(
135 model_name="mymodel",
136 name="uuid",
137 field=models.UUIDField(default=uuid.uuid4, unique=True),
138 ),
139 ]
140
141 Change ``unique=True`` to ``null=True`` -- this will create the intermediary
142 null field and defer creating the unique constraint until we've populated
143 unique values on all the rows.
144
145 * In the first empty migration file, add a
146 :class:`~django.db.migrations.operations.RunPython` or
147 :class:`~django.db.migrations.operations.RunSQL` operation to generate a
148 unique value (UUID in the example) for each existing row. Also add an import
149 of ``uuid``. For example:
150
151 .. code-block:: python
152 :caption: ``0005_populate_uuid_values.py``
153
154 # Generated by Django A.B on YYYY-MM-DD HH:MM
155 from django.db import migrations
156 import uuid
157
158
159 def gen_uuid(apps, schema_editor):
160 MyModel = apps.get_model("myapp", "MyModel")
161 for row in MyModel.objects.all():
162 row.uuid = uuid.uuid4()
163 row.save(update_fields=["uuid"])
164
165
166 class Migration(migrations.Migration):
167 dependencies = [
168 ("myapp", "0004_add_uuid_field"),
169 ]
170
171 operations = [
172 # omit reverse_code=... if you don't want the migration to be reversible.
173 migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop),
174 ]
175
176 * Now you can apply the migrations as usual with the :djadmin:`migrate` command.
177
178 Note there is a race condition if you allow objects to be created while this
179 migration is running. Objects created after the ``AddField`` and before
180 ``RunPython`` will have their original ``uuid``’s overwritten.
181
182 .. _non-atomic-migrations:
183
184 Non-atomic migrations
185 ~~~~~~~~~~~~~~~~~~~~~
186
187 On databases that support DDL transactions (SQLite and PostgreSQL), migrations
188 will run inside a transaction by default. For use cases such as performing data
189 migrations on large tables, you may want to prevent a migration from running in
190 a transaction by setting the ``atomic`` attribute to ``False``::
191
192 from django.db import migrations
193
194
195 class Migration(migrations.Migration):
196 atomic = False
197
198 Within such a migration, all operations are run without a transaction. It's
199 possible to execute parts of the migration inside a transaction using
200 :func:`~django.db.transaction.atomic()` or by passing ``atomic=True`` to
201 ``RunPython``.
202
203 Here's an example of a non-atomic data migration that updates a large table in
204 smaller batches::
205
206 import uuid
207
208 from django.db import migrations, transaction
209
210
211 def gen_uuid(apps, schema_editor):
212 MyModel = apps.get_model("myapp", "MyModel")
213 while MyModel.objects.filter(uuid__isnull=True).exists():
214 with transaction.atomic():
215 for row in MyModel.objects.filter(uuid__isnull=True)[:1000]:
216 row.uuid = uuid.uuid4()
217 row.save()
218
219
220 class Migration(migrations.Migration):
221 atomic = False
222
223 operations = [
224 migrations.RunPython(gen_uuid),
225 ]
226
227 The ``atomic`` attribute doesn't have an effect on databases that don't support
228 DDL transactions (e.g. MySQL, Oracle). (MySQL's `atomic DDL statement support
229 <https://dev.mysql.com/doc/refman/en/atomic-ddl.html>`_ refers to individual
230 statements rather than multiple statements wrapped in a transaction that can be
231 rolled back.)
232
233 Controlling the order of migrations
234 ===================================
235
236 Django determines the order in which migrations should be applied not by the
237 filename of each migration, but by building a graph using two properties on the
238 ``Migration`` class: ``dependencies`` and ``run_before``.
239
240 If you've used the :djadmin:`makemigrations` command you've probably
241 already seen ``dependencies`` in action because auto-created
242 migrations have this defined as part of their creation process.
243
244 The ``dependencies`` property is declared like this::
245
246 from django.db import migrations
247
248
249 class Migration(migrations.Migration):
250 dependencies = [
251 ("myapp", "0123_the_previous_migration"),
252 ]
253
254 Usually this will be enough, but from time to time you may need to
255 ensure that your migration runs *before* other migrations. This is
256 useful, for example, to make third-party apps' migrations run *after*
257 your :setting:`AUTH_USER_MODEL` replacement.
258
259 To achieve this, place all migrations that should depend on yours in
260 the ``run_before`` attribute on your ``Migration`` class::
261
262 class Migration(migrations.Migration):
263 ...
264
265 run_before = [
266 ("third_party_app", "0001_do_awesome"),
267 ]
268
269 Prefer using ``dependencies`` over ``run_before`` when possible. You should
270 only use ``run_before`` if it is undesirable or impractical to specify
271 ``dependencies`` in the migration which you want to run after the one you are
272 writing.
273
274 Migrating data between third-party apps
275 =======================================
276
277 You can use a data migration to move data from one third-party application to
278 another.
279
280 If you plan to remove the old app later, you'll need to set the ``dependencies``
281 property based on whether or not the old app is installed. Otherwise, you'll
282 have missing dependencies once you uninstall the old app. Similarly, you'll
283 need to catch :exc:`LookupError` in the ``apps.get_model()`` call that
284 retrieves models from the old app. This approach allows you to deploy your
285 project anywhere without first installing and then uninstalling the old app.
286
287 Here's a sample migration:
288
289 .. code-block:: python
290 :caption: ``myapp/migrations/0124_move_old_app_to_new_app.py``
291
292 from django.apps import apps as global_apps
293 from django.db import migrations
294
295
296 def forwards(apps, schema_editor):
297 try:
298 OldModel = apps.get_model("old_app", "OldModel")
299 except LookupError:
300 # The old app isn't installed.
301 return
302
303 NewModel = apps.get_model("new_app", "NewModel")
304 NewModel.objects.bulk_create(
305 NewModel(new_attribute=old_object.old_attribute)
306 for old_object in OldModel.objects.all()
307 )
308
309
310 class Migration(migrations.Migration):
311 operations = [
312 migrations.RunPython(forwards, migrations.RunPython.noop),
313 ]
314 dependencies = [
315 ("myapp", "0123_the_previous_migration"),
316 ("new_app", "0001_initial"),
317 ]
318
319 if global_apps.is_installed("old_app"):
320 dependencies.append(("old_app", "0001_initial"))
321
322 Also consider what you want to happen when the migration is unapplied. You
323 could either do nothing (as in the example above) or remove some or all of the
324 data from the new application. Adjust the second argument of the
325 :mod:`~django.db.migrations.operations.RunPython` operation accordingly.
326
327 .. _changing-a-manytomanyfield-to-use-a-through-model:
328
329 Changing a ``ManyToManyField`` to use a ``through`` model
330 =========================================================
331
332 If you change a :class:`~django.db.models.ManyToManyField` to use a ``through``
333 model, the default migration will delete the existing table and create a new
334 one, losing the existing relations. To avoid this, you can use
335 :class:`.SeparateDatabaseAndState` to rename the existing table to the new
336 table name while telling the migration autodetector that the new model has
337 been created. You can check the existing table name and constraint name through
338 :djadmin:`sqlmigrate` or :djadmin:`dbshell`. You can check the new table name
339 with the through model's ``_meta.db_table`` property. Your new ``through``
340 model should use the same names for the ``ForeignKey``\s as Django did. Also if
341 it needs any extra fields, they should be added in operations after
342 :class:`.SeparateDatabaseAndState`.
343
344 For example, if we had a ``Book`` model with a ``ManyToManyField`` linking to
345 ``Author``, we could add a through model ``AuthorBook`` with a new field
346 ``is_primary``, like so::
347
348 from django.db import migrations, models
349 import django.db.models.deletion
350
351
352 class Migration(migrations.Migration):
353 dependencies = [
354 ("core", "0001_initial"),
355 ]
356
357 operations = [
358 migrations.SeparateDatabaseAndState(
359 database_operations=[
360 # Old table name from checking with sqlmigrate, new table
361 # name from AuthorBook._meta.db_table.
362 migrations.RunSQL(
363 sql="ALTER TABLE core_book_authors RENAME TO core_authorbook",
364 reverse_sql="ALTER TABLE core_authorbook RENAME TO core_book_authors",
365 ),
366 ],
367 state_operations=[
368 migrations.CreateModel(
369 name="AuthorBook",
370 fields=[
371 (
372 "id",
373 models.AutoField(
374 auto_created=True,
375 primary_key=True,
376 serialize=False,
377 verbose_name="ID",
378 ),
379 ),
380 (
381 "author",
382 models.ForeignKey(
383 on_delete=django.db.models.deletion.DO_NOTHING,
384 to="core.Author",
385 ),
386 ),
387 (
388 "book",
389 models.ForeignKey(
390 on_delete=django.db.models.deletion.DO_NOTHING,
391 to="core.Book",
392 ),
393 ),
394 ],
395 options={
396 "constraints": [
397 models.UniqueConstraint(
398 fields=["author", "book"],
399 name="unique_author_book",
400 )
401 ],
402 },
403 ),
404 migrations.AlterField(
405 model_name="book",
406 name="authors",
407 field=models.ManyToManyField(
408 to="core.Author",
409 through="core.AuthorBook",
410 ),
411 ),
412 ],
413 ),
414 migrations.AddField(
415 model_name="authorbook",
416 name="is_primary",
417 field=models.BooleanField(default=False),
418 ),
419 ]
420
421 Changing an unmanaged model to managed
422 ======================================
423
424 If you want to change an unmanaged model (:attr:`managed=False
425 <django.db.models.Options.managed>`) to managed, you must remove
426 ``managed=False`` and generate a migration before making other schema-related
427 changes to the model, since schema changes that appear in the migration that
428 contains the operation to change ``Meta.managed`` may not be applied.