| 1 |
================================= |
| 2 |
How to create database migrations |
| 3 |
================================= |
| 4 |
|
| 5 |
This document explains how to structure and write database migrations for |
| 6 |
different scenarios you might encounter. For introductory material on |
| 7 |
migrations, see :doc:`the topic guide </topics/migrations>`. |
| 8 |
|
| 9 |
.. _data-migrations-and-multiple-databases: |
| 10 |
|
| 11 |
Data migrations and multiple databases |
| 12 |
====================================== |
| 13 |
|
| 14 |
When using multiple databases, you may need to figure out whether or not to |
| 15 |
run a migration against a particular database. For example, you may want to |
| 16 |
**only** run a migration on a particular database. |
| 17 |
|
| 18 |
In order to do that you can check the database connection's alias inside a |
| 19 |
``RunPython`` operation by looking at the ``schema_editor.connection.alias`` |
| 20 |
attribute:: |
| 21 |
|
| 22 |
from django.db import migrations |
| 23 |
|
| 24 |
|
| 25 |
def forwards(apps, schema_editor): |
| 26 |
if schema_editor.connection.alias != "default": |
| 27 |
return |
| 28 |
# Your migration code goes here |
| 29 |
|
| 30 |
|
| 31 |
class Migration(migrations.Migration): |
| 32 |
dependencies = [ |
| 33 |
# Dependencies to other migrations |
| 34 |
] |
| 35 |
|
| 36 |
operations = [ |
| 37 |
migrations.RunPython(forwards), |
| 38 |
] |
| 39 |
|
| 40 |
You can also provide hints that will be passed to the :meth:`allow_migrate()` |
| 41 |
method of database routers as ``**hints``: |
| 42 |
|
| 43 |
.. code-block:: python |
| 44 |
:caption: ``myapp/dbrouters.py`` |
| 45 |
|
| 46 |
class MyRouter: |
| 47 |
def allow_migrate(self, db, app_label, model_name=None, **hints): |
| 48 |
if "target_db" in hints: |
| 49 |
return db == hints["target_db"] |
| 50 |
return True |
| 51 |
|
| 52 |
Then, to leverage this in your migrations, do the following:: |
| 53 |
|
| 54 |
from django.db import migrations |
| 55 |
|
| 56 |
|
| 57 |
def forwards(apps, schema_editor): |
| 58 |
# Your migration code goes here |
| 59 |
... |
| 60 |
|
| 61 |
|
| 62 |
class Migration(migrations.Migration): |
| 63 |
dependencies = [ |
| 64 |
# Dependencies to other migrations |
| 65 |
] |
| 66 |
|
| 67 |
operations = [ |
| 68 |
migrations.RunPython(forwards, hints={"target_db": "default"}), |
| 69 |
] |
| 70 |
|
| 71 |
If your ``RunPython`` or ``RunSQL`` operation only affects one model, it's good |
| 72 |
practice to pass ``model_name`` as a hint to make it as transparent as possible |
| 73 |
to the router. This is especially important for reusable and third-party apps. |
| 74 |
|
| 75 |
Migrations that add unique fields |
| 76 |
================================= |
| 77 |
|
| 78 |
Applying a "plain" migration that adds a unique non-nullable field to a table |
| 79 |
with existing rows will raise an error because the value used to populate |
| 80 |
existing rows is generated only once, thus breaking the unique constraint. |
| 81 |
|
| 82 |
Therefore, the following steps should be taken. In this example, we'll add a |
| 83 |
non-nullable :class:`~django.db.models.UUIDField` with a default value. Modify |
| 84 |
the respective field according to your needs. |
| 85 |
|
| 86 |
* Add the field on your model with ``default=uuid.uuid4`` and ``unique=True`` |
| 87 |
arguments (choose an appropriate default for the type of the field you're |
| 88 |
adding). |
| 89 |
|
| 90 |
* Run the :djadmin:`makemigrations` command. This should generate a migration |
| 91 |
with an ``AddField`` operation. |
| 92 |
|
| 93 |
* Generate two empty migration files for the same app by running |
| 94 |
``makemigrations myapp --empty`` twice. We've renamed the migration files to |
| 95 |
give them meaningful names in the examples below. |
| 96 |
|
| 97 |
* Copy the ``AddField`` operation from the auto-generated migration (the first |
| 98 |
of the three new files) to the last migration, change ``AddField`` to |
| 99 |
``AlterField``, and add imports of ``uuid`` and ``models``. For example: |
| 100 |
|
| 101 |
.. code-block:: python |
| 102 |
:caption: ``0006_remove_uuid_null.py`` |
| 103 |
|
| 104 |
# Generated by Django A.B on YYYY-MM-DD HH:MM |
| 105 |
from django.db import migrations, models |
| 106 |
import uuid |
| 107 |
|
| 108 |
|
| 109 |
class Migration(migrations.Migration): |
| 110 |
dependencies = [ |
| 111 |
("myapp", "0005_populate_uuid_values"), |
| 112 |
] |
| 113 |
|
| 114 |
operations = [ |
| 115 |
migrations.AlterField( |
| 116 |
model_name="mymodel", |
| 117 |
name="uuid", |
| 118 |
field=models.UUIDField(default=uuid.uuid4, unique=True), |
| 119 |
), |
| 120 |
] |
| 121 |
|
| 122 |
* Edit the first migration file. The generated migration class should look |
| 123 |
similar to this: |
| 124 |
|
| 125 |
.. code-block:: python |
| 126 |
:caption: ``0004_add_uuid_field.py`` |
| 127 |
|
| 128 |
class Migration(migrations.Migration): |
| 129 |
dependencies = [ |
| 130 |
("myapp", "0003_auto_20150129_1705"), |
| 131 |
] |
| 132 |
|
| 133 |
operations = [ |
| 134 |
migrations.AddField( |
| 135 |
model_name="mymodel", |
| 136 |
name="uuid", |
| 137 |
field=models.UUIDField(default=uuid.uuid4, unique=True), |
| 138 |
), |
| 139 |
] |
| 140 |
|
| 141 |
Change ``unique=True`` to ``null=True`` -- this will create the intermediary |
| 142 |
null field and defer creating the unique constraint until we've populated |
| 143 |
unique values on all the rows. |
| 144 |
|
| 145 |
* In the first empty migration file, add a |
| 146 |
:class:`~django.db.migrations.operations.RunPython` or |
| 147 |
:class:`~django.db.migrations.operations.RunSQL` operation to generate a |
| 148 |
unique value (UUID in the example) for each existing row. Also add an import |
| 149 |
of ``uuid``. For example: |
| 150 |
|
| 151 |
.. code-block:: python |
| 152 |
:caption: ``0005_populate_uuid_values.py`` |
| 153 |
|
| 154 |
# Generated by Django A.B on YYYY-MM-DD HH:MM |
| 155 |
from django.db import migrations |
| 156 |
import uuid |
| 157 |
|
| 158 |
|
| 159 |
def gen_uuid(apps, schema_editor): |
| 160 |
MyModel = apps.get_model("myapp", "MyModel") |
| 161 |
for row in MyModel.objects.all(): |
| 162 |
row.uuid = uuid.uuid4() |
| 163 |
row.save(update_fields=["uuid"]) |
| 164 |
|
| 165 |
|
| 166 |
class Migration(migrations.Migration): |
| 167 |
dependencies = [ |
| 168 |
("myapp", "0004_add_uuid_field"), |
| 169 |
] |
| 170 |
|
| 171 |
operations = [ |
| 172 |
# omit reverse_code=... if you don't want the migration to be reversible. |
| 173 |
migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop), |
| 174 |
] |
| 175 |
|
| 176 |
* Now you can apply the migrations as usual with the :djadmin:`migrate` command. |
| 177 |
|
| 178 |
Note there is a race condition if you allow objects to be created while this |
| 179 |
migration is running. Objects created after the ``AddField`` and before |
| 180 |
``RunPython`` will have their original ``uuid``’s overwritten. |
| 181 |
|
| 182 |
.. _non-atomic-migrations: |
| 183 |
|
| 184 |
Non-atomic migrations |
| 185 |
~~~~~~~~~~~~~~~~~~~~~ |
| 186 |
|
| 187 |
On databases that support DDL transactions (SQLite and PostgreSQL), migrations |
| 188 |
will run inside a transaction by default. For use cases such as performing data |
| 189 |
migrations on large tables, you may want to prevent a migration from running in |
| 190 |
a transaction by setting the ``atomic`` attribute to ``False``:: |
| 191 |
|
| 192 |
from django.db import migrations |
| 193 |
|
| 194 |
|
| 195 |
class Migration(migrations.Migration): |
| 196 |
atomic = False |
| 197 |
|
| 198 |
Within such a migration, all operations are run without a transaction. It's |
| 199 |
possible to execute parts of the migration inside a transaction using |
| 200 |
:func:`~django.db.transaction.atomic()` or by passing ``atomic=True`` to |
| 201 |
``RunPython``. |
| 202 |
|
| 203 |
Here's an example of a non-atomic data migration that updates a large table in |
| 204 |
smaller batches:: |
| 205 |
|
| 206 |
import uuid |
| 207 |
|
| 208 |
from django.db import migrations, transaction |
| 209 |
|
| 210 |
|
| 211 |
def gen_uuid(apps, schema_editor): |
| 212 |
MyModel = apps.get_model("myapp", "MyModel") |
| 213 |
while MyModel.objects.filter(uuid__isnull=True).exists(): |
| 214 |
with transaction.atomic(): |
| 215 |
for row in MyModel.objects.filter(uuid__isnull=True)[:1000]: |
| 216 |
row.uuid = uuid.uuid4() |
| 217 |
row.save() |
| 218 |
|
| 219 |
|
| 220 |
class Migration(migrations.Migration): |
| 221 |
atomic = False |
| 222 |
|
| 223 |
operations = [ |
| 224 |
migrations.RunPython(gen_uuid), |
| 225 |
] |
| 226 |
|
| 227 |
The ``atomic`` attribute doesn't have an effect on databases that don't support |
| 228 |
DDL transactions (e.g. MySQL, Oracle). (MySQL's `atomic DDL statement support |
| 229 |
<https://dev.mysql.com/doc/refman/en/atomic-ddl.html>`_ refers to individual |
| 230 |
statements rather than multiple statements wrapped in a transaction that can be |
| 231 |
rolled back.) |
| 232 |
|
| 233 |
Controlling the order of migrations |
| 234 |
=================================== |
| 235 |
|
| 236 |
Django determines the order in which migrations should be applied not by the |
| 237 |
filename of each migration, but by building a graph using two properties on the |
| 238 |
``Migration`` class: ``dependencies`` and ``run_before``. |
| 239 |
|
| 240 |
If you've used the :djadmin:`makemigrations` command you've probably |
| 241 |
already seen ``dependencies`` in action because auto-created |
| 242 |
migrations have this defined as part of their creation process. |
| 243 |
|
| 244 |
The ``dependencies`` property is declared like this:: |
| 245 |
|
| 246 |
from django.db import migrations |
| 247 |
|
| 248 |
|
| 249 |
class Migration(migrations.Migration): |
| 250 |
dependencies = [ |
| 251 |
("myapp", "0123_the_previous_migration"), |
| 252 |
] |
| 253 |
|
| 254 |
Usually this will be enough, but from time to time you may need to |
| 255 |
ensure that your migration runs *before* other migrations. This is |
| 256 |
useful, for example, to make third-party apps' migrations run *after* |
| 257 |
your :setting:`AUTH_USER_MODEL` replacement. |
| 258 |
|
| 259 |
To achieve this, place all migrations that should depend on yours in |
| 260 |
the ``run_before`` attribute on your ``Migration`` class:: |
| 261 |
|
| 262 |
class Migration(migrations.Migration): |
| 263 |
... |
| 264 |
|
| 265 |
run_before = [ |
| 266 |
("third_party_app", "0001_do_awesome"), |
| 267 |
] |
| 268 |
|
| 269 |
Prefer using ``dependencies`` over ``run_before`` when possible. You should |
| 270 |
only use ``run_before`` if it is undesirable or impractical to specify |
| 271 |
``dependencies`` in the migration which you want to run after the one you are |
| 272 |
writing. |
| 273 |
|
| 274 |
Migrating data between third-party apps |
| 275 |
======================================= |
| 276 |
|
| 277 |
You can use a data migration to move data from one third-party application to |
| 278 |
another. |
| 279 |
|
| 280 |
If you plan to remove the old app later, you'll need to set the ``dependencies`` |
| 281 |
property based on whether or not the old app is installed. Otherwise, you'll |
| 282 |
have missing dependencies once you uninstall the old app. Similarly, you'll |
| 283 |
need to catch :exc:`LookupError` in the ``apps.get_model()`` call that |
| 284 |
retrieves models from the old app. This approach allows you to deploy your |
| 285 |
project anywhere without first installing and then uninstalling the old app. |
| 286 |
|
| 287 |
Here's a sample migration: |
| 288 |
|
| 289 |
.. code-block:: python |
| 290 |
:caption: ``myapp/migrations/0124_move_old_app_to_new_app.py`` |
| 291 |
|
| 292 |
from django.apps import apps as global_apps |
| 293 |
from django.db import migrations |
| 294 |
|
| 295 |
|
| 296 |
def forwards(apps, schema_editor): |
| 297 |
try: |
| 298 |
OldModel = apps.get_model("old_app", "OldModel") |
| 299 |
except LookupError: |
| 300 |
# The old app isn't installed. |
| 301 |
return |
| 302 |
|
| 303 |
NewModel = apps.get_model("new_app", "NewModel") |
| 304 |
NewModel.objects.bulk_create( |
| 305 |
NewModel(new_attribute=old_object.old_attribute) |
| 306 |
for old_object in OldModel.objects.all() |
| 307 |
) |
| 308 |
|
| 309 |
|
| 310 |
class Migration(migrations.Migration): |
| 311 |
operations = [ |
| 312 |
migrations.RunPython(forwards, migrations.RunPython.noop), |
| 313 |
] |
| 314 |
dependencies = [ |
| 315 |
("myapp", "0123_the_previous_migration"), |
| 316 |
("new_app", "0001_initial"), |
| 317 |
] |
| 318 |
|
| 319 |
if global_apps.is_installed("old_app"): |
| 320 |
dependencies.append(("old_app", "0001_initial")) |
| 321 |
|
| 322 |
Also consider what you want to happen when the migration is unapplied. You |
| 323 |
could either do nothing (as in the example above) or remove some or all of the |
| 324 |
data from the new application. Adjust the second argument of the |
| 325 |
:mod:`~django.db.migrations.operations.RunPython` operation accordingly. |
| 326 |
|
| 327 |
.. _changing-a-manytomanyfield-to-use-a-through-model: |
| 328 |
|
| 329 |
Changing a ``ManyToManyField`` to use a ``through`` model |
| 330 |
========================================================= |
| 331 |
|
| 332 |
If you change a :class:`~django.db.models.ManyToManyField` to use a ``through`` |
| 333 |
model, the default migration will delete the existing table and create a new |
| 334 |
one, losing the existing relations. To avoid this, you can use |
| 335 |
:class:`.SeparateDatabaseAndState` to rename the existing table to the new |
| 336 |
table name while telling the migration autodetector that the new model has |
| 337 |
been created. You can check the existing table name and constraint name through |
| 338 |
:djadmin:`sqlmigrate` or :djadmin:`dbshell`. You can check the new table name |
| 339 |
with the through model's ``_meta.db_table`` property. Your new ``through`` |
| 340 |
model should use the same names for the ``ForeignKey``\s as Django did. Also if |
| 341 |
it needs any extra fields, they should be added in operations after |
| 342 |
:class:`.SeparateDatabaseAndState`. |
| 343 |
|
| 344 |
For example, if we had a ``Book`` model with a ``ManyToManyField`` linking to |
| 345 |
``Author``, we could add a through model ``AuthorBook`` with a new field |
| 346 |
``is_primary``, like so:: |
| 347 |
|
| 348 |
from django.db import migrations, models |
| 349 |
import django.db.models.deletion |
| 350 |
|
| 351 |
|
| 352 |
class Migration(migrations.Migration): |
| 353 |
dependencies = [ |
| 354 |
("core", "0001_initial"), |
| 355 |
] |
| 356 |
|
| 357 |
operations = [ |
| 358 |
migrations.SeparateDatabaseAndState( |
| 359 |
database_operations=[ |
| 360 |
# Old table name from checking with sqlmigrate, new table |
| 361 |
# name from AuthorBook._meta.db_table. |
| 362 |
migrations.RunSQL( |
| 363 |
sql="ALTER TABLE core_book_authors RENAME TO core_authorbook", |
| 364 |
reverse_sql="ALTER TABLE core_authorbook RENAME TO core_book_authors", |
| 365 |
), |
| 366 |
], |
| 367 |
state_operations=[ |
| 368 |
migrations.CreateModel( |
| 369 |
name="AuthorBook", |
| 370 |
fields=[ |
| 371 |
( |
| 372 |
"id", |
| 373 |
models.AutoField( |
| 374 |
auto_created=True, |
| 375 |
primary_key=True, |
| 376 |
serialize=False, |
| 377 |
verbose_name="ID", |
| 378 |
), |
| 379 |
), |
| 380 |
( |
| 381 |
"author", |
| 382 |
models.ForeignKey( |
| 383 |
on_delete=django.db.models.deletion.DO_NOTHING, |
| 384 |
to="core.Author", |
| 385 |
), |
| 386 |
), |
| 387 |
( |
| 388 |
"book", |
| 389 |
models.ForeignKey( |
| 390 |
on_delete=django.db.models.deletion.DO_NOTHING, |
| 391 |
to="core.Book", |
| 392 |
), |
| 393 |
), |
| 394 |
], |
| 395 |
options={ |
| 396 |
"constraints": [ |
| 397 |
models.UniqueConstraint( |
| 398 |
fields=["author", "book"], |
| 399 |
name="unique_author_book", |
| 400 |
) |
| 401 |
], |
| 402 |
}, |
| 403 |
), |
| 404 |
migrations.AlterField( |
| 405 |
model_name="book", |
| 406 |
name="authors", |
| 407 |
field=models.ManyToManyField( |
| 408 |
to="core.Author", |
| 409 |
through="core.AuthorBook", |
| 410 |
), |
| 411 |
), |
| 412 |
], |
| 413 |
), |
| 414 |
migrations.AddField( |
| 415 |
model_name="authorbook", |
| 416 |
name="is_primary", |
| 417 |
field=models.BooleanField(default=False), |
| 418 |
), |
| 419 |
] |
| 420 |
|
| 421 |
Changing an unmanaged model to managed |
| 422 |
====================================== |
| 423 |
|
| 424 |
If you want to change an unmanaged model (:attr:`managed=False |
| 425 |
<django.db.models.Options.managed>`) to managed, you must remove |
| 426 |
``managed=False`` and generate a migration before making other schema-related |
| 427 |
changes to the model, since schema changes that appear in the migration that |
| 428 |
contains the operation to change ``Meta.managed`` may not be applied. |