Debian Codegraph @

docs/ref/contrib/gis/layermapping.txt

raw
1 ====================================
2 ``LayerMapping`` data import utility
3 ====================================
4
5 .. module:: django.contrib.gis.utils.layermapping
6 :synopsis: Spatial data import utility for GeoDjango models.
7
8 .. currentmodule:: django.contrib.gis.utils
9
10 The :class:`LayerMapping` class provides a way to map the contents of
11 vector spatial data files (e.g. shapefiles) into GeoDjango models.
12
13 This utility grew out of the author's personal needs to eliminate
14 the code repetition that went into pulling geometries and fields out of
15 a vector layer, converting to another coordinate system (e.g. WGS84), and
16 then inserting into a GeoDjango model.
17
18 .. note::
19
20 Use of :class:`LayerMapping` requires GDAL.
21
22 .. warning ::
23
24 GIS data sources, like shapefiles, may be very large. If you find
25 that :class:`LayerMapping` is using too much memory, set
26 :setting:`DEBUG` to ``False`` in your settings. When :setting:`DEBUG`
27 is set to ``True``, Django :ref:`automatically logs <faq-see-raw-sql-queries>`
28 *every* SQL query -- and when SQL statements contain geometries, this may
29 consume more memory than is typical.
30
31 Example
32 =======
33
34 #. You need a GDAL-supported data source, like a shapefile (here we're using
35 a simple polygon shapefile, ``test_poly.shp``, with three features):
36
37 .. code-block:: pycon
38
39 >>> from django.contrib.gis.gdal import DataSource
40 >>> ds = DataSource("test_poly.shp")
41 >>> layer = ds[0]
42 >>> print(layer.fields) # Exploring the fields in the layer, we only want the 'str' field.
43 ['float', 'int', 'str']
44 >>> print(len(layer)) # getting the number of features in the layer (should be 3)
45 3
46 >>> print(layer.geom_type) # Should be 'Polygon'
47 Polygon
48 >>> print(layer.srs) # WGS84 in WKT
49 GEOGCS["GCS_WGS_1984",
50 DATUM["WGS_1984",
51 SPHEROID["WGS_1984",6378137,298.257223563]],
52 PRIMEM["Greenwich",0],
53 UNIT["Degree",0.017453292519943295]]
54
55 #. Now we define our corresponding Django model (make sure to use :djadmin:`migrate`)::
56
57 from django.contrib.gis.db import models
58
59
60 class TestGeo(models.Model):
61 name = models.CharField(max_length=25) # corresponds to the 'str' field
62 poly = models.PolygonField(srid=4269) # we want our model in a different SRID
63
64 def __str__(self):
65 return "Name: %s" % self.name
66
67 #. Use :class:`LayerMapping` to extract all the features and place them in the
68 database:
69
70 .. code-block:: pycon
71
72 >>> from django.contrib.gis.utils import LayerMapping
73 >>> from geoapp.models import TestGeo
74 >>> mapping = {
75 ... "name": "str", # The 'name' model field maps to the 'str' layer field.
76 ... "poly": "POLYGON", # For geometry fields use OGC name.
77 ... } # The mapping is a dictionary
78 >>> lm = LayerMapping(TestGeo, "test_poly.shp", mapping)
79 >>> lm.save(verbose=True) # Save the layermap, imports the data.
80 Saved: Name: 1
81 Saved: Name: 2
82 Saved: Name: 3
83
84 Here, :class:`LayerMapping` transformed the three geometries from the shapefile
85 in their original spatial reference system (WGS84) to the spatial reference
86 system of the GeoDjango model (NAD83). If no spatial reference system is
87 defined for the layer, use the ``source_srs`` keyword with a
88 :class:`~django.contrib.gis.gdal.SpatialReference` object to specify one.
89
90 ``LayerMapping`` API
91 ====================
92
93 .. class:: LayerMapping(model, data_source, mapping, layer=0, source_srs=None, encoding=None, transaction_mode='commit_on_success', transform=True, unique=True, using='default')
94
95 The following are the arguments and keywords that may be used during
96 instantiation of ``LayerMapping`` objects.
97
98 ================= =========================================================
99 Argument Description
100 ================= =========================================================
101 ``model`` The geographic model, *not* an instance.
102
103 ``data_source`` The path to the OGR-supported data source file
104 (e.g., a shapefile). Also accepts
105 :class:`django.contrib.gis.gdal.DataSource` instances.
106
107 ``mapping`` A dictionary: keys are strings corresponding to
108 the model field, and values correspond to
109 string field names for the OGR feature, or if the
110 model field is a geographic then it should
111 correspond to the OGR geometry type,
112 e.g., ``'POINT'``, ``'LINESTRING'``, ``'POLYGON'``.
113 ================= =========================================================
114
115 ===================== =====================================================
116 Keyword Arguments
117 ===================== =====================================================
118 ``layer`` The index of the layer to use from the Data Source
119 (defaults to 0)
120
121 ``source_srs`` Use this to specify the source SRS manually (for
122 example, some shapefiles don't come with a ``'.prj'``
123 file). An integer SRID, WKT or PROJ strings, and
124 :class:`django.contrib.gis.gdal.SpatialReference`
125 objects are accepted.
126
127 ``encoding`` Specifies the character set encoding of the strings
128 in the OGR data source. For example, ``'latin-1'``,
129 ``'utf-8'``, and ``'cp437'`` are all valid encoding
130 parameters.
131
132 ``transaction_mode`` May be ``'commit_on_success'`` (default) or
133 ``'autocommit'``.
134
135 ``transform`` Setting this to False will disable coordinate
136 transformations. In other words, geometries will
137 be inserted into the database unmodified from their
138 original state in the data source.
139
140 ``unique`` Setting this to the name, or a tuple of names,
141 from the given model will create models unique
142 only to the given name(s). Geometries from
143 each feature will be added into the collection
144 associated with the unique model. Forces
145 the transaction mode to be ``'autocommit'``.
146
147 ``using`` Sets the database to use when importing spatial data.
148 Default is ``'default'``.
149 ===================== =====================================================
150
151 ``save()`` Keyword Arguments
152 ----------------------------
153
154 .. method:: LayerMapping.save(verbose=False, fid_range=False, step=False, progress=False, silent=False, stream=sys.stdout, strict=False)
155
156 The ``save()`` method also accepts keywords. These keywords are
157 used for controlling output logging, error handling, and for importing
158 specific feature ranges.
159
160 =========================== =================================================
161 Save Keyword Arguments Description
162 =========================== =================================================
163 ``fid_range`` May be set with a slice or tuple of
164 (begin, end) feature ID's to map from
165 the data source. In other words, this
166 keyword enables the user to selectively
167 import a subset range of features in the
168 geographic data source.
169
170 ``progress`` When this keyword is set, status information
171 will be printed giving the number of features
172 processed and successfully saved. By default,
173 progress information will be printed every 1000
174 features processed, however, this default may
175 be overridden by setting this keyword with an
176 integer for the desired interval.
177
178 ``silent`` By default, non-fatal error notifications are
179 printed to ``sys.stdout``, but this keyword may
180 be set to disable these notifications.
181
182 ``step`` If set with an integer, transactions will
183 occur at every step interval. For example, if
184 ``step=1000``, a commit would occur after the
185 1,000th feature, the 2,000th feature etc.
186
187
188 ``stream`` Status information will be written to this file
189 handle. Defaults to using ``sys.stdout``, but
190 any object with a ``write`` method is supported.
191
192 ``strict`` Execution of the model mapping will cease upon
193 the first error encountered. The default value
194 (``False``)
195 behavior is to attempt to continue.
196
197 ``verbose`` If set, information will be printed
198 subsequent to each model save
199 executed on the database.
200 =========================== =================================================
201
202 Troubleshooting
203 ===============
204
205 Running out of memory
206 ---------------------
207
208 As noted in the warning at the top of this section, Django stores all SQL
209 queries when ``DEBUG=True``. Set ``DEBUG=False`` in your settings, and this
210 should stop excessive memory use when running ``LayerMapping`` scripts.
211
212 MySQL: ``max_allowed_packet`` error
213 -----------------------------------
214
215 If you encounter the following error when using ``LayerMapping`` and MySQL:
216
217 .. code-block:: pytb
218
219 OperationalError: (1153, "Got a packet bigger than 'max_allowed_packet' bytes")
220
221 Then the solution is to increase the value of the ``max_allowed_packet``
222 setting in your MySQL configuration. For example, the default value may
223 be something low like one megabyte -- the setting may be modified in MySQL's
224 configuration file (``my.cnf``) in the ``[mysqld]`` section:
225
226 .. code-block:: ini
227
228 max_allowed_packet = 10M