Debian Codegraph @

docs/source/guide/dynamodb.rst

raw
1 .. _dynamodb_guide:
2
3 Amazon DynamoDB
4 ================
5 By following this guide, you will learn how to use the
6 :py:class:`DynamoDB.ServiceResource` and :py:class:`DynamoDB.Table`
7 resources in order to create tables, write items to tables, modify existing
8 items, retrieve items, and query/filter the items in the table.
9
10
11 Creating a new table
12 --------------------
13
14 In order to create a new table, use the
15 :py:meth:`DynamoDB.ServiceResource.create_table` method::
16
17 import boto3
18
19 # Get the service resource.
20 dynamodb = boto3.resource('dynamodb')
21
22 # Create the DynamoDB table.
23 table = dynamodb.create_table(
24 TableName='users',
25 KeySchema=[
26 {
27 'AttributeName': 'username',
28 'KeyType': 'HASH'
29 },
30 {
31 'AttributeName': 'last_name',
32 'KeyType': 'RANGE'
33 }
34 ],
35 AttributeDefinitions=[
36 {
37 'AttributeName': 'username',
38 'AttributeType': 'S'
39 },
40 {
41 'AttributeName': 'last_name',
42 'AttributeType': 'S'
43 },
44 ],
45 ProvisionedThroughput={
46 'ReadCapacityUnits': 5,
47 'WriteCapacityUnits': 5
48 }
49 )
50
51 # Wait until the table exists.
52 table.wait_until_exists()
53
54 # Print out some data about the table.
55 print(table.item_count)
56
57 Expected output::
58
59 0
60
61 This creates a table named ``users`` that respectively has the hash and
62 range primary keys ``username`` and ``last_name``.
63 This method will return a :py:class:`DynamoDB.Table` resource to call
64 additional methods on the created table.
65
66
67 Using an existing table
68 -----------------------
69 It is also possible to create a :py:class:`DynamoDB.Table` resource from
70 an existing table::
71
72 import boto3
73
74 # Get the service resource.
75 dynamodb = boto3.resource('dynamodb')
76
77 # Instantiate a table resource object without actually
78 # creating a DynamoDB table. Note that the attributes of this table
79 # are lazy-loaded: a request is not made nor are the attribute
80 # values populated until the attributes
81 # on the table resource are accessed or its load() method is called.
82 table = dynamodb.Table('users')
83
84 # Print out some data about the table.
85 # This will cause a request to be made to DynamoDB and its attribute
86 # values will be set based on the response.
87 print(table.creation_date_time)
88
89 Expected output (Please note that the actual times will probably not match up)::
90
91 2015-06-26 12:42:45.149000-07:00
92
93
94 Creating a new item
95 -------------------
96
97 Once you have a :py:class:`DynamoDB.Table` resource you can add new items
98 to the table using :py:meth:`DynamoDB.Table.put_item`::
99
100 table.put_item(
101 Item={
102 'username': 'janedoe',
103 'first_name': 'Jane',
104 'last_name': 'Doe',
105 'age': 25,
106 'account_type': 'standard_user',
107 }
108 )
109
110 For all of the valid types that can be used for an item, refer to
111 :ref:`ref_valid_dynamodb_types`.
112
113
114 Getting an item
115 ---------------
116 You can then retrieve the object using :py:meth:`DynamoDB.Table.get_item`::
117
118 response = table.get_item(
119 Key={
120 'username': 'janedoe',
121 'last_name': 'Doe'
122 }
123 )
124 item = response['Item']
125 print(item)
126
127
128 Expected output::
129
130 {u'username': u'janedoe',
131 u'first_name': u'Jane',
132 u'last_name': u'Doe',
133 u'account_type': u'standard_user',
134 u'age': Decimal('25')}
135
136
137 Updating an item
138 ----------------
139
140 You can then update attributes of the item in the table
141 using :py:meth:`DynamoDB.Table.update_item`::
142
143 table.update_item(
144 Key={
145 'username': 'janedoe',
146 'last_name': 'Doe'
147 },
148 UpdateExpression='SET age = :val1',
149 ExpressionAttributeValues={
150 ':val1': 26
151 }
152 )
153
154 Then if you retrieve the item again, it will be updated appropriately::
155
156 response = table.get_item(
157 Key={
158 'username': 'janedoe',
159 'last_name': 'Doe'
160 }
161 )
162 item = response['Item']
163 print(item)
164
165
166 Expected output::
167
168 {u'username': u'janedoe',
169 u'first_name': u'Jane',
170 u'last_name': u'Doe',
171 u'account_type': u'standard_user',
172 u'age': Decimal('26')}
173
174
175 Deleting an item
176 ----------------
177 You can also delete the item using :py:meth:`DynamoDB.Table.delete_item`::
178
179 table.delete_item(
180 Key={
181 'username': 'janedoe',
182 'last_name': 'Doe'
183 }
184 )
185
186
187 Batch writing
188 -------------
189 If you are loading a lot of data at a time, you can make use of
190 :py:meth:`DynamoDB.Table.batch_writer` so you can both speed up the process and
191 reduce the number of write requests made to the service.
192
193 This method returns a handle to a batch writer object that will automatically
194 handle buffering and sending items in batches. In addition, the
195 batch writer will also automatically handle any unprocessed items and
196 resend them as needed. All you need to do is call ``put_item`` for any
197 items you want to add, and ``delete_item`` for any items you want to delete::
198
199 with table.batch_writer() as batch:
200 batch.put_item(
201 Item={
202 'account_type': 'standard_user',
203 'username': 'johndoe',
204 'first_name': 'John',
205 'last_name': 'Doe',
206 'age': 25,
207 'address': {
208 'road': '1 Jefferson Street',
209 'city': 'Los Angeles',
210 'state': 'CA',
211 'zipcode': 90001
212 }
213 }
214 )
215 batch.put_item(
216 Item={
217 'account_type': 'super_user',
218 'username': 'janedoering',
219 'first_name': 'Jane',
220 'last_name': 'Doering',
221 'age': 40,
222 'address': {
223 'road': '2 Washington Avenue',
224 'city': 'Seattle',
225 'state': 'WA',
226 'zipcode': 98109
227 }
228 }
229 )
230 batch.put_item(
231 Item={
232 'account_type': 'standard_user',
233 'username': 'bobsmith',
234 'first_name': 'Bob',
235 'last_name': 'Smith',
236 'age': 18,
237 'address': {
238 'road': '3 Madison Lane',
239 'city': 'Louisville',
240 'state': 'KY',
241 'zipcode': 40213
242 }
243 }
244 )
245 batch.put_item(
246 Item={
247 'account_type': 'super_user',
248 'username': 'alicedoe',
249 'first_name': 'Alice',
250 'last_name': 'Doe',
251 'age': 27,
252 'address': {
253 'road': '1 Jefferson Street',
254 'city': 'Los Angeles',
255 'state': 'CA',
256 'zipcode': 90001
257 }
258 }
259 )
260
261 The batch writer is even able to handle a very large amount of writes to the
262 table.
263
264 ::
265
266 with table.batch_writer() as batch:
267 for i in range(50):
268 batch.put_item(
269 Item={
270 'account_type': 'anonymous',
271 'username': 'user' + str(i),
272 'first_name': 'unknown',
273 'last_name': 'unknown'
274 }
275 )
276
277 The batch writer can help to de-duplicate request by specifying ``overwrite_by_pkeys=['partition_key', 'sort_key']``
278 if you want to bypass no duplication limitation of single batch write request as
279 ``botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the BatchWriteItem operation: Provided list of item keys contains duplicates``.
280
281 It will drop request items in the buffer if their primary keys(composite) values are
282 the same as newly added one, as eventually consistent with streams of individual
283 put/delete operations on the same item.
284
285 ::
286
287 with table.batch_writer(overwrite_by_pkeys=['partition_key', 'sort_key']) as batch:
288 batch.put_item(
289 Item={
290 'partition_key': 'p1',
291 'sort_key': 's1',
292 'other': '111',
293 }
294 )
295 batch.put_item(
296 Item={
297 'partition_key': 'p1',
298 'sort_key': 's1',
299 'other': '222',
300 }
301 )
302 batch.delete_item(
303 Key={
304 'partition_key': 'p1',
305 'sort_key': 's2'
306 }
307 )
308 batch.put_item(
309 Item={
310 'partition_key': 'p1',
311 'sort_key': 's2',
312 'other': '444',
313 }
314 )
315
316 after de-duplicate:
317
318 ::
319
320 batch.put_item(
321 Item={
322 'partition_key': 'p1',
323 'sort_key': 's1',
324 'other': '222',
325 }
326 )
327 batch.put_item(
328 Item={
329 'partition_key': 'p1',
330 'sort_key': 's2',
331 'other': '444',
332 }
333 )
334
335
336 Querying and scanning
337 ---------------------
338
339 With the table full of items, you can then query or scan the items in the table
340 using the :py:meth:`DynamoDB.Table.query` or :py:meth:`DynamoDB.Table.scan`
341 methods respectively. To add conditions to scanning and querying the table,
342 you will need to import the :py:class:`boto3.dynamodb.conditions.Key` and
343 :py:class:`boto3.dynamodb.conditions.Attr` classes. The
344 :py:class:`boto3.dynamodb.conditions.Key` should be used when the
345 condition is related to the key of the item.
346 The :py:class:`boto3.dynamodb.conditions.Attr` should be used when the
347 condition is related to an attribute of the item::
348
349 from boto3.dynamodb.conditions import Key, Attr
350
351
352 This queries for all of the users whose ``username`` key equals ``johndoe``::
353
354 response = table.query(
355 KeyConditionExpression=Key('username').eq('johndoe')
356 )
357 items = response['Items']
358 print(items)
359
360
361 Expected output::
362
363 [{u'username': u'johndoe',
364 u'first_name': u'John',
365 u'last_name': u'Doe',
366 u'account_type': u'standard_user',
367 u'age': Decimal('25'),
368 u'address': {u'city': u'Los Angeles',
369 u'state': u'CA',
370 u'zipcode': Decimal('90001'),
371 u'road': u'1 Jefferson Street'}}]
372
373
374 Similarly you can scan the table based on attributes of the items. For
375 example, this scans for all the users whose ``age`` is less than ``27``::
376
377 response = table.scan(
378 FilterExpression=Attr('age').lt(27)
379 )
380 items = response['Items']
381 print(items)
382
383
384 Expected output::
385
386 [{u'username': u'johndoe',
387 u'first_name': u'John',
388 u'last_name': u'Doe',
389 u'account_type': u'standard_user',
390 u'age': Decimal('25'),
391 u'address': {u'city': u'Los Angeles',
392 u'state': u'CA',
393 u'zipcode': Decimal('90001'),
394 u'road': u'1 Jefferson Street'}},
395 {u'username': u'bobsmith',
396 u'first_name': u'Bob',
397 u'last_name': u'Smith',
398 u'account_type': u'standard_user',
399 u'age': Decimal('18'),
400 u'address': {u'city': u'Louisville',
401 u'state': u'KY',
402 u'zipcode': Decimal('40213'),
403 u'road': u'3 Madison Lane'}}]
404
405
406 You are also able to chain conditions together using the logical operators:
407 ``&`` (and), ``|`` (or), and ``~`` (not). For example, this scans for all
408 users whose ``first_name`` starts with ``J`` and whose ``account_type`` is
409 ``super_user``::
410
411 response = table.scan(
412 FilterExpression=Attr('first_name').begins_with('J') & Attr('account_type').eq('super_user')
413 )
414 items = response['Items']
415 print(items)
416
417
418 Expected output::
419
420 [{u'username': u'janedoering',
421 u'first_name': u'Jane',
422 u'last_name': u'Doering',
423 u'account_type': u'super_user',
424 u'age': Decimal('40'),
425 u'address': {u'city': u'Seattle',
426 u'state': u'WA',
427 u'zipcode': Decimal('98109'),
428 u'road': u'2 Washington Avenue'}}]
429
430
431 You can even scan based on conditions of a nested attribute. For example this
432 scans for all users whose ``state`` in their ``address`` is ``CA``::
433
434 response = table.scan(
435 FilterExpression=Attr('address.state').eq('CA')
436 )
437 items = response['Items']
438 print(items)
439
440
441 Expected output::
442
443 [{u'username': u'johndoe',
444 u'first_name': u'John',
445 u'last_name': u'Doe',
446 u'account_type': u'standard_user',
447 u'age': Decimal('25'),
448 u'address': {u'city': u'Los Angeles',
449 u'state': u'CA',
450 u'zipcode': Decimal('90001'),
451 u'road': u'1 Jefferson Street'}},
452 {u'username': u'alicedoe',
453 u'first_name': u'Alice',
454 u'last_name': u'Doe',
455 u'account_type': u'super_user',
456 u'age': Decimal('27'),
457 u'address': {u'city': u'Los Angeles',
458 u'state': u'CA',
459 u'zipcode': Decimal('90001'),
460 u'road': u'1 Jefferson Street'}}]
461
462
463 For more information on the various conditions you can use for queries and
464 scans, refer to :ref:`ref_dynamodb_conditions`.
465
466
467 Deleting a table
468 ----------------
469 Finally, if you want to delete your table call
470 :py:meth:`DynamoDB.Table.delete`::
471
472 table.delete()