| 1 |
.. _guide_resources: |
| 2 |
|
| 3 |
Resources |
| 4 |
========= |
| 5 |
|
| 6 |
Overview |
| 7 |
-------- |
| 8 |
|
| 9 |
.. note:: |
| 10 |
The AWS Python SDK team does not intend to add new features to the resources |
| 11 |
interface in boto3. Existing interfaces will continue to operate during |
| 12 |
boto3's lifecycle. Customers can find access to newer service features through |
| 13 |
the client interface. |
| 14 |
|
| 15 |
Resources represent an object-oriented interface to Amazon Web Services (AWS). |
| 16 |
They provide a higher-level abstraction than the raw, low-level calls made by |
| 17 |
service clients. To use resources, you invoke the |
| 18 |
:py:meth:`~boto3.session.Session.resource` method of a |
| 19 |
:py:class:`~boto3.session.Session` and pass in a service name:: |
| 20 |
|
| 21 |
# Get resources from the default session |
| 22 |
sqs = boto3.resource('sqs') |
| 23 |
s3 = boto3.resource('s3') |
| 24 |
|
| 25 |
Every resource instance has a number of attributes and methods. These can |
| 26 |
conceptually be split up into identifiers, attributes, actions, references, |
| 27 |
sub-resources, and collections. Each of these is described in further detail |
| 28 |
below and in the following section. |
| 29 |
|
| 30 |
Resources themselves can also be conceptually split into service resources |
| 31 |
(like ``sqs``, ``s3``, ``ec2``, etc) and individual resources (like |
| 32 |
``sqs.Queue`` or ``s3.Bucket``). Service resources *do not* have |
| 33 |
identifiers or attributes. The two share the same components otherwise. |
| 34 |
|
| 35 |
|
| 36 |
.. _identifiers_attributes_intro: |
| 37 |
|
| 38 |
Identifiers and attributes |
| 39 |
------------------------ |
| 40 |
An identifier is a unique value that is used to call actions on the resource. |
| 41 |
Resources **must** have at least one identifier, except for the top-level |
| 42 |
service resources (e.g. ``sqs`` or ``s3``). An identifier is set at instance |
| 43 |
creation-time, and failing to provide all necessary identifiers during |
| 44 |
instantiation will result in an exception. Examples of identifiers:: |
| 45 |
|
| 46 |
# SQS Queue (url is an identifier) |
| 47 |
queue = sqs.Queue(url='http://...') |
| 48 |
print(queue.url) |
| 49 |
|
| 50 |
# S3 Object (bucket_name and key are identifiers) |
| 51 |
obj = s3.Object(bucket_name='amzn-s3-demo-bucket', key='test.py') |
| 52 |
print(obj.bucket_name) |
| 53 |
print(obj.key) |
| 54 |
|
| 55 |
# Raises exception, missing identifier: key! |
| 56 |
obj = s3.Object(bucket_name='amzn-s3-demo-bucket') |
| 57 |
|
| 58 |
Identifiers may also be passed as positional arguments:: |
| 59 |
|
| 60 |
# SQS Queue |
| 61 |
queue = sqs.Queue('http://...') |
| 62 |
|
| 63 |
# S3 Object |
| 64 |
obj = s3.Object('boto3', 'test.py') |
| 65 |
|
| 66 |
# Raises exception, missing key! |
| 67 |
obj = s3.Object('boto3') |
| 68 |
|
| 69 |
Identifiers also play a role in resource instance equality. For two |
| 70 |
instances of a resource to be considered equal, their identifiers must |
| 71 |
be equal:: |
| 72 |
|
| 73 |
>>> bucket1 = s3.Bucket('amzn-s3-demo-bucket1') |
| 74 |
>>> bucket2 = s3.Bucket('amzn-s3-demo-bucket1') |
| 75 |
>>> bucket3 = s3.Bucket('amzn-s3-demo-bucket3') |
| 76 |
|
| 77 |
>>> bucket1 == bucket2 |
| 78 |
True |
| 79 |
>>> bucket1 == bucket3 |
| 80 |
False |
| 81 |
|
| 82 |
.. note:: |
| 83 |
|
| 84 |
Only identifiers are taken into account for instance equality. Region, |
| 85 |
account ID and other data members are not considered. When using temporary |
| 86 |
credentials or multiple regions in your code please keep this in mind. |
| 87 |
|
| 88 |
Resources may also have attributes, which are *lazy-loaded* properties on the |
| 89 |
instance. They may be set at creation time from the response of an action on |
| 90 |
another resource, or they may be set when accessed or via an explicit call to |
| 91 |
the ``load`` or ``reload`` action. Examples of attributes:: |
| 92 |
|
| 93 |
# SQS Message |
| 94 |
message.body |
| 95 |
|
| 96 |
# S3 Object |
| 97 |
obj.last_modified |
| 98 |
obj.e_tag |
| 99 |
|
| 100 |
.. warning:: |
| 101 |
|
| 102 |
Attributes may incur a load action when first accessed. If latency is |
| 103 |
a concern, then manually calling ``load`` will allow you to control |
| 104 |
exactly when the load action (and thus latency) is invoked. The |
| 105 |
documentation for each resource explicitly lists its attributes. |
| 106 |
|
| 107 |
Additionally, attributes may be reloaded after an action has been |
| 108 |
performed on the resource. For example, if the ``last_modified`` |
| 109 |
attribute of an S3 object is loaded and then a ``put`` action is |
| 110 |
called, then the next time you access ``last_modified`` it will |
| 111 |
reload the object's metadata. |
| 112 |
|
| 113 |
.. _actions_intro: |
| 114 |
|
| 115 |
Actions |
| 116 |
------- |
| 117 |
An action is a method which makes a call to the service. Actions may return a |
| 118 |
low-level response, a new resource instance or a list of new resource |
| 119 |
instances. Actions automatically set the resource identifiers as parameters, |
| 120 |
but allow you to pass additional parameters via keyword arguments. Examples |
| 121 |
of actions:: |
| 122 |
|
| 123 |
# SQS Queue |
| 124 |
messages = queue.receive_messages() |
| 125 |
|
| 126 |
# SQS Message |
| 127 |
for message in messages: |
| 128 |
message.delete() |
| 129 |
|
| 130 |
# S3 Object |
| 131 |
obj = s3.Object(bucket_name='amzn-s3-demo-bucket', key='test.py') |
| 132 |
response = obj.get() |
| 133 |
data = response['Body'].read() |
| 134 |
|
| 135 |
Examples of sending additional parameters:: |
| 136 |
|
| 137 |
# SQS Service |
| 138 |
queue = sqs.get_queue_by_name(QueueName='test') |
| 139 |
|
| 140 |
# SQS Queue |
| 141 |
queue.send_message(MessageBody='hello') |
| 142 |
|
| 143 |
.. note:: |
| 144 |
|
| 145 |
Parameters **must** be passed as keyword arguments. They will not work |
| 146 |
as positional arguments. |
| 147 |
|
| 148 |
.. _references_intro: |
| 149 |
|
| 150 |
References |
| 151 |
---------- |
| 152 |
A reference is an attribute which may be ``None`` or a related resource |
| 153 |
instance. The resource instance does not share identifiers with its |
| 154 |
reference resource, that is, it is not a strict parent to child relationship. |
| 155 |
In relational terms, these can be considered many-to-one or one-to-one. |
| 156 |
Examples of references:: |
| 157 |
|
| 158 |
# EC2 Instance |
| 159 |
instance.subnet |
| 160 |
instance.vpc |
| 161 |
|
| 162 |
In the above example, an EC2 instance may have exactly one associated |
| 163 |
subnet, and may have exactly one associated VPC. The subnet does not |
| 164 |
require the instance ID to exist, hence it is not a parent to child |
| 165 |
relationship. |
| 166 |
|
| 167 |
.. _subresources_intro: |
| 168 |
|
| 169 |
Sub-resources |
| 170 |
------------- |
| 171 |
A sub-resource is similar to a reference, but is a related class rather than |
| 172 |
an instance. Sub-resources, when instantiated, share identifiers with their |
| 173 |
parent. It is a strict parent-child relationship. In relational terms, these |
| 174 |
can be considered one-to-many. Examples of sub-resources:: |
| 175 |
|
| 176 |
# SQS |
| 177 |
queue = sqs.Queue(url='...') |
| 178 |
message = queue.Message(receipt_handle='...') |
| 179 |
print(queue.url == message.queue_url) |
| 180 |
print(message.receipt_handle) |
| 181 |
|
| 182 |
# S3 |
| 183 |
obj = bucket.Object(key='new_file.txt') |
| 184 |
print(obj.bucket_name) |
| 185 |
print(obj.key) |
| 186 |
|
| 187 |
Because an SQS message cannot exist without a queue, and an S3 object cannot |
| 188 |
exist without a bucket, these are parent to child relationships. |
| 189 |
|
| 190 |
.. _waiters_intro: |
| 191 |
|
| 192 |
Waiters |
| 193 |
------- |
| 194 |
A waiter is similar to an action. A waiter will poll the status of a |
| 195 |
resource and suspend execution until the resource reaches the state that is |
| 196 |
being polled for or a failure occurs while polling. |
| 197 |
Waiters automatically set the resource |
| 198 |
identifiers as parameters, but allow you to pass additional parameters via |
| 199 |
keyword arguments. Examples of waiters include:: |
| 200 |
|
| 201 |
# S3: Wait for a bucket to exist. |
| 202 |
bucket.wait_until_exists() |
| 203 |
|
| 204 |
# EC2: Wait for an instance to reach the running state. |
| 205 |
instance.wait_until_running() |
| 206 |
|
| 207 |
|
| 208 |
Multithreading or multiprocessing with resources |
| 209 |
---------------------------------- |
| 210 |
|
| 211 |
Resource instances are **not** thread safe and should not be shared |
| 212 |
across threads or processes. These special classes contain additional |
| 213 |
meta data that cannot be shared. It's recommended to create a new |
| 214 |
Resource for each thread or process:: |
| 215 |
|
| 216 |
import boto3 |
| 217 |
import boto3.session |
| 218 |
import threading |
| 219 |
|
| 220 |
class MyTask(threading.Thread): |
| 221 |
def run(self): |
| 222 |
# Here we create a new session per thread |
| 223 |
session = boto3.session.Session() |
| 224 |
|
| 225 |
# Next, we create a resource client using our thread's session object |
| 226 |
s3 = session.resource('s3') |
| 227 |
|
| 228 |
# Put your thread-safe code here |
| 229 |
|
| 230 |
In the example above, each thread would have its own Boto3 session and |
| 231 |
its own instance of the S3 resource. This is a good idea because |
| 232 |
resources contain shared data when loaded and calling actions, accessing |
| 233 |
properties, or manually loading or reloading the resource can modify |
| 234 |
this data. |