| 1 |
.. _guide_clients: |
| 2 |
|
| 3 |
Low-level clients |
| 4 |
================= |
| 5 |
Clients provide a low-level interface to AWS whose methods map close to 1:1 |
| 6 |
with service APIs. All service operations are supported by clients. Clients |
| 7 |
are generated from a JSON service definition file. |
| 8 |
|
| 9 |
Creating clients |
| 10 |
---------------- |
| 11 |
Clients are created in a similar fashion to resources:: |
| 12 |
|
| 13 |
import boto3 |
| 14 |
|
| 15 |
# Create a low-level client with the service name |
| 16 |
sqs = boto3.client('sqs') |
| 17 |
|
| 18 |
It is also possible to access the low-level client from an existing |
| 19 |
resource:: |
| 20 |
|
| 21 |
# Create the resource |
| 22 |
sqs_resource = boto3.resource('sqs') |
| 23 |
|
| 24 |
# Get the client from the resource |
| 25 |
sqs = sqs_resource.meta.client |
| 26 |
|
| 27 |
Service operations |
| 28 |
------------------ |
| 29 |
Service operations map to client methods of the same name and provide |
| 30 |
access to the same operation parameters via keyword arguments:: |
| 31 |
|
| 32 |
# Make a call using the low-level client |
| 33 |
response = sqs.send_message(QueueUrl='...', MessageBody='...') |
| 34 |
|
| 35 |
As can be seen above, the method arguments map directly to the associated |
| 36 |
`SQS API <http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html>`_. |
| 37 |
|
| 38 |
.. note:: |
| 39 |
|
| 40 |
The method names have been snake-cased for better looking Python code. |
| 41 |
|
| 42 |
Parameters **must** be sent as keyword arguments. They will not work |
| 43 |
as positional arguments. |
| 44 |
|
| 45 |
Handling responses |
| 46 |
------------------ |
| 47 |
Responses are returned as python dictionaries. It is up to you to traverse |
| 48 |
or otherwise process the response for the data you need, keeping in mind |
| 49 |
that responses may not always include all of the expected data. In the |
| 50 |
example below, ``response.get('QueueUrls', [])`` is used to ensure that a |
| 51 |
list is always returned, even when the response has no key ``'QueueUrls'``:: |
| 52 |
|
| 53 |
# List all your queues |
| 54 |
response = sqs.list_queues() |
| 55 |
for url in response.get('QueueUrls', []): |
| 56 |
print(url) |
| 57 |
|
| 58 |
The ``response`` in the example above looks something like this: |
| 59 |
|
| 60 |
.. code-block:: json |
| 61 |
|
| 62 |
{ |
| 63 |
"QueueUrls": [ |
| 64 |
"http://url1", |
| 65 |
"http://url2", |
| 66 |
"http://url3" |
| 67 |
] |
| 68 |
} |
| 69 |
|
| 70 |
Waiters |
| 71 |
------- |
| 72 |
Waiters use a client's service operations to poll the status of an AWS resource |
| 73 |
and suspend execution until the AWS resource reaches the state that the |
| 74 |
waiter is polling for or a failure occurs while polling. |
| 75 |
Using clients, you can learn the name of each waiter that a client has access |
| 76 |
to:: |
| 77 |
|
| 78 |
import boto3 |
| 79 |
|
| 80 |
s3 = boto3.client('s3') |
| 81 |
sqs = boto3.client('sqs') |
| 82 |
|
| 83 |
# List all of the possible waiters for both clients |
| 84 |
print("s3 waiters:") |
| 85 |
s3.waiter_names |
| 86 |
|
| 87 |
print("sqs waiters:") |
| 88 |
sqs.waiter_names |
| 89 |
|
| 90 |
Note if a client does not have any waiters, it will return an empty list when |
| 91 |
accessing its ``waiter_names`` attribute:: |
| 92 |
|
| 93 |
s3 waiters: |
| 94 |
[u'bucket_exists', u'bucket_not_exists', u'object_exists', u'object_not_exists'] |
| 95 |
sqs waiters: |
| 96 |
[] |
| 97 |
|
| 98 |
Using a client's ``get_waiter()`` method, you can obtain a specific waiter |
| 99 |
from its list of possible waiters:: |
| 100 |
|
| 101 |
# Retrieve waiter instance that will wait till a specified |
| 102 |
# S3 bucket exists |
| 103 |
s3_bucket_exists_waiter = s3.get_waiter('bucket_exists') |
| 104 |
|
| 105 |
Then to actually start waiting, you must call the waiter's ``wait()`` method |
| 106 |
with the method's appropriate parameters passed in:: |
| 107 |
|
| 108 |
# Begin waiting for the S3 bucket, amzn-s3-demo-bucket, to exist |
| 109 |
s3_bucket_exists_waiter.wait(Bucket='amzn-s3-demo-bucket') |
| 110 |
|
| 111 |
Multithreading or multiprocessing with clients |
| 112 |
---------------------------------------------- |
| 113 |
|
| 114 |
Unlike Resources and Sessions, clients **are** generally *thread-safe*. |
| 115 |
There are some caveats, defined below, to be aware of though. |
| 116 |
|
| 117 |
Caveats |
| 118 |
~~~~~~~ |
| 119 |
|
| 120 |
**Multi-Processing:** While clients are *thread-safe*, they cannot be |
| 121 |
shared across processes due to their networking implementation. Doing so |
| 122 |
may lead to incorrect response ordering when calling services. |
| 123 |
|
| 124 |
**Shared Metadata:** Clients expose metadata to the end user through a |
| 125 |
few attributes (namely ``meta``, ``exceptions`` and ``waiter_names``). |
| 126 |
These are safe to read but any mutations should not be considered |
| 127 |
thread-safe. |
| 128 |
|
| 129 |
**Custom** \ `Botocore Events`_\ **:** Botocore (the library Boto3 is |
| 130 |
built on) allows advanced users to provide their own custom event hooks |
| 131 |
which may interact with boto3’s client. The majority of users will not |
| 132 |
need to use these interfaces, but those that do should no longer |
| 133 |
consider their clients thread-safe without careful review. |
| 134 |
|
| 135 |
.. note:: |
| 136 |
``boto3.client('<service_name>')`` is an alias for creating a client with a |
| 137 |
shared default session. Invoking ``boto3.client()`` inside of a concurrent |
| 138 |
context may result in response ordering issues or interpreter failures |
| 139 |
from underlying SSL modules. |
| 140 |
|
| 141 |
General Example |
| 142 |
~~~~~~~~~~~~~~~ |
| 143 |
|
| 144 |
.. code:: python |
| 145 |
|
| 146 |
import boto3.session |
| 147 |
from concurrent.futures import ThreadPoolExecutor |
| 148 |
|
| 149 |
def do_s3_task(client, task_definition): |
| 150 |
# Put your thread-safe code here |
| 151 |
|
| 152 |
def my_workflow(): |
| 153 |
# Create a session and use it to make our client |
| 154 |
session = boto3.session.Session() |
| 155 |
s3_client = session.client('s3') |
| 156 |
|
| 157 |
# Define some work to be done, this can be anything |
| 158 |
my_tasks = [ ... ] |
| 159 |
|
| 160 |
# Dispatch work tasks with our s3_client |
| 161 |
with ThreadPoolExecutor(max_workers=8) as executor: |
| 162 |
futures = [executor.submit(do_s3_task, s3_client, task) for task in my_tasks] |
| 163 |
|
| 164 |
.. _Botocore Events: https://botocore.amazonaws.com/v1/documentation/api/latest/topics/events.html |