Debian Codegraph @

docs/source/guide/error-handling.rst

raw
1 .. _guide_error-handling:
2
3 Error handling
4 ==============
5
6 Overview
7 --------
8 Boto3 provides many features to assist in navigating the errors and exceptions that you might encounter when interacting with AWS services.
9
10 Specifically, this guide provides details on the following:
11
12 * How to find what exceptions could be thrown by both Boto3 and AWS services
13 * How to catch and handle exceptions thrown by both Boto3 and AWS services
14 * How to parse error responses from AWS services
15
16 Why catch exceptions from AWS and Boto
17 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
18 * *Service limits and quotas* - Your call rate to an AWS service might be too frequent, or you might have reached a specific AWS service quota. In either case, without proper error handling you wouldn’t know or wouldn’t handle them.
19 * *Parameter validation and checking* - API requirements can change, especially across API versions. Catching these errors helps to identify if there’s an issue with the parameters you provide to any given API call.
20 * *Proper logging and messaging* - Catching errors and exceptions means you can log them. This can be instrumental in troubleshooting any code you write when interacting with AWS services.
21
22 Determining what exceptions to catch
23 ------------------------------------
24 Exceptions that you might encounter when using Boto3 will come from one of two sources: botocore or the AWS services your client is interacting with.
25
26 Botocore exceptions
27 ~~~~~~~~~~~~~~~~~~~
28 These exceptions are statically defined within the botocore package, a dependency of Boto3. The exceptions are related to issues with client-side behaviors, configurations, or validations. You can generate a list of the statically defined botocore exceptions using the following code:
29
30 .. code-block:: python
31
32 import botocore.exceptions
33
34 for key, value in sorted(botocore.exceptions.__dict__.items()):
35 if isinstance(value, type):
36 print(key)
37
38 .. tip::
39
40 .. raw:: html
41
42 <details>
43 <summary>Click to see a full list of static exceptions</summary>
44
45 .. code-block:: text
46
47 AliasConflictParameterError
48 ApiVersionNotFoundError
49 BaseEndpointResolverError
50 BotoCoreError
51 ChecksumError
52 ClientError
53 ConfigNotFound
54 ConfigParseError
55 ConnectTimeoutError
56 ConnectionClosedError
57 ConnectionError
58 CredentialRetrievalError
59 DataNotFoundError
60 EndpointConnectionError
61 EventStreamError
62 HTTPClientError
63 ImminentRemovalWarning
64 IncompleteReadError
65 InfiniteLoopConfigError
66 InvalidConfigError
67 InvalidDNSNameError
68 InvalidExpressionError
69 InvalidMaxRetryAttemptsError
70 InvalidRetryConfigurationError
71 InvalidS3AddressingStyleError
72 InvalidS3UsEast1RegionalEndpointConfigError
73 InvalidSTSRegionalEndpointsConfigError
74 MD5UnavailableError
75 MetadataRetrievalError
76 MissingParametersError
77 MissingServiceIdError
78 NoCredentialsError
79 NoRegionError
80 OperationNotPageableError
81 PaginationError
82 ParamValidationError
83 PartialCredentialsError
84 ProfileNotFound
85 ProxyConnectionError
86 RangeError
87 ReadTimeoutError
88 RefreshWithMFAUnsupportedError
89 SSLError
90 ServiceNotInRegionError
91 StubAssertionError
92 StubResponseError
93 UnStubbedResponseError
94 UndefinedModelAttributeError
95 UnknownClientMethodError
96 UnknownCredentialError
97 UnknownEndpointError
98 UnknownKeyError
99 UnknownParameterError
100 UnknownServiceError
101 UnknownServiceStyle
102 UnknownSignatureVersionError
103 UnseekableStreamError
104 UnsupportedS3AccesspointConfigurationError
105 UnsupportedS3ArnError
106 UnsupportedSignatureVersionError
107 UnsupportedTLSVersionWarning
108 ValidationError
109 WaiterConfigError
110 WaiterError
111
112 .. raw:: html
113
114 </details>
115
116 .. note::
117
118 You can view available descriptions of the botocore static exceptions `here <https://github.com/boto/botocore/blob/develop/botocore/exceptions.py>`_.
119
120 AWS service exceptions
121 ~~~~~~~~~~~~~~~~~~~~~~
122 AWS service exceptions are caught with the underlying botocore exception, ``ClientError``. After you catch this exception, you can parse through the response for specifics around that error, including the service-specific exception. Exceptions and errors from AWS services vary widely. You can quickly get a list of an AWS service’s exceptions using Boto3.
123
124 For a complete list of error responses from the services you’re using, consult the individual service’s `AWS documentation <https://docs.aws.amazon.com/>`_, specifically the error response section of the AWS service’s API reference. These references also provide context around the exceptions and errors.
125
126 Catching exceptions when using a low-level client
127 -------------------------------------------------
128
129 Catching botocore exceptions
130 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
131 Botocore exceptions are statically defined in the botocore package. Any Boto3 clients you create will use these same statically defined exception classes. The most common botocore exception you’ll encounter is ``ClientError``. This is a general exception when an error response is provided by an AWS service to your Boto3 client’s request.
132
133 Additional client-side issues with SSL negotiation, client misconfiguration, or AWS service validation errors will also throw botocore exceptions. Here’s a generic example of how you might catch botocore exceptions.
134
135 .. code-block:: python
136
137 import botocore
138 import boto3
139
140 client = boto3.client('aws_service_name')
141
142 try:
143 client.some_api_call(SomeParam='some_param')
144
145 except botocore.exceptions.ClientError as error:
146 # Put your error handling logic here
147 raise error
148
149 except botocore.exceptions.ParamValidationError as error:
150 raise ValueError('The parameters you provided are incorrect: {}'.format(error))
151
152 Parsing error responses and catching exceptions from AWS services
153 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
154 Unlike botocore exceptions, AWS service exceptions aren't statically defined in Boto3. This is due to errors and exceptions from AWS services varying widely and being subject to change. To properly catch an exception from an AWS service, you must parse the error response from the service. The error response provided to your client from the AWS service follows a common structure and is minimally processed and not obfuscated by Boto3.
155
156 Using Boto3, the error response from an AWS service will look similar to a success response, except that an ``Error`` nested dictionary will appear with the ``ResponseMetadata`` nested dictionary. Here is an example of what an error response might look like::
157
158 {
159 'Error': {
160 'Code': 'SomeServiceException',
161 'Message': 'Details/context around the exception or error'
162 },
163 'ResponseMetadata': {
164 'RequestId': '1234567890ABCDEF',
165 'HostId': 'host ID data will appear here as a hash',
166 'HTTPStatusCode': 400,
167 'HTTPHeaders': {'header metadata key/values will appear here'},
168 'RetryAttempts': 0
169 }
170 }
171
172 Boto3 classifies all AWS service errors and exceptions as ``ClientError`` exceptions. When attempting to catch AWS service exceptions, one way is to catch ``ClientError`` and then parse the error response for the AWS service-specific exception.
173
174 Using Amazon Kinesis as an example service, you can use Boto3 to catch the exception ``LimitExceededException`` and insert your own logging message when your code experiences request throttling from the AWS service.
175
176 .. code-block:: python
177
178 import botocore
179 import boto3
180 import logging
181
182 # Set up our logger
183 logging.basicConfig(level=logging.INFO)
184 logger = logging.getLogger()
185
186 client = boto3.client('kinesis')
187
188 try:
189 logger.info('Calling DescribeStream API on myDataStream')
190 client.describe_stream(StreamName='myDataStream')
191
192 except botocore.exceptions.ClientError as error:
193 if error.response['Error']['Code'] == 'LimitExceededException':
194 logger.warn('API call limit exceeded; backing off and retrying...')
195 else:
196 raise error
197
198 .. note::
199
200 The Boto3 ``standard`` retry mode will catch throttling errors and exceptions, and will back off and retry them for you.
201
202 Additionally, you can also access some of the dynamic service-side exceptions from the client’s exception property. Using the previous example, you would need to modify only the ``except`` clause.
203
204 .. code-block:: python
205
206 except client.exceptions.LimitExceedException as error:
207 logger.warn('API call limit exceeded; backing off and retrying...')
208
209 .. note::
210
211 Catching exceptions through ``ClientError`` and parsing for error codes is still the best way to catch **all** service-side exceptions and errors.
212
213 Catching exceptions when using a resource client
214 ------------------------------------------------
215
216 When using ``Resource`` classes to interact with certain AWS services, catching exceptions and errors is a similar experience to using a low-level client.
217
218 Parsing for error responses uses the same exact methodology outlined in the low-level client section. Catching exceptions through the client’s ``exceptions`` property is slightly different, as you’ll need to access the client’s ``meta`` property to get to the exceptions.
219
220 .. code-block:: python
221
222 client.meta.client.exceptions.SomeServiceException
223
224 Using Amazon S3 as an example resource service, you can use the client’s exception property to catch the ``BucketAlreadyExists`` exception. And you can still parse the error response to get the bucket name that's passed in the original request.
225
226 .. code-block:: python
227
228 import botocore
229 import boto3
230
231 client = boto3.resource('s3')
232
233 try:
234 client.create_bucket(BucketName='amzn-s3-demo-bucket')
235
236 except client.meta.client.exceptions.BucketAlreadyExists as err:
237 print("Bucket {} already exists!".format(err.response['Error']['BucketName']))
238 raise err
239
240 Discerning useful information from error responses
241 --------------------------------------------------
242 As stated previously in this guide, for details and context around specific AWS service exceptions, see the individual service’s `AWS documentation <https://docs.aws.amazon.com/>`_, specifically the error response section of the AWS service’s API reference.
243
244 Botocore exceptions will have detailed error messaging when those exceptions are thrown. These error messages provide details and context around the specific exception thrown. Descriptions of these exceptions can be viewed `here <https://github.com/boto/botocore/blob/develop/botocore/exceptions.py>`_.
245
246 Outside of specific error or exception details and messaging, you might want to extract additional metadata from error responses:
247
248 * *Exception class* - You can use this data to build logic around, or in response to, these errors and exceptions.
249 * *Error message* - This data can help reason about the cause of the error and provide more context around the issue. These messages are subject to change and should not be relied upon in code.
250 * *Request ID and HTTP status code* - AWS service exceptions might still be vague or lacking in details. If this occurs, contacting customer support and providing the AWS service name, error, error message, and request ID could allow a support engineer to further look into your issue.
251
252 Using a low-level Amazon SQS client, here’s an example of catching a generic or vague exception from the AWS service, and parsing out useful metadata from the error response.
253
254 .. code-block:: python
255
256 import botocore
257 import boto3
258
259 client = boto3.client('sqs')
260 queue_url = 'SQS_QUEUE_URL'
261
262 try:
263 client.send_message(QueueUrl=queue_url, MessageBody=('some_message'))
264
265 except botocore.exceptions.ClientError as err:
266 if err.response['Error']['Code'] == 'InternalError': # Generic error
267 # We grab the message, request ID, and HTTP code to give to customer support
268 print('Error Message: {}'.format(err.response['Error']['Message']))
269 print('Request ID: {}'.format(err.response['ResponseMetadata']['RequestId']))
270 print('Http code: {}'.format(err.response['ResponseMetadata']['HTTPStatusCode']))
271 else:
272 raise err