| 1 |
.. Copyright 2010-2017 Amazon.com, Inc. or its affiliates. All Rights Reserved. |
| 2 |
|
| 3 |
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 |
| 4 |
International License (the "License"). You may not use this file except in compliance with the |
| 5 |
License. A copy of the License is located at http://creativecommons.org/licenses/by-nc-sa/4.0/. |
| 6 |
|
| 7 |
This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, |
| 8 |
either express or implied. See the License for the specific language governing permissions and |
| 9 |
limitations under the License. |
| 10 |
|
| 11 |
.. _aws-boto3-cw-subscription-filters: |
| 12 |
|
| 13 |
#################################################### |
| 14 |
Using subscription filters in Amazon CloudWatch Logs |
| 15 |
#################################################### |
| 16 |
|
| 17 |
This Python example shows you how to create and delete filters for log events in CloudWatch Logs. |
| 18 |
|
| 19 |
The scenario |
| 20 |
============ |
| 21 |
|
| 22 |
Subscriptions provide access to a real-time feed of log events from CloudWatch Logs and deliver that |
| 23 |
feed to other services, such as an Amazon Kinesis stream or AWS Lambda, for custom processing, |
| 24 |
analysis, or loading to other systems. A subscription filter defines the pattern to use for filtering |
| 25 |
which log events are delivered to your AWS resource. |
| 26 |
|
| 27 |
In this example, Python code is used to list, create, and delete a subscription |
| 28 |
filter in CloudWatch Logs. The destination for the log events is a Lambda function. |
| 29 |
The code uses the AWS SDK for Python to manage subscription filters using these methods of the |
| 30 |
CloudWatchLogs client class: |
| 31 |
|
| 32 |
* `get_paginator('describe_subscription_filters') <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html#CloudWatchLogs.Client.get_paginator>`_. |
| 33 |
|
| 34 |
* `put_subscription_filter <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html#CloudWatchLogs.Client.put_subscription_filter>`_. |
| 35 |
|
| 36 |
* `delete_subscription_filter <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html#CloudWatchLogs.Client.delete_subscription_filter>`_. |
| 37 |
|
| 38 |
For more information about CloudWatch Logs subscriptions, see |
| 39 |
Real-time `Processing of Log Data with Subscriptions <http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Subscriptions.html>`_ |
| 40 |
in the Amazon CloudWatch Logs User Guide. |
| 41 |
|
| 42 |
All the example code for the Amazon Web Services (AWS) SDK for Python is available `here on GitHub <https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/python/example_code>`_. |
| 43 |
|
| 44 |
Prerequisite tasks |
| 45 |
================== |
| 46 |
|
| 47 |
* Configure your AWS credentials, as described in :doc:`quickstart`. |
| 48 |
|
| 49 |
* Create a Lambda function as the destination for log events. You will need to use the ARN of this |
| 50 |
function. For more information about setting up a Lambda function, see |
| 51 |
`Subscription Filters with AWS Lambda <http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html#LambdaFunctionExample>`_ |
| 52 |
in the *Amazon CloudWatch Logs User Guide*. |
| 53 |
|
| 54 |
* Create an IAM role whose policy grants permission to invoke the Lambda function you created and |
| 55 |
grants full access to CloudWatch Logs or apply the following policy to the execution role you create |
| 56 |
for the Lambda function. For more information about creating an IAM role, see |
| 57 |
Creating a Role to `Delegate Permissions to an AWS Service <http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html>`_ |
| 58 |
in the *IAM User Guide*. |
| 59 |
|
| 60 |
Use the following role policy when creating the IAM role. |
| 61 |
|
| 62 |
.. code-block:: python |
| 63 |
|
| 64 |
{ |
| 65 |
"Version": "2012-10-17", |
| 66 |
"Statement": [ |
| 67 |
{ |
| 68 |
"Effect": "Allow", |
| 69 |
"Action": [ |
| 70 |
"logs:CreateLogGroup", |
| 71 |
"logs:CreateLogStream", |
| 72 |
"logs:PutLogEvents" |
| 73 |
], |
| 74 |
"Resource": "arn:aws:logs:*:*:*" |
| 75 |
}, |
| 76 |
{ |
| 77 |
"Effect": "Allow", |
| 78 |
"Action": [ |
| 79 |
"lambda:InvokeFunction" |
| 80 |
], |
| 81 |
"Resource": [ |
| 82 |
"*" |
| 83 |
] |
| 84 |
} |
| 85 |
] |
| 86 |
} |
| 87 |
|
| 88 |
List existing subscription filters |
| 89 |
================================== |
| 90 |
|
| 91 |
List the subscription filters for the specified log group. |
| 92 |
|
| 93 |
The example below shows how to: |
| 94 |
|
| 95 |
* List subscription filters using |
| 96 |
`get_paginator('describe_subscription_filters') <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html#CloudWatchLogs.Client.get_paginator>`_. |
| 97 |
|
| 98 |
For more information about paginators see, :doc:`paginators` |
| 99 |
|
| 100 |
Example |
| 101 |
------- |
| 102 |
|
| 103 |
.. code-block:: python |
| 104 |
|
| 105 |
import boto3 |
| 106 |
|
| 107 |
# Create CloudWatchLogs client |
| 108 |
cloudwatch_logs = boto3.client('logs') |
| 109 |
|
| 110 |
# List subscription filters through the pagination interface |
| 111 |
paginator = cloudwatch_logs.get_paginator('describe_subscription_filters') |
| 112 |
for response in paginator.paginate(logGroupName='GROUP_NAME'): |
| 113 |
print(response['subscriptionFilters']) |
| 114 |
|
| 115 |
|
| 116 |
|
| 117 |
|
| 118 |
Create a subscription filter |
| 119 |
============================ |
| 120 |
|
| 121 |
Create or update a subscription filter and associate it with the specified log group. |
| 122 |
|
| 123 |
The example below shows how to: |
| 124 |
|
| 125 |
* Create a subscription filter using |
| 126 |
`put_subscription_filter <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html#CloudWatchLogs.Client.put_subscription_filter>`_. |
| 127 |
|
| 128 |
Example |
| 129 |
------- |
| 130 |
|
| 131 |
.. code-block:: python |
| 132 |
|
| 133 |
import boto3 |
| 134 |
|
| 135 |
# Create CloudWatchLogs client |
| 136 |
cloudwatch_logs = boto3.client('logs') |
| 137 |
|
| 138 |
# Create a subscription filter |
| 139 |
cloudwatch_logs.put_subscription_filter( |
| 140 |
destinationArn='LAMBDA_FUNCTION_ARN', |
| 141 |
filterName='FILTER_NAME', |
| 142 |
filterPattern='ERROR', |
| 143 |
logGroupName='LOG_GROUP', |
| 144 |
) |
| 145 |
|
| 146 |
|
| 147 |
Delete a subscription filter |
| 148 |
============================ |
| 149 |
|
| 150 |
The example below shows how to: |
| 151 |
|
| 152 |
* Delete a subscription filter. using |
| 153 |
`delete_subscription_filter <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html#CloudWatchLogs.Client.delete_subscription_filter>`_. |
| 154 |
|
| 155 |
Example |
| 156 |
------- |
| 157 |
|
| 158 |
.. code-block:: python |
| 159 |
|
| 160 |
import boto3 |
| 161 |
|
| 162 |
# Create CloudWatchLogs client |
| 163 |
cloudwatch_logs = boto3.client('logs') |
| 164 |
|
| 165 |
# Delete a subscription filter |
| 166 |
cloudwatch_logs.delete_subscription_filter( |
| 167 |
filterName='FILTER_NAME', |
| 168 |
logGroupName='LOG_GROUP', |
| 169 |
) |
| 170 |
|
| 171 |
|
| 172 |
|