ArticleAWSLambda

How to test service events in AWS Organizations & AWS CloudTrail

Introduction

Today, I’d like to show you how to test CloudTrail service events (the so-called heart of AWS actions monitoring, where you can find detailed information on what and where was run and who run it – in other words, a very helpful tool when it comes to audits or verifying operations on our accounts).

In this particular example we will focus on the process of creating an account with AWS Organizations, as this service allows us to manage all accounts inside our organization with just one (“master”) account. I don’t want to go into details about both CloudTrail or Organizations, so if you want to learn more, check the AWS documentation:

  1. AWS CloudTrail – https://docs.aws.amazon.com/awscloudtrail
  2. AWS Organizations – https://docs.aws.amazon.com/organizations

In this article I want to show you how to run tests on AWS account creation events, and how to react properly to those events by invoking lambda functions with CloudWatch and SNS services. Lambda function logic was intentionally simplified to inserting short information into logs. Normally such function performs additional business operations or transfers data over to other services that are triggered with a creation of a new AWS account within the given framework.

Test architecture & event flow through different services

In this scenario we will use the following AWS services:

  1. AWS Organizations,
  2. AWS CloudTrail,
  3. AWS CloudWatch,
  4. AWS Simple Notification Service,
  5. AWS Lambda.

The event flow, in this case a setting up of an AWS account, is shown on a diagram below:

In short, our scenario will look like this:

  1. We will create a new account with AWS Organizations service from the level of our main account (we will simulate this part with an event prepared in advance in a form of a json file invoked with awscli, so we won’t have to generate new AWS accounts needlessly just for the sake of testing.),
  2. The event will be recorded with AWS CloudTrail service and then transferred to AWS CloudWatch,
  3. Using an event model prepared with CloudWatch service on the event rules level, we will filter only those events that we wish to transfer further,
  4. Specified events will be transferred to SNS service (we configured it like this for other purposes; you can transfer the event straight to lambda at this point),
  5. We will transfer the event from SNS service to lambda in order to process it (to simplify the whole process and to show the schematics we will only insert the created AWS account number into logs.)

Service configuration & testing

The configuration process will be carried out in four steps and all operations will be executed in us-east-1 region (N. Virginia):

  1. We install AWS CLI on the workstation and configure our AWS account access as users or with a permission to execute operation “aws events put-events”,
  2. Using a template, we create a new Python 3.7 function in AWS lambda service and we name it, for example: “lambda-aws-account-creation”. Then, we paste a code into function body:
import boto3
import json

def lambda_handler(event, context):
    """
    Main lambda function
    """
    message = json.loads(event['Records'][0]['Sns']['Message'])
    accountID = message['detail']['serviceEventDetails']['createAccountStatus']['accountId']
    print("AWS Account ID: ", accountID)

We save the changes, leaving other parameters as default.

3. We move to AWS SNS service and select “Topics” section. We create a new one, we give it a name – “aws-account-creation” – and then, we access it. The list in the “Subscriptions” section should be empty. We will add one with a “Create subscription” button. Next, we need to fill the blanks in the new window:
a) Protocol -> AWS Lambda
b) Endpoint -> lambda-aws-account-creation
Finally, we confirm it by pressing the “Create subscription” button one more time.

Now, we should have a lambda function subscribed to our SNS topic, like this:4. Next, we switch to AWS Cloud Watch service and in the “Events -> Rules” section we create a new rule. We chose “Event Pattern”, and change “Build event pattern to match events by service” to “Custom event pattern”. Then, we paste the following code:

{
   "source":[
       "test.aws.organizations"
   ],
   "detail":{
       "eventSource":[
           "organizations.amazonaws.com"
       ],
       "eventName":[
           "CreateAccountResult"
       ],
       "serviceEventDetails":{
           "createAccountStatus":{
               "state":[
                   "SUCCEEDED"
               ]
           }
       }
   }
}

Next, we add SNS target with an “Add target” button (on the right). Then, we set “SNS topic” and chose “aws-account-creation”. We move on by clicking on a “Configure details” button.

Finally, we name the rule, for example “aws-account-creation-cloudwatch-rule”, and finish the process by clicking on a “Create rule” button.

5. Now, we need to go back to our lambda, created in step 2, to make sure the SNS topic was subscribed correctly and to check if lambda has access to CloudWatch service logs, like on a diagram below:

If that’s the case, we can start running tests, using aws cli console and our ready-made event from json file.

To run the first test, we will need one more file with an event template equivalent to the form our lambda function takes after it completes the whole process, starting from CloudTrail service. So, we name the file, for example “cloudtrail-test-event.json”, and we paste into it the following lines:

[
  {
    "Source": "test.aws.organizations",
    "Time": "2019-08-30T21:42:18Z",
    "Resources": [],
    "DetailType": "myTestType",
    "Detail": "{ \"eventVersion\": \"1.05\",\r\n\t\t\"userIdentity\":{\r\n\t\t\t\"accountId\": \"123456789012\",\r\n\t\t\t\"invokedBy\": \"AWS Internal\"\r\n\t\t},\r\n\t\t\"eventTime\": \"2018-08-30T21:42:18Z\",\r\n\t\t\"eventSource\": \"organizations.amazonaws.com\",\r\n\t\t\"eventName\": \"CreateAccountResult\",\r\n\t\t\"awsRegion\": \"us-east-1\",\r\n\t\t\"sourceIPAddress\": \"AWS Internal\",\r\n\t\t\"userAgent\": \"AWS Internal\",\r\n\t\t\"eventID\": \"0000000-0000-0000-1111-123456789012\",\r\n\t\t\"readOnly\": \"false\",\r\n\t\t\"eventType\": \"AwsServiceEvent\",\r\n\t\t\"serviceEventDetails\": {\r\n\t\t\t\"createAccountStatus\": {\r\n\t\t\t\t\"id\": \"car-000000000000000000000000000000\",\r\n\t\t\t\t\"state\": \"SUCCEEDED\",\r\n\t\t\t\t\"accountName\": \"****\",\r\n\t\t\t\t\"accountId\": \"123456789000\",\r\n\t\t\t\t\"requestedTimestamp\": \"Aug 30, 2018 9:42:14 PM\",\r\n\t\t\t\t\"completedTimestamp\": \"Aug 30, 2018 9:42:18 PM\"\r\n\t\t\t}\r\n\t\t} }"
  }
]

We save it and, using our template, we generate the first test event with the following command:

aws --region us-east-1 events put-events --entries file://cloudtail-test-event.json

If we managed to transfer our event to AWS successfully, we should get the following result in the console:

{
    "FailedEntryCount": 0, 
    "Entries": [
        {
            "EventId": "b30dd1b6-76bb-dde6-da92-d71a59941966"
        }
    ]
}

Once we do, we can test the lambda monitoring section to see if the function was triggered at all.

As you can see on the diagram above, we triggered the function with our test event 3 times (you don’t need to run multiple times, just once). In other words, it works correctly and the tests were successful.

Additionally, we can double check if the function was triggered 3 times by looking into CloudWatch logs:

And that confirms the data presented on the lambda monitoring diagram.

I want to add that the whole scenario is focused on creating an account with AWS Organizations service and the event source for CloudWatch filter rule mentioned in step 4 was intentionally set to:

"source":[
       "test.aws.organizations"
   ],

If we had set it to “aws.organizations”, we would have got a following error:

{
    "FailedEntryCount": 1, 
    "Entries": [
        {
            "ErrorCode": "NotAuthorizedForSourceException", 
            "ErrorMessage": "Not authorized for the source."
        }
    ]
}

This would happen, because AWS reserves all sources with an “aws.*” prefix. Therefore, we couldn’t use an event with that source, but if you want to use the event template for CloudWatch rule in your AWS service environment, simply change the source to:

"source":[
       "aws.organizations"
   ],

Now, all events in your AWS Organization service will be filtered by a CloudWatch rule.

Additionally, we can check if the event template we use shows the given event in the “Show sample event(s)” section, which is a good thing to do.

And so, we conclude the first stage of verification. Now, we can run further tests in our environment, using other services.

Summary

Additional information on event types, can be found at:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html.

And if you need to implement event testing in the app’s code, you’ll find more useful information here:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/AddEventsPutEvents.html.

I hope our method of testing will help you run the events filter rules verification process in AWS more efficiently.