Migrating security groups and network access control lists across Amazon VPCs
A brief look into our approach to keeping VPC environments in sync and secure.
Amazon Virtual Private Cloud
AWS Tools and SDKs
Boto3 (AWS SDK for Python)
Python
Our everyday work often finds us in situations when we need to recreate identical environments in virtual networks (VPC) in the AWS cloud. We would like to share some of our experiences while focusing on two security features which are fundamental to secure virtual networking and access to VPC services within AWS: security groups and network access control lists. One could call them the first and second line of security.
Their key differences can be summed up as follows:
| Security group | Network access control list |
|---|---|
| Stateful – incoming traffic is allowed by default | Stateless – specific rules are required to allow incoming traffic |
| Operates on an instance level: one group can be assigned to multiple instances or multiple groups can be assigned to a single instance | Operates on the subnet level: one ACL network can be assigned to a single subnet or to multiple subnets |
| Only ‘allow’ rules control the traffic | Both ‘allow’ and ‘deny’ rules control the traffic |
| Complete set of rules is analyzed, and the analysis dictates if access is allowed | Rules are analyzed in the order of their IDs and then applied |
| Rules are activated now when the group is assigned (at the time of startup or on an active instance) | Rules are activated instantly after assigning to a subnet |
The diagram below shows where security groups and network ACLs position themselves within a network communication flow:
What’s it all for?
Usually, to simplify and automate the SG and NACL migration process between different AWS accounts or regions. Unfortunately, at present, we only have the ability to copy SG in a single region and a single account to a VPC.
To execute such an operation, you need to:
- Mark the group you want to copy in the EC2 service in the Security Groups section,
- Pick Actions and then Copy to new in the menu:
- The next window shows automatic rules from the previous SG and all you need to pick is the VPC and set the name and description of the new group.
Considering this needs to be done for every single region and every account, it’s not exactly as simple as we’d like it to be within a comprehensive migration process or the cloning of a given environment. Ideally we’d want it not just to be simple — automated would be even better.
So, how did we do it?
We wrote a tool in Python which uses the Boto3 library along with AWS’ API to simplify the migration of security groups and network access control lists. Our script, depending on the scenario, connects to one or two AWS accounts and automatically copies SG and NACL.
We incorporated two authorization methods which are used during the awscli configuration: key pairs and IAM roles. We configured the respective profiles and credentials for either of those access patterns, adjusted to either of the following scenarios:
- single AWS account migration — one profile,
- two different AWS accounts — two profiles (
scrProfileA,dstProfileB).
Profiles should be placed in the correct configuration files (the localization are for the following operating systems: Linux, OS X, Unix, while for Windows it is: %UserProfile%/.aws ):
[default]aws_access_key_id=AKIAIOSDEFAULTEXAMPLEaws_secret_access_key=wJalrXUtnFEfgddD57MDENG/bPxRfiCYEXAMPLEKEY[srcProfileA]aws_access_key_id=AKIAIOSFODNN7EXAMPLEaws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY[srcProfileB]aws_access_key_id=AKIAI44QH8DHBEXAMPLEaws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY
[profile srcProfileA]region=us-west-2output=json[profile dstProfileB]region=us-east-1output=text[profile srcProfileRoleA]role_arn = arn:aws:iam::123456789012:role/adminAregion = eu-west-2source_profile = default[profile dstProfileRoleB]role_arn = arn:aws:iam::123456789021:role/adminBregion = us-east-1source_profile = default
If you are using multi-factor authentication for your IAM accounts, you will need to add mfa_serial to the respective sections.
Having configured access from our workstation/instance, we can use our tool to execute the migration of security groups or network access control lists, for instance:
- Copying SG using
srcProfileAanddstProfileBprofiles:
python sg_nacl_migration.py –sg_migration –aws_src_profile srcProfileA –aws_dst_profile dstProfileB –sg_id SG_ID –aws_dst_vpc DST_VPC_IDAs a result, the target VPC obtains the same security group as the source one. Additionally, there is a possibility of doing a search in all the groups by tag and migrating all the selected ones at the same time.
- Copying NACL using roles on the source and target account:
python sg_nacl_migration.py -sr arn:aws:iam::SRC_ACCOUNT_ID:role/ROLE_NAME -dr arn:aws:iam::DST_ACCOUNT_ID:role/ROLE_NAME –region AWS_SOURCE_REGION –dstregion AWS_DESTINATION_REGION –nacl_migration –aws_dst_vpc VPC_DESTINATION_ID –srcnacl_id SOURCE_NACL_IDAs a result, the target account will have a network access control list with a set of rules for incoming and outgoing traffic identical to the source NACL.
If we want to assign it to appropriate subnets on the target account, then we need to add the srcnacl_id_association parameter and provide the ID of the current NACL subnet arrangement.
What did we achieve?
We managed to automate a significant chunk of the work involved in a migration process. Thanks to our solution we do not have to manually repeat the same actions, and the time needed to execute such a migration decreased considerably.
Another major benefit of implementing this solution into the migration or cloning process is the elimination of human errors that had a tendency to creep up only after the process was considered finished. Those were usually either incorrect rules or lack of rules altogether.
While using this solution on a daily basis, we also noticed its benefits for the programming environments where CloudFormation plays a major role. It forms the base infrastructure while the script syncs specified SG and NACL from a specified VPC, without the need for cyclical environment updates via CloudFormation templates.
