06.05, Katowice AWS Summit Poland
16 min readPart 1/2

Organizing AWS Organizations with Terraform

Centralizing the management of IAM groups and users with AWS Organizations defined with Terraform to keep everything well-organized.

As your cloud workloads grow, the much desired fine-granularity you had in mind when you started becomes more and more difficult to keep. This rule applies to companies and teams of any shape and size. Thankfully and thoughtfully, AWS created a bunch of services that come in handy when adding some shape and structure to cloud topology starts to feel like a necessity.

In this article, we will take a closer look at one of such services, namely AWS Organizations, and show how it can be used to help encapsulate your projects/apps/accounts.

TL;DR

Game plan

In this article, we’ll be using Terraform to show you how to create:

Prerequisites

If you want to follow along, you’ll need:

Preparation

Before we can provision any resources related to our organization, we need to do some prep work. Let us kick off by creating a folder for our project:

mkdir aws-organizations-example && cd $_

Within that folder we are going to set up the minimal Terraform configuration required and then run the initialization command:

aws-organizations-example/provider.tfhcl
provider "aws" { version = "~> 2.30" region  = "eu-west-1"}

We will store Terraform state locally for the sake of simplicity. This is generally not recommended, but fear not, you may refer to the example provided on GitHub and see how S3 can be used as a backend.

Running terraform init:

➜  aws-organizations-example$ terraform initInitializing the backend...Initializing provider plugins...- Checking for available provider plugins...- Downloading plugin for provider "aws" (hashicorp/aws) 2.54.0...Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to seeany changes that are required for your infrastructure. All Terraform commandsshould now work.If you ever set or change modules or backend configuration for Terraform,rerun this command to reinitialize your working directory. If you forget, othercommands will detect it and remind you to do so if necessary.

Since we have laid the groundwork, we are now ready to commission our organization.

Creating an AWS Organization

First, we’ll create a module for organizations:

aws-organizations-example/organizations/main.tfhcl
resource "aws_organizations_organization" "org" { feature_set                   = var.feature_set aws_service_access_principals = var.feature_set == "ALL" ? var.aws_service_access_principals : null enabled_policy_types          = var.feature_set == "ALL" ? var.enabled_policy_types : null}
aws-organizations-example/organizations/variables.tfhcl
variable "feature_set" { description = "Per Terraform docs: 'Specify ALL (default) or CONSOLIDATED_BILLING.'" default     = "ALL"}variable "aws_service_access_principals" { description = "Per Terraform docs: 'List of AWS service principal names for which you want to enable integration with your organization. This is typically in the form of a URL, such as service-abbreviation.amazonaws.com. Organization must have feature_set set to ALL. For additional information, see the AWS Organizations User Guide.'" type        = list(string) default     = null}variable "enabled_policy_types" { description = "Per Terraform docs: 'List of Organizations policy types to enable in the Organization Root. Organization must have feature_set set to ALL. For additional information about valid policy types (e.g. SERVICE_CONTROL_POLICY and TAG_POLICY), see the AWS Organizations API Reference.'" type        = list(string) default     = null}
aws-organizations-example/organizations/outputs.tfhcl
output "roots" { description = "Per Terraform docs: 'List of organization roots. (...)'" value       = aws_organizations_organization.org.roots}

Next, we’ll initialize it:

aws-organizations-example/main.tfhcl
# Per AWS docs: "An entity that you create to consolidate your AWS accounts so that you can administer them as a single unit."module "root" { source               = "./organizations" enabled_policy_types = ["SERVICE_CONTROL_POLICY"]}

We must run terraform init again

➜  aws-organizations-example$ terraform initInitializing modules...- root in organizations# other outputTerraform has been successfully initialized!# other output

And finally, we are going to run terraform plan and, if it goes without a hitch, terraform apply:

Running terraform plan:

➜  aws-organizations-example$ terraform planRefreshing Terraform state in-memory prior to plan...The refreshed state will be used to calculate this plan, but will not bepersisted to local or remote state storage.------------------------------------------------------------------------An execution plan has been generated and is shown below.Resource actions are indicated with the following symbols: + createTerraform will perform the following actions: # module.root.aws_organizations_organization.org will be created + resource "aws_organizations_organization" "org" {     + accounts             = (known after apply)     + arn                  = (known after apply)     + enabled_policy_types = [         + "SERVICE_CONTROL_POLICY",       ]     + feature_set          = "ALL"     + id                   = (known after apply)     + master_account_arn   = (known after apply)     + master_account_email = (known after apply)     + master_account_id    = (known after apply)     + non_master_accounts  = (known after apply)     + roots                = (known after apply)   }Plan: 1 to add, 0 to change, 0 to destroy.------------------------------------------------------------------------Note: You didn't specify an "-out" parameter to save this plan, so Terraformcan't guarantee that exactly these actions will be performed if"terraform apply" is subsequently run.

Running terraform apply:

$  aws-organizations-example terraform applyAn execution plan has been generated and is shown below.Resource actions are indicated with the following symbols: + createTerraform will perform the following actions: # module.root.aws_organizations_organization.org will be created + resource "aws_organizations_organization" "org" {     + accounts             = (known after apply)     + arn                  = (known after apply)     + feature_set          = "ALL"     + enabled_policy_types = [         + "SERVICE_CONTROL_POLICY",       ]     + id                   = (known after apply)     + master_account_arn   = (known after apply)     + master_account_email = (known after apply)     + master_account_id    = (known after apply)     + non_master_accounts  = (known after apply)     + roots                = (known after apply)   }Plan: 1 to add, 0 to change, 0 to destroy.Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yesmodule.root.aws_organizations_organization.org: Creating...module.root.aws_organizations_organization.org: Creation complete after 5s [id=o-eipgmnk07c]Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

When the process completes, you’ll see the following message in the AWS Management Console for AWS Organizations:

Now, log in to your mailbox, check for a message from AWS and confirm the ownership of the email account:

Splendid! You’ve just created your first AWS Organization:

Now, let’s get down to creating users, groups and accounts.

Setting up groups and users

We’ll commence by creating a module for iam-groups. They will allow us to assign our users to either the administrators or the developers group (or both, though that would not make much sense):

aws-organizations-example/iam-groups/main.tfhcl
resource "aws_iam_group" "group" { name = var.name}data "aws_iam_policy" "AdministratorAccess" { arn = "arn:aws:iam::aws:policy/AdministratorAccess"}resource "aws_iam_group_policy_attachment" "AdministratorAccess" { count      = var.enable-AdministratorAccess ? 1 : 0 group      = aws_iam_group.group.name policy_arn = data.aws_iam_policy.AdministratorAccess.arn}data "aws_iam_policy" "AWSCodeBuildDeveloperAccess" { arn = "arn:aws:iam::aws:policy/AWSCodeBuildDeveloperAccess"}resource "aws_iam_group_policy_attachment" "AWSCodeBuildDeveloperAccess" { count      = var.enable-AWSCodeBuildDeveloperAccess ? 1 : 0 group      = aws_iam_group.group.name policy_arn = data.aws_iam_policy.AWSCodeBuildDeveloperAccess.arn}
aws-organizations-example/iam-groups/variables.tfhcl
variable "name" { description = "The name of the IAM group" type        = string}variable "enable-AdministratorAccess" { description = "A flag for enabling the AWS Managed AdministratorAccess policy" default     = false}variable "enable-AWSCodeBuildDeveloperAccess" { description = "A flag for enabling the AWS Managed AWSCodeBuildDeveloperAccess policy" default     = false}

Using the above pattern, you can easily add a plethora of other groups depending on your particular needs, e.g. a group for Accountants with access only to the billing section of AWS, etc.

Next, a module for iam-users would be recommended. So, let’s add it now:

aws-organizations-example/iam-users/main.tfhcl
resource "aws_iam_user" "this" { count = var.create-aws_iam_user ? 1 : 0 name = var.name path = var.path}resource "aws_iam_user_login_profile" "this" { count = var.create-aws_iam_user && var.create-aws_iam_user_login_profile ? 1 : 0 user                    = aws_iam_user.this[count.index].name pgp_key                 = var.pgp_key password_reset_required = var.password_reset_required}resource "aws_iam_access_key" "this" { count = var.create-aws_iam_user && var.create-aws_iam_access_key ? 1 : 0 user    = aws_iam_user.this[count.index].name pgp_key = var.pgp_key}resource "aws_iam_user_group_membership" "this" { count = var.create-aws_iam_user && var.create-aws_iam_user_group_membership ? 1 : 0 user   = aws_iam_user.this[count.index].name groups = var.groups}
aws-organizations-example/iam-users/variables.tfhcl
variable "create-aws_iam_user" { description = "A flag indicating whether an IAM user should be created" default     = true}variable "create-aws_iam_user_login_profile" { description = "A flag indicating whether an IAM user login (for AWS Console) should be created for a given user" default     = true}variable "create-aws_iam_access_key" { description = "A flag indicating whether an IAM access key ('a set of credentials that allow API requests to be made as an IAM user') should be created for a given user" default     = true}variable "create-aws_iam_user_group_membership" { description = "A flag indicating whether a group membership should be created for a given user" default     = true}variable "groups" { description = "A list of groups the user should become a member of" type        = list(string)}variable "name" { description = "The name of the IAM user" type        = string}variable "path" { description = "The path in which the IAM user should be created" type        = string default     = "/users/"}variable "force_destroy" { description = "After the terraform docs: 'When destroying this user, destroy even if it has non-Terraform-managed IAM access keys, login profile or MFA devices. Without force_destroy a user with non-Terraform-managed access keys and login profile will fail to be destroyed.'" default     = false}variable "password_reset_required" { description = "Whether the user should be forced to reset the generated password on first login." default     = false}variable "pgp_key" { description = "Either a base-64 encoded PGP public key, or a keybase username in the form keybase:username. Used to encrypt the password and the access key on output to the console." default     = ""}
aws-organizations-example/iam-users/outputs.tfhcl
output "aws_iam_user-credentials" { description = "The credentials of a given IAM user" value = {   name                        = var.create-aws_iam_user ? aws_iam_user.this[0].name : null   encrypted_password          = var.create-aws_iam_user_login_profile ? aws_iam_user_login_profile.this[0].encrypted_password : null   pgp_key                     = var.pgp_key   access-key-id               = var.create-aws_iam_access_key ? aws_iam_access_key.this[0].id : null   encrypted-secret-access-key = var.create-aws_iam_access_key ? aws_iam_access_key.this[0].encrypted_secret : null }}

Finally, to make it all work add the following lines to aws-organizations-example/main.tf:

aws-organizations-example/main.tfhcl
module "administrators" { source                     = "./iam-groups" name                       = "administrators" enable-AdministratorAccess = true}module "developers" { source                             = "./iam-groups" name                               = "developers" enable-AWSCodeBuildDeveloperAccess = true}module "admin_user_1" { source                            = "./iam-users" name                              = "admin_user_1" groups                            = ["administrators"] force_destroy                     = true pgp_key                           = "keybase:admin_user_1"}output "admin_user_1-aws_iam_user-credentials" { description = "The user's credentials" value       = module.admin_user_1.aws_iam_user-credentials}

Initialize the modules:

➜ aws-organizations-example$ terraform initInitializing modules...- administrators in iam-groups- developers in iam-groups- admin_user_1 in iam-users# (...)Terraform has been successfully initialized!

Run terraform plan:

➜  aws-organizations-example$ terraform plan# (...)An execution plan has been generated and is shown below.Resource actions are indicated with the following symbols: + createTerraform will perform the following actions: # module.admin_user_1.aws_iam_access_key.this[0] will be created + resource "aws_iam_access_key" "this" {     + encrypted_secret     = (known after apply)     + id                   = (known after apply)     + key_fingerprint      = (known after apply)     + pgp_key              = "keybase:admin_user_1"     + secret               = (sensitive value)     + ses_smtp_password    = (sensitive value)     + ses_smtp_password_v4 = (sensitive value)     + status               = (known after apply)     + user                 = "admin_user_1"   } # module.admin_user_1.aws_iam_user.this[0] will be created + resource "aws_iam_user" "this" {     + arn           = (known after apply)     + force_destroy = false     + id            = (known after apply)     + name          = "admin_user_1"     + path          = "/users/"     + unique_id     = (known after apply)   } # module.admin_user_1.aws_iam_user_group_membership.this[0] will be created + resource "aws_iam_user_group_membership" "this" {     + groups = [         + "administrators",       ]     + id     = (known after apply)     + user   = "admin_user_1"   } # module.admin_user_1.aws_iam_user_login_profile.this[0] will be created + resource "aws_iam_user_login_profile" "this" {     + encrypted_password      = (known after apply)     + id                      = (known after apply)     + key_fingerprint         = (known after apply)     + password_length         = 20     + password_reset_required = false     + pgp_key                 = "keybase:admin_user_1"     + user                    = "admin_user_1"   } # module.administrators.aws_iam_group.group will be created + resource "aws_iam_group" "group" {     + arn       = (known after apply)     + id        = (known after apply)     + name      = "administrators"     + path      = "/"     + unique_id = (known after apply)   } # module.administrators.aws_iam_group_policy_attachment.AdministratorAccess[0] will be created + resource "aws_iam_group_policy_attachment" "AdministratorAccess" {     + group      = "administrators"     + id         = (known after apply)     + policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"   } # module.developers.aws_iam_group.group will be created + resource "aws_iam_group" "group" {     + arn       = (known after apply)     + id        = (known after apply)     + name      = "developers"     + path      = "/"     + unique_id = (known after apply)   } # module.developers.aws_iam_group_policy_attachment.AWSCodeBuildDeveloperAccess[0] will be created + resource "aws_iam_group_policy_attachment" "AWSCodeBuildDeveloperAccess" {     + group      = "developers"     + id         = (known after apply)     + policy_arn = "arn:aws:iam::aws:policy/AWSCodeBuildDeveloperAccess"   }Plan: 8 to add, 0 to change, 0 to destroy.# (...)

And follow it up with terraform apply:

➜  aws-organizations-example$ terraform apply# (...)Apply complete! Resources: 8 added, 0 changed, 0 destroyed.Outputs:admin_user_1-aws_iam_user-credentials = { "access-key-id" = "AKIA57MNK6ZKUWMBKF5D" "encrypted-secret-access-key" = "wcFMA/i0/Lj0oDvo(...)aQzimK8TouGhNQA=" "encrypted_password" = "wcFMA/i0/Lj0oDvo(...)Sq5z4ggd7ZDhFwoA" "name" = "admin_user_1" "pgp_key" = "keybase:admin_user_1"}

You can then grab the output and safely pass it to the user, who can then decrypt the sensitive bits (e.g. the encrypted-secret-access-key), using their PGP in the following manner:

echo wcFMA7+BmSIBAU+8(...)3NYS4ns051PhDjsA | base64 --decode | keybase pgp decrypt
Note:

keybase pgp decrypt can be swapped with pgp --decrypt

That’s a wrap of part 1. In the next (and final) article, we’ll talk more closely about organizational units and service control policies. See you there!

Let's talk about your project

We'd love to answer your questions and help you thrive in the cloud.