AWSAWS OrganizationsIAMSCPTerraform

Don’t panic, organize (part 1 of 2)

By 04/21/2020No Comments

Introduction

As your cloud workloads grow, the much-desired fine-granularity you had in mind when you started becomes more and more difficult to keep. This rule applies to companies and teams of any shape and size. Thankfully and thoughtfully, AWS created a bunch of services that come in handy when adding some shape and structure to cloud topology starts to feel like a necessity. In this article, we will take a closer look at one of such services, namely AWS Organizations, and show how it can be used to help encapsulate your projects/apps/accounts.

TL;DR

  • use the combined power of AWS Organizations and Terraform to keep your infra clean and orderly
  • all code is here

Game plan

In this article, I’ll be using Terraform to show you how to create:

  • an AWS Organization
  • sample IAM groups and users
  • an Organizational Unit (OU)
  • two AWS accounts, one for development and the other for production, under the OU
  • a Service Control Policy curbing some of the permissions of OU’s accounts

Prerequisites

If you want to follow along, you’ll need to have:

  • access to the email address associated with AWS account that will become the master account of your organization. (NB, at the time of writing AWS supported only one root in an organization.)
  • programmatic access, with adequate permissions, to your AWS account
  • Terraform v0.12 or higher installed on your local machine
  • Keybase configured locally and/or GPG Suite

Preparation

Before we can commence with provisioning any organization-related resources, we need to do some prep work. Let’s kick-off by creating a folder for our project:

➜  ~$ mkdir aws-organizations-example && cd $_

Within that folder, we are going to set up the minimal Terraform configuration required and then run the initialization command. (For the sake of simplicity, the Terraform state will be stored locally. This is not recommended, but fear not, you may refer to the example provided on Github and see how S3 can be used as a backend.)

+ aws-organizations-example/provider.tf

provider "aws" {
 version = "~> 2.30"
 region  = "eu-west-1"
}

+ running terraform init

➜  aws-organizations-example$ terraform init

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 2.54.0...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Since we have laid the groundwork, we are now ready to commission our organization.

Creating an AWS Organization

First, we’ll create a module for organizations:

+ aws-organizations-example/organizations/main.tf

resource "aws_organizations_organization" "org" {
 feature_set                   = var.feature_set
 aws_service_access_principals = var.feature_set == "ALL" ? var.aws_service_access_principals : null
 enabled_policy_types          = var.feature_set == "ALL" ? var.enabled_policy_types : null
}

+ aws-organizations-example/organizations/variables.tf

variable "feature_set" {
 description = "After the Terraform docs: 'Specify ALL (default) or CONSOLIDATED_BILLING.'"
 default     = "ALL"
}

variable "aws_service_access_principals" {
 description = "After the Terraform docs: 'List of AWS service principal names for which you want to enable integration with your organization. This is typically in the form of a URL, such as service-abbreviation.amazonaws.com. Organization must have feature_set set to ALL. For additional information, see the AWS Organizations User Guide.'"
 type        = list(string)
 default     = null
}

variable "enabled_policy_types" {
 description = "After the Terraform docs: 'List of Organizations policy types to enable in the Organization Root. Organization must have feature_set set to ALL. For additional information about valid policy types (e.g. SERVICE_CONTROL_POLICY and TAG_POLICY), see the AWS Organizations API Reference.'"
 type        = list(string)
 default     = null
}

+ aws-organizations-example/organizations/outputs.tf

output "roots" {
 description = "After the Terraform docs: 'List of organization roots. (...)'"
 value       = aws_organizations_organization.org.roots
}

Next, we’ll initialize it:

+ aws-organizations-example/main.tf

### ORGANIZATION
# After the AWS docs: "An entity that you create to consolidate your AWS accounts so that you can administer them as a single unit."
module "root" {
 source               = "./organizations"
 enabled_policy_types = ["SERVICE_CONTROL_POLICY"]
}
### ORGANIZATION - END

+ must run terraform init again

➜  aws-organizations-example$ terraform init
Initializing modules...
- root in organizations

# other output

Terraform has been successfully initialized!

# other output

And finally, we are going to run terraform plan and, if it goes without a hitch, terraform apply:

+ running terraform plan

➜  aws-organizations-example$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
 + create

Terraform will perform the following actions:

 # module.root.aws_organizations_organization.org will be created
 + resource "aws_organizations_organization" "org" {
     + accounts             = (known after apply)
     + arn                  = (known after apply)
     + enabled_policy_types = [
         + "SERVICE_CONTROL_POLICY",
       ]
     + feature_set          = "ALL"
     + id                   = (known after apply)
     + master_account_arn   = (known after apply)
     + master_account_email = (known after apply)
     + master_account_id    = (known after apply)
     + non_master_accounts  = (known after apply)
     + roots                = (known after apply)
   }

Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

+ running terraform apply

$  aws-organizations-example terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
 + create

Terraform will perform the following actions:

 # module.root.aws_organizations_organization.org will be created
 + resource "aws_organizations_organization" "org" {
     + accounts             = (known after apply)
     + arn                  = (known after apply)
     + feature_set          = "ALL"
     + enabled_policy_types = [
         + "SERVICE_CONTROL_POLICY",
       ]
     + id                   = (known after apply)
     + master_account_arn   = (known after apply)
     + master_account_email = (known after apply)
     + master_account_id    = (known after apply)
     + non_master_accounts  = (known after apply)
     + roots                = (known after apply)
   }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
 Terraform will perform the actions described above.
 Only 'yes' will be accepted to approve.

 Enter a value: yes

module.root.aws_organizations_organization.org: Creating...
module.root.aws_organizations_organization.org: Creation complete after 5s [id=o-eipgmnk07c]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

When the process completes, you’ll see the following message in the AWS Management Console for AWS Organizations:

Now, log in to your mailbox, check for a message from AWS and confirm the ownership of the email account:

Splendid! You’ve just created your first AWS Organization:

Now, let’s get down to creating users, groups and accounts.

Setting up groups and users

We’ll commence, by creating a module for iam-groups. They will allow us to assign our users to either the administrators or the developers group (or both if we wished, though that would not make much sense):

+ aws-organizations-example/iam-groups/main.tf

resource "aws_iam_group" "group" {
 name = var.name
}

### AWS MANAGED POLICIES - START
data "aws_iam_policy" "AdministratorAccess" {
 arn = "arn:aws:iam::aws:policy/AdministratorAccess"
}

resource "aws_iam_group_policy_attachment" "AdministratorAccess" {
 count      = var.enable-AdministratorAccess ? 1 : 0
 group      = aws_iam_group.group.name
 policy_arn = data.aws_iam_policy.AdministratorAccess.arn
}

data "aws_iam_policy" "AWSCodeBuildDeveloperAccess" {
 arn = "arn:aws:iam::aws:policy/AWSCodeBuildDeveloperAccess"
}

resource "aws_iam_group_policy_attachment" "AWSCodeBuildDeveloperAccess" {
 count      = var.enable-AWSCodeBuildDeveloperAccess ? 1 : 0
 group      = aws_iam_group.group.name
 policy_arn = data.aws_iam_policy.AWSCodeBuildDeveloperAccess.arn
}
### AWS MANAGED POLICIES - END

+ aws-organizations-example/iam-groups/variables.tf

### GROUPS - START
variable "name" {
 description = "The name of the IAM group"
 type        = string
}

variable "enable-AdministratorAccess" {
 description = "A flag for enabling the AWS Managed AdministratorAccess policy"
 default     = false
}

variable "enable-AWSCodeBuildDeveloperAccess" {
 description = "A flag for enabling the AWS Managed AWSCodeBuildDeveloperAccess policy"
 default     = false
}
### GROUPS - END

Using the above pattern, you can easily add a plethora of other groups depending on your particular needs, e.g. a group for Accountants with access only to the billing section of AWS, etc..

Next, a module for iam-users would be recommended. So, let’s add it now:

+ aws-organizations-example/iam-users/main.tf

resource "aws_iam_user" "this" {
 count = var.create-aws_iam_user ? 1 : 0

 name = var.name
 path = var.path
}

resource "aws_iam_user_login_profile" "this" {
 count = var.create-aws_iam_user && var.create-aws_iam_user_login_profile ? 1 : 0

 user                    = aws_iam_user.this[count.index].name
 pgp_key                 = var.pgp_key
 password_reset_required = var.password_reset_required
}

resource "aws_iam_access_key" "this" {
 count = var.create-aws_iam_user && var.create-aws_iam_access_key ? 1 : 0

 user    = aws_iam_user.this[count.index].name
 pgp_key = var.pgp_key
}

resource "aws_iam_user_group_membership" "this" {
 count = var.create-aws_iam_user && var.create-aws_iam_user_group_membership ? 1 : 0

 user   = aws_iam_user.this[count.index].name
 groups = var.groups
}

+ aws-organizations-example/iam-users/variables.tf

### USERS - START
variable "create-aws_iam_user" {
 description = "A flag indicating whether an IAM user should be created"
 default     = true
}

variable "create-aws_iam_user_login_profile" {
 description = "A flag indicating whether an IAM user login (for AWS Console) should be created for a given user"
 default     = true
}

variable "create-aws_iam_access_key" {
 description = "A flag indicating whether an IAM access key ('a set of credentials that allow API requests to be made as an IAM user') should be created for a given user"
 default     = true
}

variable "create-aws_iam_user_group_membership" {
 description = "A flag indicating whether a group membership should be created for a given user"
 default     = true
}

variable "groups" {
 description = "A list of groups the user should become a member of"
 type        = list(string)
}

variable "name" {
 description = "The name of the IAM user"
 type        = string
}

variable "path" {
 description = "The path in which the IAM user should be created"
 type        = string
 default     = "/users/"
}

variable "force_destroy" {
 description = "After the terraform docs: 'When destroying this user, destroy even if it has non-Terraform-managed IAM access keys, login profile or MFA devices. Without force_destroy a user with non-Terraform-managed access keys and login profile will fail to be destroyed.'"
 default     = false
}

variable "password_reset_required" {
 description = "Whether the user should be forced to reset the generated password on first login."
 default     = false
}

variable "pgp_key" {
 description = "Either a base-64 encoded PGP public key, or a keybase username in the form keybase:username. Used to encrypt the password and the access key on output to the console."
 default     = ""
}
### USERS - END

+ aws-organizations-example/iam-users/outputs.tf

output "aws_iam_user-credentials" {
 description = "The credentials of a given IAM user"
 value = {
   name                        = var.create-aws_iam_user ? aws_iam_user.this[0].name : null
   encrypted_password          = var.create-aws_iam_user_login_profile ? aws_iam_user_login_profile.this[0].encrypted_password : null
   pgp_key                     = var.pgp_key
   access-key-id               = var.create-aws_iam_access_key ? aws_iam_access_key.this[0].id : null
   encrypted-secret-access-key = var.create-aws_iam_access_key ? aws_iam_access_key.this[0].encrypted_secret : null
 }
}

Finally, to make it all work add the following lines to the aws-organizations-example/main.tf:

### GROUPS - START
module "administrators" {
 source                     = "./iam-groups"
 name                       = "administrators"
 enable-AdministratorAccess = true
}

module "developers" {
 source                             = "./iam-groups"
 name                               = "developers"
 enable-AWSCodeBuildDeveloperAccess = true
}
### GROUPS - END


### USERS - START
module "admin_user_1" {
 source                            = "./iam-users"
 name                              = "admin_user_1"
 groups                            = ["administrators"]
 force_destroy                     = true
 pgp_key                           = "keybase:admin_user_1"
}

output "admin_user_1-aws_iam_user-credentials" {
 description = "The user's credentials"
 value       = module.admin_user_1.aws_iam_user-credentials
}
### USERS - END

Initialize the modules:

➜ aws-organizations-example$ terraform init
Initializing modules...
- administrators in iam-groups
- developers in iam-groups
- admin_user_1 in iam-users

# other output

Terraform has been successfully initialized!

# other output

+ run terraform plan followed by terraform apply:

➜  aws-organizations-example$ terraform plan
# other output

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
 + create

Terraform will perform the following actions:

 # module.admin_user_1.aws_iam_access_key.this[0] will be created
 + resource "aws_iam_access_key" "this" {
     + encrypted_secret     = (known after apply)
     + id                   = (known after apply)
     + key_fingerprint      = (known after apply)
     + pgp_key              = "keybase:admin_user_1"
     + secret               = (sensitive value)
     + ses_smtp_password    = (sensitive value)
     + ses_smtp_password_v4 = (sensitive value)
     + status               = (known after apply)
     + user                 = "admin_user_1"
   }

 # module.admin_user_1.aws_iam_user.this[0] will be created
 + resource "aws_iam_user" "this" {
     + arn           = (known after apply)
     + force_destroy = false
     + id            = (known after apply)
     + name          = "admin_user_1"
     + path          = "/users/"
     + unique_id     = (known after apply)
   }

 # module.admin_user_1.aws_iam_user_group_membership.this[0] will be created
 + resource "aws_iam_user_group_membership" "this" {
     + groups = [
         + "administrators",
       ]
     + id     = (known after apply)
     + user   = "admin_user_1"
   }

 # module.admin_user_1.aws_iam_user_login_profile.this[0] will be created
 + resource "aws_iam_user_login_profile" "this" {
     + encrypted_password      = (known after apply)
     + id                      = (known after apply)
     + key_fingerprint         = (known after apply)
     + password_length         = 20
     + password_reset_required = false
     + pgp_key                 = "keybase:admin_user_1"
     + user                    = "admin_user_1"
   }

 # module.administrators.aws_iam_group.group will be created
 + resource "aws_iam_group" "group" {
     + arn       = (known after apply)
     + id        = (known after apply)
     + name      = "administrators"
     + path      = "/"
     + unique_id = (known after apply)
   }

 # module.administrators.aws_iam_group_policy_attachment.AdministratorAccess[0] will be created
 + resource "aws_iam_group_policy_attachment" "AdministratorAccess" {
     + group      = "administrators"
     + id         = (known after apply)
     + policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
   }

 # module.developers.aws_iam_group.group will be created
 + resource "aws_iam_group" "group" {
     + arn       = (known after apply)
     + id        = (known after apply)
     + name      = "developers"
     + path      = "/"
     + unique_id = (known after apply)
   }

 # module.developers.aws_iam_group_policy_attachment.AWSCodeBuildDeveloperAccess[0] will be created
 + resource "aws_iam_group_policy_attachment" "AWSCodeBuildDeveloperAccess" {
     + group      = "developers"
     + id         = (known after apply)
     + policy_arn = "arn:aws:iam::aws:policy/AWSCodeBuildDeveloperAccess"
   }

Plan: 8 to add, 0 to change, 0 to destroy.

# other output
➜  aws-organizations-example$ terraform apply
# other output

Apply complete! Resources: 8 added, 0 changed, 0 destroyed.

Outputs:

admin_user_1-aws_iam_user-credentials = {
 "access-key-id" = "AKIA57MNK6ZKUWMBKF5D"
 "encrypted-secret-access-key" = "wcFMA/i0/Lj0oDvoARAAdmRlRfNtMhTEM2IwgVAC68wEz8y5YWIs5IUMqKhvEl98BBj46Y1WF3SJSXD0egL3gyk/YnUnopSLjrFebAd7nLQwaakIo1bhbvTvNToic9ADw3jM1nSQzRfPWlde0rca/3SbT4/QStwtjpfOel2N+maoazS0HZ2RExGBAp1S1Xxn6HD2EwxKrcpqqpPTybrnH+kKUmTHVU48H5IN7WVVL0YGtfylPTdNYDzIkqZETYpyI2ZnwSA/iQdOKWBk0abMmd+MYLh/xtmtcU0FDk04+qvl3xS575rmEOg5Fw+MbaxxsYoYl85SwY4MWLB3e79f9AZp3lpF/8Wb0w5vIponlW+eHIfGILzHhow5sLVvNowPazvT7QxxhDRuxvX8IkxeJDdXH/PpkFqvHFLFleD/dIrIht7lXS6FdIW+D0JnhdJf00RtDBvCZvOXOGSuLHefcv9WOMJ0bc1K9kpahF6laYLLfaVKd5SR6FPXL+YAchLo9vAlvm1NdVM7MoGtIOHdF+Xs6gE9YEbadPJCW1/cT+LwrUVmtw6P409/AAetpv3jTEqA9k5NpvgQ612NhSK/6TAcb0wC9ZLUmVla+pcR4Pu9vNQy4cVfFVKALckP8DxGgqJZmg0K91WTAIf9Z30goZGbWl3yGd7Po8/oBDoTKVZWpbWe1m2iXHK4A83MncvS4AHkZZMd8hCihuhqqLxyhLE68OGvr+Cj4IvhHZjgzeKKIPvk4GnlNtXXjE7tK5rzsayCfR8CBPDDG5ZkRyWOhRfeeyasPD/gW+P6WJ+OHE1yOOA+5HnLBM1nftCM1D5n+7T0aQzimK8TouGhNQA="
 "encrypted_password" = "wcFMA/i0/Lj0oDvoARAAC9sIp1AVSTwhhjyqNtJm6x9UPBAbYGe2Rym/Ppg5C+uV0RGbXZVg0RQHpoIG3uvCStGzT7IhSjphm09QiYKpQQhAjdw3aOO/tfuftMWVhA1VkASduyKPuW89ENPLs6hKaTmvyaOazy9LOkXbfRoI5C7rhCJq+fmFBQKb5CLHAVBN2pv0Msa2yE2dc4PkEEFF8DLW1Xfi0jLbLrRI+QTXWV1sSBSkQDoNuCR8kixVcSz1OdwCymddgAg5y0g6l7IK+WF783Lq5Q6VK05s3wJhMV+GnhNFh+LSMk60Mzd2QXjR8qEP1qDBHNRKpSQIT7wBWobT52YHOLj1cYKL1ohVchdH0QlxHTWkoZgJEchujJIOMy9Bwyl2Oz+XVxdpo1zX6T9vss1xH8P5k4CHzRRCbqnBbnkwO8koW2ZYs568lEqQgodP1U0fWH79wR9Abf5ITN/senmDN68+i+obihDIALOnIkT8A9dz8sHkWTnsL0egC9Uju83784o52YaWrC/HewWMN1rnl+VM36ooylherT3X1HYtRU1lyKpa8szYC739QFubrvGzrvpQ7N0nEBCzpQ4yIOLvEqx9Fg+xfX5AmzJ5rCh1kiorQ/rCs9lcSpdq2TV4PY7z1mdq1MWW3R4NrXiw8UqCFxHoX66+OZ5If5U+bE4Kc/t4hGCApRJFbfTS4AHkMReEeStKtB5CTuzDIjCTquFNheAi4DThNU/g9uK0GbBp4JTkOhW40+GZCTS1etG6gg4JVuAr4vtO6sjgKuRsgfuk1SfF5WZhaU9XSq5z4ggd7ZDhFwoA"
 "name" = "admin_user_1"
 "pgp_key" = "keybase:admin_user_1"
}

You can grab the output and safely pass it to the user, who can then decrypt the sensitive bits (e.g. the encrypted-secret-access-key), using their PGP in the following manner:

NB, keybase pgp decrypt can be swapped with gpg –decrypt

echo wcFMA7+BmSIBAU+8ARAAQNgPEu4fkVHIfYKhzEEMTaDgWIfMUHaKPVvsiDmnYxz5Am1Y0/vk39LJY9CCw8bgKyaOZ4wzAzEShDsmNMIPtUb3+gaidiNyvsUb4dXcovKcVRIHF3KztXAeod8r5ak9Kh2g6IFcVyhJ2qUztXO1gTWzdaivqgLOAZt2spN3moJ4NZJKM4inLHcqH0rI9T5qxEhdn62wirhW43UTpukxnCJhEoeUlhvW3XvjQuYD1jZpifzuRLTU6c3qD2wn84P+PlWbKnCn5WrE3mKjdNwIn9QSPCPo6UzednjsSjUdix2LOOVsAbQCzQ866sG9m5cTIG56NUTXxQyKLkVqmdyojnB3IkspWrYgfASnyiq24by2SJqPQQiS/1aJbYVEqMGClMldbYCCOS2Uh5DyNri+cnvM0JDgVQ6pPHWK8aC7YP7ZwTqDrvwXSMh428yJ56pFwIu/d730vfagCiVPgw3wVY75yJnJWEl2MjoCpsRnDW6FgoK7NMrRMA5Gn4qZDCh2VvddjAQfW7kKIbd0MPDCMfVDFtjJfl88ubNsXNXkoUsgoDm6yIqhF3uSs0hYvZq7E8idk7a11eAenJu3BaTSQAG0qj0vjFdZ8bEoIIrsLiaRUvX8Ot4g5d0IdsUdW87UkANYtIV3zu+1O38DQfoTVJ17uqgxDOgYPdPkOZRsKzLS4AHk5nKuvQ8JYJbFSJg2rbgAouF0xOAC4MDh8ULgnuLcieAj4JDkLdrepfJyh9pG2XZ1nsUqsuBz4h5H6CPgN+TZljm8ktk+lYNiteTS3NYS4ns051PhDjsA | base64 --decode | keybase pgp decrypt

That’s a wrap of part 1. In the next and final one, we’ll talk about Organizational Units and Service Control Policies. See you there!