Terraform with AWS S3 and DynamoDB for Remote State Files

By default, Terraform state files are generated locally. This is not ideal when you have multiple people working on a project.

We want to use Amazon S3 to store remote state files with DynamoDB state locking and consistency checking.

Install Terraform

If you don’t have Terraform, you can install it using the following commands:

$ wget -O tf.zip https://releases.hashicorp.com/terraform/0.13.5/terraform_0.13.5_linux_amd64.zip
$ unzip tf.zip 
$ sudo mv terraform /usr/local/bin/
$ sudo chown root: /usr/local/bin/terraform

Create S3 Bucket and DynamoDB Table

Create an S3 bucket, e.g. my-bucket-name-for-terraform-state, and enable versioning. The S3 bucket will store your state files.

Create a DynamoDB table, e.g. my-table-name-for-terraform-state-lock, and make sure that your primary key is LockID (type is String). The DynamoDB table provides the ability to lock the state file to avoid multiple people writing to the state file at the same time.

IAM Account Permissions

Terraform IAM account will require the following S3 bucket permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::my-bucket-name-for-terraform-state"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::my-bucket-name-for-terraform-state/state.tfstate"
        }
    ]
}

Also add the following DynamoDB table permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "dynamodb:GetItem",
                "dynamodb:PutItem",
                "dynamodb:DeleteItem"
            ],
            "Resource": "arn:aws:dynamodb:*:*:table/my-table-name-for-terraform-state-lock"
        }
    ]
}

Make sure to change the name of the S3 bucket as well as the name of the DynamoDB table to suit your environment.

Configure Terraform Backend

Create a file called backend.tf with the following content:

terraform {
  backend "s3" {
    region         = "eu-west-2"
    bucket         = "my-bucket-name-for-terraform-state"
    key            = "state.tfstate"
    dynamodb_table = "my-table-name-for-terraform-state-lock"
    encrypt        = true
  }
}

The Terraform state will be written to the key state.tfstate.

References

https://www.terraform.io/docs/backends/types/s3.html