Implement Amazon S3 Cross-Region Replication With Terraform

With the information technology element finding its roots in every financial organization and across all industries, strong storage capacity forms the backbone for availability, durability, and scalability. Among these, Amazon S3 is one of the most popular services to meet these needs. As enterprises expand geographically, the need for data replication between locations starts to be felt significantly. Hence, the need arose to develop Cross-Region Replication in Amazon S3, where data replication between one bucket sourced from one AWS region to another bucket in a different AWS region is allowed.

The article will describe how to configure Cross-Region Replication in Amazon S3 using Terraform, an IaC software. It gives a general overview of how to set up SLAs, discusses why one would implement cross-region replication, and shows how to get a proof of concept running.

What Is Cross-Region Replication?

CRR stands for Cross-Region Replication, an Amazon S3 feature that replicates objects from one bucket to another bucket in a different region. This replication assists with compliance regulations, disaster recovery, and performance optimization by storing data closer to users.

Key Features of CRR

• It automatically replicates objects between buckets.
• Regions are independent; that is, source and destination buckets can be from different regions.
• Rules: This allows granular control over what gets replicated.
• Encryption and Tagging Support: Supports the replication of encrypted data and object tags.

Why Use Terraform for CRR?

Terraform enables cloud resource management through declarative configuration. While AWS Management Console has default replication policies, Terraform allows version-controlled templates.

Benefits of Terraform

• Consistency. The process reduces the chances of set-up variations.
• Automation. Enabling quick implementation, updating, and tearing down.
• Version control. It simplifies rollbacks and history tracking.

Prerequisites

Before setting up CRR with Terraform, the following should be in place:

• An AWS account with S3 and IAM permissions
• Terraform locally installed
• AWS CLI configured to use for authentication

Implementation

Step 1: Define Your S3 Buckets

You should create at least two buckets: source and destination buckets.

resource "aws_s3_bucket" "source_bucket" {
  bucket = "my-source-bucket-example"
  acl    = "private"

  versioning {
    enabled = true
  }

  tags = {
    Name        = "SourceBucket"
    Environment = "Production"
  }
}

resource "aws_s3_bucket" "destination_bucket" {
  bucket = "my-destination-bucket-example"
  acl    = "private"

  versioning {
    enabled = true
  }

  tags = {
    Name        = "DestinationBucket"
    Environment = "Production"
  }
}

Step 2: Create an IAM Role for Replication

To use CRR, there must be an IAM role with the privileges that CRR requires. Let's create one:

resource "aws_iam_role" "replication_role" {
  name = "s3-replication-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "s3.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_policy" "replication_policy" {
  name        = "s3-replication-policy"
  description = "Allows S3 to replicate objects"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = [
          "s3:GetObject",
          "s3:ListBucket",
          "s3:ReplicateObject",
          "s3:ReplicateDelete",
          "s3:ReplicateTags"
        ]
        Effect   = "Allow"
        Resource = [
          "arn:aws:s3:::my-source-bucket-example/*",
          "arn:aws:s3:::my-source-bucket-example"
        ]
      },
      {
        Action = "s3:PutObject"
        Effect = "Allow"
        Resource = "arn:aws:s3:::my-destination-bucket-example/*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "replication_role_attachment" {
  role       = aws_iam_role.replication_role.name
  policy_arn = aws_iam_policy.replication_policy.arn
}

Step 3: S3 Replication Configuration

Configure a replication configuration to associate source and destination buckets.

resource "aws_s3_bucket_replication_configuration" "replication_config" {
  bucket = aws_s3_bucket.source_bucket.id

  role = aws_iam_role.replication_role.arn

  rule {
    id     = "ReplicationRule1"
    status = "Enabled"

    filter {
      prefix = ""
    }

    destination {
      bucket        = aws_s3_bucket.destination_bucket.arn
      storage_class = "STANDARD"
    }
  }
}

Step 4: Init and Apply Configuration

Execute the Terraform configuration using these commands:

terraform init
terraform apply

During the execution of terraform apply, just review the plan and confirm its execution.

How to Test CRR Setup

Create an object in the source bucket and see if it gets replicated in the destination bucket.

# Put a file in the source bucket
aws s3 cp test-file.txt s3://my-source-bucket-example/

# Check if file is there in the destination bucket
aws s3 ls s3://my-destination-bucket-example/

Advantages of Cross-Region Replication

1. Disaster recovery. Protect data by mirroring it across different geographical regions.
2. Compliance. Meet regulatory requirements for data location and backup.
3. Improved performance. Reduce latency by storing data closer to users.
4. Backup automation. Manage backups and archives without additional tools or scripts.

Troubleshooting Tips

1. Permission errors. Verify the IAM role has correct permissions on source and destination buckets.
2. Versioning not enabled. Make sure versioning is enabled on both source and destination buckets.
3. Replication delays. Replication is eventually consistent. Monitor replication using CloudWatch.

Conclusion

Amazon S3 Cross-Region Replication is a very powerful feature for any business to achieve appropriate data redundancy and compliance. Terraform enables you to automate and streamline the implementation process so the infrastructure setup remains consistent and scalable. 

In this tutorial, we have learned how to implement CRR using Terraform, from bucket configuration to replication rule definitions. Based on the method described in this article, you can design your optimal storage architecture to handle the needs of a modern enterprise application.