DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • AWS Lambda vs. Fargate: The Battle of Cloud Giants
  • Maximizing Uptime: How to Leverage AWS RDS for High Availability and Disaster Recovery
  • A Beginner’s Guide To Building Microservices With AWS Lambda
  • Simplify Log Aggregation in AWS: A Comprehensive Guide to the Log Aggregation Pattern

Trending

  • Five Tools for Data Scientists to 10X their Productivity
  • Spring Boot and React in Harmony
  • Modern Data Backup Strategies for Safeguarding Your Information
  • Automated Testing Lifecycle
  1. DZone
  2. Software Design and Architecture
  3. Cloud Architecture
  4. Real-Time Cross-Region DynamoDB Replication Using AWS Lambda

Real-Time Cross-Region DynamoDB Replication Using AWS Lambda

Learn how to write a lambda function and configure it with a DynamoDB trigger that will invoke the lambda function on any updates to a particular DynamoDB table.

Sagar Pradhan user avatar by
Sagar Pradhan
·
Jul. 06, 17 · Tutorial
Like (9)
Save
Tweet
Share
9.02K Views

Join the DZone community and get the full member experience.

Join For Free

With the increasing amount of data that's available to be processed these days, people are looking to the cloud to resolve their scalability issues. One of the challenges is keeping the data in sync across various data centers/geographical locations, which is required for failure fallback and disaster recovery.

AWS Dynamo DB has emerged as a leader in NoSQL cloud solutions.

Recently, AWS launched a DynamoDB cross-region replication application, which runs on a dedicated EC2 instance.

In this post, we will discuss the serverless approach to achieving this using DynamoDB streams, triggers, and AWS Lambda.

To achieve this, we will write a lambda function and configure it with a DynamoDB trigger that will invoke the lambda function on any updates to a particular DynamoDB table.

1. Create 2 Similar Schemas/Tables in 2 Regions

For pre-existing tables, we might have to do a one-time export of that table to the other regions using AWS Data Pipeline.

For example:

Region:eg. Mumbai and Sydney 
Table Employee:
ID (Partition Key)
Name (Sort Key)
Job Title

2. Create a Lambda Function to Process Incoming DynamoDB Stream Events

/*Override the handle request method which serves as an 
entry point for the lambda function*/

@Override
public Integer handleRequest(DynamodbEvent event, Context context) {
 context.getLogger().log("Received event: " + event);

 DynamoDBProcessEvents dbProcessEvents = new DynamoDBProcessEvents();
 List < DynamodbStreamRecord > records = event.getRecords();
 try {
  for (DynamodbStreamRecord record: records) {
   //logic to process the incomng events
   switch (record.getEventName()) {
    case "INSERT":
     insertDynamoDBItem(record);
     break;
    case "MODIFY":
     updateDynamoDBItem(record);
     break;
    case "REMOVE":
     deleteDynamoDBItem(record);
     break;
    default:
     break;
   }
  }
 }
}
//Insert in to backup Table
private void insertDynamoDBItem(DynamodbStreamRecord record) {
 Map < String, AttributeValue > item = record.getDynamodb().getNewImage();
 PutItemRequest itemIn = new PutItemRequest( < backup_dbtable_name > , item);
 dynamoDBClient.putItem(itemIn);
}

// Delete from backup table
private void deleteDynamoDBItem(DynamodbStreamRecord record) {
 Map < String, AttributeValue > itemToBeRemoved = record.getDynamodb().getNewImage();;
 PrimaryKey keyDelete = getPrimaryKey(itemToBeRemoved);
 table.deleteItem(keyDelete);
}


// Update item in backup table. In this case we are deleting the existing record 
// and inserting the updated record. UpdateItemSpec or UpdateItemRequest can also
//be used
private void updateDynamoDBItem(DynamodbStreamRecord record) {
 deleteDynamoDBItem(record);
 insertDynamoDBItem(record);
}
private PrimaryKey getPrimaryKey(Map < String, AttributeValue > item) {
 AttributeValue valueHash = item.get(HASH_KEY);
 AttributeValue valueSort = item.get(SORT_KEY);
 PrimaryKey key = new PrimaryKey(HASH_KEY, Integer.parseInt(valueHash.getN()), SORT_KEY, valueSort.getS());
 return key;
}

3. Upload the Lambda Function

Create a role in IAM and assign it to the lambda function. Make sure to add policies to the role that allow read/write access to DynamoDB. 

4. Create Event Source Mapping in AWS Lambda

This event source mapping associates the DynamoDB stream with your lambda function. After you create this event source mapping, AWS lambda starts polling the stream.

AWS AWS Lambda Replication (computing)

Opinions expressed by DZone contributors are their own.

Related

  • AWS Lambda vs. Fargate: The Battle of Cloud Giants
  • Maximizing Uptime: How to Leverage AWS RDS for High Availability and Disaster Recovery
  • A Beginner’s Guide To Building Microservices With AWS Lambda
  • Simplify Log Aggregation in AWS: A Comprehensive Guide to the Log Aggregation Pattern

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: