Sample Architecture Using Amazon AWS, Microsoft Azure, Google GCP, MongoDB, and Couchbase

A drawing should have no unnecessary lines and a machine no unnecessary parts. 

                William Strunk Jr., Elements of Style

In the book Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, Martin Kleppmann has written about traits and trade-offs for data infrastructure while designing modern applications. He has given an example architecture for a data system that combines several components. I used this example for the article Example Architectures for Data-Intensive Applications. That article explored just the Couchbase features and functions.

A recent Twitter thread talked about how AWS has many databases, but most “half-baked” services (not my characterization) compete for the same business as DynamoDB!  Could this be a paradox of choice? The theory is, more choices do not usually lead to better outcomes. However, customers do vote with their dollars and cents. So far, they’re picking AWS for the potential savings in the short run. Could this lead to complexity and drive up the cost in the future?

In this article, I explore how we can architect that example using products and components from various Cloud database vendors.

1. Amazon AWS
2. Microsoft Azure
3. Google GCP
4. MongoDB
5. Couchbase

Example Reference Architecture

Amazon AWS

Amazon has choices and a Reddit user has created a decision tree for choosing the databases!  Here's that graphic.

Using this guide, let's look at the architecture to make our choices for various components.

For the core operational database, you can use DynamoDB (NoSQL), DocumentDB(NoSQL) or Aurora/RDS (RDBMS). Add either Elasticache or MemoryDB for caching. Use Amazon Neptune or Elastic Search service (using OpenSearch). Or use Elastic itself directly.   Either AWS SQS or Kafka provides queuing function for downstream processing.  Amazon Redshift and Amazon Quicksight provide the data warehouse and BI function.  

You'll have to make choices about sizing and workload about each one of them and find a way to stitch them together either by built-in functionality, connectors, and customer application code.  The lines from the reference architecture diagram are intact.  Choice does address your need even if you have to stitch them together.

Microsoft Azure

Microsoft has grown from a single (Sybase) SQL Server on OS/2 in the 90s plethora of products and services on Azure. This page gives you the Azure databases list and matches them with the right workload requirements. Just like AWS, the choices are plenty.  Interestingly, they don't list the "search" in the database category.  This is typical of database vendors even though most applications that use databases require search. Microsoft has the Azure cognitive search service, not tightly integrated with Cosmos or other services.  You also have the option of deploying Elasticsearch itself.

In the end, this is the same as the AWS diagram above, replaced with Microsoft Azure products.

Google GCP

Similar to AWS and Azure, GCP has plenty of choices for databases from Cloud SQL to Cloud Spanner, Memorystore to Firestore, and Oracle to Bigtable. 

Different Cloud, same result. You can pick up from a number of databases, components for cache, queues, and search from GCP and others.  The architecture is the same and the challenges are the same you saw with AWS native solutions.

MongoDB

MongoDB is a scale-out, highly available documented oriented database that meets a number of requirements for a modern database. MongoDB provides a simple get/set API, complex aggregation framework, and distributed transactions. MongoDB reduces the number of components because it has built-in MongoDB Realm functions which can execute serverside logic.  You'd still need a caching layer to meet low latency performance objectives,  While the product is advertised to have built-in text search, its functionality is limited.  So is its capability for complex analytical queries. 

This improves the situation because it has an Atlas search. Atlas search is built using Apache Lucene. This search capability is only available on Atlas and not within the MongoDB product.

Couchbase and Couchbase Capella

Conclusion