DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Related

  • Understanding the Fan-Out/Fan-In API Integration Pattern
  • How to Build a Pokedex React App with a Slash GraphQL Backend
  • Enhanced API Security: Fine-Grained Access Control Using OPA and Kong Gateway
  • Parent Document Retrieval (PDR): Useful Technique in RAG

Trending

  • Building Resilient Identity Systems: Lessons from Securing Billions of Authentication Requests
  • Building Reliable LLM-Powered Microservices With Kubernetes on AWS
  • Blue Skies Ahead: An AI Case Study on LLM Use for a Graph Theory Related Application
  • AI-Based Threat Detection in Cloud Security
  1. DZone
  2. Data Engineering
  3. Databases
  4. Tutorial: Build DynamoDB-Compatible Apps for Any Cloud (Or On-Prem)

Tutorial: Build DynamoDB-Compatible Apps for Any Cloud (Or On-Prem)

Here's how to use an open-source API to build DynamoDB-compatible applications that can be deployed wherever you want: on-premises or on any public cloud.

By 
Guy Shtub user avatar
Guy Shtub
·
Nov. 23, 22 · Tutorial
Likes (2)
Comment
Save
Tweet
Share
7.0K Views

Join the DZone community and get the full member experience.

Join For Free

Alternator is an open-source project that allows teams to build "run-anywhere" DynamoDB-compatible applications with ScyllaDB.  This means you can deploy wherever you want: on-premises or on any public cloud.

In this tutorial, we’ll start by introducing the project. Afterward, we’ll see a hands-on example of creating a one-node ScyllaDB cluster and performing some basic operations on it.

The goal of this project is to deliver an open-source alternative to DynamoDB, deployable wherever a user would want: on-premises, on other public clouds like Microsoft Azure or Google Cloud Platform, or still on AWS (for users who wish to take advantage of other aspects of Amazon’s market-leading cloud ecosystem, such as the powerful i4 instances). DynamoDB users can keep their same client code unchanged. Alternator is written in C++ and is a part of ScyllaDB.

The three main benefits ScyllaDB Alternator provides to DynamoDB users are:

  1. Cost: DynamoDB charges for read and write transactions (RCUs and WCUs). A free, open-source solution doesn’t.
  2. Performance: ScyllaDB was implemented in modern C++. It supports advanced features that enable it to improve latency and throughput significantly.
  3. Openness: ScyllaDB is open-source. It can run on any suitable server cluster regardless of location or deployment method.

Setting Up a ScyllaDB Cluster

If you haven’t done so yet, download the example from git:

git clone https://github.com/scylladb/scylla-code-samples.git

Go to the directory of the alternator example:

cd scylla-code-samples/alternator/getting-started

Next, we’ll start a one-node cluster with Alternator enabled.

By default, ScyllaDB does not listen to DynamoDB API requests. To enable such requests, we will set the alternator-port configuration option to the port (8000 in this example), which will listen for DynamoDB API requests.

Alternator uses ScyllaDB’s LWT feature. You can read more about it in the documentation.

docker run --name some-scylla --hostname some-scylla -p 8000:8000 -d scylladb/scylla:4.0.0 --smp 1 --memory=750M --overprovisioned 1 --alternator-port=8000

Wait a few seconds and make sure the cluster is up and running:

docker exec -it some-scylla nodetool status
 
[guy@localhost ~]$ docker exec -it some-scylla nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address     Load       Tokens       Owns    Host ID                               Rack
UN  172.17.0.2  79.82 KB   256          ?       1ccbb599-ce5c-4d72-9c3a-991e2567d0be  rack1

Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless
[guy@localhost ~]$ 

In this example, we will use the Python language to interact with ScyllaDB with the Boto 3 SDK for Python. It’s also possible to use the CLI or other languages such as Java, C#, Python, Perl, PHP, Ruby, Erlang, or JavaScript.

Next, if you don’t already have it set up, install the Boto 3 Python library which also contains drivers for DynamoDB:

sudo pip install --upgrade boto3

In the three scripts (create.py, read.py, and write.py) change the value for “endpoint_url” to the IP address of the node.

Create a Table

Now, we’ll use the create.py script to create a table in our newly created cluster, using Alternator.

Python
 
from __future__ import print_function
import boto3
dynamodb = boto3.resource('dynamodb',endpoint_url='http://172.17.0.2:8000',
                  region_name='None', aws_access_key_id='None', aws_secret_access_key='None')

table = dynamodb.create_table(
    BillingMode='PAY_PER_REQUEST',
    TableName='mutant_data',
    KeySchema=[
    {
        'AttributeName': 'last_name',
        'KeyType': 'HASH'
    },
    ],
    AttributeDefinitions=[
    {
        'AttributeName': 'last_name',
        'AttributeType': 'S'
    },
    ]
)

print("Finished creating table ", table.table_name ,". Status: ", table.table_status)

Authorization is not in the scope of this lesson, so we’ll use "None" and revisit this in a future lesson.

We define a table called "mutant_data" with the required properties such as the primary key “last_name” which is of a String data type. You can read about Boto 3 data types here.

The DynamoDB data model is similar to ScyllaDB’s. Both databases have a partition key (also called “hash key” in DynamoDB) and an optional clustering key (called “sort key” or “range key” in DynamoDB), and the same notions of rows (which DynamoDB calls “items”) inside partitions. There are some differences in the data model. One of them is that in DynamoDB, columns (called “attributes”), other than the hash key and sort key, can be of any type, and can be different in each row. That means they don’t have to be defined in advance. You can learn more about data modeling in Alternator in more advanced lessons.

In this simple example, we use a one-node ScyllaDB cluster. In a production environment, it’s recommended to run a cluster of at least three nodes.

Also, in this example, we’ll send the queries directly to our single node. In a production environment, you should use a mechanism to distribute different DynamoDB requests to different ScyllaDB nodes, to balance the load. More about that in future lessons.

Run the script:

python create.py
 
[guy@localhost getting-started]$ python create.py
Finished creating table  mutant_data . Status:  ACTIVE
[guy@localhost getting-started]

Each Alternator table is stored in its own keyspace, which ScyllaDB automatically creates. Table xyz will be in keyspace alternator_xyz. This keyspace is initialized when the first Alternator table is created (with a CreateTable request). The replication factor (RF) for this keyspace and all Alternator tables is chosen at that point, depending on the size of the cluster: RF=3 is used on clusters with three or more live nodes. RF=1 would be used if our cluster is smaller, as it is in our case. Using a ScyllaDB cluster of fewer than three nodes is not recommended for production.

Performing Basic Queries

Next, we will write and read some data from the newly created table.

Python
 
from __future__ import print_function
import boto3
dynamodb = boto3.resource('dynamodb',endpoint_url='http://172.17.0.2:8000',
                  region_name='None', aws_access_key_id='None', aws_secret_access_key='None')

response = dynamodb.batch_get_item(
    RequestItems={
        'mutant_data' : { 'Keys': [{ 'last_name': 'Loblaw' }, {'last_name': 'Jeffries'}] }
    }
)
for x in response:
    print (x)
    for y in response[x]:
        print (y,':',response[x][y])

In this script, we use the batch_write_item operation to write data to the table “mutant_data.” This allows us to write multiple items in one operation. Here we write two items using a PutRequest, which is a request to perform the PutItem operation on an item.

Notice that, unlike ScyllaDB (and Apache Cassandra for that matter), in DynamoDB, Writes do not have a configurable consistency level. They use CL=QUORUM.

Execute the script to write the two items to the table:

python write.py
 
[guy@localhost getting-started]$ python write.py
('Finished writing to table ', 'mutant_data')
[guy@localhost getting-started]$ 

Next, we’ll read the data we just wrote, again using a batch operation, batch_get_item.

Python
 
from __future__ import print_function
import boto3
dynamodb = boto3.resource('dynamodb',endpoint_url='http://172.17.0.2:8000',
                  region_name='None', aws_access_key_id='None', aws_secret_access_key='None')

response = dynamodb.batch_get_item(
    RequestItems={
        'mutant_data' : { 'Keys': [{ 'last_name': 'Loblaw' }, {'last_name': 'Jeffries'}] }
    }
)
for x in response:
    print (x)
    for y in response[x]:
        print (y,':',response[x][y])

The response is a dictionary with the result being the two entries we previously wrote.

Execute the read to see the results:

 
[guy@localhost getting-started]$ python read.py
Responses
mutant_data : [{u'first_name': u'Bob', u'last_name': u'Loblaw', u'address': u'1313 Mockingbird Lane'}, {u'first_name': u'Jim', u'last_name': u'Jeffries', u'address': u'1211 Hollywood Lane'}]
ResponseMetadata
RetryAttempts : 0
HTTPStatusCode : 200
HTTPHeaders : {'date': '19 Apr 2020 17:30:23 GMT', 'content-length': '219', 'content-type': 'application/json', 'server': 'Seastar httpd'}
[guy@localhost getting-started]$ 

DynamoDB supports two consistency levels for reads, “eventual consistency” and “strong consistency.” You can learn more about ScyllaDB consistency levels here and here. Under the hood, ScyllaDB implements Strongly-consistent reads with LOCAL_QUORUM, while eventually-consistent reads are performed with LOCAL_ONE.

More Resources

  • Project Alternator Wiki, with examples (GitHub)
  • ScyllaDB Cloud versus Amazon DynamoDB, benchmark
  • Alternator Design Documentation

Conclusion

In this lesson, we learned the basics of Alternator, the open-source DynamoDB ScyllaDB API. We saw how to create a cluster, connect to it, write data, and read data.

API Amazon DynamoDB Ruby (programming language) Build (game engine) Cloud clustering Data (computing) Data model (GIS) Python (language) Requests

Published at DZone with permission of Guy Shtub. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Understanding the Fan-Out/Fan-In API Integration Pattern
  • How to Build a Pokedex React App with a Slash GraphQL Backend
  • Enhanced API Security: Fine-Grained Access Control Using OPA and Kong Gateway
  • Parent Document Retrieval (PDR): Useful Technique in RAG

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!