Custom DynamoDB Docker Instance
Custom DynamoDB Docker Instance
Let's look at a custom DynamoDB Docker instance.
Join the DZone community and get the full member experience.Join For Free
Hey guys, I hope you all are doing well. I am back with another article on custom docker instances for databases. In my last post, we saw how we could have our custom docker instance for MySQL. Similarly, in this post, we will see how we can do the same with DynamoDB, so let's get started.
Just like the scenario in the previous article, I was working on a project with DynamoDB as the database due to its many features like scalability, cloud storage, etc. And I wanted to test some things and did not want to mess with the cloud instance, so I thought to make an instance of my own, so what to do?
I started searching over the internet, and the things I found are as follows:
- You can run DynamoDB miniature instance on your local, which you can download here (Great).
- Running the local version gives us an option of sharedD, which saves us from the region options and saves all the data in a file named
- A GitHub library that lets you take the dump dynamo and populate it as well (Almost there).
- And finally, building the docker instance (Bingo).
So let's get started. Once you are done downloading the tar file from the above link, just extract it somewhere. I would suggest doing so in a separate directory, and then you can run it by simply using:
java -Djava.library.path=DynamoDBLocal_lib -jar DynamoDBLocal.jar -optimizeDbBeforeStartup -sharedDb
The above command would run the local instance for DynamoDB accessible on
localhost:8000. Once you populate it with some data, you will start to see the file
Dumping the data
Now comes the tricky part; to take the dump from the remote instance. Obviously you can write your own utility for that, but thanks to the developers/contributors of the above lib, now you can simply clone the project and have a look at the documentation for the project to understand how to use the python script. Let me give you an example. I am taking an example to dump all tables, but you can also modify the command for different options.
But before that, just make sure that your DynamoDB instance is accessible from the machine from which you will run the python script.
python dynamodump.py -m backup --host dynamoHost --port 8000 -r us-west-1 -s production*
The above command will dump the data from the dynamo host for AWS region
us-west-1 from the tables that have production in the beginning of their name. You may simply choose to use
"*" to dump all tables, but while running, you may encounter failure for the dependencies, which can be installed beforehand using the
requirements.txt file from the above project. Just run:
pip install -r requirement.txt
and you will be all set. Once you are done, you will get a file structure like below. Suppose you have two tables in your remote DynamoDB, then the structure would be as follows:
dump | | +-------TableA | | | | | +------data | | | +------schema.json +-------TableB | | +------data | +------schema.json
So basically, a dump folder would be created and table data with schema would be there in JSON format.
Now the idea would be to take the dump that we already did and then use the backup part to populate the local DynamoDB instance. So then, just use:
python dynamodump.py -m restore --host localhost --port 8000 -r us-west-1 -s "*"
The above command, when run, would populate the local DynamoDB instance.
We can use this DynamoDB instance with the data of remote instances, and we are good to do experiments on it.
It is pretty handy now, but what if you want to ship it or run it in a cluster environment? For that, I took one more step and dockerized it. I tried for many days to create my own custom DynamoDB base image, but failed. So I then used a custom image from one of the forums, which you can pull using
anapsix/alpine-java. For dockerization, we can use the
The Dockerfile is below:
#BASE IMAGE FROM anapsix/alpine-java #ADD REQUIRED PACKAGES RUN apk add --update ca-certificates wget openssl #SETTING DIRECTORY WORKDIR dynamo #DOWNLOADING DYNAMO AND SETTING UP RUN wget http://dynamodb-local.s3-website-us-west-2.amazonaws.com/dynamodb_local_latest.tar.gz && tar -xzf dynamodb_local_latest.tar.gz && rm -f dynamodb_local_latest.tar.gz* #COPYING DB FILE COPY shared-local-instance.db /dynamo #SETTING UP ENTRYPOINT RUNNING THE DYNAMO ENTRYPOINT java -Djava.library.path=DynamoDBLocal_lib -jar DynamoDBLocal.jar -optimizeDbBeforeStartup -dbPath ./ -sharedDb
Using the above docker file, just use the command:
docker build -t <image-name>:<tag>
to build the docker image, assuming that Dockerfile and db file are in the same directory. Then, you are good to use it as a standalone container or maybe in cluster environments.
That's how you do it! Please share the post if you find it useful and drop any comments and suggestions below. Happy coding!
This article was first published on the Knoldus blog.
Published at DZone with permission of shubham verma . See the original article here.
Opinions expressed by DZone contributors are their own.