Running Imply Druid Distribution Inside Docker Container

Druid is an open-source data store designed for sub-second queries on real-time and historical data. Druid can scale to store trillion of events and ingest millions of events per second. Druid is best used to power user-facing data applications.

Imply is an analytics solution powered by druid. The Imply Analytics platform includes Druid bundled with all its dependencies, an exploratory analytics UI, and a SQL layer. It also provides additional tools and scripts for easy management of druid nodes.

Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, data center VMs, or the cloud.

In this post, I will demonstrate how to install Imply Druid distribution inside a Docker container.

First install Docker, if you haven’t done it yet. 

Now open a terminal and clone this repository from Github and go to the cloned repository directory.

$ git clone https://github.com/implydata/distribution-docker.git
$ cd distribution-docker

Download the latest stable Imply distribution from the imply.io, place it in the cloned repository, and then run:

$ export implyversion=2.5.16
$ tar -xzf imply-$implyversion.tar.gz
$ docker build -t imply:$implyversion --build-arg implyversion=$implyversion .

Upon the success on Docker build, you will get following in terminal:

Sending build context to Docker daemon  889.5MB
Step 1/10 : FROM ubuntu:16.04
 ---> f975c5035748
Step 2/10 : COPY setup-os.sh /root
 ---> Using cache
 ---> 3ab409279de8
Step 3/10 : RUN /root/setup-os.sh
 ---> Using cache
 ---> 612a6a67cb81
Step 4/10 : ARG implyversion
 ---> Using cache
 ---> a3f4708cd38a
Step 5/10 : COPY imply-$implyversion /opt/imply-${implyversion}
 ---> e13c382abe46
Step 6/10 : RUN mkdir -p /mnt/imply/var && ln -snf /mnt/imply/var /opt/imply-${implyversion}/var
 ---> Running in 00908d6eaebf
Removing intermediate container 00908d6eaebf
 ---> 25a229298aab
Step 7/10 : RUN ln -snf /opt/imply-${implyversion} /opt/imply
 ---> Running in 32b5097e4451
Removing intermediate container 32b5097e4451
 ---> 0e0d835e9e6b
Step 8/10 : EXPOSE 1527 2181 8081 8082 8083 8090 8091 8100 8101 8102 8103 8104 8105 8106 8107 8108 8109 8110 8200 9095
 ---> Running in 98679f4102ef
Removing intermediate container 98679f4102ef
 ---> 85b704690c3d
Step 9/10 : WORKDIR /opt/imply-$implyversion
Removing intermediate container 7334d39e08da
 ---> c2ee0a1b9c91
Step 10/10 : CMD ["bin/supervise", "-c", "conf/supervise/quickstart.conf"]
 ---> Running in 2e6162fe1398
Removing intermediate container 2e6162fe1398
 ---> 0df8c88fa274
Successfully built 0df8c88fa274
Successfully tagged imply:2.5.16

To run the image in quickstart mode (single-machine, non-clustered):

$ docker run -p 8081-8110:8081-8110 -p 8200:8200 -p 9095:9095 -d --name imply imply:$implyversion

To load the example data:

$ docker exec -it imply bin/post-index-task -f quickstart/wikipedia-index.json

You will see some log output in terminal.

Beginning indexing data for wikipedia
Task started: index_wikipedia_2018-06-18T05:36:43.392Z
Task log:     http://localhost:8090/druid/indexer/v1/task/index_wikipedia_2018-06-18T05:36:43.392Z/log
Task status:  http://localhost:8090/druid/indexer/v1/task/index_wikipedia_2018-06-18T05:36:43.392Z/status
Task index_wikipedia_2018-06-18T05:36:43.392Z still running...
Task index_wikipedia_2018-06-18T05:36:43.392Z still running...
Task index_wikipedia_2018-06-18T05:36:43.392Z still running...
Task index_wikipedia_2018-06-18T05:36:43.392Z still running...
Task finished with status: SUCCESS
Completed indexing data for wikipedia. Now loading indexed data onto the cluster...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia is 0.0% finished loading...
wikipedia loading complete! You may now query your data

Now to access Imply, go to http://localhost:9095 in any web browser.

To enter the container, if you want:

$ docker exec -it imply /bin/bash

To stop the container:

$ docker stop imply

Congratulations! You have now installed and run Imply inside a Docker container. You can now configure data cube to visualize the sample data or you can execute queries using Druid SQL.