Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Fun With Docker Swarm

DZone's Guide to

Fun With Docker Swarm

In this post, a Docker lover shows us how to quickly create a Dockerfile, and use Docker Swarm to upload an image to the web!

· Cloud Zone
Free Resource

Are you joining the containers revolution? Start leveraging container management using Platform9's ultimate guide to Kubernetes deployment.

IMG_6209.JPGYep, this is me, a crazy Docker fan. Well, I know I may sound a little crazy to you, but I'd rather be totally honest with you about my passion and love for Docker, and share my experience of using this great technology and some interesting experiments that I've been doing with it.

The major reason for experimenting with Docker Swarm is to compare it with Kubernetes, as these two have similarities. Another post is on the way to share my experience with setting up a Kubernetes cluster on my local PC using MiniKube and a remote cluster on DigitalOcean.

OK, now I'll walk you through how I set up my first Docker Swarm cluster.

Create Compute Resources

I chose DigitalOcean Droplets for setting up my Docker nodes, because it's a lot cheaper, and I love the way it charges on-demand resource usage (either monthly or per hour), and I also think it makes sense to get the "whale" in the "ocean."

droplets_image.png

You can build your own version of a distribution image, similar to EC2 AMI, but I won't bother this time because we only need to make sure the Docker daemon is running on the new droplets (VMs).

Droplets_Size.png

It's OK to choose the smallest size if you're not planning to run any fancy applications consuming a lot of resources. For my case, I only need each of my VMs to be able to run a single Docker container hosting an Apache service, but I chose the "$20/mo" one just in case I might keep stuffing and tweaking my Docker image later on.

Not_bother_volume

I won't bother attaching any extra block storages to my VMs, simply because I'm not considering data persistence.

Screen Shot 2017-11-07 at 10.53.27 pm.png

Make sure you tick "Private networking" so that all your VMs will be running within the same region. I'm not considering setting up a NAT instance for routing outbound traffic from my Docker nodes at this stage, which means all my nodes will be assigned to public IPs and publicly accessible.

SSH_restrictions.png

Well, I made my Docker nodes publicly accessible, but it doesn't mean anyone can access it. To secure SSH access, I uploaded my public SSH key, so that only those that have access to my private SSH key can SSH onto my Docker nodes (which, in this case, is only me). 

Good_to_go.png

We'll need at least 2 VMs, one will be the Docker Swarm manager, the other is the worker node. I chose 3 because I want to try Docker service scaling as well as service deployment.

Now, hit the "Create" button to spin up the 3 Droplets.

Ready_to_roll.png

Set Up Docker Engine

Remember I ticked the SSH public key when configuring my Droplets in the previous section, which allows me to SSH onto my Docker nodes by specifying my private SSH key. And now I can send the command to install a Docker engine onto each Docker node by specifying the public IP.

$ MY_DOCKER_NODES=('188.166.180.0' '188.166.186.100' '128.199.163.9')
$ for MY_DOCKER_NODE in ${MY_DOCKER_NODES[@]} ; do
> ssh -i .ssh/id_rsa root@${MY_DOCKER_NODE} \
> "curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh"
> done

You can also enable the user data option and add this command for each Droplet to automate the installation during the launch process.

Initialize Docker Swarm

I chose docker-host-01 as my Docker Swarm manager/leader, this node will manage all the Swarm nodes, you can consider it a central cluster management which gathers the state of each node, onboards the new worker nodes, and dispatches tasks such as rolling updates of an image. You can have multiple managers for redundancy purposes.

Now, let's jump on docker-host-01 to initialize a Docker Swarm.

Please note that I'm using the private IP of the manager node to initialize the swarm, considering all the communication between nodes will be within the swarm cluster, and the VMs can talk to each other directly using their private IPs.

root@docker-host-01:~# docker swarm init --advertise-addr 10.130.33.24
Swarm initialized: current node (wkrcfcd976n9hozyortjxw9pt) is now a manager.

To add a worker to this swarm, run the following command:

docker swarm join --token SWMTKN-1-26ta5kl9gkxj4iirzff24txdusqkv5u7gn5kxbqeoee0wg4p67-1eni9753bi1zkxtp22r2jxxbg 10.130.33.24:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

In the output from the initialization command (the swarm join token is provided), use the token for joining new worker nodes into the cluster. If you accidentally close your terminal and can't remember the token, not to worry, use the following command to retrieve the command with the join-token for either joining a new manager or a new worker.

root@docker-host-01:~# docker swarm join-token worker
To add a worker to this swarm, run the following command:

docker swarm join --token SWMTKN-1-26ta5kl9gkxj4iirzff24txdusqkv5u7gn5kxbqeoee0wg4p67-1eni9753bi1zkxtp22r2jxxbg 10.130.33.24:2377

OK, now we have a swarm cluster with only one manager node:

root@docker-host-01:~# docker node ls
ID                          HOSTNAME       STATUS AVAILABILITY MANAGER STATUS
wkrcfcd976n9hozyortjxw9pt * docker-host-01 Ready  Active       Leader

Onboard New Worker Nodes

Jump on "docker-host-02," and use the join command to join it to the cluster:

root@docker-host-02:~# docker swarm join --token SWMTKN-1-26ta5kl9gkxj4iirzff24txdusqkv5u7gn5kxbqeoee0wg4p67-1eni9753bi1zkxtp22r2jxxbg 10.130.33.24:2377
This node joined a swarm as a worker.

Jump on docker-host-03 and repeat the same step.

Let's verify it on the manager node:

root@docker-host-01:~# docker node ls
ID                          HOSTNAME       STATUS AVAILABILITY MANAGER STATUS
wkrcfcd976n9hozyortjxw9pt * docker-host-01 Ready  Active       Leader
65rdswe0aua3nk9mom530ilgh   docker-host-02 Ready  Active
9hmoz9bvlrtkcivnhzvzadrg2   docker-host-03 Ready  Active

Yay! Welcome, docker-host-0[2-3] onboard! My Docker Swarm cluster is ready.

Build a Customized Image

My Docker image is fairly simple, I just need to make sure the image of me wearing a Docker t-shirt will show up in the index.html page served via the Apache service (yeah, you're right, such a big deal).

Dockerfile

FROM httpd:latest

COPY index.html /usr/local/apache2/htdocs/
COPY IMG_6209.JPG /usr/local/apache2/htdocs/

The image is based on httpd which sets up the Apache service, and the two COPY commands will get my customized index.html, which has an <IMG> tag referencing my image and my image, built into the image.

File Hierarchy

Make sure you place all the files to be included in the new Docker image under the same directory as the Dockerfile.

$ ls -1
Dockerfile
IMG_6209.JPG
index.html

Cooking Time

Now, it's time to carry on cooking my customized Docker image based on httpd; exciting! Make sure you run the docker build command under the same directory as the Dockerfile and the source files.

$ docker build -t myapache .
Sending build context to Docker daemon 866.8kB
Step 1/3 : FROM httpd:latest
latest: Pulling from library/httpd
85b1f47fba49: Pull complete
45bea5eb3b59: Pull complete
d360abbf616c: Pull complete
91c7cdd03f84: Pull complete
30623dd230a8: Pull complete
cc21a2e04dd3: Pull complete
f789cd8382be: Pull complete
Digest: sha256:8ac08d0fdc49f2dc83bf5dab36c029ffe7776f846617335225d2796c74a247b4
Status: Downloaded newer image for httpd:latest
 ---> 74ad7f48867f
Step 2/3 : COPY index.html /usr/local/apache2/htdocs/
 ---> de155cc82acc
Step 3/3 : COPY IMG_6209.JPG /usr/local/apache2/htdocs/
 ---> c19556219f5c
Successfully built c19556219f5c
Successfully tagged myapache:latest

The new Docker image looks just like a 3-layered cake, which is exactly what I love about the Docker image concept, that you can see the incremental changes on top of the base (in this case, the base is httpd image). Yum!

$ docker images
REPOSITORY TAG    IMAGE ID     CREATED       SIZE
myapache   latest c19556219f5c 8 minutes ago 178MB
httpd      latest 74ad7f48867f 2 days ago    177MB

In order to make sure all the Docker nodes have access to the image, I have to push the image to an external/remote repository. I chose to use DockerHub since I don't have any sensitive data in the image (besides, I've already posted the image of me with the Docker t-shirt on Twitter), I just created a public repository in my personal DockerHub account.

Screen Shot 2017-11-07 at 6.12.00 pm.png

First, I log in to my DockerHub account:

$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: mengxuzhao
Password: ****
Login Succeeded

Second, I tag my local image with my DockerHub registry URL:

$ docker tag myapache:latest mengxuzhao/myapache:latest
$ docker images
REPOSITORY          TAG    IMAGE ID      CREATED         SIZE
myapache            latest c19556219f5c  15 minutes ago  178MB
mengxuzhao/myapache latest c19556219f5c  15 minutes ago  178MB
httpd               latest 74ad7f48867f  2 days ago      177MB

Then, I push the image tagged with the remote registry URL to the remote repository:

$ docker push mengxuzhao/myapache:latest
The push refers to a repository [docker.io/mengxuzhao/myapache]
b8f15e0a2d0e: Pushed
876337d1893b: Pushed
607cc31ddf99: Layer already exists
4bb682cd39bd: Layer already exists
aa6b8cb0f3ac: Layer already exists
9ff764a04f48: Layer already exists
607e4dd85443: Layer already exists
fc59c7018530: Layer already exists
c01c63c6823d: Layer already exists
latest: digest: sha256:80c4bf623a77cfe029694287e6f20024ac3f4fc77b6c205515dfba0c707cd9e0 size: 2197

Screen Shot 2017-11-07 at 6.22.03 pm.png

Now, the image is available and accessible from my public DockerHub repository.

Get the Service Up

Hope you're rubbing your hands together like I am at the moment. Yes, finally, we get to the stage of getting the customized Apache image onto the Docker Swarm cluster.

The following steps need to be executed on the Swarm manager node, so let's jump on docker-host-01.

Create a Service

root@docker-host-01:~# docker service create \
> --name mengapache \
> --replicas=2 \
> mengxuzhao/myapache:latest
ai7fvib9y52wgn92uu29ugra0
overall progress: 2 out of 2 tasks
1/2: running [==================================================>]
2/2: running [==================================================>]
verify: Service converged

To create a service, first, pick a cool service name, and tell it how many replicas are needed (whether you need service redundancy), and which Docker image will be used for spinning up containers which host Apache services. The service details can be dumped by running service inspect.

root@docker-host-01:~# docker service inspect --pretty mengapache

ID: ai7fvib9y52wgn92uu29ugra0
Name: mengapache
Service Mode: Replicated
 Replicas: 2
Placement:
UpdateConfig:
 Parallelism: 1
 On failure: pause
 Monitoring Period: 5s
 Max failure ratio: 0
 Update order:      stop-first
RollbackConfig:
 Parallelism: 1
 On failure: pause
 Monitoring Period: 5s
 Max failure ratio: 0
 Rollback order:    stop-first
ContainerSpec:
 Image: mengxuzhao/myapache:latest@sha256:80c4bf623a77cfe029694287e6f20024ac3f4fc77b6c205515dfba0c707cd9e0
Resources:
Endpoint Mode: vip

If you want to find out which Docker nodes are hosting the service, run the following command:

root@docker-host-01:~# docker service ps mengapache
ID           NAME         IMAGE                      NODE           DESIRED STATE  CURRENT STATE            ERROR PORTS
vgd4uuisbmdr mengapache.1 mengxuzhao/myapache:latest docker-host-02 Running        Running 10 minutes ago
6xtu04eavdfh mengapache.2 mengxuzhao/myapache:latest docker-host-01 Running        Running 10 minutes ago

To get the details of the running containers on the Docker nodes, just simply run "docker ps" on the Docker node:

On docker-host-01:

root@docker-host-01:~# docker ps
CONTAINER ID IMAGE                      COMMAND            CREATED         STATUS        PORTS  NAMES
77f9a081e6fa mengxuzhao/myapache:latest "httpd-foreground" 15 minutes ago  Up 15 minutes 80/tcp mengapache.2.6xtu04eavdfh95zoqlzcri81g

On docker-host-02:

root@docker-host-02:~# docker ps
CONTAINER ID IMAGE                      COMMAND            CREATED         STATUS        PORTS  NAMES
4cde5b4f9d3c mengxuzhao/myapache:latest "httpd-foreground" 16 minutes ago  Up 16 minutes 80/tcp mengapache.1.vgd4uuisbmdrbo518xt4wsm8d

Each container is given a name which matches up with the details from the "service inspect" (the NAMES is the last column of the output of "docker ps," please scroll to the right to find them).

Port Forwarding

Now my Docker service is up and running. To make it publicly accessible I need to publish the port of the running Apache services and mapped it to port 80 on each Docker node so that I can access the service via their public IPs.

The following step needs to be executed on the Swarm manager node as well.

root@docker-host-01:~# docker service update --publish-add 80:80 \
> mengapache
mengapache
overall progress: 2 out of 2 tasks
1/2: running [==================================================>]
2/2: running [==================================================>]
verify: Service converged

Show Time!

Open a web browser, and type in http://188.166.180.0 (the IP of docker-host-01):

TA-DA!

Screen Shot 2017-11-07 at 7.19.32 pm.png

Remember I set the replicas to 2, and "myapache" service is not running on docker-host-03, so what happens if we access via docker-host-03, should we expect a "404" error?

Screen Shot 2017-11-07 at 7.27.17 pm.png

Of course not! It still works, because the service will find the suitable Docker node which actually is running the service, even if you're hitting the one which is not. (Yeah, the purpose of doing the check is to make you look at my lovely photo again.)

Horizontal Scaling

One of the coolest things ab0ut cluster orchestration is auto-scaling, which is also a great feature of Docker Swarm.

At the moment, I've got 2 replicas hosting "mengapache" service, and I want to add one more, which can be simply done by running the following command on the swarm manager node - docker-host-01:

root@docker-host-01:~# docker service scale mengapache=3
mengapache scaled to 3
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged

Now, I've got all three of my Docker nodes hosting the mengapache service:

root@docker-host-01:~# docker service ps mengapache
ID           NAME         IMAGE                      NODE           DESIRED STATE  CURRENT STATE              ERROR PORTS
rws6j2h2jq8a mengapache.1 mengxuzhao/myapache:latest docker-host-02 Running        Running 23 minutes ago
igzcwz60k0gy mengapache.2 mengxuzhao/myapache:latest docker-host-01 Running        Running 23 minutes ago
sfpsu901y67z mengapache.3 mengxuzhao/myapache:latest docker-host-03 Running        Running about a minute ago

Pretty cool, huh?!

Rolling Update

The last thing I want to demonstrate is a rolling update of the service by making a minor change on the index.html file. Then I need to rebuild my Docker image and push it to my public DockerHub repository; please refer to the previous sections for the details.

Once I've got my new image available in the remote repository, I run the following command on the swarm manager node - docker-host-01:

root@docker-host-01:~# docker service update --image mengxuzhao/myapache:latest mengapache
mengapache
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged

Open a new browser, and this time let's try accessing our file via docker-host-02:

Screen Shot 2017-11-07 at 7.49.45 pm.png

Yay, my change has been rolled out successfully! (Sorry for making you see me wearing the Docker t-shirt one more time, this will be the last time, guaranteed.)

Quick Summary

Docker Swarm is easy to set up, configure, scale, and deliver updates continuously. And I love it. In my coming posts, I'll share my experience with using Kubernetes and then compare Docker Swarm vs Kubernetes on similar features.

Using Containers? Read our Kubernetes Comparison eBook to learn the positives and negatives of Kubernetes, Mesos, Docker Swarm and EC2 Container Services.

Topics:
docker swarm ,docker ,cloud

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}