Over a million developers have joined DZone.

Docker Services, Stack, and Distributed Application Bundle

Learn how to create a distributed application bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode.

· Cloud Zone

Build fast, scale big with MongoDB Atlas, a hosted service for the leading NoSQL database on AWS. Try it now! Brought to you in partnership with MongoDB.

The first release candidate of Docker 1.12 was announced more than two weeks ago. Several new features are planned for this release.

This blog will show how to create a Distributed Application Bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode. Many thanks to @friism for helping me understand these concepts.

Let's look at the features first:

  • Built-in orchestration: A typical application is defined using a Docker Compose file. This definition consists of multiple containers deployed on multiple hosts. This avoids a single point of failure (SPOF) and keeps your application resilient. Multiple orchestration frameworks, such as Docker Swarm, Kubernetes, and Mesos allow you to orchestrate these applications. However, it is such an important characteristic of the application that Docker Engine now has built-in orchestration. More details on this topic in a later blog.
  • Service: A replicated, distributed, and load-balanced service can be easily created using docker service create command. A "desired state" of the application, such as run three containers of Couchbase, is provided, and the self-healing Docker engine ensures that the stated number of containers are running in the cluster. If a container goes down, another container is started. If a node goes down, containers on that node are started on a different node. More on this in a later blog.
  • Zero-Configuration Security: Docker 1.12 comes with mutually authenticated TLS, providing authentication, authorization, and encryption to the communications of every node participating in the swarm, out of the box. More on this in a later blog.
  • Docker Stack and Distributed Application Bundle: Distributed Application Bundle, or DAB, is a multi-services distributable image format. Read further for more details.

So far, you can take a Dockerfile and create an image from it using the docker build command. A container can be started using the docker run command. Multiple containers can be easily started by giving that command multiple times. Or you can also use Docker Compose file and scale up your containers using the docker-compose scale command. docker-lifecycle

An image is a portable format for a single container. But Distributed Application Bundle, or DAB, is a new concept introduced in Docker 1.12, is a portable format for multiple containers. Each bundle can be then deployed as a stack at runtime.

docker-stack-lifecycle

Learn more about DAB at docker.com/dab. For simplicity, here is an analogy that can be drawn:

Dockerfile -> Image -> Container

Docker Compose -> Distributed Application Bundle -> Docker Stack

Let's use a Docker Compose file, create a DAB from it, and deploy it as a Docker Stack.

It's important to note that this is an experimental feature in 1.12-RC2.

Create a Distributed Application Bundle From Docker Compose

Docker Compose CLI adds a new bundle command. More details can be found:

docker-compose bundle --help
Generate a Docker bundle from the Compose file.

Local images will be pushed to a Docker registry, and remote images
will be pulled to fetch an image digest.

Usage: bundle [options]

Options:
    -o, --output PATH          Path to write the bundle file to.
                               Defaults to "<project name>.dsb".


Now, let's take a Docker Compose definition and create a DAB from it. Here is our Docker Compose definition:

version: "2"
services:
  db:
    container_name: "db"
    image: arungupta/oreilly-couchbase:latest
    ports:
      - 8091:8091
      - 8092:8092 
      - 8093:8093 
      - 11210:11210
  web:
    image: arungupta/oreilly-wildfly:latest
    depends_on:
      - db
    environment:
      - COUCHBASE_URI=db
    ports:
      - 8080:8080


This Compose file starts a WildFly and a Couchbase server. A Java EE application is pre-deployed in the WildFly server that connects to the Couchbase server and allows to perform CRUD operations using the REST API. The source for this file is at: github.com/arun-gupta/oreilly-docker-book/blob/master/hello-javaee/docker-compose.yml. Generate an application bundle with it:

docker-compose bundle
WARNING: Unsupported key 'depends_on' in services.web - ignoring
WARNING: Unsupported key 'container_name' in services.db - ignoring
Wrote bundle to hellojavaee.dsb


depends_on only creates dependency between two services and makes them start in a specific order. This only ensures that the Docker container is started, but the application within the container may take longer to start. So, this attribute only partially solves the problem.

container_name gives a specific name to the container. Relying upon a specific container name is tight coupling and does not allow us to scale the container.  So both the warnings can be ignored, for now. This command generates a file using the Compose project name, which is the directory name. So in our case, the hellojavaee.dsb file is generated. This file extension has been renamed to .dab in RC3. The generated application bundle looks like:

{
  "services": {
    "db": {
      "Image": "arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c", 
      "Networks": [
        "default"
      ], 
      "Ports": [
        {
          "Port": 8091, 
          "Protocol": "tcp"
        }, 
        {
          "Port": 8092, 
          "Protocol": "tcp"
        }, 
        {
          "Port": 8093, 
          "Protocol": "tcp"
        }, 
        {
          "Port": 11210, 
          "Protocol": "tcp"
        }
      ]
    }, 
    "web": {
      "Env": [
        "COUCHBASE_URI=db"
      ], 
      "Image": "arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914", 
      "Networks": [
        "default"
      ], 
      "Ports": [
        {
          "Port": 8080, 
          "Protocol": "tcp"
        }
      ]
    }
  }, 
  "version": "0.1"
}


This file provides a complete description of the services included in the application. I'm not entirely sure if Distributed Application Bundle is the most appropriate name, discuss this in #24250. It would be great if other container formats, such as Rkt or even VMs, could be supported here. But for now, Docker is the only supported format.

Initialize Swarm Mode in Docker

As mentioned above, "desired state" is now maintained by Docker Swarm. And this is now baked into Docker Engine already. Docker Swarm concepts have evolved as well and can be read at Swarm mode key concepts. A more detailed blog on this will be coming later. But for this blog, a new command docker swarm is now added:

docker swarm --help

Usage: docker swarm COMMAND

Manage Docker Swarm

Options:
      --help   Print usage

Commands:
  init        Initialize a Swarm
  join        Join a Swarm as a node and/or manager
  update      Update the Swarm
  leave       Leave a Swarm
  inspect     Inspect the Swarm

Run 'docker swarm COMMAND --help' for more information on a command.


Initialize a Swarm node (as a worker) in the Docker Engine:

docker swarm init
Swarm initialized: current node (ek9p1k8r8ox7iiua5c247skci) is now a manager.


More details about this node can be found using docker swarm inspect command.

docker swarm inspect
[
    {
        "ID": "1rcvu7m9mv2c8hiaijr7an9zk",
        "Version": {
            "Index": 1895
        },
        "CreatedAt": "2016-07-01T23:52:38.074748177Z",
        "UpdatedAt": "2016-07-02T04:54:32.79093117Z",
        "Spec": {
            "Name": "default",
            "AcceptancePolicy": {
                "Policies": [
                    {
                        "Role": "worker",
                        "Autoaccept": true
                    },
                    {
                        "Role": "manager",
                        "Autoaccept": false
                    }
                ]
            },
            "Orchestration": {
                "TaskHistoryRetentionLimit": 10
            },
            "Raft": {
                "SnapshotInterval": 10000,
                "LogEntriesForSlowFollowers": 500,
                "HeartbeatTick": 1,
                "ElectionTick": 3
            },
            "Dispatcher": {
                "HeartbeatPeriod": 5000000000
            },
            "CAConfig": {
                "NodeCertExpiry": 7776000000000000
            }
        }
    }
]


The output shows that the node is only a worker, and not a manager. This may be fine if the cluster has a single node, but a multi-node cluster should have at least a manager.

Deploy a Docker Stack

Create a stack using docker deploy command:

docker deploy -f hellojavaee.dsb hellojavaee
Loading bundle from hellojavaee.dsb
Creating network hellojavaee_default
Creating service hellojavaee_db
Creating service hellojavaee_web


The command usage can certainly be simplified as discussed in #24249. See the list of services:

docker service ls
ID            NAME             REPLICAS  IMAGE                                                                                                COMMAND
2g8kmrimztes  hellojavaee_web  1/1       arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914    
46xhlb15cc60  hellojavaee_db   1/1       arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c


The output shows that two services, WildFly and Couchbase, are running. Services is also a new concept introduced in Docker 1.12. It's what gives you the "desired state," and Docker Engine works to give you that. docker ps shows the list of containers running:

CONTAINER ID        IMAGE                                                                                                 COMMAND                  CREATED             STATUS              PORTS                                                        NAMES
622756277f40        arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c   "/entrypoint.sh /opt/"   3 seconds ago       Up 1 seconds        8091-8093/tcp, 11207/tcp, 11210-11211/tcp, 18091-18092/tcp   hellojavaee_db.1.19enwdt6i5m853m5675tx3z29
abf8703ed713        arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914     "/opt/jboss/wildfly/b"   3 seconds ago       Up 1 seconds        8080/tcp                                                     hellojavaee_web.1.70piloz6j4zt06co8htzisgyl


WildFly container starts up before the Couchbase container is up and running. This means the Java EE application tries to connect to the Couchbase server and fails. So the application never boots successfully.

Self-Healing Docker Service

Docker Service maintains the "desired state" of an application. In our case, the desired state is to ensure that one, and only one, container for the service is running. If we remove the container, not the service, then the service will automatically start the container again. Remove the container as:

docker rm -f abf8703ed713


Note, you have to put -f because the container is already running. Docker 1.12 self-healing mechanisms kick in and automatically restart the container. Now if you list the containers again:

CONTAINER ID        IMAGE                                                                                                 COMMAND                  CREATED             STATUS                  PORTS                                                        NAMES
db483ac27e41        arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914     "/opt/jboss/wildfly/b"   1 seconds ago       Up Less than a second   8080/tcp                                                     hellojavaee_web.1.ddvwdmojjysf46d4n3x4g8uv4
622756277f40        arungupta/oreilly-couchbase@sha256:f150fcb9fca5392075c96f1baffc7f893858ba763f3c05cf0908ef2613cbf34c   "/entrypoint.sh /opt/"   26 seconds ago      Up 25 seconds           8091-8093/tcp, 11207/tcp, 11210-11211/tcp, 18091-18092/tcp   hellojavaee_db.1.19enwdt6i5m853m5675tx3z29


This shows that a new container has been started. Inspect the WildFly service:

docker service inspect hellojavaee_web
[
    {
        "ID": "54otfi6dc9bis7z6gc6ubynwc",
        "Version": {
            "Index": 328
        },
        "CreatedAt": "2016-07-02T01:36:35.735767569Z",
        "UpdatedAt": "2016-07-02T01:36:35.739240775Z",
        "Spec": {
            "Name": "hellojavaee_web",
            "Labels": {
                "com.docker.stack.namespace": "hellojavaee"
            },
            "TaskTemplate": {
                "ContainerSpec": {
                    "Image": "arungupta/oreilly-wildfly@sha256:d567ade7bb82ba8f15a85df0c6d692d85c15ec5a78d8826dfba92756babcb914",
                    "Env": [
                        "COUCHBASE_URI=db"
                    ]
                }
            },
            "Mode": {
                "Replicated": {
                    "Replicas": 1
                }
            },
            "Networks": [
                {
                    "Target": "epw57lz7txtfchmbf6u0cimis",
                    "Aliases": [
                        "web"
                    ]
                }
            ],
            "EndpointSpec": {
                "Mode": "vip",
                "Ports": [
                    {
                        "Protocol": "tcp",
                        "TargetPort": 8080
                    }
                ]
            }
        },
        "Endpoint": {
            "Spec": {},
            "Ports": [
                {
                    "Protocol": "tcp",
                    "TargetPort": 8080,
                    "PublishedPort": 30004
                }
            ],
            "VirtualIPs": [
                {
                    "NetworkID": "9lpz688ir3pzexubkcb828ikg",
                    "Addr": "10.255.0.5/16"
                },
                {
                    "NetworkID": "epw57lz7txtfchmbf6u0cimis",
                    "Addr": "10.0.0.4/24"
                }
            ]
        }
    }
]


Swarm assigns a random port to the service, or we can manually update it using the docker service update command. In our case, port 8080 of the container is mapped to 30004 port on the host.

Verify the Application

Check that the application is successfully deployed:

curl http://localhost:30004/books/resources/book
[{"books":0}]


Add a new book to the application:

curl -v \
> -H "Content-Type: application/json" \
> -X POST -d '{
>   "isbn": "978-1-4919-1889-0",
>   "name": "Minecraft Modding with Forge",
>   "cost": 29.99
> }' \
> http://localhost:30004/books/resources/book
*   Trying ::1...
* Connected to localhost (::1) port 30004 (#0)
> POST /books/resources/book HTTP/1.1
> Host: localhost:30004
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 92
> 
* upload completely sent off: 92 out of 92 bytes
< HTTP/1.1 200 OK
< Connection: keep-alive
< X-Powered-By: Undertow/1
< Server: WildFly/10
< Content-Type: application/octet-stream
< Content-Length: 88
< Date: Sat, 02 Jul 2016 01:39:49 GMT
< 
* Connection #0 to host localhost left intact
{"name":"Minecraft Mhttp://localhost:30004/books/resources/book-1-4919-1889-0"}


Verify the books again:

curl http://localhost:30004/books/resources/book
[{"books":{"name":"Minecraft Modding with Forge","cost":29.99,"id":"1","isbn":"978-1-4919-1889-0"}}, {"books":1}]


Learn more about this Java EE application at github.com/arun-gupta/oreilly-docker-book/tree/master/hello-javaee.

This blog showed how to create a distributed application bundle from Docker Compose and deploy it as Docker Stack in Docker Swarm Mode.

Now it's easier than ever to get started with MongoDB, the database that allows startups and enterprises alike to rapidly build planet-scale apps. Introducing MongoDB Atlas, the official hosted service for the database on AWS. Try it now! Brought to you in partnership with MongoDB.

Topics:
docker compose ,docker swarm

Published at DZone with permission of Arun Gupta, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}