DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workkloads.

Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Docker Model Runner: Streamlining AI Deployment for Developers
  • A Guide to Container Runtimes
  • Gemma 3: Unlocking GenAI Potential Using Docker Model Runner
  • Docker vs Kubernetes: Which to Use and When?

Trending

  • AI's Dilemma: When to Retrain and When to Unlearn?
  • Debugging Core Dump Files on Linux - A Detailed Guide
  • Analyzing “java.lang.OutOfMemoryError: Failed to create a thread” Error
  • How to Convert Between PDF and TIFF in Java
  1. DZone
  2. Software Design and Architecture
  3. Cloud Architecture
  4. Deploying Docker Compose Services in a Swarm

Deploying Docker Compose Services in a Swarm

The newest version of Docker Compose comes with enhanced multi-host support. With the addition, you can have stateful services running in containers.

By 
Arun Gupta user avatar
Arun Gupta
·
Jan. 24, 17 · Tutorial
Likes (7)
Comment
Save
Tweet
Share
27.1K Views

Join the DZone community and get the full member experience.

Join For Free

Docker 1.13 introduced a new version of Docker Compose. The main feature of this release is that it allows services defined using Docker Compose files to be directly deployed to Docker Engine, enabled with Swarm mode. This enables simplified deployment of multi-container applications on multi-host setups.

Docker 1.13

This article will use a simple Docker Compose file to show how services are created and deployed in Docker 1.13.

Here is a Docker Compose v2 definition for starting a Couchbase database node:

version: "2"
services:
  db:
    image: arungupta/couchbase:latest
    ports:
      - 8091:8091
      - 8092:8092
      - 8093:8093
      - 11210:11210


This definition can be started on a Docker Engine without Swarm mode with:

docker-compose up


This will start a single replica of the service define in the Compose file. This service can be scaled with:

docker-compose scale db=2


This works fine on a single host. But if swarm mode is enabled on Docker Engine, then it shows this message:

WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.


Swarm mode allows you to create a cluster of Docker Engines. With 1.13, the docker stack deploy command can be used to deploy a Compose file to Swarm mode. Docker Compose gives us multi-container applications, but the applications are still restricted to a single host. And that is a single point of failure.

Here is a Docker Compose v3 definition:

version: "3"
services:
  db:
    image: arungupta/couchbase:latest
    ports:
      - 8091:8091
      - 8092:8092
      - 8093:8093
      - 11210:11210


As you can see, the only change is the value of the version attribute. There are other changes in Docker Compose v3. Also, read about different Docker Compose versions and how to upgrade from v2 to v3.

Enable swarm mode:

docker swarm init


Deploy the services defined in Compose file:

docker stack deploy --compose-file=docker-compose.yml couchbase


Other nodes can join this Swarm cluster, and this would easily allow us to deploy the multi-container application to a multi-host as well. A default value of Compose file here would make the command a bit shorter. #30352 should take care of that.

A list of services running can be verified using the docker service ls command:

ID            NAME          MODE        REPLICAS  IMAGE
05wa4y2he9w5  couchbase_db  replicated  1/1       arungupta/couchbase:latest


The list of containers running within the service can be seen using the docker service ps command:

ID            NAME            IMAGE                       NODE  DESIRED STATE  CURRENT STATE           ERROR  PORTS
rchu2uykeuuj  couchbase_db.1  arungupta/couchbase:latest  moby  Running        Running 52 seconds ago


In this case, a single container is running as part of the service. The node is listed as moby , which is the default name of Docker Engine running using Docker for Mac.

The service can now be scaled with:

docker service scale couchbase_db=2


The list of containers can then be seen again as:

ID            NAME            IMAGE                       NODE  DESIRED STATE  CURRENT STATE           ERROR  PORTS
rchu2uykeuuj  couchbase_db.1  arungupta/couchbase:latest  moby  Running        Running 3 minutes ago          
kjy7l14weao8  couchbase_db.2  arungupta/couchbase:latest  moby  Running        Running 23 seconds ago


Note that the containers are given the name using the format <service-name>_n. Both the containers are running on the same host.

Also note that the two containers are independent Couchbase nodes and are not configured in a cluster yet. This has already been explained in the article Couchbase Cluster using Docker and a refresh of the steps is coming soon.

A service will typically have multiple containers running spread across multiple hosts. Docker 1.13 introduces a new command, docker service logs <service-name>, to stream the log of services across all the containers on all hosts to your console. In our case, this can be seen using the command docker service logs couchbase_db and looks like:

couchbase_db.1.rchu2uykeuuj@moby    | ++ set -m
couchbase_db.1.rchu2uykeuuj@moby    | ++ sleep 15
couchbase_db.1.rchu2uykeuuj@moby    | ++ /entrypoint.sh couchbase-server
couchbase_db.2.kjy7l14weao8@moby    | ++ set -m
couchbase_db.2.kjy7l14weao8@moby    | ++ sleep 15
couchbase_db.1.rchu2uykeuuj@moby    | Starting Couchbase Server -- Web UI available at http://:8091 and logs available in /opt/couchbase/var/lib/couchbase/logs
couchbase_db.1.rchu2uykeuuj@moby    | ++ curl -v -X POST http://127.0.0.1:8091/pools/default -d memoryQuota=300 -d indexMemoryQuota=300
couchbase_db.2.kjy7l14weao8@moby    | ++ /entrypoint.sh couchbase-server
couchbase_db.2.kjy7l14weao8@moby    | Starting Couchbase Server -- Web UI available at http://:8091 and logs available in /opt/couchbase/var/lib/couchbase/logs

. . .

couchbase_db.1.rchu2uykeuuj@moby    | ++ '[' '' = WORKER ']'
couchbase_db.2.kjy7l14weao8@moby    | Content-Type: application/json
couchbase_db.1.rchu2uykeuuj@moby    | ++ fg 1
couchbase_db.2.kjy7l14weao8@moby    | Content-Length: 152
couchbase_db.1.rchu2uykeuuj@moby    | /entrypoint.sh couchbase-server
couchbase_db.2.kjy7l14weao8@moby    | Cache-Control: no-cache
couchbase_db.2.kjy7l14weao8@moby    | 
couchbase_db.2.kjy7l14weao8@moby    | ++ echo 'Type: '
couchbase_db.2.kjy7l14weao8@moby    | ++ '[' '' = WORKER ']'
couchbase_db.2.kjy7l14weao8@moby    | ++ fg 1
couchbase_db.2.kjy7l14weao8@moby    | {"storageMode":"memory_optimized","indexerThreads":0,"memorySnapshotInterval":200,"stableSnapshotInterval":5000,"maxRollbackPoints":5,"logLevel":"info"}Type: 
couchbase_db.2.kjy7l14weao8@moby    | /entrypoint.sh couchbase-server


The preamble of the log statement uses the format <container-name>.<container-id>@<host>. And then, tye actual log message from your container shows up.

At first glance, attaching the container id may seem redundant. But Docker services are self-healing. This means that if a container dies, then the Docker Engine will start another container to ensure the specified number of replicas at a given time. This new container will have a new id. And thus, it allows us to attach the log message from the right container.

So a quick comparison of commands:


 DOCKER COMPOSE V2  DOCKER COMPOSE V3
 Start services docker-compose up -d docker stack deploy --compose-file=docker-compose.yml <stack-name> 
 Scale service docker-compose scale <service>=<replicas> docker service scale <service>=<replicas>
 Shutdown docker-compose down docker stack rm <stack-name>
 Multi-host No Yes


Docker (software)

Published at DZone with permission of Arun Gupta, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Docker Model Runner: Streamlining AI Deployment for Developers
  • A Guide to Container Runtimes
  • Gemma 3: Unlocking GenAI Potential Using Docker Model Runner
  • Docker vs Kubernetes: Which to Use and When?

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!