DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Related

  • Manage Microservices With Docker Compose
  • [CSF] Using Metrics In Spring Boot Services With Prometheus, Graphana, Instana, and Google cAdvisor
  • Exploring the Horizon of Microservices With KubeMQ's New Control Center
  • Fast Deployments of Microservices Using Ansible and Kubernetes

Trending

  • The Smart Way to Talk to Your Database: Why Hybrid API + NL2SQL Wins
  • GitHub Copilot's New AI Coding Agent Saves Developers Time – And Requires Their Oversight
  • MCP Servers: The Technical Debt That Is Coming
  • How To Build Resilient Microservices Using Circuit Breakers and Retries: A Developer’s Guide To Surviving
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Deployment
  4. Monitoring Docker Swarm

Monitoring Docker Swarm

Learn how you can monitor your Docker Swarm environment with Prometheus and Grafana.

By 
Ralph Soika user avatar
Ralph Soika
·
Updated Feb. 08, 19 · Tutorial
Likes (10)
Comment
Save
Tweet
Share
54.3K Views

Join the DZone community and get the full member experience.

Join For Free

in one of my last blog posts, i explained how you can set up a lightweight docker swarm environment . the concept, which is already an open-infrastructure project on github , enables you to run your business applications and microservices in a self-hosted platform.

today, i want to explain how you can monitor your docker swarm environment. although docker swarm greatly simplifies the operation of business applications, monitoring is always a good idea. the following short tutorial shows how you can use prometheus and grafana to simplify monitoring.

prometheus is a monitoring solution to collect metrics from several targets. grafana is an open-analytics and monitoring platform that helps visualize data collected by prometheus.

both services can be easily integrated into docker swarm. there is no need to install extra software on your server nodes. you can simply start with a docker-compose.yml file to define your monitoring stack and a prometeus.yml file to define the scrape configuration. you can find the full concept explained here on github in the imixs-cloud project .

after you have defined your monitoring stack by a docker-compose.yml file, you can start the monitoring with one simple command:

$ docker-compose up  


the example files can be downloaded from here . since the docker-compose file is a little longer, i would like to briefly explain the important points now.

the networks

at the beginning of the docker-compose file, the section ‘networks’ defines two networks:

...
networks:
   frontend:
     external:
       name: imixs-proxy-net 
 backend: 
...


the external network ‘imixs-proxy-net’ is part of the core concept of the imixs-cloud project. if it is not yet available in your swarm, create it with docker:

$ docker network create --driver=overlay imixs-proxy-net


this network called ‘frontend’ is to access the prometheus and grafana services form your private network or the internet. the second network ‘backend’ is used only internally by the monitoring stack. in docker, it is always a good idea to hide as many services from external access as possible. so, this is the reason for this second network in the stack definition.

the prometheus service

the first service in the docker-compose.yml file is the prometheus service

...
prometheus:
     image: prom/prometheus
     volumes:
       - $pwd/management/monitoring/prometheus.yml:/etc/prometheus/prometheus.yml
     ports:
      - 9090:9090
     deploy:
       placement:
         constraints:
           - node.role == manager
     networks:
       - frontend
       - backend
.....


this service imports the prometheus.yml file from the host directory management/monitoring and exposes the api on port 9090, which is publicly accessible from our ‘frontend’ network. the ‘backend’ network is for later services.

the node-exporter

the next service is the ‘node-exporter.’ the node-exporter is a docker image provided by prometheus to expose metrics like disk, memory, and network from a docker host. besides some special volumes and command definitions, this service is always placed on a specific node. in the following example, the service exports hardware metrics from the node ‘manager-001’:

....
  manager-001:       
    image: prom/node-exporter       
    volumes:       
      - /proc:/host/proc:ro       
      - /sys:/host/sys:ro       
      - /:/rootfs:ro       
    command:       
      - '--path.procfs=/host/proc'       
      - '--path.sysfs=/host/sys'       
      - '--path.rootfs=/host'       
      - '--collector.filesystem.ignored-mount-points="^(/rootfs|/host|)/(sys|proc|dev|host|etc)($$|/)"'       
      - '--collector.filesystem.ignored-fs-types="^(sys|proc|auto|cgroup|devpts|ns|au|fuse\.lxc|mqueue)(fs|)$$"'       
    deploy:       
      placement:       
        constraints:       
         # hostname of the manager node!       
         - node.hostname == manager-001       
    networks:       
      - backend
...


you can replace the hostname with the corresponding hostname from your environment. the node-exporter is only connected to the ‘backend’ network. this means it is visible to the prometheus service but not accessible from outside.

you can add a separate node-exporter definition in your docker-compose.yml file for each docker node, which is part of your docker-swarm.

the cadvisor

the cadvisor is the second metrics collector. i provide metrics about docker itself. the service is maintained by google.

...
docker-manager-001:
     image: google/cadvisor:latest
     volumes:
     - /:/rootfs:ro
     - /var/run:/var/run:rw
     - /sys:/sys:ro
     - /var/lib/docker/:/var/lib/docker:ro
     deploy:
       placement:
         constraints:
          # hostname of the first manager node!
           - node.hostname == manager-001
     networks:
      - backend   
..


again, this service can be defined for each docker node within your docker-swarm network. take care of the service names here, because these are needed later in the prometheus.yml file.

grafana

the last service needed for our monitoring is grafana. this service is quite simple in its definition:

...
grafana:
     image: grafana/grafana
     volumes: 
       - grafana-data:/var/lib/grafana
     deploy:
       placement:
         constraints:
           - node.role == manager
     ports:
      - 3000:3000
     networks:
       - frontend
       - backend  
..


the service collects the data from prometheus and provides the graphical dashboard. the service is accessible via port 3000. in the example, i place the service here on the manager node from my docker swarm. but you can run the service on any other node within your swarm network.

the prometeus.yml file

now, as your docker-compose.yml file defines all services needed to monitor, you can set up your prometheus.yml file. this file tells prometheus where to collect the metric data. as defined by our docker-compose.yml file, we have two targets — the node-exporter and the cadvisor:

# my global config       

global:       
  scrape_interval:     15s # set the scrape interval to every 15 seconds. default is every 1 minute.       
  evaluation_interval: 15s # evaluate rules every 15 seconds. the default is every 1 minute.                       
  # scrape_timeout is set to the global default (10s).       
  # attach these labels to any time series or alerts when communicating with       
  # external systems (federation, remote storage, alertmanager).       
  external_labels:       
      monitor: 'codelab-monitor'       
# load rules once and periodically evaluate them according to the global 'evaluation_interval'.       
rule_files:       
  # - "first.rules"       
  # - "second.rules"       
# a scrape configuration containing exactly one endpoint to scrape:       
# here it's prometheus itself.       
scrape_configs:       
  - job_name: 'node-exporter'       
    static_configs:       
      # the targets listed here must match the service names from the docker-compose file       
      - targets: ['manager-001:9100','worker-001:9100']       

  - job_name: 'cadvisor'       
    static_configs:                       
      # the targets listed here must match the service names from the docker-compose file       
      - targets: ['docker-manager-001:8080','docker-worker-001:8080']


the important part here is the ‘targets’ section in the job descriptions for the ‘node-exporter’ and the ‘cadvisor.’ you need to add all the corresponding service names, which you have defined in your docker-compose.yml file here. this is because you need a separate node-exporter and cadvisor running on each node. in my example, i define two nodes for each service — the manager-001 node and the worker-001 node, which are part of my docker swarm.

…run your monitoring

and finally, you can now start your monitoring:

$ docker-compose up


that’s it. you can access prometheus via:

http://your-host:9090


first, check if prometheus is running and detecting all your metric providers by selecting the menu “status -> targets”:

if all endpoints are up, you can switch to the grafana ui:

http://your-host:3000


to access the metrics collected by prometheus, you just need to create a prometheus datasource. here, you can enter the internal prometheus url

 http://prometheus:9090 


which is the connect url from within the docker-swarm backend network:

next, you can import the dashboard from the imixs-cloud project:

enter the official dashboard id 9722

and select your datasource ‘prometheus,’ which you have created before:

that's it, you can now see how your docker-swarm is working:

conclusion

monitoring docker swarm is easy with the existing services prometheus and grafana. you can start your own monitoring stack with docker-compose and only one single configuration file. you will find a complete description about a lightweight docker swarm environment on github – join the imixs-cloud project !

Docker (software) microservice Network Metric (unit)

Published at DZone with permission of Ralph Soika. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Manage Microservices With Docker Compose
  • [CSF] Using Metrics In Spring Boot Services With Prometheus, Graphana, Instana, and Google cAdvisor
  • Exploring the Horizon of Microservices With KubeMQ's New Control Center
  • Fast Deployments of Microservices Using Ansible and Kubernetes

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!