A Developer's Guide to Mastering Docker Networking Concepts

As developers, our first success with Docker is typically running a single container and accessing it through localhost. While this is a solid beginning, the true strength of containers emerges when you start running multi-container applications. This is also where most developers face challenges.

You have a web application in one container and a database in another. How do they communicate?

You might attempt to find the container's IP address with docker inspect, but that IP is temporary — it will change after the next restart. You may have noticed older tutorials using the --link flag, but that's outdated and not an effective solution.

The reality is that Docker has a strong networking system that is easy to use once you grasp one key concept. This article will clarify Docker networking, show you how to properly connect your containers, and explain how Docker Compose simplifies the process.

The Problem: The Default bridge Network

When you install Docker, it creates a default network called bridge. Every container you run without a --net flag connects to it. You can see it by running docker network ls. This default bridge lets your container access the internet for tasks like apt-get or npm install. It also allows you to map ports to your host machine using -p 8080:80. This has issues with service discovery. Containers running on this default bridge can technically talk to each other, but they must be using their internal IP addresses, such as 172.17.0.2. As we've seen, these IPs can be unreliable. There is no built-in way for your web container to locate your DB container by its name. This makes the default bridge network unsuitable for nearly any multi-container application.

The Solution: User-Defined Bridge Networks

This is the main idea you need to understand. Rather than using the default network, you should always set up your own custom bridge network for each application. When you create your own network, Docker offers an important feature at no cost: automatic DNS-based service discovery. In plain language, Docker assigns an internal DNS server to your custom network. Any container you connect to this network is automatically registered with the DNS server under its container name.

A Practical CLI Example

Step 1

Create the network that we are going to use for our application.

docker network create my-app-net

Step 2

Run the database. Next, we will be running a PostgreSQL database. We'll be attaching this to our new network. This is quite important, and we also have to give it a name.

docker run -d \
  --name my-database \
  --net my-app-net \
  -e POSTGRES_PASSWORD=mysecret \
  postgres

The database container app is running and is registered with the "my-app-net" network's DSN server with the hostname "my-database."

Step 3

Run the application. Now we will have to run our web application (In this example, it will be a simple Node.js app). We will be attaching this to the same network; we will also have to tell it how to find the database.

docker run -d \
  --name my-api \
  --net my-app-net \
  -e DB_HOST=my-database \
  -p 3000:3000 \
  my-api-image

docker network create my-app-net

Look at the environment variable: DB_HOST=my-database. That's it. In your application's code, you can now use my-database as the hostname for your database connection. Docker's internal DNS will automatically change that name to the current private IP of the my-database container. It doesn't matter if you restart the containers. It doesn't matter what IP addresses they receive. The name my-database will always point to the correct container.

The "Easy Way": Using Docker Compose

While the CLI commands help us understand how things work, they can be unwieldy for everyday development. This is the problem Docker Compose aims to fix. Docker Compose handles all this network management for you, automatically. Let's define the same two-container setup in a single docker-compose.yml file:

version: '3.9' // Latest version is 3.9

services:
  # Our API service layer
  my-api:
    build: .
    image: my-api-image
    ports:
      - "5000:5000"
    environment:
	  # Connects with service name
      - DB_HOST=my-database 
    depends_on:
      - my-database

  # Our database service
  my-database:
    image: postgres
    environment:
      - POSTGRES_PASSWORD=mysecret
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:

Here is what happens when you run docker-compose behind the scenes. Docker Compose automatically spins up a new default network for the project under consideration. It then starts the my-database-service, which connects it to this network and then makes it accessible with the hostname my-database.

Furthermore, it starts the my-api service, connects it to the same network, and then adds the required environmental variables. Then the my-api container can access the database by connecting to the service name my-database. This is the standard workflow for local Docker development.

Network Driver Overview

You will use user-defined bridge networks 95% of the time. However, it's helpful to understand the purpose of the other drivers.

1. Host

• What it is: Disables network isolation. The container shares the network stack of your host machine.
• Use case: When you need high network performance and are okay with sacrificing security and port-mapping. An example is a network monitoring tool that must capture all traffic on the host. If your container binds to port 80, it connects to your host's port 80.

2. Overlay

• What it is: A distributed network that connects containers across multiple Docker hosts. 
• Use case: This is the networking model for Docker Swarm, which is Docker's native orchestrator. It creates a "flat" virtual network that covers all nodes in your cluster. This allows a web container on Host A to communicate with a database container on Host B as if they were on the same machine.

3. None

• What it is: Complete network isolation. The container has no network interface, only a loopback device.
• Use case: For tasks that require high security or for simple batch jobs that do not need to communicate with the outside world at all.

Key Takeaways for Our Software Developers

1. Stop using the default bridge network for multi-container applications. Always create user-defined bridge networks (docker network create...) when using the CLI. This enables automatic, DNS-based service discovery.

2. Use Docker Compose for local development. It automates the creation of custom networks and makes service discovery easy with just the service name. Container names are your new hostnames. In a networked Docker environment, you connect to my-database, not localhost or 172.17.0.2.