Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Microservices With Netflix OSS, Apache Kafka, and Docker: Part 1, Install Apache Kafka in a Docker Container

DZone's Guide to

Microservices With Netflix OSS, Apache Kafka, and Docker: Part 1, Install Apache Kafka in a Docker Container

We start this tutorial series on building microservices with Netflix OSS and Apache Kafka by showing you how to install Kafka on Docker.

· Microservices Zone ·
Free Resource

Learn why microservices are breaking traditional APM tools that were built for monoliths.

This is Part 1 of a blog series on building microservices with Netflix OSS and Apache Kafka.

Apache Kafka is an open source platform for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, and super fast. It aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds.

Kafka is highly used in microservices and has great value in setting up Docker. In this blog, I will show you the steps to install Kafka on Docker.

System Requirements:

  • Server: PC or server with minimum 8 GB
  • OS: any OS which supports Docker installation. In this blog, I am going to show it on Windows 7.
  • Zookeeper and Kafka images from Docker Hub

Procedure

The first step is to create the bridge network to enable communication between Kafka and Zookeeper:

$ docker network create kafka-net

Then, download and run Zookeeper from Docker Hub:

$ docker run -d --name zookeeper --network kafka-net zookeeper:latest

Image title

After setting up the container for Zookeeper, next, pull the image of Apache Kafka from Docker Hub. For this purpose, we will use the ches/kafka repository using this command:

$ docker run -d --name kafka --network kafka-net --env ZOOKEEPER_IP=zookeeper ches/kafka

Image title

Once the Kafka container is set up, check if both containers are running using the command docker ps.

Now let's create a topic to validate if Kafka is running fine:

$ docker run --rm --network kafka-net ches/kafka \kafka-topics.sh --create --topic USER_CREATED_TOPIC --replication-factor 1 --partitions 1 --zookeeper zookeeper:2181

In a separate terminal, use the below command to start the producer console. Issue the below command to produce a message to the “USER_CREATED_TOPIC” topic (created in the above step):

$ docker run --rm --interactive --network kafka-net ches/kafka \kafka-console-producer.sh --topic USER_CREATED_TOPIC --broker-list kafka:9092

The terminal will wait for our input, so enter a message and proceed.

Image title

In another terminal, start the consumer console and see that the message is displayed.

$ docker run --rm --network kafka-net ches/kafka \kafka-console-consumer.sh --topic USER_CREATED_TOPIC --from-beginning --bootstrap-server kafka:9092

The output in the terminal should be similar to this:

Image title

We have seen how to install Apache Kafka in a Docker container and make it work. In the next post, we will create a microservice and use Apache Kafka to send emails.

Stay tuned for Part 2.

Happy Coding!

Record growth in microservices is disrupting the operational landscape. Read the Global Microservices Trends report to learn more.

Topics:
microservices ,apache kafka ,tutorial ,docker

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}