Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Kafka's Cluster Replication Design

DZone's Guide to

Kafka's Cluster Replication Design

· Integration Zone ·
Free Resource

The new Gartner Critical Capabilities report explains how APIs and microservices enable digital leaders to deliver better B2B, open banking and mobile projects.

Kafka, the distributed pub-sub messaging system designed by LinkedIn and used also at Twitter and Square, is some powerful tech.  This tutorial on intra-cluster replication, a relatively new feature, is a good tutorial on the subject written by Jun Rao, a staff engineer at LinkedIn.

With replication, Kafka clients will get the following benefits:
1. A producer can continue to publish messages during failure and it can choose between latency and durability, depending on the application.
2. A consumer continues to receive the correct messages in real time, even when there is failure.

--Jun Rao, Intra-cluster Replication in Apache Kafka

For more info on the design and the implementation, check out the Kafka Replication Wiki.

The new Gartner Critical Capabilities for Full Lifecycle API Management report shows how CA Technologies helps digital leaders with their B2B, open banking, and mobile initiatives. Get your copy from CA Technologies.

Topics:

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}