Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Kafka's Cluster Replication Design

DZone 's Guide to

Kafka's Cluster Replication Design

· Integration Zone ·
Free Resource

Kafka, the distributed pub-sub messaging system designed by LinkedIn and used also at Twitter and Square, is some powerful tech.  This tutorial on intra-cluster replication, a relatively new feature, is a good tutorial on the subject written by Jun Rao, a staff engineer at LinkedIn.

With replication, Kafka clients will get the following benefits:
1. A producer can continue to publish messages during failure and it can choose between latency and durability, depending on the application.
2. A consumer continues to receive the correct messages in real time, even when there is failure.

--Jun Rao, Intra-cluster Replication in Apache Kafka

For more info on the design and the implementation, check out the Kafka Replication Wiki.

Topics:

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}