Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Kafka's Cluster Replication Design

DZone's Guide to

Kafka's Cluster Replication Design

Free Resource

Build APIs from SQL and NoSQL or Salesforce data sources in seconds. Read the Creating REST APIs white paper, brought to you in partnership with CA Technologies.

Kafka, the distributed pub-sub messaging system designed by LinkedIn and used also at Twitter and Square, is some powerful tech.  This tutorial on intra-cluster replication, a relatively new feature, is a good tutorial on the subject written by Jun Rao, a staff engineer at LinkedIn.

With replication, Kafka clients will get the following benefits:
1. A producer can continue to publish messages during failure and it can choose between latency and durability, depending on the application.
2. A consumer continues to receive the correct messages in real time, even when there is failure.

--Jun Rao, Intra-cluster Replication in Apache Kafka

For more info on the design and the implementation, check out the Kafka Replication Wiki.

The Integration Zone is brought to you in partnership with CA Technologies.  Use CA Live API Creator to quickly create complete application backends, with secure APIs and robust application logic, in an easy to use interface.

Topics:

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}