Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Kafka's Cluster Replication Design

DZone's Guide to

Kafka's Cluster Replication Design

· Integration Zone ·
Free Resource

SnapLogic is the leading self-service enterprise-grade integration platform. Download the 2018 GartnerMagic Quadrant for Enterprise iPaaS or play around on the platform, risk free, for 30 days.

Kafka, the distributed pub-sub messaging system designed by LinkedIn and used also at Twitter and Square, is some powerful tech.  This tutorial on intra-cluster replication, a relatively new feature, is a good tutorial on the subject written by Jun Rao, a staff engineer at LinkedIn.

With replication, Kafka clients will get the following benefits:
1. A producer can continue to publish messages during failure and it can choose between latency and durability, depending on the application.
2. A consumer continues to receive the correct messages in real time, even when there is failure.

--Jun Rao, Intra-cluster Replication in Apache Kafka

For more info on the design and the implementation, check out the Kafka Replication Wiki.

Download A Buyer's Guide to Application and Data Integration, your one-stop-shop for research, checklists, and explanations for an application and data integration solution.

Topics:

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}