DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Newsletter
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Related

  • What We Learned Migrating to a Pub/Sub Architecture: Real-World Case Studies from High-Traffic Systems
  • Data Pipeline Architectures: Lessons from Implementing Real-Time Analytics
  • Building Scalable AI-Driven Microservices With Kubernetes and Kafka
  • Big Data Realtime Data Pipeline Architecture

Trending

  • When One Docker Swarm Replica Became Unreachable Over an Overlay Network: A Production Case With Traefik
  • SEO Writing 101: A Simple Guide to Getting Found
  • Building Internal Developer Platforms as Products: A Practical Guide for IDP Architects
  • Best Plugins For JetBrains IDEs

Draft This article hasn't been published, so readers can't see it.

In moderation This article is waiting for review, so readers can't see it yet.

Deleted This article has been deleted, so readers can't see it.

  1. DZone
  2. Data Engineering
  3. Big Data
  4. Architectures for Distributed, Hybrid, Edge, and Global Apache Kafka

Architectures for Distributed, Hybrid, Edge, and Global Apache Kafka

Learn about several scenarios that may require multi-cluster solutions and see real-world examples with their specific requirements and trade-offs.

By 
Kai Wähner user avatar
Kai Wähner
DZone Core CORE ·
Mar. 19, 20 · Analysis
Post on X (opens in a new tab) Share on LinkedIn (opens in a new tab)
20.6K Views

Join the DZone community Save articles, join the discussion, and download Refcards and Trend Reports with a free account.

Join for free Already a member? Log in

Multi-cluster and cross-data center deployments of Apache Kafka have become the norm rather than an exception. Learn about several scenarios that may require multi-cluster solutions and see real-world examples with their specific requirements and trade-offs, including disaster recovery, aggregation for analytics, cloud migration, mission-critical stretched deployments and global Kafka.

kafka

Key Takeaways for Multi Data Center Kafka Architectures

  • In many scenarios, one Kafka cluster is not enough. Understand different architectures and alternatives for multi-cluster deployments.
  • Zero data loss and high availability are two key requirements. Understand how to realize this, including trade-offs.
  • Learn about features and limitations of Kafka for multi-cluster deployments- Global Kafka and mission-critical multi-cluster deployments with zero data loss and high availability became the normal, not an exception.
  • Learn about architectures like stretched cluster, hybrid integration and fully-managed serverless Kafka in the cloud (using Confluent Cloud), and tools like MirrorMaker 2, Confluent Replicator, Multi-Region Clusters (MRP), Global Kafka, and more.

Slide Deck

 

Architecture patterns for distributed, hybrid, edge and global Apache Kafka deployments from Kai Wähner.

Video Recording


kafka Architecture

Published at DZone with permission of Kai Wähner. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • What We Learned Migrating to a Pub/Sub Architecture: Real-World Case Studies from High-Traffic Systems
  • Data Pipeline Architectures: Lessons from Implementing Real-Time Analytics
  • Building Scalable AI-Driven Microservices With Kubernetes and Kafka
  • Big Data Realtime Data Pipeline Architecture

Partner Resources

×

Comments

Likes

Loading

The next likes didn't load. Please try again.

The likes didn't load

Please try again in a moment.

See who liked this

Log in or join DZone for free to see who liked this article.

Log in Join for free
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook