Apache Pulsar: Distributed Pub-Sub Messaging System
Apache Pulsar is an open-source distributed pub-sub messaging system originally created at Yahoo and part of the Apache Software Foundation. Pulsar is a mult...
Join the DZone community and get the full member experience.
Join For FreeApache Pulsar is an open-source distributed pub-sub messaging system originally created at Yahoo! that is part of the Apache Software Foundation.
Pulsar is a multi-tenant, high-performance solution for server-to-server messaging.
Pulsar's key features include:
- Native support for multiple clusters in a Pulsar instance, with seamless geo-replication of messages across clusters
- Very low publish and end-to-end latency
- Seamless scalability out to over a million topics
- A simple client API with bindings for Java, Python, and C++
- Multiple subscription modes for topics (exclusive, shared, and failover)
- Guaranteed message delivery with persistent message storage provided by Apache BookKeeper
Architecture Overview
At the highest level, a Pulsar instance is composed of one or more Pulsar clusters. Clusters within an instance can replicate data amongst themselves.
The diagram below provides an illustration of a Pulsar cluster:
Pulsar Comparison With Apache Kafka
The table below lists the similarities and differences between Apache Pulsar and Apache Kafka:
| Kafka | Pulsar | |
|---|---|---|
| Concepts | Producer-topic-consumer group-consumer | Producer-topic-subscription-consumer |
| Consumption | More focused on streaming, exclusive messaging on partitions. No shared consumption. | Unified messaging model and API.
|
| Acking | Simple offset management
|
Unified messaging model and API.
|
| Retention | Messages are deleted based on retention. If a consumer doesn’t read messages before retention period, it will lose data. | Messages are only deleted after all subscriptions consumed them. No data loss even the consumers of a subscription are down for a long time. Messages are allowed to keep for a configured retention period time even after all subscriptions consume them. |
| TTL | No TTL support | Supports message TTL |
Conclusion
Apache Pulsar is an effort undergoing incubation at The Apache Software Foundation (ASF) sponsored by the Apache Incubator PMC. It seems that it will be a competitive alternative to Apache Kafka due to its unique features.
References
Published at DZone with permission of Furkan Kamaci, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Trending
-
Reactive Programming
-
Scaling Site Reliability Engineering (SRE) Teams the Right Way
-
How To Use Pandas and Matplotlib To Perform EDA In Python
-
Never Use Credentials in a CI/CD Pipeline Again
Comments