Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Data Models and Consumer Idioms Using Apache Kafka

DZone's Guide to

Data Models and Consumer Idioms Using Apache Kafka

· Integration Zone ·
Free Resource

The Future of Enterprise Integration: Learn how organizations are re-architecting their integration strategy with data-driven app integration for true digital transformation.

Apache Kafka is a powerful publish subscribe messaging service that delivers high-volume messages across ad hoc topics to subscribers with message durability for offline consumers. As with any technology, Kafka comes with certain concessions, in this case, concessions that influence consumer behavior in particular at scale. Likewise, achieving optimal throughput when writing to durable storage can be optimized by distilling data storage operations into commutative, idempotent sets of operations.

This talk will detail Urban Airship's experience using Kafka to process billions of messages per day. The talk will begin with an in-depth look at Kafka's core design concepts and how they influence nuisances of writing consumers, both positive and negative. Beyond consumers, this talk will detail how Urban Airship leverages the strengths of different storage engines (Cassandra, HBase and in-house solutions) across a common consumer infrastructure for disparate goals including near real-time message routing, analytics and system measurement.

Make your mark on the industry’s leading annual report. Fill out the State of API Integration 2019 Survey and receive $25 to the Cloud Elements store.

Topics:

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}