Rethinking Stream Processing With Apache Kafka, Kafka Streams, and KSQL

I presented at JavaLand 2018 in Brühl recently. It was a great developer conference with over 1,800 attendees. The location is also awesome! It even had a theme park: Phantasialand. My talk: was titled New Era of Stream Processing With Apache Kafka's Streams API and KSQL. I just want to share the slide deck...

Abstract

Stream processing is a concept used to act on real-time streaming data. This session shows and demos how teams in different industries leverage the innovative Streams API from Apache Kafka to build and deploy mission-critical streaming real-time applications and microservices.

The session discusses important streaming concepts like local and distributed state management, exactly-once semantics, embedding streaming into any application, deployment to any infrastructure. Afterward, the session explains key advantages of Kafka’s Streams API like distributed processing and fault-tolerance with fast failover, no-downtime rolling deployments, and the ability to reprocess events so you can recalculate output when your code changes.

A demo shows how to combine any custom code with your streams application — by an example using an analytic model built with any machine learning framework like Apache Spark ML or TensorFlow.

The end of the session introduces KSQL: the open-source Streaming SQL Engine for Apache Kafka. You can write “simple” SQL streaming queries with the scalability, throughput, and fail-over of Kafka Streams under the hood.

Slide Deck

Here we go: