Akka, Spark, or Kafka? Selecting the Right Streaming Engine [Video]
A look at the criteria to consider when selecting, plus the context and background to make good decisions when it comes to adopting, streaming frameworks.
Join the DZone community and get the full member experience.Join For Free
Many engineers we talk to on a daily basis come to us with the same issue: that the batch-oriented architecture of Big Data–where data is captured in large, scalable stores, then processed later–is simply too slow. If you’re looking for a competitive advantage like our clients PayPal, HPE, Starbucks, and Capital One, then you’ve got to embrace streaming and “Fast Data” architectures, where data is processed as it arrives.
But for most people we’ve talked to, there is rarely a “one size fits all” technology that can handle all streaming use cases. With so many stream processing tools, which ones should you choose? There are several considerations when making the right selection for the specific needs of your application, such as:
- Latency: How low is necessary? What types of tasks are you processing?
- Volume: How high is required? Is Complex Event Processing involved?
- Integration with other tools: Which ones and how? We know that nothing lives in isolation in these systems.
- Data processing: What kinds? In bulk? As individual events?
In this talk by Dean Wampler, PhD., VP of Fast Data Engineering at Lightbend, we’ll look at the criteria you need to consider when selecting technologies, plus the context and background to make good decisions when it comes to adopting streaming frameworks.
Using our Fast Data Platform as an example, which supports a host of Reactive and streaming technologies like Akka Streams, Kafka Streams, Apache Flink, Apache Spark, Mesosphere DC/OS, and our own Reactive Platform, we’ll look at how to serve particular needs and use cases in both Fast Data and microservices architectures.
Watch The Full Video (40 Min)
Published at DZone with permission of Oliver White, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.