Real-Time Analytics With Kafka and MemSQL
With the demand for real-time answers coming from an ever-increasing scale, how do you deliver? Read on to find out how to accomplish this with Apache Kafka and memSQL.
Join the DZone community and get the full member experience.Join For Free
Connected devices, IoT, and on-demand user expectations push enterprises to deliver instant answers at scale. Applications that anticipate customer needs and fulfill expectations for fast, personalized services win the attention of consumers. Perceptive companies have taken note of these trends and are turning to memory-optimized technologies like Apache Kafka and MemSQL to power real-time analytics.
Building real-time systems begins with capturing data at its source and using a high-throughput messaging system like Kafka. Taking advantage of a distributed architecture, Kafka is built to scale producers and consumers by simply adding servers to a given cluster. This effective use of memory, combined with commit log on disk, provides ideal performance for real-time pipelines and durability in the event of server failure. From there, data can be transformed and persisted to a database like MemSQL.
Fast, Performant Data Storage
MemSQL persists data from real-time streams coming from Kafka. By combining transactions and analytics in a memory-optimized system, data is rapidly ingested from Kafka, then persisted to MemSQL. Users can then build applications on top of MemSQL also supplies the application with the most recent data available.
We teamed up with the folks at Confluent, the creators of Apache Kafka, to share best practices for architecting real-time systems at our latest meetup. The video recording and slides from that session are now available below.
Meetup Video Recording: Real-Time Analytics with Confluent and MemSQL
Watch now to:
- See a live demo of our new showcase application for modeling predictive analytics for global supply chain management
- Learn how to architect systems for IoT streaming data ingestion and real-time analytics
- Learn how to combine Kafka, Spark, and MemSQL for monitoring and optimizing global supply chain processes with real-time analytics
If you would like to catch upcoming tech talks and live product demonstrations, join the MemSQL meetup group here.
Published at DZone with permission of Kevin White. See the original article here.
Opinions expressed by DZone contributors are their own.