Over a million developers have joined DZone.

Real-Time Analytics With Kafka and MemSQL

DZone's Guide to

Real-Time Analytics With Kafka and MemSQL

With the demand for real-time answers coming from an ever-increasing scale, how do you deliver? Read on to find out how to accomplish this with Apache Kafka and memSQL.

· Big Data Zone
Free Resource

Learn how you can maximize big data in the cloud with Apache Hadoop. Download this eBook now. Brought to you in partnership with Hortonworks.

Connected devices, IoT, and on-demand user expectations push enterprises to deliver instant answers at scale. Applications that anticipate customer needs and fulfill expectations for fast, personalized services win the attention of consumers. Perceptive companies have taken note of these trends and are turning to memory-optimized technologies like Apache Kafka and MemSQL to power real-time analytics.

High-Speed Ingest

Building real-time systems begins with capturing data at its source and using a high-throughput messaging system like Kafka. Taking advantage of a distributed architecture, Kafka is built to scale producers and consumers by simply adding servers to a given cluster. This effective use of memory, combined with commit log on disk, provides ideal performance for real-time pipelines and durability in the event of server failure. From there, data can be transformed and persisted to a database like MemSQL.

Fast, Performant Data Storage

MemSQL persists data from real-time streams coming from Kafka. By combining transactions and analytics in a memory-optimized system, data is rapidly ingested from Kafka, then persisted to MemSQL. Users can then build applications on top of MemSQL also supplies the application with the most recent data available.

We teamed up with the folks at Confluent, the creators of Apache Kafka, to share best practices for architecting real-time systems at our latest meetup. The video recording and slides from that session are now available below.

Meetup Video Recording: Real-Time Analytics with Confluent and MemSQL
Watch now to:

  • See a live demo of our new showcase application for modeling predictive analytics for global supply chain management
  • Learn how to architect systems for IoT streaming data ingestion and real-time analytics
  • Learn how to combine Kafka, Spark, and MemSQL for monitoring and optimizing global supply chain processes with real-time analytics



If you would like to catch upcoming tech talks and live product demonstrations, join the MemSQL meetup group here.

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

real-time analytics ,kafka ,apache ,analytics ,real-time ,apache kafka ,memsql

Published at DZone with permission of Kevin White. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}