Data in Motion Shaking Up Big Data Stack

DZone 's Guide to

Data in Motion Shaking Up Big Data Stack

Over 2,400 developers say fast data development trends are driving language and framework choices, app response times, and adoption of real-time streaming technologies.

· Big Data Zone ·
Free Resource

Thanks to Mark Brewer, CEO of Lightbend for sharing the results of a new survey of 2,457 global developers. The full report explores the big data market’s rapid transformation from data at rest to data in motion. The survey shows that 83% of fast data systems are not running on Apache Hadoop and explores the new technologies and patterns for data in motion.

Key findings include:

  • Fast data value is clear: senior management buys in: Unlike its big data counterpart, fast data appears to be more intuitive from a business value perspective. Sixty percent of senior management can effectively link strategic value to projects when data is in motion. Management in some industries, however, is getting it faster than others.

  • Batch vs. streaming: where speed really matters: The move to real-time data is accelerating. Developers say ninety percent of their data processing workloads include a real-time component. The need for speed increases as use cases climb the maturity curve. Rather than batch versus streaming, enterprises will need batch and streaming to succeed with fast data.

  • Technology shifts: fast data shakes up traditional stack: Developers are in the driver’s seat with regards to tech selection. 55% say they are choosing new frameworks and languages based on fast data requirements. But where the new ecosystem of streaming engines is concerned, developers and architects say they need guidance to choose the right tools.  

Access the full report to see:

  • Breakdown of senior management buy-in of fast data by industry

  • Percentage of workloads split across batch and real-time processing

  • Correlations between data speed requirements and use case (machine learning, IoT, etc.)

  • Adoption figures for specific fast data technologies/frameworks

  • Top challenges cited by fast data early adopters

batch, big data, fast data, streaming

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}