Tractors can now talk. They send and receive signals to adjust how they operate and optimize how they perform. Location, weather and other factors are being processed in the moment (during the tractor’s operation) at the point on the planet (in the field) and in the timeframe (perhaps the only timeframe) that it matters. Remarkable.
If you haven’t heard, the Internet of Things is allowing anything and everything with the help of a a small amount of technology to have a voice in our digitized world. If you think about it, this is truly an amazing change in our lives. Smart scientists and engineers are moving very rapidly beyond the mental and physical constructs of the big data problem and putting their focus on enabling this change. It takes a different approach from any we’ve had in the past.
It isn’t just an industrial idea as with tractors or other machines — when the nearby Apple Store make you an offer when you come near (thanks to iBeacon), we’ve given a voice to a storefront separate from any human interaction. “Things” are now part of our world in digital ways like never before.
Is this big data? Sort of yes and sort of no. Much of the data infrastructure built to handle large volumes, high variety and high velocity data over the past few years has been focused on Hadoop and other batch-oriented ways of managing all of that information. The goal was to simply “deal” with the data problem that came from massive digitization of our world. It is now time to deal with data’s urgency and the connectedness of it all. That’s a whole different problem to solve.
The “fast data” era
Fast data is processing big data in real-time to gain instant awareness and instant action. Where big data’s big deal was about crunching data to spot patterns and trends, fast data is about digesting and using data anywhere and everywhere necessary. It is about taking real, intentional action in ways we’ve never been able to before. Just as with big data, there will be plenty of clumsy rebranding going on as technology vendors and consultants large and small attempt to say, “that’s what we do, too.” Don’t buy into it any more than you bought into the big data and cloud hype.
Here’s the difference to understand and where your risk lies: opportunities in this new world are very short-lived as more organizations are able to take advantage of fast data. The have’s and have nots will be defined by being able to see and act on data in those short timelines.
You’ll need to be skeptical and ask a few questions of what you’re hearing. Questions like:
- Does the technology in front of me move data in hyper fast timeframes — even world-class timeframes?
- Does it allow for analytics to be automatically and programmatically acted upon?
- Does the solution proposed scale to needs beyond today’s?
- Does it play well with many other technologies likely to rise in a new market?
Databases aren’t fast enough and data warehouses will fail in the world of fast data. Information will need to be accessible in timeframes only possible with in-memory technology, whether that’s in how information is gathered, processed, analyzed or acted upon. Companies will rise and fall for that reason, both on the tech selling and tech buying sides.
Welcome to the fast data era.