Abundant Data: The Currency for the Digital Transformation

DZone 's Guide to

Abundant Data: The Currency for the Digital Transformation

Embarking on your Big Data journey can be intimidating — but there are several ideas to consider that can help you get started.

· Big Data Zone ·
Free Resource

The idea that data is the currency for a digital transformation is a theme that came out loud and clear during the recent Gartner Data and Analytics Summit. This event, for the first time, brought together two popular conferences: Analytics and Master Data Management (MDM). The result was a very enjoyable time with tons of great conversations and thoughts being shared on our favorite topic: connected data.

Below are my top four takeaways from the event.

1. The Quest for Data Abundance

The opening keynote on Monday laid the foundation for this thought. In order for data to really take its place as the currency behind the digital transformation, it needs to become easy and really cheap. Traditional data technologies are real expensive and for anything other than structured data is not easy, either. It leads to scarcity. Consider that Business Intelligence (BI) still has not penetrated beyond 20 or 25% of the employees in any organization.

Some 2,000 years ago, the average worker had to work 50 hours to earn enough money to buy one hour of synthetic light from an oil lamp. Today, half a second of work will suffice. Similarly, for data, open-source Hadoop with its massive scale and ability to run on commodity hardware is well on its way to achieving the first part of the equation about drastic cost reductions.

The internet has been around a long time, but it was only when we got the ability to search that the internet became easy for all of us. Making Hadoop easy is lagging a little; but managed integrated distributions, cloud Hadoop, and perhaps more specifically Data Lake 3.0 are getting us to the tipping point.

2. Data in Motion to Capture Real-Time Insights

Another theme that came up is real-time data streaming: data in motion and capturing perishable insights at the edges of our landscape. Increasingly, the use cases organizations are pursuing demand support for some real-time automated decision logic pushed down to devices or sensors. Think about real-time promotions for retail or sensor data from manufacturing production lines detecting an anomaly. No one wants to find out we just produced 1,000 items that will not pass our quality assurance tests. Today’s connected data architecture need to provide the intersecting of the data-in-motion with our data-at-rest to really unlock actionable insights.

3. Achieving Meaningful Business Outcomes

One of the benefits of events like this is talking to many folks one-on-one. What is interesting is what they did not ask me as a Hadoop provider. That’s right; no questions on “what is Hadoop?” The questions are around business outcomes, what can we do with it, and how we deploy self-service BI with Hadoop. Hadoop has matured beyond the science fair project and is being employed in production to solve business critical problems.

4. Data Is About Storytelling

Data itself does not provide insights. Sam Esmail, writer and producer of the TV series Mr. Robot, suggested that it is when we combine data and analytics with our human emotions, experiences, and even gut feelings, then we create the insights that alter outcomes. The other inspiration for this point came from Tim Harford speaking of how frustration makes us creative — his point being that when we are most alert is when we are most creative, and we are most alert when we are out of our comfort zones. But he shared his talk via a fantastic story — and stories are memorable. If we want our data and insights to change behaviors and inspire, we have to tell stories with our data.

Closing Thoughts

If you have embarked on your data journey and want to measure where you are versus your peers, have a look at our Big Data Scorecard and get your own score.

big data ,data lake ,digital transformation

Published at DZone with permission of Piet Loubser , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}