It’s now two years since McKinsey Global Institute’s seminal report, “Big data: The next frontier for innovation, competition, and productivity”, was published. If there was a moment when the big data industry was born, it was with the publication of this report. No, McKinsey weren’t the first to talk about big data and they certainly weren’t the first to coin it (1), but they were the first to raise big data to the boardroom.
So, two years on, how does their report hold up? Well, it remains an easy and compelling read, but maybe now we’d focus on what it said rather more differently and more carefully. For example, when MGI wrote in the introduction to the report “$600 billion potential annual consumer surplus from using personal location data globally” most people focused on the wow factor of a $600 billion surplus and probably skipped over “potential.” Now, two years on, I’m guessing many people would focus on how many times MGI used the word “potential” in that much quoted introduction.
For many of us, big data is starting to look a lot like Data Warehousing Part Deux. Although no-one denies the potential of big data, we’re starting to get bogged down in some very familiar sounding problems, for example: data loading and ETL and some equally familiar sounding solutions; and exporting data to easy-to-use databases to provide big data BI. Please see my colleague Tobin Gilman’s “Why Aren’t BI Users Analyzing Hadoop Data?” blog for more on the subject of big data and BI.
Speaking of Hadoop, part of the problem is that Hadoop and big data have become synonymous in many people’s minds. This has refocused the big data debate away from business and onto technology. And, when the technology in question is Hadoop, it’s a polarizing debate between true believers and skeptics with both parties fuelled by multi-million dollar marketing budgets.
So, two years on, when I look at the big data market world, I’d echo the McKinsey keyword – the “potential” of big data is there and remains as strong as ever but, for now at least, the industry seems to be bogged down in big data technology issues not business solutions. Maybe McKinsey needs to re-enter the fray and focus people on what really matters, the business value of big data.
(1) That honor probably goes to John Mashey of Silicon Graphics according to “The Origins of Big Data An Etymological Detective Story” <link to:http://bits.blogs.nytimes.com/2013/02/01/the-origins-of-big-data-an-etymological-detective-story/>