Over a million developers have joined DZone.

Is the Big Data Shakeout Coming in 2013?

· Big Data Zone

Learn how you can maximize big data in the cloud with Apache Hadoop. Download this eBook now. Brought to you in partnership with Hortonworks.

Is the inevitable Big Data shakeout coming?  If you are an enterprise customer, how do you prepare for this? What strategies do you adopt to take advantage of the situation?

The Big Data ecosystem is exploding with exciting start-ups, new divisions and new initiatives from established vendors.  Everyone wants to be the vendor/platform of choice in assisting firms deal with the data deluge (Data growth curve: Terabytes -> Petabytes -> Exabytes -> Zettabytes -> Yottabytes -> Brontobytes -> Geopbytes), translate data to information to insight, etc.

In both U.S and Europe, several billion dollars of venture money has been invested in the past three years alone in over 300+ firms.  In the MongoDB space alone – a small market of less than 100M total revenue right now, over $2 Billion is said to have been invested in the past few years.

An interesting sampling of start-up vendors and the amount of venture capital each has raised, including lead investors is listed at Wikibon here.  This list includes only Big Data pure-plays delivering products and/or services in one of the following markets: Hadoop, NoSQL, Next Generation (MPP) Data Warehousing, predictive analytics and/or advanced data visualization.

A narrow slice of the Big Data Market but illustrates the vibrant big data startup activity taking place. This list doesn’t include all the Social Intelligence and Analytics firms that are living off Facebook or Twitter data that represent a different slice of Big Data.  Several hundred firms easily in that segment alone.

The temporal clustering of major innovations under the banner of big data is definitely one of the catalysts for the next wave of innovation and economic growth.  We have seen this pattern before where advances in technology have combined to bring about a series of coordinated technological transformations that are correlated with waves of investment and business efficiency. (Joseph Schumpeter’s studied these business cycle patterns in the 1930s. These were later labeled creative destruction in innovation management circles.)

Most recently we saw this innovation pattern in the late 1990s around e-commerce.  Thousands of new companies were created, bought, and merged during the 1997 – 2000. At the end of the cycle we saw a massive creative destruction period with a washout that lasted from late 2000 to end of 2003.

My hypothesis is that in 2013 we are going to see a similar shakeout pattern around Big Data and Social Intelligence/Analytics.  The evidence… we have too many startup companies chasing customers.  Most of the projects are low-cost ($100K or less) or free pilots.  Enterprise customers who are innovative are piloting technologies to understand the business value but are having a hard time moving these into production deployments.

The shakeout will start slowly but will pickup pace towards the end of this year.  The catalyst for creative destruction is always the same – lack of next round of funding, lack of enterprise customers,  declining valuations that prompt investors to pull back, and finally big established firms like Oracle, SAP and others moving to protect their turf by creating fear, uncertainty and doubt.

So what does this mean if you are a Big Data startup firm?  It means managing your funding aggressively and making sure it lasts till you get paying customers.  A simply analogy – a car with gas in the tank will win always against another car that is running out of gas.

What does this mean if you are an enterprise customer?  Create a roadmap and continuously learn. If you are in the experimental mode, it’s ok to do several pilots. But make sure you are learning something and bringing this knowledge back into the organization.  I see a lot of companies that are doing interesting pilots but have no plans to assimilate, scale or leverage the insights.  So lot of effort is wasted.

Birth and death of firms is a natural phenon in entrepreneurship.  The survivors in Big Data will be those that are actively planning for the impending shakeout and acquiring assets – customers, technology, IP, patents — from the weaker players.

Big Data Start-up Ecosystem

Innovation in Big Data – Hadoop, NoSQL, Next Generation (MPP) Data Warehousing, predictive analytics and/or advanced data visualization… How many of these will survive?

Consolidation via mergers/acquisitions has already started.  Vertica (HP),  Kitenga (Dell), Salesforce.com (Buddymedia, Radian6), Oracle (Vitrue), EMC (Greenplum), IBM (too many to name).

If you know of other firms that should be added to this list…

Big Data Start-up Funding by Vendor (adapted from Wikibon)

VendorFoundedFunding (in $US mil.)# of Institutional RoundsInvestors
Palantir2004$3017SAC Capital, The Founders Fund, Glynn Capital, In-Q-Tel, Reed Elsevier Ventures, Ulu Ventures, Youniversity Ventures and Jeremy Stoppelman
Cloudera2008$1465Accel Partners, Greylock Partners, Ignition Partners, In-Q-Tel and Meritech Capital Partners
Mu Sigma2004$1332General Atlantic and Sequoia Capital
Opera Solutions2004$841Silver Lake Sumeru, Accel-KKR, Invus Financial Advisors, JGE Capital and Tola Capital
10gen2008$816Intel Capital, Red Hat, New Enterprise Associates, Sequoia Capital, Flybridge Capital and Union Square Ventures
Guavus2006$783Investor Growth Capital, QuestMark Partners, Intel Capital, Artiman Ventures and Sofinnova Ventures
ParAccel2005$735Amazon, Menlo Ventures, Mohr Davidow Ventures, Bay Partners, Walden International, Tao Venture Capital Partners and Silicon Valley Bank
Talend2005$61.65Silver Lake Partners, Balderton Capital, Galileo Partners and IDInvest Partners
GoodData2007$53.53Andreesen Horowitz, General Catalyst, O’Reilly AlphaTech Ventures, Windcrest Partners, Tenaya Capital and Next World Capital
Splunk2003$403Ignition Partners, August Capital, JK&B and Sevin Rosen Funds
DataStax2010$38.73Meritech Capital, Lightspeed Venture Partners, Sequoia Capital and Crosslink Capital
1010data2000$351Norwest Venture Partners
Couchbase2009$313Ignition Partners, Accel Partners, Mayfield Fund, and North Bridge Venture Partners
MapR2009$292Redpoint Ventures, Lightspeed Venture Partners and NEA
Tidemark2011$282Andreesen Horowitz, Redpoint Ventures and Greylock Partners
Factual2007$271Andreesen Horowitz and Index Ventures
Platfora2011$25.72Battery Ventures, Andreessen Horowitz, Sutter Hill Ventures and In-Q-Tel
MetaMarkets2010$232Khosla Ventures, IA Ventures, AOL Ventures, Neu Venture Capital, Joshua Stylman, Village Ventures and True Ventures
Hopper2007$223Atlas Venture, OMERS Ventures and Brightspark Ventures
Lattice Engines2006$21.62Battery Ventures and New Enterprise Associates
SumoLogic2010$20.52Sutter Hill Ventures, Greylock Partners, Shlomo Kramer
Hortonworks2011$201Benchmark Capital, Yahoo and Index Ventures
RainStor2004$19.23Storm Ventures, Doughty Hanson Technology Ventures, Informatica, Rogers Venture Partners and The Dow Company
DataXu2009$18.82Menlo Ventures, Atlas Venture and Flybridge Capital Partners
Datameer2009$17.83Kleiner Perkins Caufield & Byers and Redpoint Ventures
Revolution Analytics2007$17.62North Bridge Venture Partners and Intel Capital
Hadapt2010$16.21Atlas Venture, Norwest Venture Partners and Bessemer Venture Partners
Lucid Imagination2007$162Shasta Ventures, Granite Ventures, In-Q-Tel and Walden International
Continuity2011$12.52Andreessen-Horowitz, Ignition Ventures, Battery Ventures, Data Collective and Amplify Partners
Connotate2000$12.32Castile Ventures, Prism VentureWorks and .406 Ventures
ClearStory Data2012$12.251Kleiner Perkins Caufield & Byers, Andreessen Horowitz, Google Ventures and Khosla Ventures
Karmasphere2005$112Presidio Ventures, Hummer Winblad and US Venture Partners
Loggly2009$101True Ventures, Trinity Ventures, Matrix Partners
Oragami Logic2012$81Accel Partners
Alpine Data Labs2010$7.51Sierra Ventures, Mission Ventures, Sumitomo Corporation Equity Asia and Stanford University
SpaceCurve2009$7.51Triage Ventures, Reed Elsevier Ventures and Divergent Ventures
ParStream2008$5.61Khosla Ventures, Baker Capital, Crunch Fund, Data Collective and Tola Capital
SpaceCurve2011$5.22Reed Elsevier, Divergent Ventures, and Triage Ventures
MemSQL2011$51First Round Capital, SV Angel, Y Combinator, IA Ventures and Ashton Kutcher
WibiData2010$51New Enterprise Associates, SV Angel, Mike Olson and Eric Schmidt
InsightSquared2010$4.51Atlas Venture, Bessemer Venture Partners, NextView Ventures and salesforce.com
Chartio2010$4.41Avalon Ventures, Bullpen Capital, Y Combinator, Crosslink Capital and Jeff Hammerbacher
Trifacta2012$4.31Accel Partners, X/Seed Capital, Data Collective LLC, Dave Goldberg, Venky Harinarayan and Anand Rajaraman
Digital Reasoning2000$4.22In-Q-Tel and Silver Lake Sumeru
SiSense2008$41Opus Capital, Genesis Partners and Eli Farkash
Calpont2000$3.271Austin Ventures and GF Private Equity
StackIQ2006$31Anthem Venture Partners and Avalon Ventures
Zettaset2009$31Draper Fisher Jurvetson and Epic Ventures
GridGain2005$2.51RTP Ventures
NGDATA2011$2.51ING, Sniper Investments, Plug and Play Ventures
Sqrrl2012$21Atlas Venture
Feedzai2008$21Espirito Santo Ventures and Novabase Capital
Nodeable2011$21True Ventures and Matrix Partners
RelateIQ2012$1.251Accel Partners, Morgenthaler and SV Angel
Zoomdata2012$1.11Hemang Gadhia
AppEnsure2011$11Citrix Accelerator, TiE, Ignition Partners
DataHero2012$11Neu Venture Capital, The Foundry Group, David G. Cohen and Tasso Argyros
Drawn to Scale2009$0.931RTP Ventures, IA Ventures, and SK Ventures

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks


Published at DZone with permission of Ravi Kalakota, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}