The Future of Big Data
The Future of Big Data
More data, faster, in more formats, from more sources with faster analysis and real-time integration and decision making to solve problems before they occur.
Join the DZone community and get the full member experience.Join For Free
Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.
Here's who we talked to:
Uri Maoz, Head of U.S. Sales and Marketing, Anodot | Dave McCrory, CTO, Basho | Carl Tsukahara, CMO, Birst | Bob Vaillancourt, Vice President, CFB Strategies | Mikko Jarva, CTO Intelligent Data, Comptel | Sham Mustafa, Co-Founder and CEO, Correlation One | Andrew Brust, Senior Director Marketing Strategy, Datameer | Tarun Thakur, CEO/Co-Founder, Datos IO | Guy Yehiav, CEO, Profitect | Hjalmar Gislason, Vice President of Data, Qlik | Guy Levy-Yurista, Head of Product, Sisense | Girish Pancha, CEO, StreamSets | Ciaran Dynes, Vice Presidents of Products, Talend | Kim Hanmark, Director, Professional Services, TARGIT | Dennis Duckworth, Director of Product Marketing, VoltDB.
We asked these executives, "What’s the future for big data from your point of view - where do the greatest opportunities lie?"
Here's what they told us:
- The most disruptive force is big data in the cloud enabling real-time analysis. Splunk went from offline to real-time IT intelligence running on a classical scale-out database.
- Data is the oil of the 21st century. It will impact all of our lives and how we make choices – especially healthcare, policy, and municipal operations. The innovation gap is getting smaller than it used to be. Companies in China are starting to adopt some of the municipal practices we have here in the U.S. Public policy will be affected because of how rich the data is and how frequently we collect it. The nature of elections will change completely like the Obama data science team used data for campaign outreach and GOTV campaigns. Many of them are now working for the Clinton campaign. Campaigns of the future will have a data analytics team or they will lose. There are more tools available to quantify decisions. The highest value choice we make is how to choose elected officials. The impact is only going to go up given the rate at which we’re collecting data and we have faster tools to process changing every choice we make.
- Fast data is as important as big data for Visa, MasterCard, and New York Stock Exchange. How can we make the data more relevant to the users within a few seconds? Use data intelligently and provide value. Fast = relevant.
- More data in more formats needing faster analysis to facilitate real-time decision making.
- Move to real-time versus ad hoc exploratory analytics. Operationalizing analytics – getting control of all the data that’s available.
- Real-time integration streams. Data resides in Teradata, Hadoop comes along and there’s a need for analytics, a front-end cluster that will provide insights from the data and then send it back out to the end user. Opportunities to do things in real time. Currently only 8 to 15% of projects but this will increase dramatically. Metadata with security is the hottest issue right now – governances, oversight, and policies.
- More business focus on what is needed to achieve to get ROI. What’s the value proposition – where do we excel and where do we not do so well? How can big data be used to improve? We talk to clients who want to use ERP and CRM data but aren’t sure what they want to do or what they can do. Model vendors in one country to move to other countries. Get companies to think about what they need and what they’d like to have. There’s a lot of potential in the developer community to think out of the box and solve business problems. We need insight teams with business people, statisticians, insights, and developers collaborating to solve problems.
- Create interdependent ecosystems in the early stages of development. Big data needs to sit somewhere but B.I. is the way to deliver value from big data. You will see the acquisition of B.I. companies and open APIs to build out the ecosystems and integrate in a seamless manner. The number of data sources and structures will continue to grow fast. You will need to be able to scale and feed the data into B.I. systems for insights.
- More real-time decision making. Depends on the needs of the customer. More focus on the P&L and providing guidance.
- “Big data” just becomes “data.” Tools can deal with larger volumes and greater velocity. People are able to access and explore the data. IoT will drive an order of magnitude increase in the amount of data where it is not economical to store all of the data produced.We’ll have to decide on the fly what to analyze, store, and throw away. The opportunity and challenge is identifying what data is worth storing, highlighting, and throwing away.
- How to deal with more data at scale with regards to analysis and prediction. Ability to predict events based on past anomalies to predict future problems. Solve the problem before it happens. Maturity in the collection of data. Will need someone to deliver the insights.
- Continued exponential growth with the cost of data generation declining and IoT amplifying the amount of data. Combining data sources together to provide more valuable insights for the next five to 10 years.
- Some companies are already there using big data for real-time decision making. They have instant access to the data and visibility into the application’s performance.
- Markets outside the United States. For the most part, big data has been around in some way in the U.S. for decades, but that is still not the case in other countries around the world.
What do you see as the future of big data?
Opinions expressed by DZone contributors are their own.