Data science is critical for success, but Continuum Analytics finds that just 49 percent have data science teams in place.
I had a great discussion with Michele Chambers, E.V.P. Anaconda Business Unit and CMO for Continuum Analytics about the research they commissioned. They found that 96 percent of data science and analytics decision-makers agree that data science is critical to the success of their business, yet 22 percent are failing to make full use of the data available. These findings are included in Continuum Analytics’ new eBook, Winning at Data Science: How Teamwork Leads to Victory, based on the company’s inaugural study that explores the state of Open Data Science in the enterprise.
The research, conducted by independent research firm Vanson Bourne, surveyed 200 data science and analytics decision makers at U.S. organizations of all sizes and industries to examine the state of open data science in the enterprise. Continuum Analytics also surveyed more than 500 data scientists to uncover similarities and disparities between the two groups. Topics ranged from the value of data science, challenges around adoption and how data science is being utilized in the enterprise.
Key takeaways and findings from the research are consistent with my recent qualitative research on big data with 22 executives from 20 companies:
- The benefits of data science in the enterprise are undisputed. 73 percent of respondents ranked it as one of the top three most valuable technologies they use. Conversely, findings show that a disparity exists between understanding the impact of data science and executing it in the enterprise––62 percent said data science is used at least on a weekly basis, but just 31 percent of that group are using it daily.
- When comparing the beliefs of executives/IT managers with data scientists, nearly all respondents from both groups agree on the critical impact of data science in the enterprise. However, a divide exists around where companies are in the data science lifecycle.. Just 24 percent of data scientists feel their companies have reached the “teen” stage––developed enough to hold its own with room to mature––as opposed to the 40 percent of executives who feel confident they have arrived at this stage of development.
- Despite the benefits offered by data science, 22 percent of enterprise respondents report that their teams are failing to use the data to its potential. What’s more, 14 percent use data science very minimally or not at all, due to three primary adoption barriers: executive teams that are satisfied with the status quo (38 percent), a struggle to calculate ROI (27 percent) and budgetary restrictions (24 percent).
While obstacles persist, an increasingly data-driven world calls for data science teams in the enterprise — it’s not a one-person job. Though 89 percent of organizations have at least one data scientist, less than half have data science teams. Findings revealed that 69 percent of respondents associate Open Data Science with collaboration, proving that teamwork is essential to exploit the power of the data, requiring a combination of skills best tackled by a strong team.
According to Michele, a lot of data science is still ad hoc with people working in silos. A team is needed to succeed and see the ROI of a big data initiative. This team is typically responsible for data cleansing and preparation, data analysis, application development, and production deployment.
Much like the DevOps initiative in the software development lifecycle, a similar philosophy is needed in big data and analysis whereby teams of people are creating and testing multiple models to determine which work, which do not, and which perform the best. Continuum Analytics has a lot of clients in hedge funds continually looking at different investment models that will out-perform the current model. These companies know more data results in greater insights and better predictive and prescriptive modeling.
The hurdle rates for developing and testing models used to be high; however, automation has lowered the cost, the time to see meaningful results, and the ability to provide alerts when the current model begins to drift from past performance. Biotech is using big data analytics to uncover patterns in complex images and building models that test drug cocktails to attack specific diseases.
“Over 94 percent of the enterprises in the survey rely on open source for data science. Open Data Science is the Rosetta Stone to unlocking the value locked away in data, especially Big Data,” said Michele Chambers, EVP Anaconda Business Unit, Continuum Analytics. “Our research shows that data science is no longer just for competitive advantage; it needs to be infused into day-to-day operations to maximize the value of data. Data science is business and the best run businesses run Open Data Science.”
Michele sees a bifurcation between poor and rich companies using data with banks leading the way. Smart businesses have studied the opportunities to collect and use data intelligently. Developers with computer science and math degrees, along with domain expertise are few and far between, as such, companies need to combine data engineers, data scientists, business analysts, DevOps, and developers to solve business problems. This requires looking at data differently and collaborating to find the best solutions to the problems.
There's a disconnect in that data is used by C-level executives and data scientists but it’s not making its way down to the front-line business personnel who are working with customers and are making real-time decisions. Companies will realize the full benefits of big data analytics when they are able to empower front-line employees to make real-time recommendations and decisions that benefit customers.