3 Takeaways From the 2019 Gartner Market Guide for Data Prep
After searching the report myself, we thought we'd share some takeaways.
Join the DZone community and get the full member experience.Join For Free
Gartner has recently released its 2019 Market Guide for Data Preparation (), its fourth edition of a guide that was first published in the early days of the market, back in 2015 when Data Preparation was mostly intended to support self-service uses cases. Compared to Magic Quadrants, the Market Guide series generally covers early, mature, or smaller markets, with less detailed information about competitive positioning between vendors, but more information about the market itself and how it evolves over time.
While everyone's priority with these kinds of documents might be to check the vendor profiles (where you'll find Talend Data Preparation listed with a detailed profile), I would recommend focussing on the thought leadership and market analysis that the report provides. Customers should consider the commentary delivered by the authors, Ehtisham Zaidi and Sharat Menon, on how to successfully expand the reach and value of Data Preparation within their organization.
After searching the report myself, I thought I'd share three takeaways addressing our customers' requirements in that exciting market.
Data Preparation Turns Data Management Into a Team Sport
Self-service was the trend that started the data preparation market. This happened at a time when business users had no efficient way to discover new data sources before they could get insights, even after they were empowered with modern data discovery tools such as Tableau or Power BI. They had to depend on IT... or alternatively create data silos by using tools like Microsoft Excel in an ungoverned way.
Data preparation tools addressed these productivity challenges, an area where reports have shown that data professionals and business analysts spend 80 percent of their time searching, preparing, and protecting data before they could actually turn it into insights. Data preparation came to the rescue by enabling a larger audience with data integration and data quality management.
This was the challenge in the early days of the 21st-century, but since that time data has turned into a bigger game. It is not only about personal productivity but also about creating a corporate culture for data-driven insights. Gartner's Market Guide does a great job at highlighting that trend: as disciplines and tools are maturing, the main challenge is now to turn data preparation into a team sport where everybody in the business and IT can collaborate to reap the benefits of data.
As a result, what's critical is operationalization. In order to capture what lines of business users, business analysts, data scientists, or data engineers are doing ad-hoc and turn it into an enterprise-ready asset that can run repetitively in production in a governed way. Ultimately, this approach can benefit enterprise-wide initiatives such as data integration, analytics, and Business Intelligence, data science, data warehousing, or data quality management.
Smarter People With Smarter Tools... and Vice-Versa
Gartner's market report also highlights how tools are embedding most modern technologies, such as data cataloging, pattern recognition, schema on read, or machine learning. This empowers the less skilled users to do complex activities with their data, while automating tasks such as transformation, integration, reconciling, or remediation as soon as they become repetitive.
What's even more interesting is that Gartner relates those technologies' innovation with a market convergence, as stated in this prediction: "By 2024, machine-learning-augmented data preparation, data catalogs, data unification, and data quality tools will converge into a consolidated modern enterprise information management platform."
In fact, a misconception might have been to consider Data Preparation as a separate discipline geared toward a targeted audience of business users. Rather, it should be envisioned as a game-changing technology for information management due to its ability to enable potentially anyone to participate. Armed with innovative technologies, enterprises can organize their data value chain in a new collaborative way, a discipline that we refer to at Talend as collaborative data management, and sometimes also referred to as DataOps by some analysts, including by Gartner in the market guide.
Take data quality management as an example. Many companies are struggling to address their data quality issues because their approaches rely too heavily on a small number on data quality experts from a central organization, such as central IT or the office of the CDO. Although those experts can play a key role in orchestrating data quality profiling and remediation, they are not the ones in the organization that know the data the best. They need to delegate some of the data cleansing effort to colleagues that are working closer to where the data is sourced. Empowering those people with simple data preparation tools makes data quality management much more efficient.
The Value of the Hybrid Cloud
Gartner also looked at growing customer demands for data preparation being delivered through innovative Platform as a Service deployment models. What they highlight are requirements for much more sophisticated deployment models that go beyond basic SaaS. The report notes that, "organizations need the flexibility to perform data preparations where it makes the best sense, without necessarily having to move data first." They need a hybrid model to meet their constraints, both technical (such as pushing down the data preparation so that it runs where the data resides) and business (such as limiting cross borders data transfers for data privacy compliance).
This is a brilliant highlight, one that we are seeing very concretely at Talend: we are hearing sophisticated requirements in our data preparation tool with respect to hybrid deployments. Some of our cloud customers are required to run their preparations on premises. Others want a cloud deployment, but with the ability to access remotely to data inside the company's firewalls through our remote engines. Others want to be able to operationalize their data preparations so they can run natively inside big data clusters.
Published at DZone with permission of Jean-Michel Franco, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.