Give all users access to the data they need to make informed decisions.
Join the DZone community and get the full member experience.Join For Free
I had the opportunity to meet with Bob Eve, Senior Director for Data Intelligence in the Thought Leadership Group at TIBCO during TIBCO NOW 2019 in Chicago. The day before, I attended Bob's breakout, Democratize Data for Better Insight in which he presented the need for a data virtualization strategy, explained how data virtualization helps support multiple analytical projects in an organization and provides self-service access to data.
Less than 50% of structured data in an organization is used for decision-making. More than 70% of employees in an organization have access to data they shouldn't, and 80% of data analysts' time is spent searching for the data they need to perform analysis.
According to Bob, data virtualization can solve each of these problems. Data virtualization is an integration technique that provides complete, high-quality, actionable information through virtual data across multiple, disparate internal and external data sources.
I asked Bob, "Why is data virtualization important as a data management strategy?" According to him, organizations today are facing a supply of data bottleneck. There is expansive demand for data throughout the organization as people realize that innovation and insights are driven by data.
Every business today is facing the challenge of getting insights to make decisions. At the same time, the data supply has gotten more siloed in the cloud, systems, and data lakes. It's a challenge to provide access for analysis and consumption. The distribution and diversity of data have made it harder while the security and compliance needs around the data have become more severe.
The old ways of handling and accessing data cannot continue if companies are going to achieve their objectives. People, process, politics, and technology have to come together to do some industrial grade problem solving.
Data virtualization is proving to be a primary route to success in dealing with this challenge. We're starting to understand that we have to break the problem up and think about how to manage access to the data technically and identify how to surface the data in a friendly way to the business.
Data virtualization enables the establishment of a set of standard services so scientists can consume the data "as is" or prepare it for "the last mile." Rather than spending 80% of the time preparing data, you can now spend 20% of the time doing that.
If what you need doesn’t exist, you can call IT to gin up the data you need. Use metadata to provide a new data set in less than an hour. Get the data sets to the business and see if it's good enough. Refine more if needed, run faster, put more engineering to get more value.
ETL to data warehouse required heavy engineering on everything. With data virtualization, you're able to use open source to combine a good piece of code quickly and iterate on it.
Bob has just written the first book on Data Virtualization and it includes 10 detailed case studies from IT leaders in multiple industries with different use cases.
I asked, "What do developers need to know about data virtualization?" Bob assured me that if a developer took a SQL class, then they are more than skilled enough to use data virtualization since it involves classic tabular relational work. He encouraged developers to not be afraid of data virtualization but to let go of old ideas that everything needs to be physically consolidated to do anything as this is a dated 30-year-old architectural principle.
Data virtualization is like Agile, it took years to let go of Waterfall and then see the benefits. Be open to new ways of doing things. Be willing to adopt a new, more agile approach. Deliver fast and improve as much as you need. Learn about use cases and customers. Read up on data virtualization, check out videos on YouTube, get comfortable with the art of the possible.
Adopt best practices, think about the long-term vision for a layer over all your data sources with a standard data service. Start with small projects and get some payback. See the speed with which you can do things.
What am I missing? The critical success factors when people do adopt data virtualization happen with top-line customer experience. Your efforts pay out quickly. You see the top line impact. It serves as the foundation for doing proactive customer engagement. You are able to plumb the data to get a 360-view of the customers to do insightful things. Customers that move the fastest are trying to drive topline revenue with CX. Some are driving CX in companies like pharma with innovation. You're able to get new products to market faster. You know where to invest – the next best decision for the company.
Opinions expressed by DZone contributors are their own.