Data Virtualization: A Secret Weapon for Businesses
Data is no longer all neatly delivered from relational tables, but can originate from a nearly unlimited number of sources, both structured and unstructured, including sensors, email, social media, and RESTful APIs. Data virtualization is here to help!
Join the DZone community and get the full member experience.Join For Free
Data is growing every day. It is no longer all neatly delivered from relational tables, but can originate from a nearly unlimited number of sources, both structured and unstructured, including sensors, email, social media, and RESTful APIs.
That is why data virtualization is so useful—it brings clarity and simplicity to the data scattered across silos and the globe, so you can solve business problems.
Why Is Data Virtualization a Secret Weapon for Businesses?
Data virtualization enables an enterprise’s simple relationship with complex data by abstracting away the complexities of diverse underlying data stores and providing a single source of access to all data in a consistent format. It allows data to be joined across all stores and individual stores to be switched in and out behind the scenes, without the awareness of end users.
Licensing costs can often be scaled back when using data virtualization because full copies aren’t made of every last bit of data like they are with solutions purely based on extract, transform, and load (ETL). Finally, a data virtualization server’s single point of access allows the easy management of security policies because each enterprise team logs in through the same entry point.
The data virtualization layer sits between an enterprise’s data stores and the applications that need access to them. The source stores can be local or in the cloud, and can include relational databases, data warehouses, NoSQL databases, Hadoop installations, and more.
Data is transformed at various points in the installation: transformative wrappers can be defined for data in the virtualization layer itself; in some cases, however, it is preferable to perform transformation closer to, or in a data store itself—using stored procedures.
How Can I Quickly Deploy and Get Results?
Similar to the way in which big data solutions like Hadoop are made possible by innovations like horizontally scaled commodity servers, data virtualization has been enabled by recent developments in the field of in-memory databases. One major provider of data virtualization services is the developer of one of the more unique in-memory database offerings currently on the market—Tarantool.
Tarantool’s data virtualization solution is based on its open source product which offers fully ACID transactions in a fast in-memory and disk database with horizontal scaling and persistence. An enterprise’s IT staff can configure the connectors to a Tarantool virtualization layer as well as the representation of data in the tool. Data doesn’t need to fit into RAM, as Tarantool’s Vinyl engine allows disk to be used in conjunction with RAM.
Veon, a large global telecom, found that Tarantool’s virtualization solution enabled them to access customer billing information ten times faster than they could before. They were also able to generate better service offers for customers based on profiles. Veon’s Tarantool implementation combines dozens of data sources, with a total volume of two to ten terabytes on nine machines—six front facing and three used for storage. Each of the machines has 24 cores and 300 gigabytes of RAM.
Because it facilitates the mixing of old and new data sources and enables quick access and reconfigurations, data virtualization can refresh an enterprise’s business intelligence practices. It can also be tested relatively easily for effectiveness without disturbing an existing data ecosystem.
Visit the Tarantool Challenge to connect with actual developers for answers.
Opinions expressed by DZone contributors are their own.