The Rise of the Data Fabric
The Rise of the Data Fabric
Data fabrics, still in their infancy, are still a bit fluid. But see what they offer to solutions that use IoT and edge computing and what providers are doing with them.
Join the DZone community and get the full member experience.Join For Free
Digi-Key Electronics’ Internet of Things (IoT) Resource Center Inspires the Future: Read More
As enterprises and suppliers adopt cloud computing, edge computing has also become increasingly important. Smartphones, tablets, laptop computers, traditional PCs and, perhaps most importantly, IoT devices are now being pressed into service as part of major applications. This has created an imperative to deploy technologies designed to bring these devices “into the fold.”
This means technology designed to serve in the following ways must be deployed as well. This technology must make it easily possible to:
Leverage data across devices and locations and format that data in appropriate ways so that it can be used at the edge, in the data center, and in the cloud.
Coordinate the flow of data to support real-time processes.
Support application processing as the data is flowing at the edge, in the data center, and in the cloud.
Gain insight centrally and push intelligence out to the edge.
Support ways for the environment to react quickly to improve service, reduce costs, mitigate risk, ensure security, and, of course, make it easy for enterprises to manage an increasingly complex environment.
Some suppliers have coined the phrase "data fabric" to describe this comprehensive mix of technology. One of the challenges the industry faces has been seen before as new approaches are introduced — not all parties to the conversation agree on the definition of the phrase “data fabric.” As the business needs I’ve described are increasingly felt, many suppliers have packaged up their current products and have described them as being the ideal data fabric. Here are a few examples.
NetApp has been pointing out that a data fabric is really just a way to move data around inside of their storage systems. They are now offering tools and services that make it more easily possible for enterprises that use NetApp storage to reformat data for specific applications. They haven’t described, as of the moment, how edge computing devices can easily take part in computing solutions.
Talend has been using the phrase to support its development tool technology and is describing a data fabric as a platform that “meets all of your integration needs — batch, streaming, real-time, and cloud — in a single platform.” This platform consists of a set of open source tools that “generates optimized, native code (Java/Spark/SQL).” It is not at all clear how these tools can be utilized to help edge devices cooperate with enterprise computing environments.
It is clear that the industry needs to coalesce on a comprehensive definition that includes the following things:
Combine data from established systems as well as these new edge computing solutions.
Provide necessary speed, scale, and reliability to support enterprise-grade applications.
Support multiple locations that can include many data centers, cloud service providers, and, of course, the systems on the edge of the network.
Support the notion that execution/computing can be anywhere — not just describing support for the flow of data.
Create a unified data environment even though systems at the edge and systems in the data center might view data differently.
Support redundancy to support high levels of reliability and availability.
Once a final definition is agreed upon, then the industry can go about the business of creating international standards describing how Android, iOS, Windows, Linux, UNIX, and other computing environments can work together.
In my industry research, I’ve come across a number of definitions of the data fabric notion. So far, the one being put forward by MapR Technologies seems to be the most comprehensive. Furthermore, the company has been enjoying some level of success as it persuades customers to adopt its approach. The company appears to understand the importance of distributed processing in the era of mobile, cloud, and the fast proliferation of IoT and how all of these are important members of enterprise computing applications. They also appear to understand that these devices also represent potential threats and have discussed how to bring them in safely. The company has also considered where and how data should be stored so that it is available broadly, but still secure and reliable.
It would be wise for enterprise developers and decision-makers to consider the concept of a data fabric and finding ways to bind edge computing, data center computing, and cloud computing into a reliable, scalable tool now before the need is thrust upon them by changing market conditions. It would also be wise for them to take the time to review what suppliers are saying and determine if their view of the topic is comprehensive enough to provide needed solutions to the problems they’re likely to experience as their customers, suppliers, and competitors are defining the digital future.
Opinions expressed by DZone contributors are their own.