Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

3 Predictions for the Future of Data Storage

DZone's Guide to

3 Predictions for the Future of Data Storage

Enterprises can expect to see new strategies and tactics for storing data that deliver faster access, greater protection, and significantly improved collaboration.

· Database Zone
Free Resource

Learn how to create flexible schemas in a relational database using SQL for JSON.

An often-cited statistic is that 90 percent of world’s data was created in the past two years. With such exponential data growth, managing storage demands has become more challenging than ever — and many companies are turning to the cloud, where data can be safely stored and managed at massive scale.

Going forward, enterprises can expect to see new strategies and tactics for storing data that deliver faster access, greater protection, and significantly improved collaboration.

1. The Hybrid Model Will Dominate

Increasingly, IT organizations are adopting cloud technology as they seek ways to reduce costs while meeting ever-increasing application performance requirements and service levels. The public cloud’s strength is its elasticity, enabling organizations to dynamically provision resources on-demand. By leveraging the AWS spot market, these organizations can also improve the economics of batch processing.

On-premises (private) clouds are effective when high utilization levels of compute and storage can be maintained. However, many enterprise applications have resource requirements that vary throughout the year. Examples of highly variable workloads include simulations prior to IC tape-out, quarterly business results reporting, and annual inventory recordkeeping. IT organizations must provision infrastructure based on peak needs just to meet customer service-level agreements. The result is that these resources are idle much of the time, yielding a low return on investment.

An increasing number of organizations will move to a model that leverages the best of both worlds; their private cloud will run core application services, while non-mission critical applications will be on the public cloud. Such hybrid cloud adoption is already occurring at a remarkable rate. In fact, the hybrid cloud market is expected to nearly triple from $33.28 billion in 2016 to $91.74 billion by 2021.

2. Automation Will Push the Limits of Machine Learning and IoT

These new automation platforms will have an IoT portion that will use REST APIs (S3-like) to not only access stored sensor data but also to get model information to be able to process and make decisions at the edge. The machine learning part that trains the “brain” of the automation solution will have to effectively process vast quantities of data from hundreds to many thousands of large end-point IoT devices while iteratively improving the model on which these operate.

Storage systems will need to support not only a REST based API with adequate security controls but also a very high throughput POSIX interface to the data that will be served to the machine learning compute nodes. These compute nodes may be GPU-based or standard CPU-based. New hybrid server nodes that incorporate both processor types will also be common, leading to increasingly complex and resource intensive workloads.

3. NAS and Legacy IT Infrastructure Use Will Decline

Current NAS-based storage solutions were designed in the era of fast ethernet, magnetic media (hard disk drives), and sub-petabyte scale. However, modern datacenters use 10-100Gb/s ethernet, use solid-state drives (SSDs), and contain several to hundreds of petabytes (PB) of data. When these legacy systems attempt to access all this data, they do so using outdated communication protocols and data protection schemes that suffer from performance bottlenecks and put valuable data at risk.

Modern shared data infrastructure solutions that were designed for SSD, fast networking, and exascale eliminate the bottlenecks of the legacy systems, which provide much higher levels of data integrity and resiliency at much better economics.

Organizations wishing to improve their competitive advantage will adapt their data centers to be more cloud-like and will turn to using storage solutions that are cloud-native with scalability, performance, economics, protection, and API-based provisioning all built in.

Create flexible schemas using dynamic columns for semi-structured data. Learn how.

Topics:
data storage ,hybrid cloud ,automation ,nas ,machine learning ,iot ,database

Published at DZone with permission of Liran Zvibel. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}