How the Internet of Things Will Affect Database Management

There's no denying the rise of the Internet of Things will challenge existing database systems to adapt to accommodate huge volumes of unstructured data from diverse sources. Some analysts question whether RDBMSs have the scalability, flexibility, and connectivity required to collect, store, and categorize the disparate data types organizations will be dealing with in the future. Others warn against counting out RDBMSs prematurely, pointing out that there's plenty of life left in the old data structures.

Imagine billions of devices of every type flooding data centers with information: a secured entryway reporting on people's comings and goings; a smart shelf indicating a shortage of key production supplies; a pallet sensor reporting an oversupply of stocked items.

The Internet of Things poses unprecedented challenges for database administrators in terms of scalability, flexibility, and connectivity. How do you collect, categorize, and extract business intelligence from such disparate data sources? Can RDBMSs be extended to accommodate the coming deluge of device-collected data? Or are new, unstructured data models required?

As you can imagine, there's little consensus among experts on how organizations should prepare their information systems for these new types and sources of data. Some claim that RDBMSs such as MySQL can be extended to handle data from unconventional sources, many of which lack the schema, or preconditioning, required to establish the relations that are the foundation of standard databases. Other analysts insist that only unstructured, "schema-less" DBMSs such as NoSQL are appropriate for data collection from intelligent devices and sensors.

The standard application model is transformed to encompass the cloud by the need to accommodate tomorrow's diverse data sources and types. Source: Technische Universität München.

In a November 28, 2014 article, ReadWrite's Matt Asay reports on a recent survey conducted by Machine Research that found NoSQL is the key to "managing more heterogeneous data generated by millions and millions of sensors, devices, and gateways." Not surprisingly, the two primary reasons for the assessment are NoSQL's flexibility in handling unstructured data, and its scalability, which the researchers claim RDBMSs simply can't match.

Reports of the Death of RDBMSs Are Slightly Exaggerated

There are a couple of problems with this claim, however. First, there's an acute shortage of NoSQL developers, according to statistics compiled by research firm VisionMobile. The company pegs the current number of NoSQL developers at 300,000, but it estimates that by the year 2020 the number will jump to 4.5 million.

VisionMobile's research indicates that demand for NoSQL developers will explode in coming years. Source: VisionMobile.

Many experts posit that the forecast demise of RDBMSs is premature because the forecasters underestimate the ability of RDBMS vendors to extend their products to enhance their scalability and their ability to accommodate unstructured data. For example, MySQL vendor DeepDB improved the scalability of its product by replacing the default InnoDB storage engine with an alternative that it claims improved server performance by a factor of 100.

At this point, IT managers can be excused for thinking "Here we go again." Does the Internet of Things signal yet another sea-change for their data centers? Or is this the most recent case of the hype outpacing the substance? According to a December 2014 TechTarget article by Alan R. Earls, corporate bandwidth will be overwhelmed by the rush of small data packets coming from Internet-connected devices.

In particular, distributed data centers will be required that move data security — and the bulk of data analysis — to the edge of the network. This will require pushing the application layer to the router and integrating a container with logic. As Accenture Technology Lab researcher Walid Negm points out, the cloud is increasingly serving as either an extension of or replacement for the conventional network edge.