SQL, the Technology That Never Left, Is Back!

The history of SQL, or Structured Query Language, dates back to 1970, when E.F. Codd, then of IBM Research, published a seminal paper titled, “A Relational Model of Data for Large Shared Data Banks.”

Since then, SQL has remained the lingua franca of data processing, helping build the relational database market into a $36 billion behemoth.

The Rise And Fall of NoSQL

Starting in 2010, many companies developing datastores tossed SQL out with the bathwater after seeing the challenges in scaling traditional relational databases. A new category of datastores emerged, claiming a new level of scalability and performance. But without SQL they found themselves at a loss for enabling easy analytics. Before long, it was clear that there were many hidden costs of NoSQL.

The Comeback That Never Left

More recently, many point to a SQL comeback, although the irony is that it never left.

In a piece last week on 9 enterprise tech trends for 2017 and beyond, InfoWorld Editor in Chief Eric Knorr notes on trend number 3:

The incredible SQL comeback
For a few years it seemed like all we did was talk about NoSQL databases like MongoDB or Cassandra. The flexible data modeling and scale-out advantages of these sizzling new solutions were stunning. But guess what? SQL has learned to scale out, too – that is, with products such as ClustrixDB, DeepSQL, MemSQL, and VoltDB, you can simply add commodity nodes rather than bulking up a database server. Plus, such cloud database-as-a-service offerings as Amazon Aurora and Google Cloud SQL make the scale-out problem moot.

At the same time, NoSQL databases are bending over backward to offer SQL interoperability. The fact is, if you have a lot of data then you want to be able to analyze it, and the popular analytics tools (not to mention their users) still demand SQL. NoSQL in its crazy varieties still offers tremendous potential, but SQL shows no sign of fading. Everyone predicts some grand unification of SQL and NoSQL. No one knows what practical form that will take.

Taking the ‘No’ Out of NoSQL

In an article, Who took the ‘no’ out of NoSQL?, Matt Assay writes,

In the wake of the NoSQL boom, we’re seeing a great database convergence between old and new.

Everybody wants to speak SQL because that’s where the primary body of skills reside, given decades of enterprise build-up around SQL queries.

The article, interviewing a host of NoSQL specialists, reminds us of the false conventional wisdom that SQL doesn’t scale. Quoting a former MongoDB executive, Assay notes,

But the biggest benefit of NoSQL, and the one that RDBMSes have failed to master, is its distributed architecture.

The reality is that legacy vendors have had trouble applying scale to their relational databases. However, new companies using modern techniques have shown it is very possible to build scalable SQL systems with distributed architectures.

SQL Reigns Supreme in Amazon Web Services

There is no better bellwether for technology directions these days than Amazon Web Services. And the statistics shared by AWS tell the story.

In 2015, Andy Jassy, CEO of Amazon Web Services, noted that the fastest growing service in AWS was the data warehouse offering Redshift, based on SQL.

In 2016, he noted that the fastest growing service in AWS was the database offering Aurora, based on SQL.

And one of the newest services, AWS Athena, delivers SQL on S3. This offering is conceptually similar to the wave of ‘SQL as a layer’ solutions developed by Hadoop purveyors so customers could have easy access to unstructured data in HFDS. Lo and behold there were simply not enough MapReduce experts to make sense of the data.

AWS has recognized a similar analytics conundrum with S3 growth, which has been so strong it appears that objects stores are becoming the new data lakes. And what do you do when you have lots of data to examine and want to do so easily? You add SQL.

SQL Not Fading Away

Nick Heudecker, Research Director in the Data and Analytics group at Gartner, put his finger on it recently,

Each week brings more SQL into the NoSQL market subsegment. The NoSQL term is less and less useful as a categorization.

— Nick Heudecker (@nheudecker) November 8, 2016

Without a doubt, the data management industry will continue to debate the wide array of approaches possible with today’s tools. But if we’ve learned one thing over the last 5 years, SQL never left, and it remains as entrenched and important as ever.