Why We Pull Back the Curtain on Technology

I was trying to explain to a business analyst this week the difference between SDK and API, which he said was often used interchangeably by people he worked with. In my opinion, SDK and API can be the same thing depending on how you see this layer of our web, mobile, and device connectivity. The Internet has been rapidly expanding this layer for some time now, and unless you are watching it, you really don't see any difference between API and SDK; it's just where the software connects everything.

For me, an SDK is where the data, content, and algorithmic production behind the curtain is packaged up for you, giving you a predefined look at what is possible, prepared for you with a specific language or platform in mind. Most of the hard work of understanding what is going on has been translated and crafted, providing you with a set of instructions of what you can do with this resource in your application. Your integration is pretty rigidly defined — not much experimentation or hacking encouraged.

An API has many of the same characteristics as an SDK, but the curtain is pulled back on the production a little bit more — not entirely, but you do get a little more of a look at how things work, what data and content are available, and algorithmic resources are accessible. You still get the view which a provider intends you to have, but there are fewer assumptions about what you'll do with the resources put on the interface, leaving you to do more of the heavy lifting with how these resources will get put to use.

Most of the early motivations behind choosing an open approach to web APIs over more closed and as proprietary SDK, pulling back the curtain on how we develop software, weren't entirely intentional. Companies like Flickr and Twitter weren't trying to make their mark on the politics of how we integrate software, they were busy and looking to encourage third-party developers to do the hard work of crafting the SDKs, and other platform integrations. The reasons for pulling back the curtain on how the sauce is made was purely about furthering their own needs and not necessarily about moving the needle forward regarding how we talk about software integration. It was just business, enabled by tech (HTTP), and the politics came later, as a sort of side-effect.

Many traditional software developers and software-enabled hardware manufacturers have a hard time seeing this expansion in how we integrate with software and are usually still very SDK oriented, even if there are many APIs right behind their SDK curtain. They do this for a variety of technical, business, and political reasons. It is my personal mission to help these folks understand a little more about this expansion in the software connectivity layer and the benefits brought to the table by being more open. We need the client integration (SDK) and API to be loosely coupled from a technical, business, and political stance to make things work in a web-enabled environment.

It isn't easy to help business folk see the importance of leaving this layer open. This is the damaging effects of the Oracle vs. Google Java Copyright. It gums up and slows this expansion, something we need to encourage and keep open, even if the big tech companies don't fully get it. We are going to need this momentum to not just keep the web, mobile, and device integration accessible and loosely coupled, but we will also need to help make sure the growing number of algorithms that are impacting our worlds are more observable, as well. Providers aren't going to be willing to pull back the curtain on the smoke and mirrors that are AI, machine learning, and other algorithmic varietals infecting our lives.

There are many reasons why we pull back (or don't) the curtain on technology at the application, SDK, API, or algorithmic levels. I don't count on companies, institutions, and government agencies to ever do this for the right reasons. I'm counting on them doing it for all the wrong reasons. I am looking at incentivizing their competitors to do it, helping influence policy or law to direct the systems to behave in a certain way, and encouraging companies to be lazy and keep the curtain pulled back because it's easier. Getting a peek behind the curtain or convincing some to pull it back is never a straightforward conversation — you often have to use many of the same tactics and voodoo employed by tech providers to get what you want.