APIs, Connectors and Integration Applications

[This article was originally written by Darko Vukovic.]

In this post, I wanted to give an analogy around how to think about API’s, connectors, and integration applications. This is something that can be confusing when you first start working with or building integrations since the definitions of applications and connectors are relative terms which means that they differ in the application space vs the integration space. Let’s say that you want to connect your laptop to your TV so that you can watch some YouTube videos. What would this look like using the terminology used in the integration space?

APIs

An API is a way to access information from a system using the exposed system functionality rather than the UI, if you are not satisfied with the out of box solution. APIs allow people to create applications that interface with the system or create new applications/UIs on top of existing systems. The easiest way to visualize an API is to look at the back of your desktop computer or the sides of your laptop. You will notice all kinds of sockets, maybe a few USB slots, a headphone jack, microphone input, power input and some form of video output. All of these sockets can be thought of as APIs for your computer.

For the most basic things, like video, audio, and text input, you get an out of the box solution like the screen, speakers, and keyboards. I may be happy watching my YouTube videos on the screen provided by my Mac, listening to sound through the built-in speakers, and typing search terms using the built in keyboard. But what if I wanted to watch it on my big screen TV, listen to the sound on my headphones, and type commands from 10 ft away with an external keyboard? To do this, I would need to access the data at the API layer for video, audio, and text input. So, to build this Mac to TV integration application from scratch, I have a few choices, ranked in order of degree of difficulty:

• I can go down to the memory, where the video is being buffered and hack into my computer’s RAM, process the data, and turn it into a format that my TV accepts.
• I could use one of the multiple video out APIs that my Mac provides me out of the box. For example, I can build an thunderbolt mini display to HDMI conversion cable, which is much easier than going to the RAM.
• For an even easier approach, I can go to go to Amazon.com and buy an HDMI conversion cable for $11 dollars and use one of my Mac’s API.

There are a couple of things at play here that make getting this integration to work easier. One of which is the API which supports the HDMI protocol. Since it’s a well known protocol, I can use connectors that support that protocol to easily consume those APIs. And because I am one of millions of people that needs this integration, it has been solved and packaged for $11.

Connectors

A connector is a thing that connects you from the point of your reference to a destination system. For example, a Salesforce connector within the context of MuleSoft means a connection between the Anypoint Platform and Salesforce. A Salesforce connector within the context of Atlassian, means a way to connect JIRA to Salesforce. In the integration space, you typically will not find something like a pre-packaged “Salesforce to JIRA” integration because that application would need to run either in Salesforce or in JIRA. However, you may find a “Salesforce to JIRA” connector both in the Salesforce AppExchange as well as the Atlassian Marketplace. When an integration platform is used to avoid point to point integrations, you usually will have something like a Salesforce to our Anypoint connector, then some message processing logic and then aa Anypoint to JIRA connector. This way you don’t tightly couple those systems as you can much more easily modify the integration application that lives outside both systems in play. So to an integration platform, a connector is always from the reference of our integration application.

Here is a diagram of the three points of reference when thinking about the term connector:

To continue on my Mac to TV integration analogy above, lets assume my TV only had an HDMI API and my Mac only has a thunderbolt mini display API. The connector is the piece that is going to connect to the API and pass that data to the next message processor as a data stream. In this example, the connector is the piece that you hold when you plug the cable into the Mac. Likewise, there is a connector on the other end which plugs into the HDMI slot in my TV. Another thing to note is that the same connector can support different functionality. For example, the connector which plugs into my HDMI slot can support handling video as well as sound.

Connectors in the integration space are usually categorized either by the system that they connect to or the protocol that they support. So you will see names like a HTTP connector, a JDBC connector, a .NET connector, a Salesforce connector, or SAP connector. Likewise in our analogy you have connectors based on protocol like HDMI, or specific platforms like Thunderbolt for Apple, or custom ones that are built for a specific computer or phone.

Integration Application

An integration application is the encoding of the business logic, processes, and mappings to enable a solution for a specific integration use case. It is usually comprised of one or more connectors that obtain the data from APIs, message processors to route, modify, and change the mapping of the data, and one or more connectors to pass the data to the final APIs that the data was intended for. For example, in the Salesforce to JIRA example of case to service request, synchronization is an integration application.

To round off my analogy, I cannot just stick an HDMI plug into my TV and a Thunderbolt plug into my Mac and connect them with a solid wire. There is a lot more going on in the Mac to TV integration app than meets the eye. Generally, one one of the two ends, or potentially on both sides where the plug meets cable, data mapping occurs, which converts from one format to the other. This makes it so that any HDMI system, not just my TV, can take that signal and believe that its a true HDMI signal, not caring about where that signal originated or how it became a HDMI signal. So this application would look something like “Mini Display Connector -> data mapper -> HDMI signal.”

So this brings me to the final question, albeit facetious, of “why are all integration solutions not $11?” What is it that makes integrations more complicated and thereby expensive relative to other integrations? To answer this question, lets looks at some other types of out of box integrations between Macs and TVs. You can generally find the standard, in the sense that they are common, ones that convert from Thunderbolt to VGA, DVI, HDMI and all of these tend to run under $15 online. Now as your needs become a little more custom or less typical, or as the complexity of the mapping increases, or when it takes multiple steps and formats to process a request, the price is bound to increase. For example, lets say that instead of a new TV, I wanted to watch the youtube videos on my vintage TV from the 60’s, I would have to convert from my mini thunderbolt display to a coaxial cable.

If you search the web for “thunderbolt to coaxial converter” you will see that its not intuitive to find it and I was not able to find any of of the box solutions. I had to solicit help from a friend to design the application that can solve this problem. And at the end of it, we end up with a solution that is much more expensive and degrades video quality, as well as convert my sound from stereo to mono. The solution we came up with is below.

And when you compare the price of the solution below, you end up with:

• Thunderbolt to Composite = ~$50
• Composite to Coax = ~$57
• Cables = ~$10
• Professional Services to design and install = ~$50

So the price of the total solution comes out to be over 15x higher than a simple thunderbolt to HDMI, in addition to the needed knowledge to manage it going forward and the degraded quality.

In summary, integration applications use connectors to connect into systems via the APIs that those systems expose. Connectors make it easier for the creator of the integration application to work with those APIs since they don’t have to program directly against those APIs. APIs make it easier to get data in and out of the systems and they are usually included with the systems.

Hopefully this gives you a clearer picture on how these three concepts work together in the integration space.