Integration Reboot: RIP ESB
Thanks to the explosion of cloud technology, ESBs may not be necessary for your new applications.
Join the DZone community and get the full member experience.Join For Free
Traditional integration is undergoing a seismic shift due to the explosion of cloud-based services and the ubiquity of APIs in what’s being dubbed the Internet of APIs (IoA). An essential component in modern integration, APIs allow systems, SaaS services, sensors and things to connect, communicate and in combination, actually do something useful. By linking APIs together, previously disconnected business functions like sales and legal can finally interact and operate together seamlessly and efficiently. However, despite the inherent business opportunities that connections like this offer, many organizations are still not capitalizing on APIs and thus, cannot integrate properly. The days of legacy integration are over — now is the time for organizations to consider rebooting their integration strategy.
Legacy Enterprise Integration
Much of the old school enterprise integration paradigm revolved around the concept of the “Enterprise Service Bus” (ESB), a fundamental building block to connect legacy systems like Oracle, Siebel or PeopleSoft. Its basic principle was to offer a neutral communications platform that could connect to different systems and pass messages or events between them. Prior to ESBs, developers would directly connect two systems (i.e. Oracle to PeopleSoft). All of these connections were difficult to implement, required a lot of specialized knowledge about both systems, provided only one-to-one integration capabilities and were extremely brittle — any changes to either application would typically result in the connection failing and needing to be updated or re-engineered.
Although developers experienced issues with ESBs at the time, they worked because they provided a more flexible framework, allowing for one-to-many connections and components to remain completely independent of each other. By using ESBs, application vendors and middleware providers could wrap their applications into “services” to be shared with other systems. Multiple systems could connect to the bus, communicate about internal events, listen for relevant events triggered in other systems and even respond to them.
Integration pioneers like Vivek Ranadive from TIBCO Software deserve a lot of credit, as they helped establish the ESB as the default method of integration in the enterprise. Vivek in particular pushed the boundaries of what was possible at the time by introducing the concept of “realtime” integration, which became widely successful in industries where the communication of information or events between systems was time sensitive. An example of this would be stock prices relayed between systems at a financial services institution.
Enterprises today have many more applications and systems to orchestrate than ever before and simply cannot keep up with the pace of digital change if they’re running on ESB infrastructure alone. For businesses that still use this classic approach, it’s important to realize that it comes with significant tradeoffs:
Deploying an ESB can take 8–12 weeks to set up for even the simplest projects, or a couple of years if you’re a behemoth like Walmart.
There are limited standards, so there is no out-of-the-box way to simply plug in applications to different ESB vendors. Middleware vendors bridge the gap and provide the connecting “glue” between the apps and the ESB, introducing additional cost and complexity into this type of architecture.
Effective ESB integration is inherently limited to systems deployed on-premise, behind the corporate firewall and not suited for connecting cloud applications.
Integration in the Cloud
The IT landscape and types of systems companies rely on have changed significantly. Of course there are still plenty of on-premise solutions out there, but today, organizations depend on cloud-based SaaS providers and deploy systems like Salesforce for customer relationship management, Marketo for marketing automation, NetSuite for finance and accounting, Slack as their enterprise collaboration platform—the list goes on.
A natural evolution connects these cloud systems together to enable end-to-end business processes like quote-to-cash, support or supply chain management. Since cloud-based software tends to provide REST-based APIs, which provide an easy “hook” for developers to connect into, these connections are much easier to establish.
There is a great opportunity and need for integration and the idea of horizontal connections remains an attractive one. However, instead of the traditional ESB, you need something that’s more lightweight, API-enabled, cloud-first and internet-aware. We call this the Internet Bus (IB). The IB is made up of two parts: an infrastructure layer that provides the ability to publish and subscribe to services and the so-called pub/sub model. Providers like PubNub, Pusher and Google have created popular offerings, often revolving around standardized protocols such as MQTT.
While applications can directly connect to this IB infrastructure, integration Platform-as-a-Service (iPaaS) vendors make it easier by processing incoming events and outbound messages. Since there are no real standards with integration and it’s up to the app to connect to this IB, the iPaaS layer provides the “glue” like enterprise middleware used to. But since this layer is in the cloud, it requires no setup, it’s lightweight, immediately usable and massively scalable.
We are still in the early days, with companies typically connecting five or fewer services and cloud apps per business process. Over the next year, I expect that we’ll see as many as 50 connected services become the norm, all working together to orchestrate and automate sophisticated and powerful business processes, fundamentally changing the way businesses operate.
What to Do Now
Choosing the right iPaaS can unlock efficiencies, enable new digital experiences and boost innovation across the entire organization. Here are some guidelines for evaluating and selecting the best iPaaS for your business:
Look for an iPaaS that is not reliant on an ESB. If you don’t already have an ESB and/or your business was founded in the last 10 years, you can comfortably skip over this chapter of legacy IT
Look for an iPaaS that is truly cloud-native and cloud-first. If you find yourself installing things locally, run the other direction
Look for enterprise case studies where the time to deployment was counted in days or weeks, not months
Integration technology has come a long way. Once considered to be the most difficult realm of IT, attracting only the technical elite to solve complex problems for rich and powerful corporations, we’re now in the midst of a veritable revolution. A revolution that is delivering integration to the masses. You no longer need to hire an army of integration ninjas and integrating new cloud services like Salesforce doesn’t have to kill your budget. What’s most astonishing, perhaps, is the added value you can unlock from your existing IT investments by simply connecting systems you’ve owned for 25 years with the systems you’re adopting now. If you’re looking for the most efficient, most effective, most direct path to doing just that, it’s high time to look forward to the cloud and to finally put the pursuit of the ESB to rest.
Opinions expressed by DZone contributors are their own.