Understanding the Importance of EAI

In my experiences as an EAI consultant, I have come across many people who asked me questions like:

• What is an EAI and why is it necessary?
• Why not simply follow the client-server model?
• Why doesn't the request-response model work?

Frankly, it was really difficult for me to convince them with some quick and short responses to those questions. But after taking some time and some proper enterprise use cases to explain it to them, they usually get the clear picture and understand the reason. So, I thought I should share my views on this to help fresh integration developers understand the concept of EAI and how it helps organizations in both business and technical aspects.

Background

We take for granted that applications in large enterprises cut across heterogeneous systems (like systems to manage Billing, CRM, Stock Handling, Provisioning, etc.). This is kind of inevitable and it is very uncommon to find an application that runs in isolation. The reason behind this is obvious when someone understands the simple logic behind it, which is nothing but the necessity of interactions between the systems and data exchange between them. When I say systems, in this context, they could be source systems (Customer Interaction Interface, Point of sale, Kiosk, exposed API, etc.), backend systems (CRM, ICMS, Ericsson, SAP, Comptel, etc.), or data storing and processing systems (Database, Filesystem, Message processing server, etc.)

A naïve organization without a proper architect may come up with an integration design where source systems interact with rest of the systems directly.

This may look fine when the number of systems involved in the design is a single digit number. But sometimes, the number of these heterogeneous systems might go to double digits based on how large the organization is. In such scenarios, following challenges are to be addressed with this approach:

1. Tight coupling between the interacting systems.
2. Change management is a nightmare.
3. Code refactoring is almost impossible.
4. Huge cost and effort occur when there are breaking change in the system service contracts.
5. Tedious work is involved in handling and mapping multiple data formats between systems.
6. Adopting new protocols or transport methodologies will cost a fortune.

Above, I mentioned that if there is a direct coupling between 1 source system and 1 backend system, the effect will be directly multiplied by ‘N’ if the same source system is interacting with ‘N’ backend systems.

This is not over. The same challenge could get worse, i.e., N*N = N² when more than one (N) source system starts interacting with ‘N’ backend systems.

So, what is the answer to this problem? Who can address these challenges and provide a solution?

EAI Is a Need, Not a Want

This is when EAI (Enterprise Application Integration) comes into the picture.

The difference between normal application integration and enterprise application integration is that EAI separates the concerns about data exchange formats, target (backend) system details, methodologies or protocols used for interactions, and platforms or technologies used for interactions from the source system. To do this, EAI uses an effectively proven system called ESB (Enterprise Service Bus).

Considering these, EAI could turn our previous monolithic application integration architecture into below. Here, source systems only interact with ESB in standard canonical format regardless of target system data format. It is the job of ESB after that to transport the collected request or event from source to target, concealing all of the details about what transport or technology is being used to do this job.

So, with this in place, typical challenges with normal application integration will be gone.

This means that:

1. Source systems are no longer tightly coupled with backend systems.

2. Any change in handling data or logic will now be in a single place, ESB.

3. Code refactoring related to interactions will now be in a single place, ESB.

4. All breaking changes now only have to be addressed in ESB, which brings the cost and effort involvement from ‘N’ source systems to ‘1.’

5. With ESB in place, source systems always use canonicals that are system-independent. Thus, mapping efforts are always one to one rather than many to one or many to many.

6. Any technology or transport protocol changes will now have to be taken into consideration only inside ESB.

Now we know why EAI is a necessity.

How an EAI Should Be

So, is that it? Is our integration problem completely solved just by this short documentation with a couple of fancy architectural diagrams?

NO!

The above EAI pattern solves only half of the bigger problem. The other half of the problem lies among the below:

• What about network interruptions?
• How can you achieve guaranteed delivery?
• What approach do you use to handle asynchronous scenarios in a reliable fashion?
• What do you do about administration, standardization, and discipline inside ESB?

These problems can be solved by adopting the blend of some successful patterns into EAI such as Message Oriented Middleware (M-O-M), Event Driven Architecture (EDA), Service Oriented Architecture (SOA), etc.

The proportion of these patterns depends on the type of applications and interfaces the organization needs.

Message Oriented Middleware fits right into the places where Point to Point interactions do not make sense, like in fire and forget scenarios. This makes the application experience faster. All network reliability issues and guaranteed delivery issues can be handled by MOM using the retry mechanism and inbuilt short term data stores.

Event Driven Architecture is another successful architectural approach in ESB that is completely opposed to the request and response model and everything is driven by an event. While EAI separates the concerns between source and target systems, EDA separates the concerns between everything. All of the needed jobs will be events, and any interested parties can be consumers to those events. So, no one knows who gives what and who takes what. This enhances the true power of loose coupling.

Service Oriented Architecture helps to bring more discipline, standards, and rules inside your ESB. It must be clear and obvious by now that ESB is a common box between all the interacting systems. And it is often common that “common” places like this get chaotic and dirty with time unless until rules or policies are enforced. SOA helps in enforcing such policies in both design time and runtime of the application.

Conclusion

I want to conclude this using the same line with which I started this document. Larger organizations have to take it for granted that their applications will have to cut across multiple heterogeneous systems. This is inevitable and obvious, as every large organization wants their application to be a huge success. For the same reason, they have to properly segregate, distribute, and manage data in various systems. This distribution involves huge maintenance, proper integration, and easy data interactions to make everyone’s effort successful. However, this may end up as big failure involving huge cost and effort if proper integration strategy, architecture, and teams are not in place. EAI has proved to be a successful application integration approach for many larger enterprises involving an enormous number of systems.