Article Retrospective Integration – All You Can Find in Corporate It!
Application integration is a relatively old subject now, which finds in API Management its latest fashionable offspring.
Join the DZone community and get the full member experience.Join For Free
Application integration is a relatively old subject now, which finds in API Management its latest fashionable offspring. However, in the past, we have not only done API Management. And certain ancestors are still being used in the various IT departments. So let's see what exists on earth :)
The integration was first achieved through file exchanges from one application to another. Banal FTP upload is not enough. This is how MFT, for Managed File Transfer was invented. MFT manages files to transfer between applications through the network, internally and externally, with specific security, integrity, visibility, and non-repudiation features.
It supports multiple file transfer protocols and has proven solutions for many scenarios (consolidation, synchronization ...). At the same time, it is made for batch integration, not real-time, and is aimed for mass integration. And the logic we can add is globally very technical.
We saw then the emergence of ETL (Extract, Transform, Load), which is made to transfer data from one database to another. It is then possible to perform business logic on it, such as aggregation, filtering, conversions. ETL is used a lot for BI needs, but remains limited to batch use and not interactive, just like MFT.
Then EAI is coming (Enterprise Application Integration), with the ability to do real-time processing and to exchange messages in a unitary way. It also brings a functional view by using canonical exchange formats. But unlike ETL, it is not designed to handle large amounts of data.
With the emergence of SOAP web services in the early 2000s, ESB added web services to EAI, allowing a standardized and open exposure to all applications. It also added several specific connectors and became the off-the-shelf integration tool for unitary integration. Even if in practice we continued to use ESBs for medium size file exchange, via ESBs dedicated to this kind of use (say hello to JVM tuning).
The eternally forgotten, yet one of my favorites, CDC, for Change Data Capture. It's a type of tool that was historically used to duplicate databases, very often for backup purposes. However, this technology needs to be studied seriously. It's nothing more than the real-time synchronization of databases between them.
In a master/slave scheme, this is a very useful technique when you have a legacy system that does not offer the possibility of interactive exchange, and which should not be overloaded because the system is crucial. If you have data in an old mainframe that you want to show to your customers, you just need to synchronize your mainframe with a modern database and then display data from that database to your customers. It's almost as simple as that!
API Management is a bit of a rejection of the ESB. The ESBs pronounced SOAP web services, which was a concern. Verbose, too strict, not so compatible depending on whether you had two different libraries between consumer and producer, the API has helped free you from the SOAP straitjacket. Moreover, everyone understood that it was necessary to avoid putting management rules in an integration tool!
It is also the result of the generalization of REST/JSON in new developments, and the maturity of ESB. There is always an API on its backend to expose its services. All that's left to do is to secure and manage this via the Management API!
Or commonly known as "ESB in the cloud", allows you to globally manage an ESB regardless of the type of integration. Cloud to the cloud, Cloud to on-site, On-site to On-Site, the iPaaS can do it all. I would advise the iPaaS if you have complex integrations to achieve. Otherwise, by default try to use the API Management!
What About MQs?
MQs have been used as integration middleware, but their use can be complex. Moreover, I recommend using the underlying protocols rather than dedicated tools, which are only useful for very specific use cases; however, MQ has many advantages. It is extremely fast, which makes its asynchronous character almost invisible, but above all, it is very reliable! If you ever think that what is asynchronous is has been, tell your brain that it works asynchronously!
In short, I hope this article can help you see more clearly, and for the old of the old refreshed your memory!
Opinions expressed by DZone contributors are their own.