Integration Technologies: From EAI to API Gateways

A Gartner Predicts report stated that by 2018, more than 50 percent of the cost of implementing 90 percent of new large systems would be spent on integration. I don’t know if this has come to fruition or not but I think we can all agree that companies spend a lot of time and energy on integration solutions.

A Brief History of Integration Technologies

Before I dive too deeply into the role an API gateway can play in integration, let’s take a quick look at the more recent history of enterprise integration technologies. I’m not going to provide a thorough discussion of the various solutions and vendors, there are plenty of analyst reports that do that, I’m simply going to look at the types of systems that have been in use for the last 20 years or so.

EAI: Enterprise Application Integration Technologies

In the nineties, most companies had some form of investment, often quite substantial, in what I’ll call traditional Enterprise Application Integration (EAI) solutions. EAI solutions packaged a few things together including adapter frameworks (there were lots of systems and applications with proprietary interfaces), messaging systems, content transformation modules, and a variety of mechanisms for stitching solutions together from all these parts. This last part is where it gets most interesting, because, over time, companies found themselves building more and more logic into the EAI solutions and almost ended up delivering full-blown applications built entirely in their integration platforms.

ESB: Enterprise Service Bus

In the mid 2000s, the idea of web services started to become enmeshed with integration (I talked about this a bit in the opening post of this series), and we witnessed the birth of the Enterprise Service Bus (ESB). ESBs ranged from putting a thin layer of lipstick on the EAI pig, to all-new solutions built entirely around web services stacks. Even the newer versions still deliver the same core elements, although the adapter framework was either replaced by, or augmented with the web services stack. Companies still found themselves writing a lot of code to create integrations, in many cases to simply ensure the interoperability of the services they were integrating with.

iPaaS & Hybrid Integration Technologies

As the popularity of Service Oriented Architecture (SOA) waned, so did the desire to have one or more ESBs. Some newer ideas started emerging and we saw the advent of terms like citizen integrator and integration Platform-as-a-Service (iPaaS). The idea here is that integration should be the realm of business people, and projects shouldn’t require deep technology expertise. As these ideas gained traction, Gartner introduced the concept of the Hybrid Integration Platform, combining traditional EAI services, ESB, API management, Citizen Integration Tools, and even Data Integration solutions into a single platform.

Standardization

The biggest change in integration over the years, however, has been the standardization of interfaces and content types. What this means for integration platforms and tools is that the need to maintain libraries of proprietary adapters is rapidly diminishing. This, in turn, diminished the value of a traditional integration solution. More recently, with the popularity of microservices-based approaches to application architecture, we’re seeing companies moving away from their investments in ESB as they try to follow the “smart endpoints, dumb pipes” guideline espoused in the seminal blog post by Martin Fowler and James Lewis. While we might hope that all this standardization and componentization of applications might eliminate the need for integration tools completely, all it really does is change the nature of those tools, introducing a different set of requirements.

See where Akana scored highly in The Forrester Wave™: API Management Solutions, Q3 2020 report.

Download Report

Why Do I Still Need Integration? Can I Use My API Gateway?

All that history is all very well, but let’s take a quick look at what all these integration solutions were/are really doing. The primary function of all these solutions is to allow applications to communicate with each other to share data or participate in transactions. In most cases, the integration server (especially in the case of an ESB) is providing a standardized interface to a proprietary system, sometimes delivering a higher-level capability by orchestrating multiple of the more granular functions provided by the backend application.

This raises an interesting question: Given that most back-end applications now expose their functions via some form of service interface, why should I need any form of integration server?

There Are Standards, and There Are Standards

Just because an application exposes a standards-based interface, it doesn’t mean your consumers will want to, or be able to use it. Standards-based comes in a lot of flavors from SOAP, XMLRPC, JMS with XML, simple HTTP with structured or unstructured content, and that’s not even getting into all the security models and variants that come into play. To make these interfaces play nicely with the rest of your enterprise, you really need to define an enterprise standard and ensure all your applications conform. Perhaps your enterprise standard will be REST/JSON using OAuth with JWT, where end users are involved, and a simpler app/API key model in other cases. Or perhaps you want to use SOAP with WS-Security for some applications.

You could use your ESB to create these “proper” standard interfaces from all your applications, but you’ll end up writing code and dealing with all the other issues associated with ESBs, e.g., do you really want to deploy an ESB at the network edge?

This is where an API Gateway comes into its own. A good gateway will have the ability to take the interfaces from your applications in whatever form of “standards” they use and create clean, elegant services or APIs that conform to your defined enterprise standards. See my earlier blog on the difference between an API Proxy and a Gateway for some input on what defines a good gateway.

That's probably enough for now, in my next post I'll explain how an API Gateway fills the integration role better than any other technology today, including the differences between an ESB and API Gateway, along with scenarios where you should (and should not) use one.

More About Akana

Take our developer portal for a test spin with a free 30- trial. Click below to see how easy it is to create, secure, monitor, and publish APIs with Akana.

Start Trial