Hybrid integration platforms: the evolution of integration beyond ESB and API Management

27 May 2020

With no time frame, adapting the organisation and the infrastructure to the growth of new line management and the elimination of those that are not profitable, elastically and transparently- what is the role of integration in this scenario? 

To answer this question, we will run through the successive stages, which the world of integration has experienced, and we will present VASS’ vision for the future of this discipline. 

The initial chaos and the boom of integration solutions

In the beginning, it was chaotic, and each project had to deal with integrating other systems. The integrations were difficult, complicated, and expensive as each system had its protocol and paradigm of communication. To make a new integration system with a new consumer involved a high cost, and substituting a system provoked a significant impact, this was difficult to predict most of the time as there was no control of the consumers that depended on a network. 

All of this led to platforms which centralised communications and offered advanced capacities in mediation, integration, orchestration, cataloguing and service management. Thus, architectures orientated towards services were born. 

The fall of integration solutions 

Nevertheless, it has been quite a few years since the weight of specialised solutions of integrations in our technological ecosystems has fallen. Its biggest reasons for being and the problems that came to create answers had disappeared. The heterogeneity of protocols and paradigms of communication between systems stepped up to a bigger homogeneity which did not require the capacity of mediation and integration needed. Furthermore, cataloguing had been waived and the management which had failed the harsh model of control and the technical offices of architecture ended up being a bottleneck which drowned and slowed down our projects instead of being an aid and an accelerator. Finally, the needs of orchestration by themselves did not justify the necessity of tools and expensive resources specialised and, it had opted to assume it as part of the development of services. The ESB died out, this reflected upon the investment made in this type of solutions and the number of projects addressed in them. 

The time of API Management

When the integration as such and the creation and exposure of new services stopped being a technical problem, we could raise the question of the API as a product.

However, the API Management solution focuses precisely on that vision of negotiation. It doesn’t cover many aspects of the integration which appear in the current scenarios which we will see in the next paragraph. 

The Situation today 

Recently our CTO spoke on our blog on what businesses will be like in the future and presented five aspirations (Customer Anywhere, Context&Cognitive, Datamorphosys, Elastic Enterprise y Outstanding People).

Although we can find the integration present in any of them (discovering it in detail would lead to a whole blog entry), the most relevant is Elastic Enterprise: Businesses of the future will adapt naturally to changes. They will organise flexibly and transparently, which will mould how they function, their infrastructure, their services, depending on the businesses growth or downfall. In this way, we will optimise the companies processes through the automation or reengineering of functions, so together with different types of cloud- private, hybrid or public- with which we will be able to operate and now propose laaS, PaaS and SaaS.

It is precisely this increase of unfolding the services in the cloud, joined by the rise in the consume of services and APIs of third parties and combined with the rise in modal services Saas which conforms a hybrid scenario for which we need new architectures of integration, beyond the ESB and API Management. The centralised vision of both does not fit in with services which can be so on-premise like in different clouds from different providers. Due to limitations of the present solutions in the market, the API Management ignored the protocols and asynchronous architectures. Its volatile management cannot apply to the boom of microservices, which are present in current ecosystems. 

Welcome to the era of new integration

With all the information presented previously, we welcome the new era of integration. If you still have not heard of the acronym FIP (Hybrid Integration Platforms), we will briefly explain what you should ask request your next architecture of integration: 

· A light approach and execution model with native support towards hybrid scenarios in a way in which your services of integration could be close to the sources of data.

· Classification and volatile management of services- exposed for external consumption as well as internal.

· The capacity to manage APIs facilitating its publishing, develop and supervise in a safe and scalable surrounding.

· Support integration with legacy systems in which we still need connectors with mediation and coping abilities with different paradigms and communications.

· Consider architectures orientated towards events and support publication and subscription protocols.