What is Enterprise Integration and where is it needed?
Today, organizations rely heavily on technologies – which are often of different nature to each other - and applications, running in different business units and likely based on different technologies. So, integrating these applications into a unified set of business processes has emerged as a priority. Enterprise Integration provides a mechanism to build a composite application and share processes and data.
As several studies indicate (here and here), Enterprise integration started as point-to-point integration and evolved to cloud based enterprise service bus architectures. In a point-to-point integration, integrating between two applications requires code development and creating information you could exchange between them. This may prove cost-saving and effective on small networks, but it results in exponential growth of cost and required effort for enterprises. Forrester Research estimates that up to 35% of development time is spent creating application interfaces.
In an Enterprise Integration platform, enterprises integrate between applications through an Enterprise Service Bus (a middleware platform) which orchestrates the information exchanged between the integration endpoints, reducing the number of interconnections. Additionally, Enterprise Integration platforms usually provide features such as
- Scalability, high availability and sizing.
- Use of industry standards as interface technologies (no custom development-universal connectivity).
- Transactional processing, since the information should be consistent within multiple applications.
- Process driven approach embedding logic into integration flows.
- Code abstraction, lowering development cost.
- Monitoring and managing components, making all transaction executions accessible and clear.
- Deployment management components.
Why should you use an ESB?
The business world is undergoing a revolution. With the proliferation of technology across our lives and work, industries around the world are realising that the tried and trusted methods that have kept them moving forward might no longer suffice. Consumers today are connected, informed, and unprecedentedly technologically savvy, and as a result, the companies they deal with are expected to be the same.
Digital transformation — the process by which organizations modernize themselves to incorporate new digital technologies throughout their entire business — is becoming necessary to survive.
Modern clients want instant service and support from the brands they deal with. They want brands that make themselves available to the client in a way that suits the client, not in a way that suits the organization. And they want this experience to be offered seamlessly across every channel. In order for an organization to meet these needs and wants of the consumer, it is absolutely necessary for companies to be innovative and to stay abreast of digital updates and changes. With this in mind, decision makers in businesses are emphasizing the need for digital transformation within their organizations.
With the rise of tools such as CRM, ERP, SRP and others, it has become necessary to integrate these tools in order to have a complete view of your customers data and to provide them with the level of service they expect. Using an ESB will ensure all of your tools talk to each other and give you the power to understand your customers and drive your business forwards.
Platform Challenges
A typical integration project has the following phases:
- Project planning and conceptual solution definition (business review, project scoping and planning)
- Detailed analysis and design of the solution (architecture and design, development environment setup)
- Development and testing of the solution (Development, testing, rollout planning, production environment setup)
- Go live to production (Rollout preparation, go live)
Most of the challenges that may arise come from over-focusing on the development (phase c), compared to project planning analysis and design (a and b). While this is common on all IT projects, it is essential for enterprise integration.
Technical challenges can be grouped to the following categories:
Incomplete or Not Well-Defined Analysis
Business people should know exactly what they are looking for in an integration project, and what the deliverables will be. The delivered solution may include processes for several departments, so the project team should have an overall knowledge of business processes.
This may also apply for IT people, since applications and systems involved are numerous and may run on different platforms and heterogeneous environments. In terms of technical analysis and design and project management, this makes the composition of individual systems and application into a composite one, difficult to manage. Additionally high consulting rates often prevent external support.
Not open platforms
There are systems that provide limited interface or non-standard capabilities. Other systems require a license to upgrade and extra costs to customize. In such cases, you might need a custom development (from integration platform side) solution, resulting in extra effort, time and cost.
If a system is unsupported, closed and does not have an interface, you should dispose of it.
Vendor management
Clear vendor management is critical for an Enterprise Service Bus Integration project when:
- Vendors cannot make modifications to application for several reasons, or require a new license.
- The result of a requested modification may not fulfil time specifications.
- Arguments and conflicts regarding areas of responsibility may arise.
- Application vendor may not support custom modifications in future upgrades.
In these cases, you must strictly define your requirements, responsibilities, milestones, and deliverables within your vendor agreement.
Performance, Sizing and High Availability
You should know your performance objectives upfront. Some of things you should consider include:
- What is the most important factor, number of supported concurrent users, response time or system throughput?
- Is a stress test required?
- How can bottlenecks or poor performance be measured?
As a general guide to face performance issues:
- In design, components that talk frequently should be placed in the same network.
- If possible, workload should be distributed.
- Bottlenecks should be located during tests.
- Focus on processes that take 80%+ of processing power.
- Perform step by step changes. Measure impact before application.
- Keep historical logs.
Scalability is the ability of a system to handle increased workload. A basic question to ask in the first step is: Is vertical or horizontal scalability needed?
The first is the quick, less expensive solution (e.g. increase memory in a server). The second is the long term: adding a new server in a clustered environment, implementing an external load balancer, etc. Usually, a combination of both is applicable.
Every business that runs an enterprise service bus integration platform wants their integration platform to be available as much as possible. Since this strongly relates to cost and performance, you should figure out how much downtime your business can tolerate prior to design and implementation.
Transaction Processing
Transactional integrity: while it is very common for multiple integration services to execute automatically, we require either an “all complete” or “all no complete”. (Most RDBMs and ERPs provide such functionality)
Most integration platforms provide transactional processing capabilities with some additionally guaranteeing the processing of documents.
Setting Up Test Environments. Incomplete Tests
Setting up integration project test environments requires duplicating (simulating) your production ones, including all systems and applications involved. Additionally, it requires that you establish all network connections between test components.
This may prove difficult in some cases:
- Integration between real time applications or applications that require hardware resources not available (e.g. control systems)
- Integration between cloud applications or integration between applications that may require extra licensing for implementing a test environment
Usually, you perform tests in batches on specific dates -according to the project plan-, with the intermediate time dedicated to corrections and improvements. This requires management effort for not having any incomplete tests.
Security
Since data can come from different applications, you need data access control in your integration platform when that Integration platform has an end user component.
Data mapping
If some of your applications include entity information, data mapping becomes a major priority when you integrate these applications.
Usually, you resolve this by applying the “canonical” model, a defined superset of the entity’s fields and attributes that the integration endpoints should follow.
Monitoring Tools are not Implemented
You often find integration management software missing from your final solution, usually for cost reasons.
Management and monitoring tools provide information to the end user, regarding integration executions, execution statuses, and timestamps, which are often valuable in troubleshooting
In conclusion, in an enterprise application and process integration project based on enterprise service bus architecture, your biggest challenge is to focus on analysis and design -both operational and technical. This is because these projects often deal with different applications, technologies and processes that make these projects complex.
Contact us today to find out for yourself why our customers praise the effectiveness of our product at improving their IT networks and systems.
Leave a Comment