Post date: Jul 30, 2015 10:13:46 PM
I write this blog as interface lead on a large Customer Information System project. During the project my team designed and built over 100 interfaces between applications internal to the solution and 14 third party vendors.
Message queues and service buses are commonplace in enterprise technology architectures but they do not spell integration success and in many cases core principles will be key to working interfaces. I will share with you my lessons learned from my most recent project and key considerations when architecting a solution.
Understand Out of the Box Integration First - products from the same vendor may be built to integrate out of the box. Understand how this works prior to making any decisions around required tools or service buses especially, prior to installation. The team installing the software will follow the user guide which will detail default configurations and may not make full use of additional products purchased for the project.
Ask do we really need middleware (SB or MQ)? - there are good reasons to purchase a middleware product such as: enterprise SOA, message throttling, consolidated logging and monitoring, guaranteed delivery. However, from experience enterprise products offer these features without middleware and most interfaces are file based. Prior to deciding on a tool consider whether a combination of the core system and a batch scheduling tool cannot meet the requirement. A majority of the interfaces on my project were file based and used the batch tool for file transfers.
Architecture first, implementation second - interfaces almost define themselves after guiding principles and common patterns are defined.
Define a pattern (aligned to your infrastructure) for file based interfaces, real time interfaces and queue based interface - we had minor variations on these and ended with ~15 patterns.
Define archiving requirements and how they will be achieved within the infrastructure. For example, how long must payment files be stored after receipt from a vendor?
Define logging and alerting requirements and how they will be achieved within the infrastructure. For example, what should be logged and when should people be notified.
Keep business rules out of integration - integration should be transporting and routing of messages. Unless a business rule is required to route a message then all other logic should sit within the core applications. For example, payment eligibility rules should be within Customer Care and Billing and not OSB. The core system owns the data and the respective rules. (there are exceptions to this rule)
Prototype prior to full execution - this one goes without saying, prototype and test the build of each pattern prior to full implementation across all interfaces. Thorough testing during the prototype phase will greatly reduce defects in later phases.
Most importantly it all comes down to schedules, directories, certificates, file names, message formats and credentials. Agree these six details with your interfacing partners and the build team can make it work.
This was of course only my experience I would be interested in the experience of people on other projects.