One of the most overlooked functions of an ESB Designer toolset is the ability to do mapping & transformation of data between a source document and a target document. As much as Application Integration is about connecting two or more application interfaces together, the end goal is to provide a seamless way to synchronize data or at least view data as a single source of truth. In order to be able to do so, when one application’s data needs to be synchronized with another, a series of data augmentation / transformation steps need to be done. And that is the mapping & transformation function.
Going back to my first statement, I did mention that this is an often overlooked function primarily because customers tend to think that this is a very standard featureset amongst all integration middleware vendors. When evaluating ESBs across multiple vendors, customers need to spend a good amount of time in this topic since they may end up spending most of their time doing integration work in this area. Let us take a couple of examples and understand why ease of use is such a big deal with this function.
Let us assume we are mapping customer order information from a CRM system to an ERP system. There may or may not be a canonical based on your needs and how you have structured your document types. As a side note, it is highly advisable to use canonicals to make your modeling and mapping highly extensible and reusable. Back to the example, the CRM order document may have customer name as two fields – First Name and Last Name. If that is how it is represented within the ERP document as well, our map just became easier. Just drag and drop First Name in the source doc to the First Name in the target doc. Evaluation check time – some vendor tools do not even offer drag and drop capabilities to connect up the two documents. Imagine doing a mapping exercise for over 200+ fields without drag and drop capability!
Key features to understand when evaluating an ESB for this capability –
- Ability to understand a wide range of document formats (XML, flatfiles, CSV, EDI etc)
- Support for canonicals
- Drag-and-drop capability between source and target documents
- Pre-built transformation functions to implement common data transformations quickly
- Ability to create custom transformation services quickly
- Ability to create reusable maps / segment maps that can be repurposed across multiple maps
- Minimal or no code to do mapping / transformation
- Ability to parse complex document structures (such as nested or referenced documents)
- Ability to do looping over a collection of documents to extract and map data into another document
With that, I will leave you with one more pointer. Pick your most complex (yet often used) document type and use it for the evaluation. That will be a good reality check for you and your team. Adios!