Enterprise Service Bus (ESB): To Adapt(er) or Not to Adapt(er)

One of the most commonly touted features of commercial Enterprise Service Bus products (ESB’s) is the out-of-box adapters for other COTS products (eg. SAP, PeopleSoft, Siebel). But organizations who become dependent on these adapters and take a “use ‘em if you’ve got ‘em” approach inevitably find that the implementations are a lot more complex than advertised. This is because the decision on whether or not to use adapters should be driven by the alignment of the organization’s skillsets and support structure, not the perceived simplicity in using out-of-box components.

I’ve created a simple decision tree to help clients determine whether they’re better off using the adapters provided by their ESB vendor or whether they should consider some alternative method of integrating with the ESB:

  1. Is the integrating system a data component with no business logic layer and with no built-in ability to be exposed as a web service (eg. Database, LDAP, MQueue)?

    Yes – Use the provided adapters. There’s no point adding an additional layer of complexity when simple data integration is very mature among all the leading ESB products. Also ask yourself if you’re doing data mappings and translations within the adapters? Whether there will be a DBA on the ESB support team or not. ESB’s are excellent at handling simple data translations (ie. Field-to-field). Any complex data translations such as multi-table selects and data merges should be handled in the data tier where you have DBA teams who are equipped to manage them.

    No – Go to question 2.

  2. Does the integrating system have the built-in ability to generate a web service or ESB-compatible API?

    Yes – Go to question 3.

    No – Use the provided adapters. The whole point of SOA is to leverage existing assets as much as possible and to avoid building new custom code for integration. Putting in a custom coded service layer between the ESB and the target defeats the objective.

  3. Will the support team for the integrating system be cross-trained on configuring the ESB adapters?

    Yes – Use the provided adapters. The support team of the integrating system will ensure that the adapters and reconfigured as necessary based on any changes in the application in question.

    No – Expose web services and/or API’s on the integrating system. The support team for that system are the experts on that application. They know the data model and the implications of seemingly inconsequential details on the business logic in the backend. I’ve seen many an adapter implementation go off the rails because the team implementing the ESB simply are not experts on every system they’re connecting to. Let the team with the expertise manage the abstraction and register their endpoint on the ESB so you can still take advantage of features like version management and BAM.

Next time you’re opening the box to your shiny new ESB, resist the urge to take all the adapters out and deploy them everywhere possible. Ask yourself in the long run, who’s going to own the integration point? And whether their lives will be made easier with the adapter or without? An IT landscape that is well supported is much more important than an IT landscape that matches exactly to the vendor’s product sheet.

  • Shan
  • Gu

Agile Governance in SOA

I know, at first glance it looks like the cramming together of 3 loosely defined and oft-debated buzzwords into the title, but got your attention didn’t it? So if you will bear with me for a few minutes, I will explain how applying Agile to a much more abstract of an activity such as building an SOA Governance Framework works.

SOA Governance is a fluid concept, with many differing, complementary, and overlapping point of views. Is it a framework focused on lifecycle management of service assets? How does it integrate with existing governance frameworks around other IT components such as infrastructure, database, packaged applications? Or should it be focused around the monitoring and management of business capabilities. The discussion could go on forever.

Because of this fluidity, I’ve seen two major problems come up again and again with clients when it comes to creating an SOA Governance Framework:

  1. The client has invested in a shiny new SOA Governance Framework complete with templates, registries and repositories, architectural review boards, the works. It takes a year to roll out. And the moment it’s done, people start finding scenarios that don’t fit within the framework; do not apply it consistently; or refuse to use it altogether.
  2. Various architects within the client organization can’t agree on the scope, focus, or business case around such an ambiguous concept as SOA Governance, so it keeps on getting pushed off.

The reason why the first problem occurs is that organizations are treating Governance like a traditional application build project. You need to define it all up front and build it once. And that can only lead to an SOA Governance Framework, or any Governance Framework for that matter, which is out-of-date before it’s even delivered. Governance needs to constantly evolve to adapt to the changes in an organization. Is the organization moving from being IT focused to being business focused? Is there an increased uptake in COTS implementations over custom development? Is the organization moving to an offshore support model? All of these changes in the organization will impact how the services should be governed. In short, the way the SOA Governance Framework is developed and managed over time needs to be flexible and adaptable to change. Sounds like Agile, doesn’t it?

The same general mentality also leads to the second problem: the idea that the SOA Governance Framework has to be perfect and all-encompassing the first time around. The fact of the matter is that instead of being stuck in analysis paralysis, we’re always better off with having something rather than nothing. Even an imperfectly managed SOA landscape is better than a completely unmanaged landscape. One of the key tenets of SOA is to enable continuous improvement of business capabilities and efficiency by building it out in loosely coupled modules. So why can’t we implement a SOA Governance Framework by starting small and building up additional components as we learn more about the landscape and become more experienced in working with it. Isn’t it better to solve a real problem that we encounter than to create a solution in anticipation of a problem? Once again, sounds a bit like Agile, doesn’t it?

That was a long preamble. So here’s our approach:

  1. Get the organization to agree on a core set of SOA Governance deliverables, creating a baseline of the framework. In Agile terms, this list is our backlog and includes:
    – a Service Definition template
    – a Service Contract template
    – a Taxonomy
    – a Service Registry and Repository
    – and some kind of org chart from at least a support perspective
  2. For each deliverable defined in the backlog, analyze whether something similar already exists in the organization today or needs to be created. Group similar/relevant deliverables together (ie. Service Definition template with Service Contract template). Implement it and deliver it to the organization. These are our sprints.
  3. While operating our SOA landscape, constantly be on the lookout for areas to improve. Did we have any services that didn’t fit the definition template? Did we have a project that identified inefficiencies within the architectural review process? Is the service inventory becoming too cumbersome to manage? Are people adhering to the change management process and if not, why? Was there a re-org that requires us to rethink the key roles and responsibilities? Frame these into deliverables and put them into the backlog.

The key to success here is continuously looking at the applicability and appropriateness of the framework and keep it aligned to the rest of the organization. If the users of the framework start to feel like they have to shoehorn something into a structure that no longer works for them, they will cease to use it. So don’t fall into that trap and let’s apply some agility to our SOA Governance initiatives.

  • Shan
  • Gu