The need for an SOA Ecosystem

By Steve Craggs

Attracted by the flexibility, agility and cost effectiveness of SOA, with its ability to meld disparate systems and technologies into a collection of reusable resources many businesses are getting involved. But as they do so, there are many traps along the way, and one in particular is the failure to grasp the concept of and the need for the SOA ecosystem. Lack of an appropriate ecosystem is one of the major reasons for the failure of SOA to deliver on business and IT expectations.

The problem stems partially from the fact that companies often treat SOA as just another technology, without giving it the thought it deserves. This issue is exacerbated by the insidious nature of web services, the most common basis for deploying SOAs. Many tools exist that make it easy to create a web service and publish it for others to use, and as a result it seems only logical to start turning individual pieces of IT functionality into web services as rapidly as possible – after all, once a piece of code is made into a web service it can be reused across the enterprise, so that must be a good thing from a reusability point of view at least, mustn’t it?

In reality, however, this road frequently leads to disaster. Typically, companies experience some success at the beginning, but as the number of web services grows more and more rapidly, levels of reuse fail to increase, operational service levels are impacted, systems become less rather than more agile and business risk grows. In order to avoid these problems, companies are realizing that they need to consider SOA more carefully, giving thought to the need for an SOA ecosystem that can ensure higher rates of success. The following diagram depicts a broad-brush overview of the main components in this SOA ecosystem concept.

First, a registry will be required to store information on the services available within the SOA and their attributes. As already mentioned, most SOA implementations today use web services as the mechanism for making components accessible to all, but without a registry it rapidly becomes impossible to keep track of these services. This, in turn, reduces the likelihood of component reuse because programmers have no way to discover them. Also, the attributes recorded in the registry provide important information about the nature of the services and how they fit with corporate policy. So, for example, a service might be marked as sensitive enough to require encryption of data flows between the components, or it may be associated with a particular service-level agreement.

The communications layer represents the wiring to connect different technologies and systems together, while the gateways and adapters reflect the need for software to enable all components to be viewed as an SOA service. Typically the communications layer may be a JMS-based messaging bus or some other form of middleware. The gateways and adapters layer includes tools for purposes such as making legacy applications and packages appear as services. Mediation services are required to carry out various support functions such as data format transformations, rules-based routing and orchestration, data enrichment and logging.

One common approach to supplying much of this functionality is to implement an enterprise service bus (ESB). ESBs deliver the communications and mediation services, and also partially address the gateways and adapters requirement for the most common options. For non-standard adapters, however, additional tools will usually be required.

Wrapped around these core SOA functions are the development and operational toolsets. On the development side, it is important to appreciate that tools will be needed both to develop, configure and deploy the SOA infrastructure itself, and also to develop and deploy the callable services. So, a tool will be needed to support the creation of mediations, for example, as well as to handle the process of turning a legacy application into a service and making that service available to all programs within the scope of the SOA.

One particularly important area here is the need for tools to manage the registry described earlier. Since reuse is a major goal of SOA, it becomes essential to ensure that developers can easily find out about existing services, so that they don’t end up repeating development activities. But more than this, it is important to make this process as easy as possible. Developers have a natural aversion to reuse – they prefer the challenge of developing their own software, given the chance. If the registry search and browse tools are not really simple and intuitive, developers may use this as an excuse to give up on reuse and do the work themselves.

The area of operations is impacted considerably by the adoption of SOA architecture. In an SOA environment, services and components will be blended together from all over the enterprise and beyond, linked together by the communications layer of the ecosystem and with a flow that is orchestrated by the mediation layer. This leads to all sorts of issues in terms of understanding what is happening where and for whom. This immediately throws up questions about security and problem determination. It is imperative that tools are made available to ensure that only authorized access to services is allowed, and to trace, detect and resolve problems on an end-to-end basis.

Another important on the operational front is visibility of activity through the SOA. This is not only for the purposes of monitoring and tracking the system performance and responsiveness, but also to drive more value from the investment in a business-oriented architecture. Since SOA is all about creating business services representing real business activities, this presents the ideal opportunity to exert a stronger level of business governance and management on the IT implementation. Individual business services could be monitored, using such technologies as business activity monitoring (BAM) to measure performance against specified service-level agreements and flagging out of line situations. Dashboards will be able to give an increasingly business-oriented view of operational performance, with the ability to track corporate key performance indicators (KPIs) in real time. Of course, this only deals with part of the governance issue – overall SOA governance will include corporate policy, process and organizational implications too.

So, in summary, anyone embarking on the long SOA journey should take care to ensure that the elements that make up a proper SOA ecosystem are in place before getting too far down the road. If this issue is not addressed, then early success will quickly be reversed as the SOA expands. But as long as the SOA ecosystem is developed an early stage, it will continue to be an attractive route for many companies that really does promise to generate additional value from existing IT investments.

Steve Craggs is founder and president of Saint Consulting Limited and European chairman of the Integration Consortium.