Top 10 SOA Pitfalls: #6 - SOA does not solve complexity automatically

After discussing #7: Incorrect granularity of services , let's move on to #6.

In organizations data and functionality/processes are often fragmented, but are needed centrally. What are the causes of this fragmentation. Does a SOA solve this complexity automatically? Most companies start with a SOA and are confronted with this complexity during the implementation of the SOA.

First let’s take a look at the problem.

• Applications are often built using a silo model. The application contains a set of business functionality exposed through a user interface. The business functionality is intended for its own context without intentions of reuse.
• Applications or services have different views on an entity. For instance, an "amount" entity can include or exclude tax. Combining these different views into a single entity is difficult.
• Heterogeneous data environments with varied schema's contain redundant data elements.
• When a service is centrally used, data replication is applied in order to combine the different entities and fragmented data into one service.
• Organizations with multiple business lines, spread over multiple locations, have stored core data fragmented in databases/systems. In many cases the size of the IT portfolio has a relation to the scatter.

The real problem is the way the data is stored and how functionality/processes are organized. The SOA architecture style won’t change these problems automatically. In some cases a SOA can make it worse due to the complex integration.

The complexity is caused by the functionality and data fragmentation over different systems, processes and locations. In many cases the fragmentation of core data is caused by organizational growth (for instance mergers and acquisitions). In most cases these companies have a heterogeneous data environment with redundant data elements such as static customer information, common business data or common external data (for instance government, market, or statistical data). Due to the fragmented data, it is often hard to create a complete view of the core business data such as customer information. In portals however, a complete view is needed. An incomplete view can lead to an inconsistent user experience. This might introduce risks and in the long term could lead to being untrustworthy as a company. In order to create a complete view, different complex and inefficient mechanism are needed for accessing/updating the systems/databases. This can lead to inefficiency and higher cost due to the overhead in accessing/updating data in multiple databases using different mechanisms. The real problem is the way the data is stored and how functionality/processes are organized.

Some organizations introduce a SOA in order to reduce the complexity, but what they really do is introduce JBOWS (Just a bunch of web services) and don’t solve the complexity at all. The existing business processes and business functionality remain the same.

In order to reduce complexity the following things need to be organized:

• Eliminate redundant data by gradual reduction in the scatter of core data
• Use the right service granularity (see #7: Incorrect granularity of services )
• Move from the silo/application-based approach to a business process approach.
• Introduce a canonical data model for common data across the enterprise (something we will come back to later).
• Select the appropriate tools to manage and monitor the processes/services.

Complexity is not easy to reduce. SOA can help, but won’t solve the problem automatically. The root cause of the complexity is the way data, functionality and processes are organized. Solving these problems will really reduce complexity.

Next week, Vincent Partington will continue with pitfall #5.

Reference: Top 10 SOA Pitfalls: #6 - SOA does not solve complexity automatically at the Xebia blog