SOA anti-pattern: Transactional Integration
Join the DZone community and get the full member experience.
Join For Freetransactional integration
it all starts with a business requirement – as it
always should. we have an ordering system (say the same one from the
knot anti-pattern) and the business says they only want to confirm an
order to the user if the item is already secured for that order in the
stock. from the technical point of view we have 2 separate services –
one handles orders the other handles the stock – now what?
figure 10.1 a vanilla ordering scenario. an ordering service needs to confirm item in stock before confirming order for customer.
this sounds like a text book case for using
transactions but in reality it isn’t. i am going to explain why in a
short while but before we go there let’s do a (very) short recap on
transactions and distributed transactions.
transactions basically build on four basic tenets:
atomicity, consistency, isolation and durability (acid):
- atomic – “all or nothing” meaning that once a transaction ends the state is either completely done (commit) or undone (abort)
- consistent – the actions included in the transaction are done together so the state is kept consistent. if you were to remove something from stock and add it to shipment in the same transaction you won’t have a situation where the item was removed from stock and not added to shipping list
- isolated – while the transaction is in progress, logic which is not with part of the transaction will not see the world in its inconsistent form
- durable – the consequences of transaction are saved to persistent storage so that they are available after a system restart
the simplest way for transactions is what is known as pessimistic locking. in this case a writer can only write if no one else is reading or writing and a reader can only read if no one else is writing (for a specific piece or block of data). in order to ensure acidness you need to write the data twice; once where you want it to end up and a second to a log file. this double bookkeeping ensures that if a crash occurs before the transaction was finalized (committed or aborted) you can check to see that both copies match and if not either (re)apply the log or rollback the data.
unfortunately pessimistic locks rarely work in real
life scenarios so more advanced ways of locking and still maintaining
acidness were developed. however, all the mechanisms holds resources for
the transactions and all locking mechanism build on the assumption that
the time spent inside the transaction is short.
the plot thickens further, when it comes to
distributed transactions. now we have at least two transactional
resources and not only do each of them has to handle the transaction we
also need to coordinate the state between them – since if one commits
the transaction and the other rolls it back the overall transaction is
incomplete. still computer scientist where smart enough to come up
with several solutions to achieving distributed consensus and gave us
two-phase commits, three-phase commits, paxos commits and whatnot. case
closed we can use transactions in soa and life is beautiful.
or is it?
consequences
well first off the transactions, even the
distributed ones are not a problem in themselves. for instance chapter 2
introduced the transactional service pattern to allow handling incoming
messages in a reliable manner. the problems begin when the transaction
scope involve more than one service or in other words:
transactional integration is an anti-pattern where transactions extend across services boundaries (i.e. not isolated inside services)
so what sorts of problems can transactional
integration introduce to your soa? quite a few actually, with the main
three being performance problems, security threats and rigidity let’s
take a look at them one by one.
with all the goodness transactions does it also
introduce temporal coupling i.e. the need for all the involved actions
to finalize on or about the same time. even if the locks held while the
transaction enfolds are permissive (optimistic), the coordination that
is needed to ensure consistency needs to be synchronized. when you
develop a non-soa you may be able to take all the performance
considerations at design time and make sure the system behaves. i’d
still say distributed transactions are not highly recommended even then,
since the rigidity of the consistency needed when trying to achieve a
distributed consensus can still mean holding locks for a long time in
cases of partial failures.
the situation is much worse in an soa since each
service can and will evolve independently both in terms of deployment
and functionality. for instance, what will happen when the stock
service moves to another datacenter (e.g. ported to the cloud). what if
the designers of the stock service decide that when the stock level hits
a thresh-hold they want to automatically order new supplies and they
want that in a transaction – now you cannot secure an item in the stock
until new supplies are ordered. all of a sudden our transaction expanded
and now includes the ordering service, the stock service and a
supplier’ service(s). so one risk is that designers of services
participating in our transaction extend the transaction to handle
business rules they need to comply with.
another risk highlighted by the above mentioned scenario is related
to security. in the example we added the suppliers’ services into our
transaction. this means that we now run the risk that external systems
will now hold locks on our system, either maliciously or by neglect,
effectively creating a denial of service scenario on our services.
service boundary, its edge, should also be a trust boundary,
externalizing transactions to third parties might be far-fetched but
externalizing it to other teams within the organization which work on
their own services with their own priorities is not, the same risk
applies there.
the last risk we can highlight with this example is connected
directly to the knot anti-pattern (which is one reason the same sample
scenario is used) – having transactions between services increases the
coupling between them and increased coupling increases the risk of
ending up with a knot, which effectively kills soa.
one can argue that most of even all of these are
hypothetical situations and that when we design the soa we can take the
real constraints into consideration and plan for them – isn’t that why
we have enterprise architects for? though the scenarios are
over-simplified to illustrate the problems in a clear-cut way, real life
scenarios manifest the same problem in subtler ways. the main point is
that evolvability and flexibility are the hallmark of soa – that’s why
we want an soa in the first place – so that we can evolve the it of the
organization to better match the changing needs of the business. the
end result is that regardless of how we plan it out on the onset, in the
long term it is hard to control who participate in the transactions
which means that adding distributed transactions to the mix is an
accident waiting to happen.
causes
the main reason transactional integration happens
was already mentioned above, when we start out and design our soa we
have a relatively good grasp on the enterprise business. again, the
problem is that an soa solution is not static – if it is than it is
probably a needless overhead to architect it as an soa anyway.
even if you do have a good initial understanding of
the business flows, that understanding can deteriorate pretty fast. it
isn’t just that requirements change over time – an even greater force of
change is getting deeper understanding as to what is exactly needed. to
side track a little, the pragmatic way to implement an enterprise-wide
soa is not to do a multi-month (if not multi-year) project just to
documents and design the overall architecture and services and only then
begin the transition to the new architecture. doing that is like
sending a message that new things should just sit there and wait until
you’d be ready – i am yet to see a business that can afford that. no,
the more realistic and cost-effective way is to do some upfront design
but also begin developing real services and work them into the existing
software portfolio. this is sort of like building a new intersection
where you also have to build detours, keep some of the lanes open -
anything to keep the traffic going. when you work on an soa in this
manner the rework, the growing understanding of the business and the
requirements changes means you can expect a lot of evolution to happen,
as mentioned above, transactional integration will make that unlikely or
very hard.
other forces pushing to the transactional
integration anti-pattern are the marketing organization of technology
vendors. that’s an odd statement so let me explain. technology vendors
provide, well, er technology. thus when a vendor ships a technology
frameworks, it is usually geared to answer a broad variety of needs.
take for example, microsoft’s windows communication foundation (wcf)
which is a unified infrastructure for remote communications between
components. wcf offers message based communications along support for
names-pipes; it is built to replace rpc technologies like remoting and
it supports soap (ws*) web services, some support for rest style
services and what not – yet wcf is by and large is marketed as an “soa
foundation”. this isn’t to say you can’t use wcf for soa but it does a
lot more, it also does transactions… other vendors follow the same path,
whenever there’s a new buzzword, their marketing organizations take
whatever technology they currently have and slap that on it. the end
result of that is a lot of confusion in regards to what right and what
is not. the use of transactions for cross service integration is,
unfortunately, just one sample of this effect.
well, if transactions are not the way to go, what
can we do instead?
refactoring
there are several options to get around the problem
either using orchestration or sagas, inversion communications pattern
and similar means to achieve eventual consistency.
but wait, what exactly is the problem we’re trying
to solve? we are trying to achieve distributed consensus and
consistency in the data and business picture as seen by several
services. let’s take a quick look at the business scenario presented
above. we have an ordering system and the business says they only want
to confirm an order to the user if the item is already secured for that
order in the stock.
one way to solve this would be to externalize both
the transaction scope and the business flow to an orchestration engine
(see orchestration pattern in chapter 6). the advantage over
transactions directed from within the services is that the orchestration
engine has the full picture both of the services involved (and their
various trust levels) and the flow of which service calls to which
service so there’s more control on who does what and when. still
services participating need to be transaction aware and need to retain
internal locks for external constraints so use this with caution.
another alternative is to use sagas (see saga
pattern in chapter 5), sagas basically means running a long running
interaction (where messages are related and belong to the same
“conversation) but without holding the same transactional guarantees as
acid transactions. for instance in case of a stock problem the ordering
service will have to do a compensating action to handle the error. in
order for this to work in a reasonable manner the services may need to
hold some data about the world such as a some data about stock levels so
it can receive a reasonable decision on its own.
sagas can be augmented by the inversion
communications pattern to make the services send events of their actions
and other events to subscribe to them to create choreography scenarios.
in our example the order service would publish that it has a new order
that needs handling and the stock service would listen on that. once the
stock secures the items it will publish an event that says that so the
ordering service can notify the customer that the order is ready (maybe
there would be additional steps like actually shipping the product etc.)
both sagas and the inversion communication pattern
actually implement an eventually consistent system – we basically relax
the temporal constraints on decision making by the various services.
this can, and usually does, translate into how the business works in
general. in our ordering example it may mean that it would be better to
send an additional notification to the customer that her order was
received, when the order service processed the order and sent the event
to that affect
known exceptions
i can’t think of a lot of soa solutions that would
benefit from cross service transactions. transactional integration is
usually a bad idea for most distributed systems anyway – from similar
reasons to the ones mentioned above.
a rare exception to this rule might be for a closed
solution (i.e. a system and not an organization) that is building on soa
principles. in a closed environment where everything is controlled it
might be possible to pull it off without suffering from the rigidity and
performance problems induced by transactional integration. even in
these rare cases it would still be preferable to control the transaction
scope outside of the service by using an orchestration engine. using
orchestration means that at least the scope of transactions and the
general flow of the business processes will be handled in the same
place.
i would still be wary of going down this path since
even closed control system tend to evolve over time so be forewarned.
a related anti-pattern, which bears some resemblance
to transactional integration, is the whitebox services anti-pattern. an
anti-pattern which occurs when services expose their innards too much.
Anti-pattern
Web Service
Integration
SOA
Published at DZone with permission of Arnon Rotem-gal-oz, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Trending
-
Integrating AWS With Salesforce Using Terraform
-
Getting Started With the YugabyteDB Managed REST API
-
Cucumber Selenium Tutorial: A Comprehensive Guide With Examples and Best Practices
-
5 Key Concepts for MQTT Broker in Sparkplug Specification
Comments