Over a million developers have joined DZone.

Immutable Pointers—A Pattern for Modular Design

Suppose we have a project where we use Lombok annotations. To use it we must change the compiler configuration for our project and enable annotation processing.

· Java Zone

Microservices! They are everywhere, or at least, the term is. When should you use a microservice architecture? What factors should be considered when making that decision? Do the benefits outweigh the costs? Why is everyone so excited about them, anyway?  Brought to you in partnership with IBM.

Modular design has a lot of benefits, including:

  • making it easier to predict the impacts from a change
  • helping developers work in parallel

But, it is much much much harder than people think.  For a start, the abstractions have to be at the right level.  Too high and they can become meaningless and too low and they cease to be abstractions, as they will end up having way too many dependencies ( efferent and afferent).

In a recent greenfield project, which was in the area of digital commerce, I was intent on achieving good modular design in the back end for the reasons outlined above and because the project had potential to grow into a platform ( more than likely microservices) that would be used by multiple teams. To achieve the modular design, after some white boarding the back end was split up into a bunch of conceptual components to achieve shopping functionality.  The core components are:

  • Shopping component - Core shopping functionality e.g. Shopping Cart management
  • User component - user management, names, addresses etc
  • Purchases - details about previous transactions
  • Merchant - Gateway to merchant API to get information about merchant products etc. 
  • Favourites - Similar to an Amazon wishlist

Now, every software project uses the word " component" differently. In this project, a component was strictly defined as something that did something useful and contained:

  • domain classes 
  • services
  • a database schema  (could be its own database, but the idea was to isolate it from any other components persistence) 
  • its own configuration
  • its own dedicated tests
  • its own exception codes domain 

The outside world could access a component via a bunch of ReSTful endpoints. 

Any component could be individually packaged, deployed etc.  Now, the astute out there will  be thinking,  " this sounds like microservices", well they almost were.  For this project,  some of them were co-located but they were architected so that deploying them out into  individual  deployed artifacts (and hence a microservices approach would be easy). 

Ok, to reiterate, the goal was to achieve a very clean modular design.  This meant, that I didn't want  any dependencies from one component's database scheme to another and for this blog post we are only going to focus on how that aspect of the modularity was achieved. 

Now, looking at the above components, it doesn't take long to see that isn't going to be so easy.  For example: 

  • A shopping cart (in the Shopping component) will have a reference to a User (in the User component)
  • A shopping cart item (Shopping component) will have a reference to a Product (Merchant component)
  • A Shopping cart (Shopping component) will have a reference to a shipping Address (User component

So the challenge of achieving modularity in the persistence tier should now be becoming more clear. References of some sort across components need to be persisted.  Immediately, any developer will ask,  " Wait a sec, if we just use foreign keys for these inter-schema references we get ACID and referential integrity for free!"  True. But then you are losing modularity and introducing coupling.  Say you want to move your products (in the Merchant Component) away from a relational database and use something like Elastic or Mongo DB instead - to leverage their searching capabilities.  That foreign key isn't so useful now is it?  

Ok, so first of all in looking for a solution here, I thought about all the references that were across  components to see if there was anything in common with them.  One thing that was obvious was that  they were generally all  immutable in nature.  For example:

  • When a Cart Item (Shopping component) points to a Product (Merchant component) it points to that product only.  It never changes to point to another product.  
  • When a Shopping Cart (Shopping component) points to a User (User component), it is also immutable.  My shopping cart is always mine, it never changes to be someone else.

So I was now starting to think about preferences:

  1. Avoid cross-component dependencies if you can (this should be kinda obvious)
  2. If you have to have them, strive for immutable references.

So, next up was to have a name for this type of relationship - which I was calling "Immutable pointer" in design and architecture documents. But, for actual code I needed something more succinct. The database schema was already using "id" for primary keys and "{name_other_relationship}_id" for foreign keys. So I decided all cross component relationships were named as the name of the entity being pointed to and "ref".  

So, some concrete examples:

  • userRef   (ShoppingCart pointing to the user)
  • productRef (CartItem pointing to the product)
  • shippingAddressRef (ShoppingCart pointing to the ShippingAddress)

This meant anytime anyone saw something like "xyzRef" in code, schema or logfiles they knew it was a cross-component reference. In case it wasn't obvious, Ref was an abbreviation for Reference.

Next up was to decide on the format for the actual Refs.  I took a bit of inspirational from that thing call the internet, which of course has similar concepts where abstractions:  websites of web pages contain immutable pointers to other abstractions: hyperlinks to web pages in other websites.

So similar to hyperlinks and URLs, the refs would follow a hierarchical namespacing format. Some good team input from senior technical member suggested to continue the inspiration from the Web and start the hierarchical names with cp://.  CP for Commerce Platform the name of the project. This was analogous to http://  I thought this was good idea, as it indicates that our platform generated the reference.  Again, this meant they stood out in logfiles etc and could be differentiated from any downstream components also using hierarchical type references but in a different context. 

The key point of the ref was that when generated it should of course be unique.  To achieve this  a mixture of database primary keys or something unique about the data (e.g. product SKUs were used).

So some examples: 

  • userRef -> cp://user/{uuid of user}
  • productRef -> cp://merchant/{uuid of merchant}/product/{sku of product}
  • cardRef -> cp://user/{uuid of card}/card/{uuid of card}

Always immutable?

As stated, the first preference was to avoid the cross-component reference. The second preference was to use the immutable pointer (ref pattern). However, what about an edge case where the cross-component reference could be mutable. Could this happen? Well, it could.  Easiest to explain by an example.   

Every shopping cart doesn't just have a User, it also has a selected shipping address where the contents purchased will be shipped to. In the domain model, the user's address lived in the User component.  But unlike the other cross-component references the shipping address could change. Consider your Amazon shopping cart.  Imagine you are on the checkout screen with your selected card and your selected address, but before you proceed to checkout you go into your user preferences and delete your card and address.  This project had to facilitate similar scenarios.  So the User deletes the address that a shopping card id pointing to what should happen?

So for inspiration for a solution here, we can look to the various NoSQL patterns and in particular one of the most popular is eventual consistency. What this says is that unlike ACID you don't  always need consistency straight away, all the time.  In certain cases, it is okay to allow inconsistency on the basis that the system is able to reconcile itself. 

So in this case:

  1. The shopping cart is pointing to a specific shopping address using an addressRef. 
  2. The user deletes that address by hitting a ReST endpoint in the user component.
  3. This means the shopping cart will point to an address that doesn't exist. The system is inconsistent.
  4. The next time the user reads the shopping cart, in the request handling the shopping component asks the user component if the address with this address ref sill exists and if it doesn't remove the pointer.  
  5. This system is now consistent

So with the architect hat on it is really important we get this all right. Otherwise, the goal  of modular design falls apart. 

In this case, it is worth reiterating the strategy one more time:

  1. Avoid cross-component references.  Doesn't matter how great your pattern is, if you have a lot of cross-component references,  it is more than likely you have got the component abstractions at too low a level.
  2. Favour immutability. In general, immutability means fewer code paths, fewer edge cases, and less complexity in code. 
  3. Eventual consistency.  

Software Architecture is about trade-offs, and finding the right balance.  In this case, it was the balance between clean modular design but not going overboard with it so that it become impossible to achieve. 

For anyone trying to do microservices, I would strongly recommend trying to master how you would do modular design in your architecture first.  If you can't do this, when you add in the complexity of the network things get very complicated. 

Discover how the Watson team is further developing SDKs in Java, Node.js, Python, iOS, and Android to access these services and make programming easy. Brought to you in partnership with IBM.

Topics:
design patterns

Published at DZone with permission of Alex Staveley, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}