API Implementation: Using Idempotent Filter to Prevent Message Twinning

[This article was written by Adrian Hsieh.]

 When it comes to API implementation, message “twinning” is one of those annoying pitfalls that in the worst case scenario, can lead to major problems downstream, such as duplicate orders being submitted etc. However, the client side is often not equipped to prevent duplicate messages from being sent, especially for javascript based user interface calling REST APIs, where user actions such as fat-fingering and unintentional double-clicking are virtually impossible to control.

Fortunately, the idempotent filter comes standard with Mule ESB, thus greatly simplifying the tasks of preventing message twinning for developers who are implementing REST APIs. The example here will illustrate how the filter can be used in an APIKit project. As illustrated in the application diagram below, first the message must be serialized so that the payload can be properly evaluated for comparison purpose. In this case a Byte-Array-to-String transformer is used immediately after the HTTP listener. Then, an idempotent filter is added in front of the APIkit Router for uniqueness check.

Note the configuration of the idempotent filter. For APIs, the id expression has to be a combination of both the payload as well as the operation being invoked, since the same data can be valid for different operations. Therefore, we used an id expression that concatenates the request path (operation) with the payload.

<idempotent-message-filter idExpression="#[message.inboundProperties['http.request.path']+message.payload]" doc:name="Idempotent Message">
  <in-memory-store entryTTL="120000" expirationInterval="1800000"/>
</idempotent-message-filter>

Also keep in mind that for the object store configuration, the time-to-live (TTL) parameter unit is in millisecond. TTL value should be set large enough to provide effective filtering but not too large to block legitimate messages from coming through. (For high availability (HA) implementation, choose a cluster-aware object store.)

Twinning is among the more popular activities of millennials hipsters. However, when it comes to API implementation, it can be a unwanted evil that should be exorcised. And for us Gen Xers with the benefits of wisdom from life experience, we also remember that you can have fun twinning when each member of the twin is as unique as it comes…