Processing Elements for Enterprise Integration Needs

In February, AdroitLogic released UltraESB-X, a completely redesigned ESB which is developed by the very same team who developed the UltraESB six years ago. UltraESB-X is developed in line with UltraStudio, a user-friendly graphical tool to easily create, manage, and test integration flows. From the user’s point of view, UltraESB-X has two new concepts, namely connectors and processing elements.

What Is a Processing Element?

A processing element is an element which performs a specific processing requirement or business logic on the message. It can be one of a wide range of operations, such as filtering using conditional branching, transformation, flow control, scope handling such as a transactional scope, and many more. Usually, processing elements in the UltraStudio editor are represented by an icon with a blue background, except for a few exceptions. One of those exceptions is the scope processing element family, which uses a purple background, and another set of special flow control processing elements, which uses a reddish background.The structure of a processing element is as shown in the above image. There is an In Port (gray) which takes a message in, then Out Ports (light yellow) which send a processed message out, and finally, there is the On Exception Port (red) which is used to forward the message in case an exception occurs during the processing logic of the element. UltraESB-X ships with dozens of built-in processing elements which covers a wide range of integration processing requirements and more processing elements are continuously added with each release cycle. The current list of processing elements can be browsed through the developer portal of AdroitLogic here. Apart from the inbuilt processing elements, UltraStudio allows users to write their own custom processing elements, as explained in the documentation.

What Is a Connector?

A connector is an element which is used to receive or send messages between the ESB and an external system via a given protocol, such as HTTP/S, JMS, files, databases, or web services. Connectors are categorized into two categories, ingress and egress connectors, based on the message flow. Ingress connectors, represented by an icon with a green background, are components dedicated for receiving messages into the integration flow, and egress connectors, represented by an icon with an orange background, are the connectors for sending out messages from the integration flow.

Ingress Connector

The structure of an ingress connector is as shown in the above diagram. There is a Processor Port (light yellow) which is basically an out port used to forward the receiving message into a processing element. Next, there is the On Exception Port (red) which will be used to forward the message in case of an exception occurs during the message receiving process. Then, an optional In Port (gray) is presented depending on the message exchange pattern of the protocol supported by the connector. For example, the In Port is there with the HTTP connectors, which are 2-way (request-response MEP) but it is not there in one-way connectors such as JMS, File, SFTP, etc.

Egress Connector

Let's move on to the egress connectors, the structure of which is shown above. The In Port (gray) takes in a message to be sent out via the connector. Again, depending on the message exchange pattern of the protocol used by the connector, there may or may not be a Response Processor Port (light yellow). For example, the HTTP connectors again have the Response Processor Port (as it is 2-way) but most of the other one-way connectors do not have that port. Then, the On Exception Port is used to forward the message in case of a failure in the send-out process. Additionally, egress connectors have a port called the Connector Operation Port, which is a special kind of a port which accepts an operation specific to the connector where special logic will be applied in message preparation before the send-out process. The following sections of this article will focus on introducing the use of a few processing elements leading into a complete sample scenario at the final stage.

XML Validation - XSD Validator Processing Element

XML Validation is the process of checking a document written in XML to confirm that it is both well-formed and also "valid" in that it follows a defined structure. Generally, this structure is defined using an XSD (XML Schema Definition) file. The XSD Validator processing element is focusing on this task of validating an XML payload against one or more given schemas.

The diagram above shows the port structure of the XSD Validator element. Notice the On Successful Validation Port and On Failed Validation Port, which will be used to forward the message when the XML validation results in a success or a failure respectively. The following diagram shows a very basic flow to depict the use of the XSD Validator element, captured from AdroitLogic UltraStudio, the integration flow designer. In this flow, there is an HTTP ingress connector which takes in a message and then passes it through an XSD Validator and forwards the message into two different HTTP egress connectors depending on the validation result.

The next diagram below shows the configuration view of the validator element in UltraStudio.

XML to JSON Conversion - XML to JSON Transformer

The XML To JSON Transformer processing element handles conversion of XML payloads to JSON. Similarly, UltraESB-X includes built-in processing elements covering most of the common payload conversion requirements.

If-Else Condition Evaluation - Conditions Evaluator

When implementing business logic, the requirement for conditional operators is inevitable. The Conditions Evaluator processing element is the if-then-else implementation of UltraESB-X. Following is a sample XML configuration of the Conditions Evaluator processing element. UltraStudio generates this configuration when we configure the processing element through the user interface.

<bean id="0bd5ecf6-6b51-7733-49f9-48b4feaac455" class="org.adroitlogic.x.processor.flowControl.ConditionsEvaluator">
  <property name="ifOutPort" ref="7e0fe076-3d8c-208b-76b9-bfb818b77429"/>
  <property name="elseOutPort" ref="a92a2828-b20b-3fa1-0dd3-fdde9812faec"/>
  <property name="predicateType" value="VARIABLE"/>
  <property name="varKey" value="id_one"/>
  <property name="varType" value="String"/>
  <property name="predicateFunction" value="EQUALS"/>
  <property name="matchingValue" value="3"/>
</bean>

As you can see, there are several configuration parameters. One is predicate type, which basically defines the type on which the evaluation is done. We can evaluate on a header, a scope variable, a message property, or on the payload itself. Upon selecting the predicate type, the name of the variable of the selected type must be given. Then we can select the type, which could be one of Boolean, Integer, Long, Float, Double, or String. Next, the predicate function should be selected, and it could be one of EQUALS, MATCH, CONTAINS, NULL, NOT NULL, or EXISTS.

EQUALS evaluates Java Object.equals() operations. MATCH evaluates String.matches() operations. CONTAINS evaluates String.contains() functions. NOT_NULL and EXISTS simply evaluate the existence of such values where NULL evaluates the opposite. Predicate functions EQUALS, MATCH, and CONTAINS require a matching value to be configured.

Apart from the Conditions Evaluator, UltraESB-X also consists of processing elements that support Switch-Case, For-Loop, and While-Loop as well.

JSON Path Extraction - JSON Path Extractor

The JSON Path Extractor processing element allows us to extract a JSON element and store it as a scope variable easily by providing a JSON path to the configuration. We can use that extracted element to make decisions such as using the extracted value in a conditions evaluator.

Failover and Load Balancing - Egress Load Balancer

Load balancing and failover are among the most sought after features in an ESB. In the context of an ESB, load balancing means distribution of workloads across multiple endpoints, and failover means basically attempting to send to redundant or alternative end points in case of failure. UltraESB-X supports this requirements through the Egress Load Balancer processing element. A sample XML configuration of the Egress Load Balancer processing element is shown below.

<bean id="7b7f6ccc-acba-c2e6-5125-e9aa49e5e944" class="org.adroitlogic.x.processor.flowControl.lb.EgressLoadBalancer">
  <property name="endpointList">
      <list>
          <ref bean="dcec8a87-3922-d62d-0da7-dc87e52ccae6"/>
          <ref bean="36faacbf-f4fa-723b-3170-e5c84e7ab262"/>
          <ref bean="14defbf2-cb22-24c6-d717-17ad9b091182"/>
      </list>
  </property>
  <property name="loadBalancingAlgorithm" value="round-robin"/>
  <property name="retryAttempts" value="3"/>
  <property name="safeToRetryErrorCodes" value="E0001"/>
</bean>

When configuring the Egress Load Balancer through the UI, there are three main parameters. One is the load balancing algorithm. Egress Load Balancer supports a list of load balancing algorithms including round robin, round robin with failover, failover, weighted, weighted with failover, random, and random with failover. Retry attempts parameters must be configured when the LB algorithm is weighted with failover or random with failover. The safe to retry error codes parameter accepts a comma separated list of error codes and is only required when the LB algorithm is a failover one. Using this parameter, we basically say the processing element that it can proceed to retry if the error code is one on the given list.

Getting All The Pieces Together

Here is our complete sample scenario using the above-mentioned elements. Note that connections of handling exception paths (On Exception Port connections) are omitted for simplicity.

Sample Input

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  <soapenv:Header><routing xmlns="http://someuri">xadmin;server1;community#1.0##</routing></soapenv:Header>
  <soapenv:Body>
      <m:buyStocks xmlns:m="http://services.samples/xsd">
          <promiseList>
              <id>1</id>
              <orderId>1234-5678-2345-5467-SH1</orderId>
              <comments>grade-1 quality</comments>
          </promiseList>
          <promiseList>
              <id>2</id>
              <orderId>2345-1222-6778-9999-SH2</orderId>
              <comments>grade-2 quality</comments>
          </promiseList>
          <promiseList>
              <id>3</id>
              <orderId>4568-7678-9876-1211-SH2</orderId>
              <comments>grade-3 quality</comments>
          </promiseList>
      </m:buyStocks>
  </soapenv:Body>
</soapenv:Envelope>

Output From XML to JSON Transformer

{
  "Body":{
    "buyStocks":{
      "promiseList":{
        "id":"1",
        "orderId":"1234-5678-2345-5467-SH1",
        "comments":"grade-1 quality"
      },
      "promiseList":{
        "id":"2",
        "orderId":"2345-1222-6778-9999-SH2",
        "comments":"grade-2 quality"
      },
      "promiseList":{
        "id":"3",
        "orderId":"4568-7678-9876-1211-SH2",
        "comments":"grade-3 quality"
      }
    }
  }
}

Message Flow Path

Highlighting the message flow path is an important feature of UltraStudio which can be more than handy in development and debugging. The following diagram shows the message flow path of the sample scenario for a single request.

Inspecting Message Details

Another useful feature of UltraStudio is the ability to inspect message details after a message has been sent simply by clicking the message icon on the message path, as shown below.

Following are the message details of the results of the XML to JSON Transformer: