Hystrix Part 1: Getting Started With Hystrix

What is Hystrix?

Hystrix is a latency and fault tolerance java library designed to isolate points of access to remote systems, services, and 3rd-party libraries in a distributed environment. It helps to stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.

How to Add Hystrix As Library to Your Java Application

Hystrix can be added as a Maven library dependency to your Java application by adding below dependency declaration into your pom.xml.

Code Listing 1.1 : Showing Hystrix library 's maven dependency declaration

<dependency>
 <groupId>com.netflix.hystrix</groupId>
 <artifactId>hystrix-core</artifactId>
 <version>1.3.18</version>
</dependency>

The above declaration should pull all its transitive dependencies and make the programmer life easier. To know more about how to work with a Maven-enabled Java project, please go to Apache Maven in 5 minutes.

Code Your First Java Generics Based Hystrix Command

Creating your first Hystrix command involves creating a subclass of the HystrixCommand<T> API.

Let's take an example to start with. We know that in a distributed enterprise application, it is quite common to see service-oriented architecture where one service calls another service. Enterprise portal service is an SSO login that integrates multiple remote services. Here, the enterprise portal service calls one such service called employee information service to fetch employee related information from an employee directory based on search terms.

For portal service, employee information service is a point of access to a remote system, which only creates a point of possible failure resulting in cascading failure for the portal service as well. To prevent such a scenario where the portal service continues to stay resilient despite employee information service being down is something we can achieve via hystrix.

The interaction between portal service and employee information service is depicted in the below diagram.

Diagram 1.1: Showing interaction between Portal Service and Employee Service

Let us assume that URL to access employee service is as shown below,

http://<domain>:<port>/app/employees?terms="alex"

Currently I have hosted the employee information service as a RESTful web service API. I have used the Jersey client library to access the RESTful web service. Client code to access the employee information service is as shown below without using Hystrix.

Code Listing 1.2: Showing Client side code snippet on fetching employee information for given search terms

Client client = Client.create();
WebResource webResource = client.resource("http://localhost:9999/app/employees");
webResource.queryParam("terms", params);
String employeesList = webResource.get(String.class);

Now, we know that Portal Service is bound to resiliency issues. If the employee information service goes down or service is not responding or a call times out, then these failures get cascaded to the portal service as well, which is not desirable. It would be better for the portal service to build resiliency around all outbound calls such as the employee information service, so that it can continue to serve the portal users need despite failures in some of the dependent remote services such as this one.

How to Make Portal Service Resilient to Failures on Remote Service Side?

The component that calls the remote employee information service should be made resilient by wrapping it in a Hystrix Command class so that you can define behavior on what should happen when the remote service crosses Service Level Agreement (SLA) on response time. Let us say there is an SLA for the employee information service to respond with necessary data within 2 seconds. If the remote employee information service fails to respond within 2 seconds, then what should the portal service should do? Should the portal service send a timeout request to the employee service or do something else, like sending responses from the cache? Response caching is one way to solve it. Another way would be asking the portal service to come back after some time. As long as the portal is responsive by wrapping outbound network calls like the remote employee service call, users are not frustrated.

The code listing 1.3 shows how to wrap a client code snippet in a Hystrix command. This should be simple as long as you have the basics of how to use Hystrix library in the right way.

Code Listing 1.3: Client code wrapped in HystrixCommand class to introduce resiliency around outbound network call.

public class EmployeeInformationCommand extends HystrixCommand<String> {

    public EmployeeInformationCommand() {
        super(HystrixCommandGroupKey.Factory.asKey("EmployeeServiceGroup"));
    }

    @Override
    protected String run() throws Exception {
        Client client = Client.create();
        WebResource webResource = client.resource("http://localhost:9999/app/employees");
        webResource.queryParam("terms", params);
        String employeesList = webResource.get(String.class);
        return employeesList;
    }
}

As shown in the above code snippet, create a class that extends HystrixCommand<T>, where T stands for generic return type. In our case, we return a String type object from the run() method. One has to move client-side code inside the run method as shown. It is the same exact code just executed inside another class, which provides much more capabilities when we do it.

If you have already observed EmployeeInformationCommand's default constructor, there is one important thing I am doing that is essential to running as a Hystrix Command. Gather all employee related commands under one group, 'EmployeeServiceGroup'. It is necessary to provide such logic a group name to run the code via Hystrix library.

How to Run Hystrix Command

Please keep in mind that calling a Hystrix command execution is a blocking call. There are many other ways to run it as well. This is just for starters.

Code Listing 1.4: Calling a Hystrix command

new EmployeeInformationCommand().execute();

As shown above, it is that simple. When the portal service wants to call a Hystrix-wrapped outbound call, it is just one line of code.

Peek Into Hystrix'S Runtime Configuration Per Command

Configuring a Hystrix command is quite simple, you just have to follow the convention on how you pass configuration key names per command.

For example, by default, a Hystrix command times out command execution if it doesn't complete its execution within 1 second (1000 milliseconds).

The default property name across all commands is:

Code Listing 1.5: default timeout property key

hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds

Also, the per command-level property to configure the timeout has the pattern:

Code Listing 1.6: Command instance level property key pattern

hystrix.command.HystrixCommandKey.execution.isolation.thread.timeoutInMilliseconds

When you want to override this particular property for our EmployeeInformationCommand class, replace "HystrixCommandKey" with group name "EmployeeInformationCommand". You are done. Go ahead and mention that inside the default constructor as shown below:

Code Listing 1.7: Actual property override declaration

public EmployeeInformationCommand() {
    super(HystrixCommandGroupKey.Factory.asKey("EmployeeServiceGroup"));

    ConfigurationManager.getConfigInstance().setProperty(
            "hystrix.command.EmployeeInformationCommand.execution.isolation.thread.timeoutInMilliseconds",
            new Long(2000));
}

The above code listing informs Hystrix to timeout if a response is NOT received within 2 seconds (2000 milliseconds).

Please find the below code listing for complete Hystrix command class:

Code Listing 1.8: Complete Hystrix command source code

public class EmployeeInformationCommand extends HystrixCommand<String> {

    private String params;

    public void setParams(String params) {
        this.params = params;
    }

    public EmployeeInformationCommand() {
        super(HystrixCommandGroupKey.Factory.asKey("EmployeeServiceGroup"));

        ConfigurationManager.getConfigInstance().setProperty(
                "hystrix.command.EmployeeServiceGroup.execution.isolation.thread.timeoutInMilliseconds",
                new Long(2000));
    }

    @Override
    protected String run() throws Exception {
        Client client = Client.create();
        WebResource webResource = client.resource("http://localhost:9999/app/employees");
        webResource.queryParam("terms", params);
        return webResource.get(String.class);
    }
}

When we are overriding configuration properties, it is always best to externalize them to a properties file, so that we can modify them for running application to pick new configuration without recompiling source code again and again. An application restart will do the job.