Circuit Breaker Pattern With Netflix-Hystrix: Java

If you are familiar with the circuit breaker pattern then you may have heard about Netflix-hystrix. But, before diving into our topic, let's make sure we have an understanding of what circuit breaker is and how Netflix-hystrix is implementing this pattern.

What Problem Are We Trying to Solve?

When in a distributed environment, our service may interact with other services or applications via remote call and these external services may be unavailable anytime for several reasons such as downtime, unavailable resources, etc. This unavailability due to the fault in an external application will also affect our perfectly working application. 

For example, we know that in a microservice architecture number of services interact with each other, so if we have a service A whose sole purpose is to process and pass on data to service B via a rest call then, ideally, it will be assumed that service B will always be available for service A to send data and return a proper response. But, we don’t live in an ideal world, do we? It may very possibly happen that service B goes down due to some reason like pod failure, resource issues, etc, and when its service A’s turn to send data, it finds that the endpoint it’s trying to send data to is not available anymore. So, what does it do with that piece of data and the data coming after that? 

Well, it isn't service A’s fault that service B is not available, correct? It’s doing its best, but it also does not know what to do now. The solution is to make service A fault-tolerant. We want the service to handle the failures gracefully and take action on the failure, which won’t affect the working of the service itself. Irrespective of what happens to service B, service A should not suffer, and it should keep on processing the data to pass on to service B. We can achieve this by using the circuit breaker pattern.

Circuit Breaker Pattern

The circuit breaker is a pattern that prevents our service from repeatedly retrying the operation that is failing and allows it to work continuously irrespective of what the fault is and how much time it can take to fix, therefore saving us CPU resources. This pattern also helps us monitor if the fault has been resolved and, if it is, it would again continue making calls through the original operation. You can find a more detailed and better explanation here in the Microsoft docs.

What Is Netflix-hystrix?

In very simple terms, a circuit breaker is a pattern, and Netflix-hystrix is an implementation of this pattern. This library was designed to control the interactions between distributed services by adding latency tolerance and fault tolerance logic. Hystrix does this by isolating points of access between the services, stopping cascading failures across them, and providing fallback options -- all of which improve your system’s overall resiliency.

Features of Netflix-hystrix:

• Give protection from and control over latency and failure from dependencies
• Stop cascading failures in a complex distributed system
• Fallback Mechanism
• Enable near real-time monitoring, alerting, and operational control

You can get more detail about its works in their documentation here.

Let’s implement this ourselves for better understanding and use of netflix-hystrix with a Java Springboot Application.

Here, we have created a sample Hystrix Java service that does the following:

• Exposes a GET endpoint that provides information about the service.
• The service internally calls another endpoint to retrieve this information. This is again a rest call.
• When we hit the service endpoint, the controller will execute this method.

   public List<ServiceInformation> showServiceInformation() {
        List<ServiceInformation> information = restTemplate.exchange(
                "http://localhost:8081", HttpMethod.GET, null, new
                        ParameterizedTypeReference<List<ServiceInformation>>(){}).getBody();
        for(ServiceInformation serviceInformation : information) {
            logger.trace(serviceInformation.getServiceName());
            logger.trace(serviceInformation.getMessage());
        }
        return information;
    }

Now, we can see that the method makes a rest call to another service, so we will only be able to receive information if the external service returns List<ServiceInformation>

But, if this rest endpoint is unavailable, we won't receive any data. This is where we can implement the circuit breaker pattern using the Netflix-hystrix.

The library provides us with a fallback method mechanism, which is a method that will be invoked in case the call to external service fails. The logic to the fallback method will depend on your use case; here, we are just going to log for demo purposes that the service failed and the application moved to a fallback method.

    @HystrixCommand(fallbackMethod = "defaultStatus")
    public List<ServiceInformation> showServiceInformation() {
        List<ServiceInformation> information = restTemplate.exchange(
                "http://localhost:8081", HttpMethod.GET, null, new
                        ParameterizedTypeReference<List<ServiceInformation>>(){}).getBody();
        for(ServiceInformation serviceInformation : information) {
            logger.trace(serviceInformation.getServiceName());
            logger.trace(serviceInformation.getMessage());
        }
        return information;
    }
    
      public List<ServiceInformation> defaultStatus() {
        logger.error("circuit-breaker-proxy is down, running fallback method");
        return Collections.emptyList();
    }

@HystrixCommand() enables us to provide the fallback method. The name of the fallback method is passed in the parameter of this annotation.

Here, the default status is the fallback method, and the name of this method is passed as a parameter in the @HystrixCommand annotation which will execute the fallback method if the call to external service fails. We can also have fallbacks for the fallback method.

The Annotation can have several more parameters depending on the property you want to provide. For now, we will just look at the fallback method. 

Hystrix is no longer in active development and is currently in maintenance mode. There are more alternatives to Hystrix such as Resilience4j, which we will cover in future blogs.

For more detailed implementation and understanding of the annotation check out the documentation here.

The link to the project is here.

Refer to the readme file to run the project and see how the library actually works. Fork/clone, and play around to have a better understanding.