{{announcement.body}}
{{announcement.title}}

Java Advent Calendar: Take Control of Your Slow Producers

DZone 's Guide to

Java Advent Calendar: Take Control of Your Slow Producers

Take control of your slow producers... with READ-BEHIND-CACHE!

· Java Zone ·
Free Resource
snailTake control of your slow producers... with READ-BEHIND-CACHE!

In our connected world, we often use data from APIs that we don't own or don't have access to improve. If all things go right, their performance is good and everybody's happy. But too many times we have to use APIs that have less than optimal latency.

Of course, the answer is to cache that data. But a cache that you don't know when it gets stale is a dangerous thing, so that's not a proper solution.

Therefore... we're stuck. We'll need to get used to waiting for our page to load or invest in a really nice spinner to entertain the users while they wait for the data. Or... are we? What if, for a small, calculated compromise, we could have our desired performance using the same slow producer.

You may also like:  Java Advent Calendar: Celebrating 20 Years of Java

I think everybody heard about the write-behind cache. It's an implementation of a cache that registers a write that will happen asynchronously; the caller is free to continue its business while the write is performed on a background task.

What if we adopt this idea for the read side of the issue. Let's have a read-behind cache for our slow producers.

Fair warning: This technique applies only to data that we can afford to be stale for a limited number of requests. So if you can accept that your data will be "eventually fresh," you can apply this.

I'll use Spring Boot to build my application. All the code presented can be accessed on GitHub. There are three branches for different stages of the implementation.

The code samples contain only the relevant lines for brevity.

Status Quo

branch: status-quo

So, we'll start with the status quo. Firstly, we have a slow producer that receives a URL parameter. To simplify this, our producer will sleep for five seconds and then return a timestamp (of course, this is not a good example of low-change data, but for our purposes, it's useful to detect that the data is fresh as soon as possible).


public static final SimpleDateFormat SIMPLE_DATE_FORMAT = new SimpleDateFormat("HH:mm:ss.SSS");

    @GetMapping
    String produce(@RequestParam String name) throws InterruptedException {
        Thread.sleep(5000);
        return name + " : " + SIMPLE_DATE_FORMAT.format(new Date());
    }


In the consumer, we just make a call to the producer:


//ConsumerController.java  
  @GetMapping
    public String consume(@RequestParam(required = false) String name) {
        return producerClient.performRequest(ofNullable(name).orElse("default"));
    }

//ProducerClient.java
@Component
class ProducerClient {
    public String performRequest(String name) {
        return new RestTemplate().getForEntity(
                "http://localhost:8888/producer?name={name}",
                String.class, name)
                .getBody();
    }
}


Simple Cache

branch: simple-cache

To enable simple cache in Spring, we need to add the following:

  • Dependency to org.springframework.boot:spring-boot-starter-cache
  • Enable cache in application.properties: spring.cache.type=simple
  • Add @EnableCaching annotation to your Spring application main class
  • Add @Cacheable("cacheName") to your method to cache

Now, we have a simple cache represented. This will work with distributed cache also, but for this example, we'll stick with in-memory one. The consumer will cache the data, and after the first call, the latency is gone. But the data will become stale fast, and nobody evicts it. We can do better!

Intercept the Call

branch: master

The next thing we need to do is to intercept the call when it happens, regardless if it is cached or not.

In order to do this, we need to:

  • Create a custom annotation: @ReadBehind
  • Register an aspect that will intercept the method call annotated with @ReadBehind

So, we create the annotation and add it to performRequest method:


    @ReadBehind
    @Cacheable(value = CACHE_NAME, keyGenerator = "myKeyGenerator")
    public String performRequest(String name) {


As you see, a CACHE_NAME constant was defined. If you need to dynamically set the cache name, you can use a CacheResolver and a configuration. Also, in order to control the key structure, we need to define a key generator.


  @Bean
    KeyGenerator myKeyGenerator() {
        return (target, method, params) -> Stream.of(params)
                .map(String::valueOf)
                .collect(joining("-"));
    }


Furthermore, in order to add the aspect, we need to:

  • Add the dependency to org.springframework.boot:spring-boot-starter-aop
  • Create the aspect class
  • We need to implement the Ordered interface and return 1 for getOrder method. This is needed for the aspect to kick in even if the cache mechanism will suppress the call of the method when the value is already in the cache.

@Aspect
@Component
public class ReadBehindAdvice implements Ordered {
    @Before("@annotation(ReadBehind)")
    public Object cacheInvocation(JoinPoint joinPoint) {
...
    @Override
    public int getOrder() {
        return 1;
    }


Now we have a way to intercept all the calls to the  @ReadBehind method.

Remember the Call

Now that we have the call, we need to save all the needed data to be able to call it from another thread.

For this, we need to retain:


    @Before("@annotation(ReadBehind)")
    public Object cacheInvocation(JoinPoint joinPoint) {
        invocations.addInvocation(new CachedInvocation(joinPoint));
        return null;
    }

    public CachedInvocation(JoinPoint joinPoint) {
        targetBean = joinPoint.getTarget();
        arguments = joinPoint.getArgs();
        targetMethodName = joinPoint.getSignature().getName();
    }


We'll keep these objects in a different bean:


@Component
public class CachedInvocations {
    private final Set<CachedInvocation&gt; invocations = synchronizedSet(new HashSet<&gt;());

    public void addInvocation(CachedInvocation invocation) {
        invocations.add(invocation);
    }
}


The fact that we're keeping the invocations in a set and we have a scheduled job that processes those invocations at a fixed rate will give us also a nice side-effect of throttling the calls to the external API.

Schedule the Read-Behind Job

Now that we know what calls were performed, we can start a scheduled job to take those calls and refresh the data in the cache

In order to schedule a job in Spring Framework, we need to:

  • Add annotation@EnableScheduling to your Spring application class
  • Create a job class with a method annotated with @Scheduled

@Component
@RequiredArgsConstructor
public class ReadBehindJob {
    private final CachedInvocations invocations;

    @Scheduled(fixedDelay = 10000)
    public void job() {
        invocations.nextInvocations()
                .forEach(this::refreshInvocation);
    }
}


Refresh the Cache

Now that we have all the information collected, we can make the real call on the read-behind thread and update the information in the cache.

Firstly, we need to call the real method:


    private Object execute(CachedInvocation invocation) {
        final MethodInvoker invoker = new MethodInvoker();
        invoker.setTargetObject(invocation.getTargetBean());
        invoker.setArguments(invocation.getArguments());
        invoker.setTargetMethod(invocation.getTargetMethodName());
        try {
            invoker.prepare();
            return invoker.invoke();
        } catch (Exception e) {
            log.error("Error when trying to reload the cache entries ", e);
            return null;
        }
    }


Now that we have the fresh data, we need to update the cache.

Firstly, calculate the cache key. For this, we need to use the key generator defined for the cache.

Now that we have all information to update the cache, let's take the cache reference and update the value:


    private final CacheManager cacheManager;
    ...
    private void refreshForInvocation(CachedInvocation invocation) {
        var result = execute(invocation);
        if (result != null) {
            var cacheKey = keyGenerator.generate(invocation.getTargetBean(),
                    invocation.getTargetMethod(),
                    invocation.getArguments());
            var cache = cacheManager.getCache(CACHE_NAME);
            cache.put(cacheKey, result);
        }
    }


And with this, we finished the implementation of our read-behind idea. Of course, there are other concerns still remaining that you need to address.

For example, you could do this implementation and trigger the call on a thread immediately. That will ensure the cache refresh at the first possible time. If the staleness time is a major concern for you, you should do it.

I like the scheduler because it acts also as a throttling mechanism. Therefore, if you make the same call over and over again, the read-behind scheduler will collapse those calls in a single call.

Running the Sample Code

WARNING

As I said at the beginning of this article, there are some things you should be aware of when implementing read-behind.

Also, if you can't afford eventual consistency, don't do it! This is suitable for high-frequency reads with low-frequency changes APIs.

If the API has a sort of ACL implemented, you need to add the username with which you make the request in the cache key. Otherwise, very bad things can happen.

Therefore, analyze your application carefully and use this idea only where appropriate.

This post was originally published as part of the Java Advent series. If you like it, please spread the word by sharing it on Twitter, Facebook, etc.!

Want to write for the Java Advent blog? We are looking for contributors to fill all 24 slot and would love to have your contribution! Contact the  to contribute!

Further Reading

Java Advent Calendar: Celebrating 20 Years of Java

[DZone Refcard] Java Caching

Topics:
java ,java performance ,cache ,read-behind ,read-behind cache ,tutorial ,java advent calendar

Published at DZone with permission of Stefan Bulzan . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}