Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Benchmark series on simple caching solutions in Java

DZone's Guide to

Benchmark series on simple caching solutions in Java

· Java Zone
Free Resource

Try Okta to add social login, MFA, and OpenID Connect support to your Java app in minutes. Create a free developer account today and never build auth again.

Caching is a very common solution when you don't want to repeat CPU intense tasks. The last days I was benchmarking options to do caching with ConcurrentHashMap. In this blog I publish the first results. I have used Heinz Kabutz' Performance Checker to do this. I also added some features based on my readings of this article series.

My Conclusions up front

I am testing three different cache implementations: "Check null", "check map" and "putIfAbsent" cache. The code is listed below the results. Also, I am using three different cache sizes: 10 (small), 100000 (large) and 1000000 (very large) possible key values. Again: The 10 units cache can have 10 different key values. The 100000 units cache can have 100000 different key values etc.

None of the implementation options show significant performance differences assuming cache size is equivalent. This is disappointing (!!) but also good to know. It's also a little surprising I think, 'cause for example "putIfAbsent" cache appears to be more complex then the others. All the solutions seem to decrease logarithmical in performance when the cache increases to very large sizes. The main reason for that should be the increased time for cache initialization, 'cause in my test harness I start with an empty cash and a fixed set of possible key values (i.e. 10, 100000 or 1000000). I'll come up with a benchmark series for fully initialized cache later.

Knowing what I know now, I would carefully recommend to use the "putIfAbsent" cache solution. That's because it has equivalent performance but gives you great flexibility to design the behaviour of your cache in highly concurrent scenarios. See the pattern solution of my last article about multithreading as an example of more complex use cases.

If you're interested in the test harness take a look at the implementations: CacheSolution_CheckMap.java, CacheSolution_CheckNull.java and CacheSolution_PutIfAbsent.java. I appreciate your comments very much!

My JVM was in mixed JIT mode and with -server option set. OK, let's go into this!

Here are the results

-XX:InitialHeapSize=49759808 -XX:MaxHeapSize=796156928 -XX:ParallelGCThreads=2 -XX:+PrintCommandLineFlags -XX:-UseLargePagesIndividualAllocation -XX:+UseParallelGC 
Check null - small cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
1981447,2            8210,4428       381         381        651        651       
1956580,4            9408,1773       381         381        651        651       
1991336,4            13575,1192      381         381        651        651       
1947666,2            8564,2068       381         381        651        651       
Check map - small cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
1794618,4            11201,6421      381         381        651        651       
1790090,6            10716,0937      381         381        651        651       
1824689,2            3967,281        381         381        651        651       
1788582,6            3523,9609       381         381        651        651       
putIfAbsent - small cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
1708336,4            6037,146        475         475        654        654       
1706098,8            1804,7957       584         584        654        654       
1738511,4            6360,8573       584         584        654        654       
1711581              11009,8691      584         584        654        654       
Check null - large cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
1243431,8            4671,9216       602         602        654        654       
1231862,8            3457,7805       613         613        654        654       
1243344,8            6136,7727       613         613        654        654       
1220573,8            22964,319       613         613        654        654       
Check map - large cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
1116376,6            2488,8944       625         625        654        654       
1069693,2            73327,6869      636         636        654        654       
1112667,4            7746,2007       636         636        654        654       
1102652,4            1648,8972       636         636        654        654       
putIfAbsent - large cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
966868               2609,4972       642         642        654        654       
968215,4             1688,7719       653         653        654        654       
982656,8             6600,9725       659         659        654        654       
973622,2             3884,5465       659         659        654        654       
Check null - very large cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
757436               54966,9938      659         659        654        654       
778233,2             24416,1994      659         659        654        654       
783227               23614,2381      659         659        654        654       
777516,2             22406,1194      659         659        655        655       
Check map - very large cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
741914,2             22100,5588      659         659        655        655       
738258               22120,656       659         659        655        655       
741697               22608,6525      659         659        655        655       
738903,8             21126,0641      665         665        656        656       
putIfAbsent - very large cache - mixed mode
Mean exec count      Deviation       JIT before  JIT after  CL before  CL after  
703047,4             14771,0501      665         665        656        656       
698942,8             15885,6482      665         665        656        656       
702126,6             16192,2101      665         665        656        656       
699374,8             16967,6066      665         665        656        656       

5 test runs each / 500 ms each test run
CL before/after = Classes Loaded before and after test harness
JIT before/after = Total JIT time before and after test harness
Small cache = 10 units
Large cache 100000 units
Very large cache = 1000000 units

Check null cache 

private Map<String, String> commandCache = new ConcurrentHashMap<String, String>();

@Override
public Object[] execute(Object... arguments) {
String clientCommand = (String) arguments[0];
String serverCommand = commandCache.get(clientCommand);
if (serverCommand == null) {
lock.lock();
try {
if (commandCache.containsKey(clientCommand))
serverCommand = commandCache.get(clientCommand);
else {
// do something CPU intensive (is not relevant for test result here)
serverCommand = "dummy string";
commandCache.put(clientCommand, serverCommand);
}
} finally {
lock.unlock();
}
}
Object[] result = new Object[] { serverCommand };
return result;
}

Check map cache

private Map<String, String> commandCache = new ConcurrentHashMap<String, String>();

@Override
public Object[] execute(Object... arguments) {
String clientCommand = (String) arguments[0];
String serverCommand = null;
if (commandCache.containsKey(clientCommand)) {
serverCommand = commandCache.get(clientCommand);
} else {
lock.lock();
try {
if (commandCache.containsKey(clientCommand)) {
serverCommand = commandCache.get(clientCommand);
} else {
// do something CPU intensive (is not relevant for test result)
serverCommand = "dummy string";
commandCache.put(clientCommand, serverCommand);
}
} finally {
lock.unlock();
}
}
Object[] result = new Object[] { serverCommand };
return result;
}

The putIfAbsent cache

private ConcurrentHashMap<String, SomeCPUIntenseTask> commandCache = new ConcurrentHashMap<String, SomeCPUIntenseTask>();

@Override
public Object[] execute(Object... arguments) {
String clientCommand = (String) arguments[0];
SomeCPUIntenseTask newServerCommand;
SomeCPUIntenseTask serverCommand = commandCache.putIfAbsent(clientCommand, newServerCommand = new SomeCPUIntenseTask());
if (serverCommand == null) {
serverCommand = newServerCommand;
}
return new Object[]{serverCommand.doCPUIntenseTask()};
}

public class SomeCPUIntenseTask {
private Object taskResult = null;
public Object doCPUIntenseTask() {
if (taskResult != null)
return taskResult; // avoid locking overhead
else {
try {
lock.lock();
if (taskResult != null) // two threads may have been in race condition
return taskResult;
// perform CPU intense task
taskResult = new Object();
return taskResult;
} finally {
lock.unlock();
}
}
}
}

From http://niklasschlimm.blogspot.com/2011/09/benchmark-series-on-simple-caching.html

Build and launch faster with Okta’s user management API. Register today for the free forever developer edition!

Topics:

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}