Benchmark series on simple caching solutions in Java
by
·
Join the DZone community and get the full member experience.
Join For FreeCaching is a very common solution when you don't want to repeat CPU
intense tasks. The last days I was benchmarking options to do caching
with ConcurrentHashMap. In this blog I publish the first results. I have used Heinz Kabutz' Performance Checker to do this. I also added some features based on my readings of this article series.
My Conclusions up front
I am testing three different cache implementations: "Check null", "check map" and "putIfAbsent" cache. The code is listed below the results. Also, I am using three different cache sizes: 10 (small), 100000 (large) and 1000000 (very large) possible key values. Again: The 10 units cache can have 10 different key values. The 100000 units cache can have 100000 different key values etc.
None of the implementation options show significant performance differences assuming cache size is equivalent. This is disappointing (!!) but also good to know. It's also a little surprising I think, 'cause for example "putIfAbsent" cache appears to be more complex then the others. All the solutions seem to decrease logarithmical in performance when the cache increases to very large sizes. The main reason for that should be the increased time for cache initialization, 'cause in my test harness I start with an empty cash and a fixed set of possible key values (i.e. 10, 100000 or 1000000). I'll come up with a benchmark series for fully initialized cache later.
Knowing what I know now, I would carefully recommend to use the "putIfAbsent" cache solution. That's because it has equivalent performance but gives you great flexibility to design the behaviour of your cache in highly concurrent scenarios. See the pattern solution of my last article about multithreading as an example of more complex use cases.
If you're interested in the test harness take a look at the implementations: CacheSolution_CheckMap.java, CacheSolution_CheckNull.java and CacheSolution_PutIfAbsent.java. I appreciate your comments very much!
My JVM was in mixed JIT mode and with -server option set. OK, let's go into this!
Here are the results
-XX:InitialHeapSize=49759808 -XX:MaxHeapSize=796156928 -XX:ParallelGCThreads=2 -XX:+PrintCommandLineFlags -XX:-UseLargePagesIndividualAllocation -XX:+UseParallelGC Check null - small cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 1981447,2 8210,4428 381 381 651 651 1956580,4 9408,1773 381 381 651 651 1991336,4 13575,1192 381 381 651 651 1947666,2 8564,2068 381 381 651 651 Check map - small cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 1794618,4 11201,6421 381 381 651 651 1790090,6 10716,0937 381 381 651 651 1824689,2 3967,281 381 381 651 651 1788582,6 3523,9609 381 381 651 651 putIfAbsent - small cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 1708336,4 6037,146 475 475 654 654 1706098,8 1804,7957 584 584 654 654 1738511,4 6360,8573 584 584 654 654 1711581 11009,8691 584 584 654 654 Check null - large cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 1243431,8 4671,9216 602 602 654 654 1231862,8 3457,7805 613 613 654 654 1243344,8 6136,7727 613 613 654 654 1220573,8 22964,319 613 613 654 654 Check map - large cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 1116376,6 2488,8944 625 625 654 654 1069693,2 73327,6869 636 636 654 654 1112667,4 7746,2007 636 636 654 654 1102652,4 1648,8972 636 636 654 654 putIfAbsent - large cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 966868 2609,4972 642 642 654 654 968215,4 1688,7719 653 653 654 654 982656,8 6600,9725 659 659 654 654 973622,2 3884,5465 659 659 654 654 Check null - very large cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 757436 54966,9938 659 659 654 654 778233,2 24416,1994 659 659 654 654 783227 23614,2381 659 659 654 654 777516,2 22406,1194 659 659 655 655 Check map - very large cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 741914,2 22100,5588 659 659 655 655 738258 22120,656 659 659 655 655 741697 22608,6525 659 659 655 655 738903,8 21126,0641 665 665 656 656 putIfAbsent - very large cache - mixed mode Mean exec count Deviation JIT before JIT after CL before CL after 703047,4 14771,0501 665 665 656 656 698942,8 15885,6482 665 665 656 656 702126,6 16192,2101 665 665 656 656 699374,8 16967,6066 665 665 656 656
5 test runs each / 500 ms each test run
CL before/after = Classes Loaded before and after test harness
JIT before/after = Total JIT time before and after test harness
Small cache = 10 units
Large cache 100000 units
Very large cache = 1000000 units
Check null cache
private Map<String, String> commandCache = new ConcurrentHashMap<String, String>();
@Override
public Object[] execute(Object... arguments) {
String clientCommand = (String) arguments[0];
String serverCommand = commandCache.get(clientCommand);
if (serverCommand == null) {
lock.lock();
try {
if (commandCache.containsKey(clientCommand))
serverCommand = commandCache.get(clientCommand);
else {
// do something CPU intensive (is not relevant for test result here)
serverCommand = "dummy string";
commandCache.put(clientCommand, serverCommand);
}
} finally {
lock.unlock();
}
}
Object[] result = new Object[] { serverCommand };
return result;
}
Check map cache
private Map<String, String> commandCache = new ConcurrentHashMap<String, String>();
@Override
public Object[] execute(Object... arguments) {
String clientCommand = (String) arguments[0];
String serverCommand = null;
if (commandCache.containsKey(clientCommand)) {
serverCommand = commandCache.get(clientCommand);
} else {
lock.lock();
try {
if (commandCache.containsKey(clientCommand)) {
serverCommand = commandCache.get(clientCommand);
} else {
// do something CPU intensive (is not relevant for test result)
serverCommand = "dummy string";
commandCache.put(clientCommand, serverCommand);
}
} finally {
lock.unlock();
}
}
Object[] result = new Object[] { serverCommand };
return result;
}
The putIfAbsent cache
private ConcurrentHashMap<String, SomeCPUIntenseTask> commandCache = new ConcurrentHashMap<String, SomeCPUIntenseTask>();
@Override
public Object[] execute(Object... arguments) {
String clientCommand = (String) arguments[0];
SomeCPUIntenseTask newServerCommand;
SomeCPUIntenseTask serverCommand = commandCache.putIfAbsent(clientCommand, newServerCommand = new SomeCPUIntenseTask());
if (serverCommand == null) {
serverCommand = newServerCommand;
}
return new Object[]{serverCommand.doCPUIntenseTask()};
}
public class SomeCPUIntenseTask {
private Object taskResult = null;
public Object doCPUIntenseTask() {
if (taskResult != null)
return taskResult; // avoid locking overhead
else {
try {
lock.lock();
if (taskResult != null) // two threads may have been in race condition
return taskResult;
// perform CPU intense task
taskResult = new Object();
return taskResult;
} finally {
lock.unlock();
}
}
}
}
From http://niklasschlimm.blogspot.com/2011/09/benchmark-series-on-simple-caching.html
Cache (computing)
Java (programming language)
Opinions expressed by DZone contributors are their own.
Comments