A couple of months ago I was reading up on Complex Event Processing in Java and ways to achieve low latency. At the end of my hour long research I figured out that even if your application is well written and your methods run mostly in 0(log n) time, and you are using some bleeding edge hardware solutions, there is still some time consumed by the VM during its interpretation of the bytecode.

On the good side of things, Java is interpreted and its bytecode is cross-JVM compatible but we also know that because of this we are bound to lose something somewhere. Our JVM reads the interpreted bytecode and runs it every time. Obviously, this takes time.

But its not that bad considering our friendly neighbourhood JIT compiler (server or client) watches out for commonly used methods and when it figures out that the method is called just too many times, it compiles it into native machine code instead of making the JVM rely on bytecode all the time.

The number for ‘too many times’ is configured using the vm argument

-XX:CompileThreshold

And the default is 1500. One’s natural guess would be that reducing the number would mean more methods are converted to native code faster and that would mean faster application but it may not be. Considerably low numbers would mean that the server starts considerably slower because of the time taken by the JIT to compile too many methods (which may not be used that often after all) and since the native machine code resides in the memory, your application will be awarded the “Memory killer” award and die a slow painful death. A little googling shows that numbers around 100 is not that bad. Again, it depends on your application and the usage patterns and traffic.

Forgot to mention, that the smallest compilation unit to become a JIT native compilation candidate is a method. Not a block. So, long running fat methods - good luck !!!

In reality, this JIT compilation is not in one go. It has two neat phases :

1) Everytime a method is called, its counter gets increased by 1 and soon after it reaches the threshold, the JIT does its first compilation.

2) After the first compilation, the counter is reset to 0 and incremented again. In this second cycle, when it reaches the threshold, JIT does an second round of compilation - this time with more aggressive and awesome optimizations (sorry - unable to provide you much details here)

If you are using JDK 7 and your machine runs on multi-core (I don’t see why not), then you could use the following flag to speed up your native compilation process

-server -XX:+TieredCompilation

I can’t claim to be an expert in JVM tuning considering the amount of options available. So, please leave your comments if you find this useful or is incorrect.