Java Holiday Calendar 2016 (Day 15): Don't Optimize Now!

Today's tip is about how the JVM can optimize our code. Back in the good ol' Java 1.0 days, I remember running programs that replaced the names of variables, classes, and methods so they would become shorter, presumably resulting in faster execution. When things blew up in our faces, we were clueless what really happened because the NPE that method C threw in class D that inherited from class A just didn't provide adequate information as to what really went wrong. It was better before. Not.

These days, the present JVM has been incredibly improved over its first incarnations. The JVM will collect statistics on how methods are used and then use that piece of information when it eventually compiles the method with its Just-In-Time (JIT) compiler. This often takes place when a method has been called for about 10,000 times. This is one reason why we should "warm up" our code properly before we run benchmarks.

The JVM is also using a scheme called Code Flattening , which means that it is able to "roll up" method calls and consider the code as a larger flow of operations. This way, it is able to determine the possible paths the program can take and consider these paths individually and optimize them. For example, identical range checking that takes place in several methods that call each other can be eliminated. Another example is that dead paths (e.g. where an "if"-branch is using a constant expression) can be eliminated altogether.

Another means of optimization is Escape Analysis, which is used to determine if an object is visible outside a certain scope. If not, the object can be allocated on the stack rather than on the heap, making the program run much faster. Consider the following code snippet:

@Override 
public String toString() {
    final StringBuilder sb = new StringBuilder()
    .append("(")
    .append(x)
    .append(", ")
    .append(y)
    .append(")");
    return sb.toString();
}

Once this code is compiled and the method has been called a predetermined number of times, the JVM will allocate the StringBuilder on the stack, rather than on the heap. Once the method returns, the StringBuilder is cleaned up automatically when the stack is popped upon its return. Because the object cannot be observed from outside, it is also possible to use a simplified representation of the StringBuilder on the stack. So, effectively, the StringBuilder never exists, and thus putting in a lot of work to eliminate it is wasted work and will only result in code that looks bad.

One cool thing is that the JVM can combine its optimization schemes. For example, with code flattening, much larger scopes can be used for escape analysis, and objects that actually do escape a method might be re-catch on a higher "rolled up" level. Such objects may be subject to stack allocation too. Amazing stuff! 

There are two golden rules when it comes to optimization:

1. Don't optimize

2. Don't optimize now...

That said, when we want to write performance-critical applications, it is important to have a basic understanding of the JVM to really know what is going on under the hood. Sometimes, it really pays off to rewrite and optimize code.

Speedment, an open-source stream ORM tool and runtime, relies on a number of JVM features to be able to provide efficient execution of its own and application Java code. In particular, the stream implementations benefit substantially from code flattening and escape analysis.

Know your JVM and put your effort where it matters — and not in places the JVM will optimize anyhow.

Follow the Java Holiday Calendar 2016 with small tips and tricks all the way through the winter holiday season.