Performance Test: Groovy 2.0 vs. Java

At the end of July 2012, Groovy 2.0 was released with support for static type checking and some performance improvements through the use of JDK7 invokedynamic and type inference as a result of type information now available through static typing.

I was interested in seeing some estimate as to how significant the performance improvements in Groovy 2.0 have turned out and how Groovy 2.0 would now compare to Java in terms of performance. In case the performance gap had become minor, or at least acceptable, in the meantime, it would certainly be time to take a serious look at Groovy. Groovy has been ready for production for a long time. So, let's see whether it can compare with Java in terms of performance.

The only performance measurement I could find on the Internet was this little benchmark measurment on jlabgroovy. The measurement only consists of calculating Fibonacci numbers with and without the @CompileStatic annotation. That's it; i.e., it's certainly not very meaningful in striving to get an overall impression. I was only interested in obtaining some rough estimate of how Groovy compares to Java as far as performance is concerned.

Alas, no measurement was included in this little benchmark as to how much time Java takes to calculate Fibonacci numbers. So I "ported" the Groovy code to Java (here it is) and repeated the measurements. All measurements were done on an Intel Core2 Duo CPU E8400 3.00 GHz using JDK7u6 running on Windows 7 with Service Pack 1. I used Eclipse Juno with the Groovy plugin using the Groovy compiler version 2.0.0.xx-20120703-1400-e42-RELEASE. These are the figures I obtained without having a warm-up phase:

I also did measurements with a warm-up phase of various length with the conclusion that there is no benefit for either language with or without the @CompileStatic. Since the Fibonacci algorithm is that recursive the warm-up phase seems to be "included" for any Fibonacci number that is not very small.

We can see that the performance improvements due to static typing have made quite a difference. This little comparison does little justice, though. To me, the impression that static typing in Groovy has had in conjunction with type inference has led to significant performance improvements—and in the same way it has led to Groovy++ becoming very strong. With the @CompileStatic, the performance of Groovy is about 1-2 times slower than Java, and without Groovy, it's about 3-5 times slower. Unhappily, the measurements of "instance ternary" and "instance if" are the slowest. Unless we want to create masterpieces in programming with static functions, the measurements for "static ternary" and "static if" are not that relevant for most of the code with the ambition to be object-oriented (based on instances).

Conclusion

When Groovy was about 10-20 times slower than Java (see benchmark table almost at the end of this article) it is questionable whether the @CompileStatic was used or not. This means to me that Groovy is ready for applications where performance has to be somewhat comparable to Java. Earlier, Groovy (or Ruby, Closure, etc.) could only serve as a plus on your CV because of the performance impediment (at least here in Europe).

New JVM kid on the block: Kotlin 

I added the figures for Kotlin as well (here is the code). Kotlin is a relatively new statically typed JVM-based Java-compatible programming language. Kotlin is more concise than Java by supporting variable type inferences, higher-order functions (closures), extension functions, mixins and first-class delegation, etc. Contrary to Groovy, it is more geared towards Scala, but also integrates well with Java. Kotlin is still under development and has yet to be  officially released. So the figures have to be taken with caution as the guys at JetBrains are still working on the code optimization. Ideally, Kotlin should be as fast as Java. The measurements were done with the current "official" release 0.1.2580.

And what about future performance improvements? 

At the time when JDK1.3 was the most recent JDK, I still earned my pay with Smalltalk development. At that time the performance of VisualWorks Smalltalk (now Cincom Smalltalk) and IBM VA for Smalltalk (now owned by Instantiations) was very good comparable to Java. And Smalltalk is a dynamically typed language, like pre-Goovy 2.0 and Ruby, where the compiler cannot make use of type inference to do optimizations. Because of this, it always appeared strange to me that Groovy, Ruby and other JVM-based dynamic languages had such a big performance penalty compared to Java when Smalltalk had not. From that point of view I think there's still room for Groovy performance improvements beyond @CompileStatic.