Java Microbenchmark Harness (JMH)
A quick hands-on lesson to learn about Java Microbenchmark Harness (JMH). The article helps you get started and configure JMH project.
Join the DZone community and get the full member experience.Join For Free
In my previous article, we established that microbenchmarking is hard with
jvm. It is not enough to surround the code in a loop with
System.out.println() and gather the time measurements. While benchmarking, a developer should consider warm-up cycles, JIT compilations, JVM optimizations, avoiding usual pitfalls, and even more.
Thankfully, OpenJDK has a great tool Java Microbenchmark Harness (JMH) that can help us generated benchmarking stats. In this article, we will discuss how JMH can help us avoid the pitfalls that we have discussed earlier.
Getting Started With JMH
A quick way to start with JMH is to use the Maven archetype. The command below will generate a new Java project
benchmark. The project will have
com/gaurav/MyBenchmark.java class and
pom.xml. The Maven
pom.xml includes all the required dependencies to support JMH.
mvn archetype:generate -DarchetypeGroupId=org.openjdk.jmh -DarchetypeArtifactId=jmh-java-benchmark-archetype -DinteractiveMode=false -DgroupId=com.gaurav -DartifactId=benchmark -Dversion=1.0
Good Benchmarks With JMH
Let us discuss how JMH can help us to write better microbenchmarks.
- JMH by default makes several warm-up cycles before collecting the stats. Thus, it makes sure that the results are not completely random and
jvmhas performed optimizations.
@benchmarkruns iteration over the code, and then collects the average. The more runs you make through the code, the better stats you will collect.
Blackholeclass of JMH can avoid deal code elimination by
jvm. If you pass the calculated results to
blackhole.consume(), it would trick the
jvmwill never drop the code, thinking that
consume()method uses the result.
Writing First Benchmark
Maven has already provided with us a template in
MyBenchmark class to fill in. I am going to utilize the same class.
I would like to keep my first benchmark simple. Let us start by iterating over all the elements of a list and sum them up using a conventional
for loop. As discussed, I will use
Blackhole to fool the compiler and return the result. Here, I am asking JMH to calculate the average time, using
@BenchmarkMode, it takes to run the
Compiling the JMH Project
You can compile and build the project like any other Maven project, using the below Maven command:
mvn clean install
The command will create a fully executable
jar file under
benchmark/target the directory. Please note that Maven will always generate a
jar file named
benchmarks.jar, regardless of the project name.
Next step is to execute the
jar using the below command:
java -jar target/benchmarks.jar
It produced the below result for me. It means that test operation is taking approx. 0.053 seconds on the current hardware.
In the previous example, I used
@BenchmarkMode(Mode.AverageTime). If you try to decompile the JMH jar, you will find
enum Mode with the below values:
||It will calculate the number of times your method can be executed with in a second|
||It will calculate the average time in seconds to execute the test method|
||It randomly samples the time spent in the test method calls|
||It works on single invocation of the method and is useful in calculating cold performance|
||Calculates all the above|
The default Mode is
It is evident from the console output above that calculations are in seconds. But, JMH allows you to configure the time units, using
@OutputTimeUnit annotation. The
java.util.concurrent.TimeUnit, as shown below:
TimeUnit enum has the following values:
Configure Fork, Warmup, and Iterations
The benchmark is currently executing 5 times, with 5 warmup iterations and 5 measurement iterations. JMH even allows to configure these values using
@Measurement annotations. The code snippet below would execute the test method twice, with a couple of warmup iterations and 3 measurement iterations.
@Measurement annotations also accepts parameters:
batchSize- configures the number of test method calls to be performed per operation.
time- time spent for each iteration.
You can play around to compare execution times of different
for loops i.e. a conventional
for loop, a
forEach loop and an
stream iterator. Something like:
In this article, we have gone through a hands-on example of creating a JMH project. We have seen how can we configure our JMH project to suit our needs. You can refer to JMH Github Samples for more in-depth examples.
We have seen that JMH is a
jvm tool. In the next article, we will try to explore if it can help us with other
jvm based languages.
Published at DZone with permission of Gaurav Gaur. See the original article here.
Opinions expressed by DZone contributors are their own.