Measuring Performance of Your Methods Using JMH in Java

From JDK-12 onwards, the JDK comes with JMH (Java Microbenchmark Harness), It is a toolkit that helps you implement Java microbenchmarks correctly. JMH is developed by the same people who implement the Java virtual machine (JVM) so they know the internals and how Java makes optimizations at run time.

You may also like: JMH: Benchmark REST APIs

Why JMH?

There are many optimizations that the JVM  or underlying hardware may apply to your component when the benchmark executes that component in isolation. These optimizations may not be possible to apply when the component is running as part of a larger application. Badly implemented microbenchmarks may thus make you believe that your component’s performance is better than it will be in reality.

Whenever we come across a problem, we tend to solve it recursively, iteratively or by using inbuilt methods provided in our language. After coding we tend to confuse that for a given set of data(large, small, Fixed) will I have a better edge to use inbuilt methods or method 1 or method 2 or...method 3.

In this article, I will be referring method as a component in isolation, As my whole agenda is to show you how to test your methods using JMH as per their performances.

Before starting there are few prerequisites, If you don’t have JDK 12 then you need to import the dependencies in your project for that you can either download the JARs of making a maven project and adding below dependencies:

  <dependencies>

<dependency>

<artifactId>jmh-core</artifactId>

<groupId>org.openjdk.jmh</groupId>

<version>${jmh.version}</version>

</dependency>

<dependency>

<artifactId>jmh-generator-annprocess</artifactId>

<groupId>org.openjdk.jmh</groupId>

<version>${jmh.version}</version>

</dependency>

</dependencies>

<p>

<jmh.version>1.21</jmh.version>

  </properties>

For the sake of simplicity, let's take an example to generate N natural numbers and store them in a List.

• A simple or traditional approach will be to use a loop which will execute from 1 to N and store each number to List<Integer> list by list.add() .
• A more functional approach will be to use stream to generate a Intstream from 1 to N and then store than to list.

xxxxxxxxxx

import java.util.ArrayList;

import java.util.List;

import java.util.stream.Collectors;

import java.util.stream.IntStream;

import org.openjdk.jmh.annotations.Benchmark;

import org.openjdk.jmh.annotations.BenchmarkMode;

import org.openjdk.jmh.annotations.Mode;

import org.openjdk.jmh.annotations.Param;

import org.openjdk.jmh.annotations.Scope;

import org.openjdk.jmh.annotations.State;

import org.openjdk.jmh.runner.Runner;

import org.openjdk.jmh.runner.RunnerException;

import org.openjdk.jmh.runner.options.Options;

import org.openjdk.jmh.runner.options.OptionsBuilder;

@State(Scope.Thread)

public class Testing {

  @Param({"10"})

  private int number;

  @Benchmark

  @BenchmarkMode({Mode.All})

  //traditional method to generate N numbers and store it in List

  public List<Integer> traditionalMethod(Testing T) {

    List<Integer> list = new ArrayList<>();

    for (int i = 1; i <= number; i++) {

      list.add(i);

    }

    return list;

  }

  @Benchmark

  @BenchmarkMode({Mode.All})

  //using #stream to generate N numbers and store it in List

  public List<Integer> streamMethod(Testing T) {

    return IntStream.rangeClosed(1, number).boxed().collect(Collectors.toList());

  }

  public static void main(String[] args) throws RunnerException {

    Options opt = new OptionsBuilder()

        .include(Testing.class.getSimpleName())

        .forks(1)

        .build();

    new Runner(opt).run();

  }

}

After running the program the results are as follows:

The throughput of the stream method is better for the small range of 10, it might or might not be the same for the larger data set.

The detailed project has been uploaded to GitHub, you can fork it and run in your own local environment and can do modify or add your own methods and check how they are working in terms of performance.

The following article used @BenchmarkMode({Mode.All}) which will benchmark all the following:

Throughput("thrpt", "Throughput, ops/time"),
AverageTime("avgt", "Average time, time/op"),
SampleTime("sample", "Sampling time"),
SingleShotTime("ss", "Single shot invocation time"),
 All("all", "All benchmark modes");

In case you need the only Throughput to be measured then use Mode.Throughput.

GitHub Link

Further Reading

How to Improve the Performance of a Java Application