Changing the Order of Tests in JUnit5

Multiple tests within a suite are expected to be independent in most cases. If changing the order that the tests are run subsequently causes a different outcome, such as a failed test or different failed tests, it is possible, or even likely, a sign of an underlying bug in the test class or application under test.

A test could cause an inconsistency in the database that is only detected when tests are executed in a different order. Another possibility is that a test positioned at the end could inadvertently take into account a change in the data from an earlier test and fail when it is run in isolation or at the beginning of the suite.

To detect such an issue, the order that the tests are run can be modified with JUnit's TestMethodOrder annotation. It includes a parameter to designate an alternative order. The order can also be modified by implementing and using a custom orderer class.

TestMethodOrder is a type-level annotation on a class or interface that configures the MethodOrderer. It affects the order that the test methods are run, which are those methods annotated Test, RepeatedTest, ParameterizedTest, TestFactory, or TestTemplate. The MethodOrderer is an API for choosing the specific order and includes three implementations: Alphanumeric, OrderAnnotation, and Random.

By default, with no TestMethodOrder annotation on the class, parent class, or interface, the test methods are run in a consistent order but not necessarily in the order in which they exist in the class. The Alphanumeric option sorts the test methods by their names with the String’s compareTo function. The OrderAnnotation implementation causes the test methods to be run according to the number in the Order annotation on each test. The last built-in implementation,  MethodOrder.Random runs the tests randomly, although it can be seeded for repeatability. 

If a different order is required, the MethodOrderer class can be implemented for additional customization. The class consists of a single method that provides context from which the test methods are accessed and repositioned through the list of descriptors. For example, the following tests are run in a reverse alphabetic sequence with the ReverseAlphaOrderer class that implements the MethodOrderer interface.

import org.junit.jupiter.api.TestMethodOrder;
import org.junit.jupiter.api.Test;

@TestMethodOrder(ReverseAlphaOrderer.class)
class CustomOrderTests{

    @Test
    void alphaTest() {
      System.out.println("alpha");
    }

    @Test
    void bravoTest() {
      System.out.println("bravo");
    }

    @Test
    void charlieTest() {
      System.out.println("charlie");
    }
}

import org.junit.jupiter.api.MethodDescriptor;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.MethodOrdererContext;
import java.util.Comparator;

class ReverseAlphaOrderer implements MethodOrderer {

private Comparator<MethodDescriptor> comparator = new Comparator<MethodDescriptor>() {

@Override
public int compare(MethodDescriptor one, MethodDescriptor two) {
String twoName = two.getMethod().getName();
String oneName = one.getMethod().getName();
return twoName.compareTo(oneName);
}
};

    @Override
public void orderMethods(MethodOrdererContext context) {
context.getMethodDescriptors().sort(comparator);     
}
}