Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

JUnit 5 Extension Models Explained

DZone's Guide to

JUnit 5 Extension Models Explained

An in-depth look at the JUnit 5 extension model, how it's improved over JUnit 4, and how users can implement their own additions to JUnit.

· DevOps Zone
Free Resource

The Nexus Suite is uniquely architected for a DevOps native world and creates value early in the development pipeline, provides precise contextual controls at every phase, and accelerates DevOps innovation with automation you can trust. Read how in this ebook.

We already know quite a lot about the next version of Java’s most ubiquitous testing framework. Let’s now look at the JUnit 5 extension model, which will allow libraries and frameworks to add and implement their own additions to JUnit.

Overview

Most of what you will read here and more can be found in the emerging JUnit 5 user guide. Note that it is based on an alpha version and hence subject to change.

Indeed, we are encouraged to open issues or pull requests so that JUnit 5 can improve further. Please make use of this opportunity! It is our chance to help JUnit help us, so if something you see here could be improved, make sure to take it upstream.

This post will be updated when necessary. The code samples I've shown here can be found on GitHub.

JUnit 4 Extension Model

Let’s first look at how JUnit 4 solved the problem. It has two, partly competing extension mechanisms: runners and rules.

Runners

Test runners manage a test’s life cycle: instantiation, calling setup and teardown methods, running the test, handling exceptions, sending notification, etc., and JUnit 4 provides an implementation that does all of that.

In 4.0 there was only one way to extend JUnit: Create a new runner and annotate your test class with @RunWith(MyRunner.class) so that JUnit uses it instead of its own implementation.

This mechanism is pretty heavyweight and inconvenient for little extensions. And it had a very severe limitation: There could always only be one runner per test class, which made it impossible to compose them. So there was no way to benefit from the features of, e.g., both the Mockito and the Spring runners at the same time.

Rules

To overcome these limitations, JUnit 4.7 introduced rules, which are annotated fields of the test class. JUnit 4 wraps test methods (and other actions) into a statement and passes it to the rules. They can then execute some code before and after executing the statement. Additionally, test methods usually call methods on rule instances during execution.

An example is the temporary folder rule:

publicstaticclassHasTempFolder{

@Rule

publicTemporaryFolder folder=newTemporaryFolder();

@Test

publicvoidtestUsingTempFolder()throwsIOException{

File createdFile=folder.newFile("myfile.txt");

File createdFolder=folder.newFolder("subfolder");

// ...

Due to the @Rule annotation, JUnit calls folder with a statement wrapping the method testUsingTempFolder. This specific rule is written so that folder creates a temporary folder, executes the test, and deletes the folder afterwards. The test itself can then create files and folders in the temporary folder.

Other rules might run the test in Swing’s Event Dispatch Thread, set up and tear down a database, or let the test time out if it ran too long.

Rules were a big improvement but are generally limited to executing some code before and after a test is run. They can not help with extension that can’t be implemented within that frame.

State Of Affairs

JUnit has two competing extension mechanisms, each with its own limitations.

So since JUnit 4.7 there were two competing extension mechanisms, each with its own limitations but also with quite an overlap. This makes clean extension difficult. Additionally, composing different extensions can be problematic and will often not do what the developer hoped it would.

JUnit 5 Extension Model

The JUnit Lambda project has a couple of core principles and one of them is to “prefer extension points over features”. This translated quite literally into an integral mechanism of the new version — not the only one, but the most important one for extending JUnit 5.

Extension Points

JUnit 5 extensions can declare interest in certain junctures of the test life cycle. When the JUnit 5 engine processes a test, it steps through these junctures and calls each registered extension. In rough order of appearance, these are the extension points:

  • Test Instance Post Processing
  • BeforeAll Callback
  • Conditional Test Execution
  • BeforeEach Callback
  • Parameter Resolution
  • Exception Handling
  • AfterEach Callback
  • AfterAll Callback

(Don’t worry if it’s not all that clear what each of them does. We will look at some of them later.)

Each extension point corresponds to an interface. Their methods take arguments that capture the context at that specific point in the test’s lifecycle, e.g. the test instance and method, the test’s name, parameters, annotations, and so forth.

An extension can implement any number of those interfaces and will get called by the engine with the respective arguments. It can then do whatever it needs to implement its functionality. One detail to consider: The engine makes no guarantees when it instantiates extension and how long it keeps instances around, so they have to be stateless. Any state they need to maintain has to be written to and loaded from a store that is made available by JUnit.

After creating the extension all that is left to do is tell JUnit about it. This is as easy as adding @ExtendWith(MyExtension.class) to the test class or method that needs the extension.

Actually, a slightly less verbose and more revealing option exists. But for that we first have to look at the other pillar of JUnit’s extension model.

Custom Annotations

The JUnit 5 API is driven by annotations and the engine does a little extra work when it checks for their presences: It not only looks for annotations on classes, methods and parameters but also on other annotations. And it treats everything it finds as if it were immediately present on the examined element. Annotating annotations is possible with so-called meta-annotations and the cool thing is, all JUnit annotations are totally meta.

This makes it possible to easily create and compose annotations that are fully functional within JUnit 5:

/**

 * We define a custom annotation that:

 * - stands in for '@Test' so that the method gets executed

 * - has the tag "integration" so we can filter by that,

 *   e.g. when running tests from the command line

@Target({ElementType.TYPE,ElementType.METHOD})

@Retention(RetentionPolicy.RUNTIME)

@Test

@Tag("integration")

public@interfaceIntegrationTest{}

We can then use it like this:


@IntegrationTest

voidrunsWithCustomAnnotation(){

// this gets executed

// even though `@IntegrationTest` is not defined by JUnit

Or we can create more succinct annotations for our extensions:

@Target({ElementType.TYPE,ElementType.METHOD,ElementType.ANNOTATION_TYPE})

@Retention(RetentionPolicy.RUNTIME)

@ExtendWith(ExternalDatabaseExtension.class)

public@interfaceDatabase{}

Now we can use @Database instead of @ExtendWith(ExternalDatabaseExtension.class). And since we added ElementType.ANNOTATION_TYPE to the list of allowed targets, it is also a meta-annotation, and we or others can compose it further.

An Example

Let’s say we want to benchmark how long certain tests run. First, we create the annotation we want to use:

@Target({ElementType.TYPE,ElementType.METHOD,ElementType.ANNOTATION_TYPE})

@Retention(RetentionPolicy.RUNTIME)

@ExtendWith(BenchmarkCondition.class)

public@interfaceBenchmark{}

It already points to BenchmarkCondition, which we will implement next. This is our plan:

  • To measure the runtime of the whole test class, store the time before any test is executed.
  • To measure the runtime of individual test methods, store the time before each test,
  • After a test method executed retrieve the test’s launch time, compute, and print the resulting runtime.
  • After all tests are executed retrieve the class’s launch time, compute, and print the resulting runtime.
  • Only do any of this if the class or method is annotated with @Benchmark.

The last point might not be immediately obvious. Why would a method not annotated with @Benchmark be processed by the extension? This stems from the fact that if an extension is applied to a class, it automatically applies to all methods therein. So if our requirements state that we might want to benchmark the class but not necessarily all individual methods, we need to exclude them. We do this by checking whether they are individually annotated.

Coincidentally, the first four points directly correspond to the life cycle callbacks BeforeAll, BeforeEach, AfterEach, AfterAll, so all we have to do is implement the four corresponding interfaces. The implementations are pretty trivial, they just do what we said above:

public class BenchmarkCondition implements
BeforeAllExtensionPoint, BeforeEachExtensionPoint,
AfterEachExtensionPoint, AfterAllExtensionPoint {

private static final Namespace NAMESPACE =
Namespace.of("BenchmarkCondition");

@Override
public void beforeAll(ContainerExtensionContext context) {
if (!shouldBeBenchmarked(context))
return;

writeCurrentTime(context, LaunchTimeKey.CLASS);
}

@Override
public void beforeEach(TestExtensionContext context) {
if (!shouldBeBenchmarked(context))
return;

writeCurrentTime(context, LaunchTimeKey.TEST);
}

@Override
public void afterEach(TestExtensionContext context) {
if (!shouldBeBenchmarked(context))
return;

long launchTime = loadLaunchTime(context, LaunchTimeKey.TEST);
long runtime = currentTimeMillis() - launchTime;
print("Test", context.getDisplayName(), runtime);
}

@Override
public void afterAll(ContainerExtensionContext context) {
if (!shouldBeBenchmarked(context))
return;

long launchTime = loadLaunchTime(context, LaunchTimeKey.CLASS);
long runtime = currentTimeMillis() - launchTime;
print("Test container", context.getDisplayName(), runtime);
}

private static boolean shouldBeBenchmarked(ExtensionContext context) {
return context.getElement().isAnnotationPresent(Benchmark.class);
}

private static void writeCurrentTime(
ExtensionContext context, LaunchTimeKey key) {
context.getStore(NAMESPACE).put(key, currentTimeMillis());
}

private static long loadLaunchTime(
ExtensionContext context, LaunchTimeKey key) {
return (Long) context.getStore(NAMESPACE).remove(key);
}

private static void print(
String unit, String displayName, long runtime) {
System.out.printf("%s '%s' took %d ms.%n", unit, displayName, runtime);
}

private enum LaunchTimeKey {
CLASS, TEST
}
}

Interesting details are shouldBeBenchmarked, which uses JUnit’s API to effortlessly determine whether the current element is (meta-)annotated with @Benchmark, and writeCurrentTime/loadLaunchTime, which use the store to write and read the launch times.

You can find the code on GitHub.

The next posts will talk about conditional test execution and parameter injection and show examples for how to use the corresponding extension points. If you can’t wait, check out this post, which shows how to port two JUnit 4 rules (conditional disable and temporary folder) to JUnit 5.

Summary

We have seen that JUnit 4’s runners and rules were not ideal to create clean, powerful, and composable extensions. JUnit 5 aims to overcome their limitations with the more general concept of extension points. They allow extensions to specify at what points in a test’s life cycle they want to intervene. We have also looked at how meta-annotations enable easy creation of custom annotations.

The DevOps Zone is brought to you in partnership with Sonatype Nexus.  See how the Nexus platform infuses precise open source component intelligence into the DevOps pipeline early, everywhere, and at scale. Read how in this ebook

Topics:
junit ,unit testing

Published at DZone with permission of Nicolai Parlog, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}