Test Parameterization With JUnit 5.7: A Deep Dive Into @EnumSource

Parameterized tests allow developers to efficiently test their code with a range of input values. In the realm of JUnit testing, seasoned users have long grappled with the complexities of implementing these tests. But with the release of JUnit 5.7, a new era of test parameterization enters, offering developers first-class support and enhanced capabilities. Let's delve into the exciting possibilities that JUnit 5.7 brings to the table for parameterized testing!

Parameterization Samples From JUnit 5.7 Docs

Let's see some examples from the docs:

@ParameterizedTest
@ValueSource(strings = { "racecar", "radar", "able was I ere I saw elba" })
void palindromes(String candidate) {
    assertTrue(StringUtils.isPalindrome(candidate));
}

@ParameterizedTest
@CsvSource({
    "apple,         1",
    "banana,        2",
    "'lemon, lime', 0xF1",
    "strawberry,    700_000"
})
void testWithCsvSource(String fruit, int rank) {
    assertNotNull(fruit);
    assertNotEquals(0, rank);
}

@ParameterizedTest
@MethodSource("stringIntAndListProvider")
void testWithMultiArgMethodSource(String str, int num, List<String> list) {
    assertEquals(5, str.length());
    assertTrue(num >=1 && num <=2);
    assertEquals(2, list.size());
}

static Stream<Arguments> stringIntAndListProvider() {
    return Stream.of(
        arguments("apple", 1, Arrays.asList("a", "b")),
        arguments("lemon", 2, Arrays.asList("x", "y"))
    );
}

The @ParameterizedTest annotation has to be accompanied by one of several provided source annotations describing where to take the parameters from. The source of the parameters is often referred to as the "data provider."

I will not dive into their detailed description here: the JUnit user guide does it better than I could, but allow me to share several observations:

• The  @ValueSource is limited to providing a single parameter value only. In other words, the test method cannot have more than one argument, and the types one can use are restricted as well.
• Passing multiple arguments is somewhat addressed by @CsvSource, parsing each string into a record that is then passed as arguments field-by-field. This can easily get hard to read with long strings and/or plentiful arguments. The types one can use are also restricted — more on this later.
• All the sources that declare the actual values in annotations are restricted to values that are compile-time constants (limitation of Java annotations, not JUnit).
• @MethodSource and @ArgumentsSource provides a stream/collection of (un-typed) n-tuples that are then passed as method arguments. Various actual types are supported to represent the sequence of n-tuples, but none of them guarantee that they will fit the method's argument list. This kind of source requires additional methods or classes, but it provides no restriction on where and how to obtain the test data.

As you can see, the source types available range from the simple ones (simple to use, but limited in functionality) to the ultimately flexible ones that require more code to get working.

• Sidenote — This is generally a sign of good design: a little code is needed for essential functionality, and adding extra complexity is justified when used to enable a more demanding use case.

What does not seem to fit this hypothetical simple-to-flexible continuum, is @EnumSource. Take a look at this non-trivial example of four parameter sets with 2 values each.

• Note — While @EnumSource passes the enum's value as a single test method parameter, conceptually, the test is parameterized by enum's fields, that poses no restriction on the number of parameters.

    enum Direction {
        UP(0, '^'),
        RIGHT(90, '>'),
        DOWN(180, 'v'),
        LEFT(270, '<');

        private final int degrees;
        private final char ch;

        Direction(int degrees, char ch) {
            this.degrees = degrees;
            this.ch = ch;
        }
    }

    @ParameterizedTest
    @EnumSource
    void direction(Direction dir) {
        assertEquals(0, dir.degrees % 90);
        assertFalse(Character.isWhitespace(dir.ch));
        
        int orientation = player.getOrientation();
        player.turn(dir);
        assertEquals((orientation + dir.degrees) % 360, player.getOrientation());
    }

Just think of it: the hardcoded list of values restricts its flexibility severely (no external or generated data), while the amount of additional code needed to declare the enum makes this quite a verbose alternative over, say, @CsvSource.

But that is just a first impression. We will see how elegant this can get when leveraging the true power of Java enums.

• Sidenote: This article does not address the verification of enums that are part of your production code. Those, of course, had to be declared no matter how you choose to verify them. Instead, it focuses on when and how to express your test data in the form of enums.

When To Use It

There are situations when enums perform better than the alternatives:

Multiple Parameters per Test

When all you need is a single parameter, you likely do not want to complicate things beyond @ValueSource. But as soon as you need multiple -— say, inputs and expected results — you have to resort to @CsvSource, @MethodSource/@ArgumentsSource or @EnumSource.

In a way, enum lets you "smuggle in" any number of data fields.

So when you need to add more test method parameters in the future, you simply add more fields in your existing enums, leaving the test method signatures untouched. This becomes priceless when you reuse your data provider in multiple tests.

For other sources, one has to employ ArgumentsAccessors or ArgumentsAggregators for the flexibility that enums have out of the box.

Type Safety

For Java developers, this should be a big one.

Parameters read from CSV (files or literals), @MethodSource or @ArgumentsSource, they provide no compile-time guarantee that the parameter count, and their types, are going to match the signature.

Obviously, JUnit is going to complain at runtime but forget about any code assistance from your IDE.

Same as before, this adds up when you reuse the same parameters for multiple tests. Using a type-safe approach would be a huge win when extending the parameter set in the future.

Custom Types

This is mostly an advantage over text-based sources, such as the ones reading data from CSV — the values encoded in the text need to be converted to Java types.

If you have a custom class to instantiate from the CSV record, you can do it using ArgumentsAggregator. However, your data declaration is still not type-safe — any mismatch between the method signature and declared data will pop up in runtime when "aggregating" arguments. Not to mention that declaring the aggregator class adds more support code needed for your parameterization to work. And we ever favored @CsvSource over @EnumSource to avoid the extra code.

Documentable

Unlike the other methods, the enum source has Java symbols for both parameter sets (enum instances) and all parameters they contain (enum fields). They provide a straightforward place where to attach documentation in its more natural form — the JavaDoc.

It is not that documentation cannot be placed elsewhere, but it will be — by definition — placed further from what it documents and thus be harder to find, and easier to become outdated.

But There Is More!

Now: Enums. Are. Classes.

It feels that many junior developers are yet to realize how powerful Java enums truly are.

In other programming languages, they really are just glorified constants. But in Java, they are convenient little implementations of a Flyweight design pattern with (much of the) advantages of full-blown classes.

Why is that a good thing?

Test Fixture-Related Behavior

As with any other class, enums can have methods added to them.

This becomes handy if enum test parameters are reused between tests — same data, just tested a little differently. To effectively work with the parameters without significant copy and paste, some helper code needs to be shared between those tests as well.

It is not something a helper class and a few static methods would not "solve."

• Sidenote: Notice that such design suffers from a Feature Envy. Test methods — or worse, helper class methods — would have to pull the data out of the enum objects to perform actions on that data.

While this is the (only) way in procedural programming, in the object-oriented world, we can do better.

Declaring the "helper" methods right in the enum declaration itself, we would move the code where the data is. Or, to put in OOP lingo, the helper methods would become the "behavior" of the test fixtures implemented as enums. This would not only make the code more idiomatic (calling sensible methods on instances over static methods passing data around), but it would also make it easier to reuse enum parameters across test cases.

Inheritance

Enums can implement interfaces with (default) methods. When used sensibly, this can be leveraged to share behavior between several data providers — several enums.

An example that easily comes to mind is separate enums for positive and negative tests. If they represent a similar kind of test fixture, chances are they have some behavior to share.

The Talk Is Cheap

Let's illustrate this on a test suite of a hypothetical convertor of source code files, not quite unlike the one performing Python 2 to 3 conversion.

To have real confidence in what such a comprehensive tool does, one would end up with an extensive set of input files manifesting various aspects of the language, and matching files to compare the conversion result against. Except for that, it is needed to verify what warnings/errors are served to the user for problematic inputs.

This is a natural fit for parameterized tests due to the large number of samples to verify, but it does not quite fit any of the simple JUnit parameter sources, as the data are somewhat complex.

See below:

    enum Conversion {
        CLEAN("imports-correct.2.py", "imports-correct.3.py", Set.of()),
        WARNINGS("problematic.2.py", "problematic.3.py", Set.of(
                "Using module 'xyz' that is deprecated"
        )),
        SYNTAX_ERROR("syntax-error.py", new RuntimeException("Syntax error on line 17"));
        // Many, many others ...

        @Nonnull
        final String inFile;
        @CheckForNull
        final String expectedOutput;
        @CheckForNull
        final Exception expectedException;
        @Nonnull
        final Set<String> expectedWarnings;

        Conversion(@Nonnull String inFile, @Nonnull String expectedOutput, @NotNull Set<String> expectedWarnings) {
            this(inFile, expectedOutput, null, expectedWarnings);
        }

        Conversion(@Nonnull String inFile, @Nonnull Exception expectedException) {
            this(inFile, null, expectedException, Set.of());
        }

        Conversion(@Nonnull String inFile, String expectedOutput, Exception expectedException, @Nonnull Set<String> expectedWarnings) {
            this.inFile = inFile;
            this.expectedOutput = expectedOutput;
            this.expectedException = expectedException;
            this.expectedWarnings = expectedWarnings;
        }

        public File getV2File() { ... }

        public File getV3File() { ... }
    }

    @ParameterizedTest
    @EnumSource
    void upgrade(Conversion con) {

        try {
            File actual = convert(con.getV2File());
            if (con.expectedException != null) {
                fail("No exception thrown when one was expected", con.expectedException);
            }
            assertEquals(con.expectedWarnings, getLoggedWarnings());
            new FileAssert(actual).isEqualTo(con.getV3File());
        } catch (Exception ex) {
            assertTypeAndMessageEquals(con.expectedException, ex);
        }
    }

The usage of enums does not restrict us in how complex the data can be. As you can see, we can define several convenient constructors in the enums, so declaring new parameter sets is nice and clean. This prevents the usage of long argument lists that often end up filled with many "empty" values (nulls, empty strings, or collections) that leave one wondering what argument #7 — you know, one of the nulls — actually represents.

Notice how enums enable the use of complex types (Set, RuntimeException) with no restrictions or magical conversions. Passing such data is also completely type-safe.

Now, I know what you think. This is awfully wordy. Well, up to a point. Realistically, you are going to have a lot more data samples to verify, so the amount of the boilerplate code will be less significant in comparison.

Also, see how related tests can be written leveraging the same enums, and their helper methods:

    @ParameterizedTest
    @EnumSource
    // Upgrading files already upgraded always passes, makes no changes, issues no warnings.
    void upgradeFromV3toV3AlwaysPasses(Conversion con) throws Exception {
        File actual = convert(con.getV3File());
        assertEquals(Set.of(), getLoggedWarnings());
        new FileAssert(actual).isEqualTo(con.getV3File());
    }

    @ParameterizedTest
    @EnumSource
    // Downgrading files created by upgrade procedure is expected to always pass without warnings.
    void downgrade(Conversion con) throws Exception {
        File actual = convert(con.getV3File());
        assertEquals(Set.of(), getLoggedWarnings());
        new FileAssert(actual).isEqualTo(con.getV2File());
    }

Some More Talk After All

Conceptually, @EnumSourceencourages you to create a complex, machine-readable description of individual test scenarios, blurring the line between data providers and test fixtures.

One other great thing about having each data set expressed as a Java symbol (enum element) is that they can be used individually; completely out of data providers/parameterized tests. Since they have a reasonable name and they are self-contained (in terms of data and behavior), they contribute to nice and readable tests.

@Test
void warnWhenNoEventsReported() throws Exception {
    FixtureXmls.Invalid events = FixtureXmls.Invalid.NO_EVENTS_REPORTED;
    
    // read() is a helper method that is shared by all FixtureXmls
    try (InputStream is = events.read()) {
        EventList el = consume(is);
        assertEquals(Set.of(...), el.getWarnings());
    }
}

Now, @EnumSource is not going to be one of your most frequently used argument sources, and that is a good thing, as overusing it would do no good. But in the right circumstances, it comes in handy to know how to use all they have to offer.