Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Writing Custom Suppliers for Your JUnit @Theory

DZone's Guide to

Writing Custom Suppliers for Your JUnit @Theory

· DevOps Zone
Free Resource

The DevOps Zone is brought to you in partnership with Sonatype Nexus. The Nexus Suite helps scale your DevOps delivery with continuous component intelligence integrated into development tools, including Eclipse, IntelliJ, Jenkins, Bamboo, SonarQube and more. Schedule a demo today

Introduction

JUnit's "Theories"  runner is an (mostly) undocumented experimental runner that allows you to run your @Test method (your @Theory)  against all possible combinations of parameters (your @Datapoints) you can throw at it.

You can leverage Theories tool on critical, complex services to ensure they are "bullet-proof" against any parameter and parameter combinations you can think of.

This is particulary useful to test legacy, badly-coded services, that take a lot of parameters, with some of them functionnaly excluding some others, although this is "technically possible".

A simple example

Numerous articles have been written on the subject, refer to them for a proper introduction, but here is a short example:

@RunWith(Theories.class)
public class GuessTheMurdererTest {

    public static class Suspect {
        private String name;
        private String color;

        @Override
        public String toString() {
            return String.format("%s (%s)", name, color);
        }

        public Suspect(String name, String color) {
            this.name = name;
            this.color = color;
        }

        public String getName() {
            return name;
        }

        public String getColor() {
            return color;
        }
    }

    @DataPoints
    public static Suspect[] whoMadeIt = {
        new Suspect("Miss Rose", "Pink"),
        new Suspect("Mrs. Peacock", "Blue"),
        new Suspect("Professor Plum", "Purple")
    };
    @DataPoints
    public static String[] whereWasIt = {
        "Library", "Lounge"
    };

    
    @Theory
    public void dummyTest(Suspect murderer, String murderScene) {
        System.out.println(String.format("%s murdered Colonel Mustard in the %s", murderer, murderScene));
    }
}

The @Theory method here is just a dummy method that prints all possible combinations of parameters specified by @DataPoints.

It will be executed with every possible combination of Suspect and room, hence the following output:

Miss Rose (Pink) murdered Colonel Mustard in the Library
Miss Rose (Pink) murdered Colonel Mustard in the Lounge
Mrs. Peacock (Blue) murdered Colonel Mustard in the Library
Mrs. Peacock (Blue) murdered Colonel Mustard in the Lounge
Professor Plum (Purple) murdered Colonel Mustard in the Library
Professor Plum (Purple) murdered Colonel Mustard in the Lounge
Professor Plum (Purple) murdered Colonel Mustard in the Lounge

Ok, that's all fine, but here's the catch...

The catch: multiple parameters with the same type

Suppose we're trying to test some old legacy service method, in order to find that unique combination of parameters that's been causing a "random" bug for just too long...

The old method does not take a "Suspect" and "Room" object as parameters, but just 2 strings...

Applied to our example, that would be something like this:

@RunWith(Theories.class)
public class GuessTheMurdererTest {
    @DataPoints
    public static String[] whoMadeIt = {
        "Miss Rose", "Mrs. Peacock", "Professor Plum"
    }
    ;
    @DataPoints
    public static String[] whereWasIt = {
        "Library", "Lounge"
    }
    ;
    @Theory
    public void dummyTest(String murderer, String murderScene) {
        System.out.println(String.format("%s murdered Colonel Mustard in the %s", murderer, murderScene));
    }
}

No big deal right? Let's run it, then...

Miss Rose murdered Colonel Mustard in the Miss Rose
Miss Rose murdered Colonel Mustard in the Mrs. Peacock
Miss Rose murdered Colonel Mustard in the Professor Plum
Miss Rose murdered Colonel Mustard in the Library
Miss Rose murdered Colonel Mustard in the Lounge
Mrs. Peacock murdered Colonel Mustard in the Miss Rose
Mrs. Peacock murdered Colonel Mustard in the Mrs. Peacock
Mrs. Peacock murdered Colonel Mustard in the Professor Plum
Mrs. Peacock murdered Colonel Mustard in the Library
Mrs. Peacock murdered Colonel Mustard in the Lounge
Professor Plum murdered Colonel Mustard in the Miss Rose
Professor Plum murdered Colonel Mustard in the Mrs. Peacock
Professor Plum murdered Colonel Mustard in the Professor Plum
Professor Plum murdered Colonel Mustard in the Library
Professor Plum murdered Colonel Mustard in the Lounge
Library murdered Colonel Mustard in the Miss Rose
Library murdered Colonel Mustard in the Mrs. Peacock
Library murdered Colonel Mustard in the Professor Plum
Library murdered Colonel Mustard in the Library
Library murdered Colonel Mustard in the Lounge
Lounge murdered Colonel Mustard in the Miss Rose
Lounge murdered Colonel Mustard in the Mrs. Peacock
Lounge murdered Colonel Mustard in the Professor Plum
Lounge murdered Colonel Mustard in the Library
Lounge murdered Colonel Mustard in the Lounge

Wait, what?! A suspect murdered this poor Colonel in another suspect?! A Library murdered someone? Huh? Sounds weird!

In fact, the Theories runner picks potential parameter assignements based on their type.
Here, the "murderer" and "murderScene" both are String... that's why the list of possible value is the union of both datapoints values.

This can really be cumbersome in a real-life project.
Although you can get around this using enums, or custom providers, that's a lot of boiler-plate...

Workaround: a "named" custom supplier

What if we could add a "name" to the @Datapoints, and use it as a reference for possible values of a specific parameter?

Thankfully, JUnit let's you define custom data suppliers.

Understand the custom supplier: a dummy supplier

We need to define a new annotation, that specifies the supplier class that will be used for a parameter annotated with this annotation

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.PARAMETER)
@ParametersSuppliedBy(FooBarSupplier.class)
public @interface FooBar{
}

Then we need to define the Supplier class.

public static class FooBarSupplier extends ParameterSupplier {
    @Override
    public List<PotentialAssignment> getValueSources(ParameterSignature sig) {
        List<PotentialAssignment> list = new ArrayList<PotentialAssignment>();
        list.add((PotentialAssignment.forValue("foo", "bar1"));
        list.add((PotentialAssignment.forValue("foo", "bar2"));
        list.add((PotentialAssignment.forValue("foo", "bar2"));
        return list;
   }
}

This supplier will just supply 3 hard-coded values, for demonstration purpose

In order to use this supplier:

@Theory
public static void fooBarTheory(@FooBar String bar) {
...
}

This method will be called 3 times, with "bar1", "bar2", "bar3"

Write the custom supplier

Let's use this feature and add our own supplier:
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface DataPointsRef {
    String value();
}
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.PARAMETER)
@ParametersSuppliedBy(WithDataPointsSupplier.class)
public @interface WithDataPoints {
    Class<?> clazz();
    String name();
}
public static class WithDataPointsSupplier extends ParameterSupplier {
    @Override
    public List<PotentialAssignment> getValueSources(ParameterSignature sig) {
        List<PotentialAssignment> list = new ArrayList<PotentialAssignment>();
        WithDataPoints ref = (WithDataPoints) sig.getAnnotation(WithDataPoints.class);
        Field[] fields = ref.clazz().getFields();
        for (Field field : fields) {
            DataPointsRef dpSupplier = field.getAnnotation(DataPointsRef.class);
            if (dpSupplier != null) {
                if (dpSupplier.value().equals(ref.name())) {
                    Class<?> fieldType = field.getType();
                    if (!fieldType.isArray()) {
                        throw new RuntimeException("Referenced Datapoint must be an array.");
                    }
                    try {
                        Object values = field.get(null);
                        for (int i = 0; i < Array.getLength(values); i++) {
                            Object value = Array.get(values, i);
                            list.add(PotentialAssignment.forValue(withTag.name(), value));
                        }
                    } catch (IllegalArgumentException e) {
                        throw new RuntimeException(e);
                    }
                    catch (IllegalAccessException e) {
                        throw new RuntimeException(e);
                    }
                }
            }
        }
        return list;
    }
}
This is a just sample implementation that tries to find a public field in a class (cpecified by the @WithDataPoints annotation on the parameter), with an array type, that has a @DataPointsRef annotation.

You could easily modify this to load a bean from a Spring Context, read a properties file, get data from a database, ...

Use the  custom supplier

Let's use this new supplier and fix our test case:

@RunWith(Theories.class)
public class GuessTheMurdererTest {
    @DataPointsRef("suspects")
    public static String[] whoMadeIt = {
        "Miss Rose", "Mrs. Peacock", "Professor Plum"
    };
    @DataPointsRef("rooms")
    public static String[] whereWasIt = {
        "Library", "Lounge"
    };

    @Theory
    public void dummyTest(
    @WithDataPoints(clazz=GuessTheMurdererTest.class, name="suspects")
    String murderer,
    @WithDataPoints(clazz=GuessTheMurdererTest.class, name="rooms")
    String murderScene) {
        System.out.println(String.format("%s murdered Colonel Mustard in the %s", murderer, murderScene));
    }
}

Our @Datapoints can now have a name, and the @Theory parameters get a new annotation referencing this name.

If we run this test, we get this output:

Miss Rose murdered Colonel Mustard in the Library
Miss Rose murdered Colonel Mustard in the Lounge
Mrs. Peacock murdered Colonel Mustard in the Library
Mrs. Peacock murdered Colonel Mustard in the Lounge
Professor Plum murdered Colonel Mustard in the Library
Professor Plum murdered Colonel Mustard in the Lounge

Much better...

Let's push it just a little more...
We just purchased the new edition of the game, new suspects are available...

@RunWith(Theories.class)
public class GuessTheMurdererTest {
    @DataPointsRef("suspects")
    public static String[] whoMadeIt = {
        "Miss Rose", "Mrs. Peacock", "Professor Plum"
    }
    ;
    @DataPointsRef("suspects")
    public static String[] orMaybeItWasSomeoneElse = {
        "Mr. Slate-Grey", "Captain Brown"
    }
    ;
    @DataPointsRef("rooms")
    public static String[] whereWasIt = {
        "Hall", "Kitchen"
    }
    ;
    @Theory
    public void dummyTest(
    @WithDataPoints(clazz=GuessTheMurdererTest.class, name="suspects")
    String murderer,
    @WithDataPoints(clazz=GuessTheMurdererTest.class, name="rooms")
    String murderScene) {
        System.out.println(String.format("%s murdered Colonel Mustard in the %s", murderer, murderScene));
    }
}
Miss Rose murdered Colonel Mustard in the Hall
Miss Rose murdered Colonel Mustard in the Kitchen
Mrs. Peacock murdered Colonel Mustard in the Hall
Mrs. Peacock murdered Colonel Mustard in the Kitchen
Professor Plum murdered Colonel Mustard in the Hall
Professor Plum murdered Colonel Mustard in the Kitchen
Mr. Slate-Grey murdered Colonel Mustard in the Hall
Mr. Slate-Grey murdered Colonel Mustard in the Kitchen
Captain Brown murdered Colonel Mustard in the Hall
Captain Brown murdered Colonel Mustard in the Kitchen

Good! Datapoints values are now aggregated by name rather than type.

Enhancing the custom supplier: introducing "Tags"

Suppose we buy yet another game's extention, the murderer now has an accomplice.
Some suspects can only be "murderers", some can only be "accomplices", some can be both "murderer" or "accomplice"

We could easily imagine extending the Supplier to add multiple "tags" to a @Datapoint.
Let's modify our annotations and supplier:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface TaggedDataPoints {
    //Multiple values allowed
    String[] value();
}

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.PARAMETER)
@ParametersSuppliedBy(TaggedDataPointsSupplier.class)
public @interface WithTags {
    Class<?> clazz();
    String name();
}
public static class TaggedDataPointsSupplier extends ParameterSupplier {
    @Override
    public List<PotentialAssignment> getValueSources(ParameterSignature sig) {
        List<PotentialAssignment> list = new ArrayList<PotentialAssignment>();
        WithTags withTag = (WithTags) sig.getAnnotation(WithTags.class);
        Field[] fields = withTag.clazz().getFields();
        for (Field field : fields) {
            TaggedDataPoints taggedDataPoints = field.getAnnotation(TaggedDataPoints.class);
            if (taggedDataPoints != null) {
                //browse tags, an select values if one matches
                for (String tag : taggedDataPoints.value()) {
                    if (tag.equals(withTag.name())) {
                        Class<?> fieldType = field.getType();
                        if (!fieldType.isArray()) {
                            throw new RuntimeException("Referenced Datapoint must be an array.");
                        }
                        try {
                            Object values = field.get(null);
                            for (int i = 0; i < Array.getLength(values); i++) {
                                Object value = Array.get(values, i);
                                list.add(PotentialAssignment.forValue(withTag.name(), value));
                            }
                        }
                        catch (IllegalArgumentException e) {
                            throw new RuntimeException(e);
                        }
                        catch (IllegalAccessException e) {
                            throw new RuntimeException(e);
                        }
                    }
                }
            }
        }
        return list;
    }
}

We can now distinguish those new "suspects" and "accomplices", and add an accomplice to the @Theory:

@RunWith(Theories.class)
public class GuessTheMurdererTest {
    @TaggedDataPoints("suspects")
    public static String[] suscpectsOnly = {
        "Miss Rose", "Mrs. Peacock", "Professor Plum"
    };
    @TaggedDataPoints({ "suspects", "accomplices" })
    public static String[] suspectsOrAccomplices = {
        "Mr. Slate-Grey"
    };
    @TaggedDataPoints("accomplices")
    public static String[] accomplicesOnly = {
        "Captain Brown"
    };
    @TaggedDataPoints("rooms")
    public static String[] whereWasIt = {
        "Hall", "Kitchen"
    };

    @Theory
    public void dummyTest(
    @WithTags(clazz = GuessTheMurdererTest.class, name = "suspects") String murderer,
    @WithTags(clazz = GuessTheMurdererTest.class, name = "accomplices") String accomplice,
    @WithTags(clazz = GuessTheMurdererTest.class, name = "rooms") String murderScene) {
        // Ensure no-one is accompliced to himself...
        Assume.assumeTrue(!murderer.equalsIgnoreCase(accomplice));
        System.out.println(String.format("%s and his/her accomplice %s both murdered Colonel Mustard in the %s", murderer, accomplice, murderScene));
        // Assert that Captain brown cannot be the murderer
        Assert.assertFalse(murderer.equalsIgnoreCase("Captain Brown"));
        // Assert that Professor Plum cannot be an accomplice
        Assert.assertFalse(accomplice.equalsIgnoreCase("Professor Plum"));
    }
}

If everything went fine, asserts are all ok, and you get:

Miss Rose and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Hall
Miss Rose and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Kitchen
Miss Rose and his/her accomplice Captain Brown both murdered Colonel Mustard in the Hall
Miss Rose and his/her accomplice Captain Brown both murdered Colonel Mustard in the Kitchen
Mrs. Peacock and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Hall
Mrs. Peacock and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Kitchen
Mrs. Peacock and his/her accomplice Captain Brown both murdered Colonel Mustard in the Hall
Mrs. Peacock and his/her accomplice Captain Brown both murdered Colonel Mustard in the Kitchen
Professor Plum and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Hall
Professor Plum and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Kitchen
Professor Plum and his/her accomplice Captain Brown both murdered Colonel Mustard in the Hall
Professor Plum and his/her accomplice Captain Brown both murdered Colonel Mustard in the Kitchen
Mr. Slate-Grey and his/her accomplice Captain Brown both murdered Colonel Mustard in the Hall
Mr. Slate-Grey and his/her accomplice Captain Brown both murdered Colonel Mustard in the Kitchen

Pretty nice isn't it?

A few final words...

This could give you a neat way of dividing values of a same list according to a fonctional property.

Suppose we get a "Folder" list, some of them are "Input folders", some others are "Output folders", and a few are "Input/Output folders"
Of course, ideally, you would create different objects, but if the tested method just takes two String, that's a lot of boilerplate...

Additionally, this could be used as an easy way to qualify the test dataitself, some information that does not belong to the "Domain", but belongs to the test itself.
eg: you might want to test some theories against only a subset of the @Datapoints.

Well, that's it for today, I hope this article gave you some insights on how you can use @Theory, and how you can improve the data you feed to it using custom suppliers.

The DevOps Zone is brought to you in partnership with Sonatype Nexus. Use the Nexus Suite to automate your software supply chain and ensure you're using the highest quality open source components at every step of the development lifecycle. Get Nexus today

Topics:

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}