Writing Custom Suppliers for Your JUnit @Theory
Writing Custom Suppliers for Your JUnit @Theory
Join the DZone community and get the full member experience.
Join For FreeDiscover How Software Testing Kills DevOps and how software testing is still dominated by yesterday's tools and outdated processes. Brought to you in partnership with Tricentis.
Introduction
JUnit's "Theories" runner is an (mostly) undocumented experimental runner that allows you to run your @Test method (your @Theory) against all possible combinations of parameters (your @Datapoints) you can throw at it.
You can leverage Theories tool on critical, complex services to ensure they are "bullet-proof" against any parameter and parameter combinations you can think of.
This is particulary useful to test legacy, badly-coded services, that take a lot of parameters, with some of them functionnaly excluding some others, although this is "technically possible".
A simple example
Numerous articles have been written on the subject, refer to them for a proper introduction, but here is a short example:
@RunWith(Theories.class)
public class GuessTheMurdererTest {
public static class Suspect {
private String name;
private String color;
@Override
public String toString() {
return String.format("%s (%s)", name, color);
}
public Suspect(String name, String color) {
this.name = name;
this.color = color;
}
public String getName() {
return name;
}
public String getColor() {
return color;
}
}
@DataPoints
public static Suspect[] whoMadeIt = {
new Suspect("Miss Rose", "Pink"),
new Suspect("Mrs. Peacock", "Blue"),
new Suspect("Professor Plum", "Purple")
};
@DataPoints
public static String[] whereWasIt = {
"Library", "Lounge"
};
@Theory
public void dummyTest(Suspect murderer, String murderScene) {
System.out.println(String.format("%s murdered Colonel Mustard in the %s", murderer, murderScene));
}
}
The @Theory method here is just a dummy method that prints all possible combinations of parameters specified by @DataPoints.
It will be executed with every possible combination of Suspect and room, hence the following output:
Miss Rose (Pink) murdered Colonel Mustard in the Library Miss Rose (Pink) murdered Colonel Mustard in the Lounge Mrs. Peacock (Blue) murdered Colonel Mustard in the Library Mrs. Peacock (Blue) murdered Colonel Mustard in the Lounge Professor Plum (Purple) murdered Colonel Mustard in the Library Professor Plum (Purple) murdered Colonel Mustard in the Lounge Professor Plum (Purple) murdered Colonel Mustard in the Lounge
Ok, that's all fine, but here's the catch...
The catch: multiple parameters with the same type
Suppose we're trying to test some old legacy service method, in order to find that unique combination of parameters that's been causing a "random" bug for just too long...
The old method does not take a "Suspect" and "Room" object as parameters, but just 2 strings...
Applied to our example, that would be something like this:
@RunWith(Theories.class)
public class GuessTheMurdererTest {
@DataPoints
public static String[] whoMadeIt = {
"Miss Rose", "Mrs. Peacock", "Professor Plum"
}
;
@DataPoints
public static String[] whereWasIt = {
"Library", "Lounge"
}
;
@Theory
public void dummyTest(String murderer, String murderScene) {
System.out.println(String.format("%s murdered Colonel Mustard in the %s", murderer, murderScene));
}
}
No big deal right? Let's run it, then...
Miss Rose murdered Colonel Mustard in the Miss Rose Miss Rose murdered Colonel Mustard in the Mrs. Peacock Miss Rose murdered Colonel Mustard in the Professor Plum Miss Rose murdered Colonel Mustard in the Library Miss Rose murdered Colonel Mustard in the Lounge Mrs. Peacock murdered Colonel Mustard in the Miss Rose Mrs. Peacock murdered Colonel Mustard in the Mrs. Peacock Mrs. Peacock murdered Colonel Mustard in the Professor Plum Mrs. Peacock murdered Colonel Mustard in the Library Mrs. Peacock murdered Colonel Mustard in the Lounge Professor Plum murdered Colonel Mustard in the Miss Rose Professor Plum murdered Colonel Mustard in the Mrs. Peacock Professor Plum murdered Colonel Mustard in the Professor Plum Professor Plum murdered Colonel Mustard in the Library Professor Plum murdered Colonel Mustard in the Lounge Library murdered Colonel Mustard in the Miss Rose Library murdered Colonel Mustard in the Mrs. Peacock Library murdered Colonel Mustard in the Professor Plum Library murdered Colonel Mustard in the Library Library murdered Colonel Mustard in the Lounge Lounge murdered Colonel Mustard in the Miss Rose Lounge murdered Colonel Mustard in the Mrs. Peacock Lounge murdered Colonel Mustard in the Professor Plum Lounge murdered Colonel Mustard in the Library Lounge murdered Colonel Mustard in the Lounge
Wait, what?! A suspect murdered this poor Colonel in another suspect?! A Library murdered someone? Huh? Sounds weird!
In fact, the Theories runner picks potential parameter assignements based on their type.
Here, the "murderer" and "murderScene" both are String... that's why the list of possible value is the union of both datapoints values.
This can really be cumbersome in a real-life project.
Although you can get around this using enums, or custom providers, that's a lot of boiler-plate...
Workaround: a "named" custom supplier
What if we could add a "name" to the @Datapoints, and use it as a reference for possible values of a specific parameter?
Thankfully, JUnit let's you define custom data suppliers.
Understand the custom supplier: a dummy supplier
We need to define a new annotation, that specifies the supplier class that will be used for a parameter annotated with this annotation
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.PARAMETER)
@ParametersSuppliedBy(FooBarSupplier.class)
public @interface FooBar{
}
Then we need to define the Supplier class.
public static class FooBarSupplier extends ParameterSupplier {
@Override
public List<PotentialAssignment> getValueSources(ParameterSignature sig) {
List<PotentialAssignment> list = new ArrayList<PotentialAssignment>();
list.add((PotentialAssignment.forValue("foo", "bar1"));
list.add((PotentialAssignment.forValue("foo", "bar2"));
list.add((PotentialAssignment.forValue("foo", "bar2"));
return list;
}
}
This supplier will just supply 3 hard-coded values, for demonstration purpose
In order to use this supplier:
@Theory
public static void fooBarTheory(@FooBar String bar) {
...
}
This method will be called 3 times, with "bar1", "bar2", "bar3"
Write the custom supplier
Let's use this feature and add our own supplier:@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface DataPointsRef {
String value();
}
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.PARAMETER)
@ParametersSuppliedBy(WithDataPointsSupplier.class)
public @interface WithDataPoints {
Class<?> clazz();
String name();
}
public static class WithDataPointsSupplier extends ParameterSupplier {
@Override
public List<PotentialAssignment> getValueSources(ParameterSignature sig) {
List<PotentialAssignment> list = new ArrayList<PotentialAssignment>();
WithDataPoints ref = (WithDataPoints) sig.getAnnotation(WithDataPoints.class);
Field[] fields = ref.clazz().getFields();
for (Field field : fields) {
DataPointsRef dpSupplier = field.getAnnotation(DataPointsRef.class);
if (dpSupplier != null) {
if (dpSupplier.value().equals(ref.name())) {
Class<?> fieldType = field.getType();
if (!fieldType.isArray()) {
throw new RuntimeException("Referenced Datapoint must be an array.");
}
try {
Object values = field.get(null);
for (int i = 0; i < Array.getLength(values); i++) {
Object value = Array.get(values, i);
list.add(PotentialAssignment.forValue(withTag.name(), value));
}
} catch (IllegalArgumentException e) {
throw new RuntimeException(e);
}
catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
}
}
}
return list;
}
}
You could easily modify this to load a bean from a Spring Context, read a properties file, get data from a database, ...
Use the custom supplier
Let's use this new supplier and fix our test case:
@RunWith(Theories.class)
public class GuessTheMurdererTest {
@DataPointsRef("suspects")
public static String[] whoMadeIt = {
"Miss Rose", "Mrs. Peacock", "Professor Plum"
};
@DataPointsRef("rooms")
public static String[] whereWasIt = {
"Library", "Lounge"
};
@Theory
public void dummyTest(
@WithDataPoints(clazz=GuessTheMurdererTest.class, name="suspects")
String murderer,
@WithDataPoints(clazz=GuessTheMurdererTest.class, name="rooms")
String murderScene) {
System.out.println(String.format("%s murdered Colonel Mustard in the %s", murderer, murderScene));
}
}
Our @Datapoints can now have a name, and the @Theory parameters get a new annotation referencing this name.
If we run this test, we get this output:
Miss Rose murdered Colonel Mustard in the Library Miss Rose murdered Colonel Mustard in the Lounge Mrs. Peacock murdered Colonel Mustard in the Library Mrs. Peacock murdered Colonel Mustard in the Lounge Professor Plum murdered Colonel Mustard in the Library Professor Plum murdered Colonel Mustard in the Lounge
Much better...
Let's push it just a little more...
We just purchased the new edition of the game, new suspects are available...
@RunWith(Theories.class)
public class GuessTheMurdererTest {
@DataPointsRef("suspects")
public static String[] whoMadeIt = {
"Miss Rose", "Mrs. Peacock", "Professor Plum"
}
;
@DataPointsRef("suspects")
public static String[] orMaybeItWasSomeoneElse = {
"Mr. Slate-Grey", "Captain Brown"
}
;
@DataPointsRef("rooms")
public static String[] whereWasIt = {
"Hall", "Kitchen"
}
;
@Theory
public void dummyTest(
@WithDataPoints(clazz=GuessTheMurdererTest.class, name="suspects")
String murderer,
@WithDataPoints(clazz=GuessTheMurdererTest.class, name="rooms")
String murderScene) {
System.out.println(String.format("%s murdered Colonel Mustard in the %s", murderer, murderScene));
}
}
Miss Rose murdered Colonel Mustard in the Hall Miss Rose murdered Colonel Mustard in the Kitchen Mrs. Peacock murdered Colonel Mustard in the Hall Mrs. Peacock murdered Colonel Mustard in the Kitchen Professor Plum murdered Colonel Mustard in the Hall Professor Plum murdered Colonel Mustard in the Kitchen Mr. Slate-Grey murdered Colonel Mustard in the Hall Mr. Slate-Grey murdered Colonel Mustard in the Kitchen Captain Brown murdered Colonel Mustard in the Hall Captain Brown murdered Colonel Mustard in the Kitchen
Good! Datapoints values are now aggregated by name rather than type.
Enhancing the custom supplier: introducing "Tags"
Suppose we buy yet another game's extention, the murderer now has an accomplice.
Some suspects can only be "murderers", some can only be "accomplices", some can be both "murderer" or "accomplice"
We could easily imagine extending the Supplier to add multiple "tags" to a @Datapoint.
Let's modify our annotations and supplier:
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface TaggedDataPoints {
//Multiple values allowed
String[] value();
}
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.PARAMETER)
@ParametersSuppliedBy(TaggedDataPointsSupplier.class)
public @interface WithTags {
Class<?> clazz();
String name();
}
public static class TaggedDataPointsSupplier extends ParameterSupplier {
@Override
public List<PotentialAssignment> getValueSources(ParameterSignature sig) {
List<PotentialAssignment> list = new ArrayList<PotentialAssignment>();
WithTags withTag = (WithTags) sig.getAnnotation(WithTags.class);
Field[] fields = withTag.clazz().getFields();
for (Field field : fields) {
TaggedDataPoints taggedDataPoints = field.getAnnotation(TaggedDataPoints.class);
if (taggedDataPoints != null) {
//browse tags, an select values if one matches
for (String tag : taggedDataPoints.value()) {
if (tag.equals(withTag.name())) {
Class<?> fieldType = field.getType();
if (!fieldType.isArray()) {
throw new RuntimeException("Referenced Datapoint must be an array.");
}
try {
Object values = field.get(null);
for (int i = 0; i < Array.getLength(values); i++) {
Object value = Array.get(values, i);
list.add(PotentialAssignment.forValue(withTag.name(), value));
}
}
catch (IllegalArgumentException e) {
throw new RuntimeException(e);
}
catch (IllegalAccessException e) {
throw new RuntimeException(e);
}
}
}
}
}
return list;
}
}
We can now distinguish those new "suspects" and "accomplices", and add an accomplice to the @Theory:
@RunWith(Theories.class)
public class GuessTheMurdererTest {
@TaggedDataPoints("suspects")
public static String[] suscpectsOnly = {
"Miss Rose", "Mrs. Peacock", "Professor Plum"
};
@TaggedDataPoints({ "suspects", "accomplices" })
public static String[] suspectsOrAccomplices = {
"Mr. Slate-Grey"
};
@TaggedDataPoints("accomplices")
public static String[] accomplicesOnly = {
"Captain Brown"
};
@TaggedDataPoints("rooms")
public static String[] whereWasIt = {
"Hall", "Kitchen"
};
@Theory
public void dummyTest(
@WithTags(clazz = GuessTheMurdererTest.class, name = "suspects") String murderer,
@WithTags(clazz = GuessTheMurdererTest.class, name = "accomplices") String accomplice,
@WithTags(clazz = GuessTheMurdererTest.class, name = "rooms") String murderScene) {
// Ensure no-one is accompliced to himself...
Assume.assumeTrue(!murderer.equalsIgnoreCase(accomplice));
System.out.println(String.format("%s and his/her accomplice %s both murdered Colonel Mustard in the %s", murderer, accomplice, murderScene));
// Assert that Captain brown cannot be the murderer
Assert.assertFalse(murderer.equalsIgnoreCase("Captain Brown"));
// Assert that Professor Plum cannot be an accomplice
Assert.assertFalse(accomplice.equalsIgnoreCase("Professor Plum"));
}
}
If everything went fine, asserts are all ok, and you get:
Miss Rose and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Hall Miss Rose and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Kitchen Miss Rose and his/her accomplice Captain Brown both murdered Colonel Mustard in the Hall Miss Rose and his/her accomplice Captain Brown both murdered Colonel Mustard in the Kitchen Mrs. Peacock and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Hall Mrs. Peacock and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Kitchen Mrs. Peacock and his/her accomplice Captain Brown both murdered Colonel Mustard in the Hall Mrs. Peacock and his/her accomplice Captain Brown both murdered Colonel Mustard in the Kitchen Professor Plum and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Hall Professor Plum and his/her accomplice Mr. Slate-Grey both murdered Colonel Mustard in the Kitchen Professor Plum and his/her accomplice Captain Brown both murdered Colonel Mustard in the Hall Professor Plum and his/her accomplice Captain Brown both murdered Colonel Mustard in the Kitchen Mr. Slate-Grey and his/her accomplice Captain Brown both murdered Colonel Mustard in the Hall Mr. Slate-Grey and his/her accomplice Captain Brown both murdered Colonel Mustard in the Kitchen
Pretty nice isn't it?
A few final words...
This could give you a neat way of dividing values of a same list according to a fonctional property.
Suppose we get a "Folder" list, some of them are "Input folders", some others are "Output folders", and a few are "Input/Output folders"
Of course, ideally, you would create different objects, but if the tested method just takes two String, that's a lot of boilerplate...
Additionally, this could be used as an easy way to qualify the test dataitself, some information that does not belong to the "Domain", but belongs to the test itself.
eg: you might want to test some theories against only a subset of the @Datapoints.
Well, that's it for today, I hope this article gave you some insights on how you can use @Theory, and how you can improve the data you feed to it using custom suppliers.
Get a reality check on the Best (and Worst) Use Cases of Testing in AI and how AI will soon become part of our day-to-day quality engineering process. Brought to you in partnership with Tricentis.
Like This Article? Read More From DZone
Opinions expressed by DZone contributors are their own.
{{ parent.title || parent.header.title}}
{{ parent.tldr }}
{{ parent.linkDescription }}
{{ parent.urlSource.name }}