How to Create Runtime JUnit Test Cases

Sometimes, we have a situation when we need to create the JUnit test-cases at runtime. For example, we do have a file which has n number of test cases with different inputs, so we can create n number of JUnit test cases at runtime. So we have to extend the features of JUnit to provide the reusability of tests. So, for the same piece of test logic, we can have n number of JUnit test cases placed with different inputs. I have introduced the custom runner to achieve the same.

Basically, we need to extend the org.junit.runners.BlockJUnit4ClassRunner concrete class in my JUnitFactoryRunner concrete class. JUnitFactoryRunner is a runner that collects and processes methods. There are two concerns — one is collecting the tests and the second is executing them. JUnitFactoryRunner has the ability to hold the number of tests from our test class.

The JUnitFactoryRunner has the advantage of collecting all the tests in one go. JUnitFactoryRunner has the flexibility to collect tests, which are marked with the annotation @Test (by default), and to collect tests that are marked with annotation @JUnitTestFactory (my implementation). So it will look for test classes for any methods marked with @JUnitTestFactory.

I have one more of my own annotations — @JUnitFactoryTest. So, any method in the factory-generated objects marked with @JUnitFactoryTest will be executed.

I have extended the org.junit.runners.model.FrameworkMethod concrete class in my JUnitFrameworkFactory concrete class. This class knows how to execute a method on an object. In JUnit, the class that handles @Test methods is called org.junit.runners.model.FrameworkMethod. So here, I have followed the same tactic to follow the JUnit API. Here, my JUnitFrameworkFactory takes the supplied target instance rather than auto-generated target created by FrameworkMethod.

Technical Implementation

JUnitFactoryRunne.java

import java.lang.reflect.Modifier;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedList;
import java.util.List;
import org.junit.After;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.runners.BlockJUnit4ClassRunner;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.InitializationError;
import org.junit.runners.model.TestClass;

/**
* Custom JUnit Runner
* This is the base class to create the runtime JUnit TC.
*/
public class JUnitFactoryRunner extends BlockJUnit4ClassRunner {
    protected LinkedList<FrameworkMethod> tests = new LinkedList<FrameworkMethod>();
    /**
    * Creates a customized BlockJUnit4ClassRunner to run cls.
    * It throws the InitializationError if the test class is malformed.
    **/
    public JUnitFactoryRunner(Class<?> cls) throws InitializationError {
        super(cls);
        try {
            computeTests();
        } catch (Exception e) {
        }
    } 
    /**
    * To ensure the test class constructor is called at least
    * once during testing. This is adding all the tests
    * which will be entertained as new fixture for running a test.
    *
    * @throws Exception
    */
    protected void computeTests() throws Exception {
        tests.addAll(super.computeTestMethods());
        tests.addAll(computeFactoryTests());
    }
    /**
    * Find all methods in our test class marked with @JUnitTestFactory,
    * and for each @JUnitTestFactory, find any methods marked with @JUnitFactoryTest,
    * and add them to the List.
    *
    * @return
    * @throws Exception
    */
    protected Collection<? extends FrameworkMethod> computeFactoryTests() throws Exception {
        List<JUnitFrameworkFactory> tests = new LinkedList<JUnitFrameworkFactory>();
        for (FrameworkMethod method: getTestClass().getAnnotatedMethods(JUnitTestFactory.class)) {
            if (! Modifier.isStatic(method.getMethod().getModifiers())) {
                throw new InitializationError("Exception during initialization as method must be static.");
            }
            Object instances = method.getMethod().invoke(getTestClass().getJavaClass());
            if (instances.getClass().isArray()) {
                instances = Arrays.asList((Object[]) instances);
            }
            if (! (instances instanceof Iterable<?>)) {
                instances = Collections.singletonList(instances);
            }
            for (Object instance: (Iterable<?>) instances) {
                for (FrameworkMethod m: new TestClass(instance.getClass()).getAnnotatedMethods(JUnitFactoryTest.class))
                tests.add(new JUnitFrameworkFactory(m.getMethod(), instance, method.getName()));
            }
        }
        return tests;
    }
    /**Returns the methods that run tests.**/
    @Override
    protected List<FrameworkMethod> computeTestMethods() {
        return tests;
    }
    @Override
    protected void validateInstanceMethods(List<Throwable> errors) {
        validatePublicVoidNoArgMethods(After.class, false, errors);
        validatePublicVoidNoArgMethods(Before.class, false, errors);
        validatePublicVoidNoArgMethods(BeforeClass.class, true, errors);
        validateTestMethods(errors);
    }
} 

JUnitFactoryTest.java

import java.lang.annotation.ElementType; 
import java.lang.annotation.Retention; 
import java.lang.annotation.RetentionPolicy; 
import java.lang.annotation.Target;   
/**
* Here RetentionPolicy.RUNTIME annotations are to be recorded
* in the class file by the compiler and retained by the VM at
* runtime, so they may be read reflectively.
*/
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME) 
public @interface JUnitFactoryTest {}  

JUnitFrameworkFactory.java

import java.lang.reflect.Method;
import org.junit.runners.model.FrameworkMethod;
/**
* Represents a method on a test class to be invoked at the appropriate point in test
* execution. These methods are usually marked with an annotation (such as @Test, @Before,
* @After, @BeforeClass, @AfterClass, etc.)
*/
public class JUnitFrameworkFactory extends FrameworkMethod {
    private Object target;
    private String name;
    /** Returns a new FrameworkMethod for method **/
    public JUnitFrameworkFactory(Method method, Object target, String name) {
        super(method);
        this.target = target;
        this.name = name;
    }
    /**
    * Returns the result of invoking this method on target with parameters params.
    * Executes the test method on the supplied target (returned by the JUnitTestFactory)
    * and not the instance generated by FrameworkMethod.
    */
    @Override
    public Object invokeExplosively(Object target, Object... params) throws Throwable {
        return super.invokeExplosively(this.target, params);
    }
    /**
    * Returns the method's name.
    */
    @Override
    public String getName() {
        return String.format("%s=%s.%s[%s]", name, target.getClass().getSimpleName(),
         getMethod().getName(), target.toString());
    }
} 

JUnitTestFactory.java

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
/**
* Here RetentionPolicy.RUNTIME annotations are to be recorded
* in the class file by the compiler and retained by the VM at
* run time, so they may be read reflectively.
*/
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface JUnitTestFactory {
}

Now the framework is ready to create the runtime JUnit test cases. Now we need to create your own classes to test. Below are the ways to write your customized classes.

RuntimeTestCase.java

import java.io.IOException;
import java.text.ParseException;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

public class RuntimeTestCase {

    private InputData testInput;
    private ExpectedResultData expectedResultData;

    public RuntimeTestCase(TestInputData testInput,
        ExpectedResultData expectedResultData) {
        this.testInput = testInput;
        this.expectedResultData = expectedResultData;
    }
    /**
    * This is the actual TC, which we are expecting to test. Number of same
    * TCs with different input data will be placed in air.
    */
    @JUnitFactoryTest
    public void test() throws IOException, InterruptedException, ParseException {

        assertEquals(expectedResultData.getName(), testInput.getName());
    }
} 

TextFileInputDataTest.java

import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;
import java.util.Map;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.junit.After;
import org.junit.Before;
import org.junit.runner.RunWith;

/**
* This class is a JUnit TC, which we need to run to test the given file,
* which may have n number of test-cases.
*/
@RunWith(JUnitFactoryRunner.class)
public class TextFileInputDataTest {

    /** Here's where to set up test case inputs, and their expected result will be 
    * read from a text-file, and will be populated as an map 
    * (Map<TestInputData, ExpectedResultData>) and need to compare values with 
    * their corresponding expected result fetched from map.
    * This basically reads the text input file and prepare the List<RuntimeTestCase>.
    * and provides this List as an input to RuntimeTestCase
    * @return
    */
    @JUnitTestFactory
    public static Collection<?> tests() {

        Map<TestInputData, ExpectedResultData> testInputAndResultMap = getTestInputAndResultMap();
        Iterator<MarkupTestInputData> testInputAndResultMapItr = testInputAndResultMap.keySet().iterator();
        ArrayList<RuntimeTestCase> tests = new ArrayList<RuntimeTestCase>(testInputAndResultMap.size());
        while(testInputAndResultMapItr.hasNext()){

            TestInputData testInput = testInputAndResultMapItr.next();
            ExpectedResultData expectedResultData = testInputAndResultMap.get(testInput);
            tests.add(new RuntimeTestCase(testInput, expectedResultData));
        }
        return tests;
    }
}

And with that, we are all set to create JUnit test cases on the fly. Enjoy!