Maven Plugin Testing - in a Modern Way - Part I

You have decided to write a Maven Plugin. Now the important part comes around the corner:
How to test a Maven Plugin? There are in general several options to test a Maven Plugins for example:

• Maven Invoker Plugin
• Maven Verifier
• Maven Plugin Testing Harness
• etc.

You can read about different reasons not to use one of those options.

Coming back to the subject and think about to write integration tests for a Maven plugin. Writing integration test means having three parties involved:

1. The component you would like to test (typically the Maven plugin/Extension etc.).
2. The testing code by which you check the functionality.
3. The Project you would like to test with (where your Maven plugin usually is configured in to be used.)

Let us start with a simple test case which simply requires that the build which contains your new plugin to run successful. This looks like this by using the Integration Testing Framework:

xxxxxxxxxx

@MavenJupiterExtension

class FirstMavenIT {

  @MavenTest

  void the_first_test_case(MavenExecutionResult result) {

    assertThat(result).build().isSuccessful();

  }

}

The written tests is easy to understand? Isn't it? The foundation of the Integration Testing Framework is the JUnit Jupiter extension mechanism, which allows to easily write extensions which can do many things. 

Going back to the given simple integration test we need now two other parts to get a working integration test. The code of the plugin (which I assume already exists) and of course the project where you would like to test your plugin with.

The above is an integration test written by using the Integration Testing Framework (ITF for short) which exactly does that. 

So a bit more nifty details here. The @MavenJupiterExtension is the annotation to activate the ITF extension. The @MavenTest is the equivalent for unit tests which are annotated with @Test of course it makes it very clear that we are in the context of a test for Maven.

Ah finally we have the parameter of the test method MavenExecutionResult result which gives you access to the result of the build and many more. The assertThat(..) is a customer assertions for AssertJ to make writing assertions more convenient.

So now we have to define the project which is used to use our plugin. As always in Apache Maven you should follow convention over configuration and the ITF is not an exception of that. We locate the integration itself into the usual location of tests like src/test/java/.... I recommend to name your integration tests like *IT.java but of course you can change that if you like.

xxxxxxxxxx

.

└── src/

    └── test/

        └── java/

            └── org/

                └── it/

                    └── FirstMavenIT.java

So now we need to put the project into a particular location like this:

xxxxxxxxxx

.

└── src/

    └── test/

        └── resources-its/

            └── org/

                └── it/

                    └── FirstMavenIT/

                        └── the_first_test_case/

                            ├── src/

                            └── pom.xml

It is important to mention that the directory FirstMavenIT represents the integration test class, and the the_first_test_case directory represents the method name of the integration test class. This will give us the opportunity to define several test cases within the class. Based on the association between method name and directory name it is necessary to write method names in lowercase and separate by using an underscore. If you use camel case method names this could cause issues with case insensitive file systems. 

As you can see that the directory the_first_test_case contains a full fledged project which comprises a pom.xml file and of course the usual structures for source code and maybe unit test etc.

Let us take a look into the pom.xml of the project which is used to test our plugin:

xxxxxxxxxx

<build>

  <plugins>

    <plugin>

      <groupId>com.soebes.itf.jupiter.extension</groupId>

      <artifactId>itf-failure-plugin</artifactId>

      <version>@project.version@</version>

      <executions>

        <execution>

          <id>the_first_test_case</id>

          <phase>initialize</phase>

          <goals>

            <goal>failure</goal>

          </goals>

        </execution>

      </executions>

    </plugin>

  </plugins>

</build>

One interesting thing is important here: @project.version@ which represents the current version of your plugin project. 

There are some requirements to use the ITF:

• JDK 8+
• Apache Maven 3.1.0 or above.

To get the integration tests running we need to add some parts to the projects `pom.xml` which is at first the dependencies to ITF:

The dependency com.soebes.itf.jupiter.extension:itf-assertj contains custom assertions of AssertJ in case you want to use AssertJ as your assertion framework. This means you have to include org.assertj:assertj-core as well. If you don’t want to use AssertJ as assertion framework you can omit them both.

xxxxxxxxxx

    <dependency>

      <groupId>com.soebes.itf.jupiter.extension</groupId>

      <artifactId>itf-extension-maven</artifactId>

      <version>0.9.0</version>

      <scope>test</scope>

    </dependency>

    <dependency>

      <groupId>org.junit.jupiter</groupId>

      <artifactId>junit-jupiter-engine</artifactId>

      <scope>test</scope>

    </dependency>

    <dependency>

      <groupId>com.soebes.itf.jupiter.extension</groupId>

      <artifactId>itf-assertj</artifactId>

      <version>0.9.0</version>

      <scope>test</scope>

    </dependency>

    <dependency>

      <groupId>com.soebes.itf.jupiter.extension</groupId>

      <artifactId>itf-jupiter-extension</artifactId>

      <version>0.9.0</version>

      <scope>test</scope>

    </dependency>

    <dependency>

      <groupId>org.assertj</groupId>

      <artifactId>assertj-core</artifactId>

      <scope>test</scope>

    </dependency>

You need to add the resource filtering like this:

xxxxxxxxxx

<testResources>

  <testResource>

    <directory>src/test/resources</directory>

    <filtering>false</filtering>

  </testResource>

  <testResource>

    <directory>src/test/resources-its</directory>

    <filtering>true</filtering>

  </testResource>

</testResources>

The first one for src/test/resources might be change based on your own requirements. The second is needed to copy the test projects to the appropriate location. So now we need to go for the itf-maven-plugin like this:

xxxxxxxxxx

<plugin>

  <groupId>com.soebes.itf.jupiter.extension</groupId>

  <artifactId>itf-maven-plugin</artifactId>

  <version>0.9.0</version>

  <executions>

    <execution>

      <id>installing</id>

      <phase>pre-integration-test</phase>

      <goals>

        <goal>install</goal>

      </goals>

    </execution>

  </executions>

</plugin>

which is responsible to copy your plugin/extension to the correct location and finally the maven-failsafe-plugin to execute the integration tests like this:

xxxxxxxxxx

<plugin>

  <groupId>org.apache.maven.plugins</groupId>

  <artifactId>maven-failsafe-plugin</artifactId>

  <configuration>

    <systemProperties>

      <maven.version>${maven.version}</maven.version>

      <maven.home>${maven.home}</maven.home>

    </systemProperties>

  </configuration>

  <executions>

  <execution>

    <goals>

      <goal>integration-test</goal>

      <goal>verify</goal>

    </goals>

  </execution>

</executions>

</plugin>

So now you can run the integration tests simply by using:

xxxxxxxxxx

mvn clean verify

After the execution you will find a directory a structure like this:

.

└──target/

   └── maven-it/

       └── org/

           └── it/

               └── FirstMavenIT/

                   └── the_first_test_case/

                       ├── .m2/

                       ├── project/

                       │   ├── src/

                       │   ├── target/

                       │   └── pom.xml

                       ├── mvn-stdout.log

                       ├── mvn-stderr.log

                       └── mvn-arguments.log

The directory the_first_test_case represents the test case from our original integration test class FirstMavenIT. The project which has been used to test the plugin is put into the directory project which also contains the target folder after a run of such project. The directory project is the location where you can go into and start manually Maven by using the command line arguments which are stored in mvn-arguments.log. The result of this execution is redirected into mvn-stdout.log and the redirection of stderr is logged into mvn-stderr.log. The directory .m2/repository contains the local cache (aka maven repository) of that build. 

So this it is for Part I. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.