Maven + JavaScript Unit Test: Running in a Continuous Integration Environment

So you're still interested in unit testing JavaScript (good). This post is an extension of my much more indepth first posting on how to unit test JavaScript using JS Test Driver. Please check it out here.

Recap Last Posting

In the last posting we successfully unit tested JavaScript using Maven and JsTest Driver. This allowes us to test JavaScript when on an environment that has a modern browser installed and can be run.

Problem with typical CI environments

So what happens when the test are passing on your local box, but you go to check in your code and the Continuous Integration (CI) server pukes on the new tests becasue there is no "screen" to run chrome or firefox? As of this posting, none of the top-tier browsers have a "headless" or an in-memory only browser window. There are alternatives to running JavaScript in a browser, such as rhino.js, env.js or HtmlUnit, however, these are just ports of browsers and the JavaScript and DOM representation are not 100% accurate which can lead to problems with your code when rendered in a client's browser.

Approach

What we need to do is to run JSTestDriver's browser in a Virtual X Framebuffer (Xvfb) which is possible on nearly all Linux based systems. The example below uses a Solaris version of Linux, however, Debian and RedHat linux distrubutions come with the simplified bash script to easily run an appliation in a virtual framebuffer. This solution was derived from one posted solution on the JS Test Driver wiki. The given example is also a full working example that is in use at my current client.

Here is the quick list of what we will accomplish. Note, several of these steps are discussed in depth in the previous post and are not covered in depth here.

1. Create a profile to run Js Unit-Tests
2. Copy JsTestDriver library to a known location for Maven to use
3. Copy JavaScript main and test files to known locations
4. Use ANT to start JsTestDriver and pipe the screen into xvfb

Here is a sample profile to use. You will need to adjust the properties at the top of the profile to match your system.

<profile>
    <!-- Runs JS Unit Test inside of Xvfb -->
    <id>ci-jstests</id>
    <properties>
        <firefox.location>/opt/swf/bin/firefox</firefox.location>
        <js-test-driver.version>1.3.2</js-test-driver.version>
        <xvfb.location>/opt/X11R6/xvfb-run</xvfb.location>
    </properties>
    <build>
        <plugins>
            <!-- Copy JS Test Driver JAR to target folder -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>2.1</version>
                <executions>
                    <execution>
                        <id>copy</id>
                        <phase>generate-resources</phase>
                        <goals>
                            <goal>copy</goal>
                        </goals>
                        <configuration>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>com.google.jstestdriver</groupId>
                                    <artifactId>jstestdriver</artifactId>
                                    <version>${js-test-driver.version}</version>
                                    <type>jar</type>
                                    <overWrite>true</overWrite>
                                    <destFileName>jsTestDriver.jar</destFileName>
                                </artifactItem>
                            </artifactItems>
                            <outputDirectory>${project.build.directory}/jstestdriver</outputDirectory>
                            <overWriteReleases>false</overWriteReleases>
                            <overWriteSnapshots>true</overWriteSnapshots>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <!-- Copy JavaScript files and Tests files  -->
            <plugin>
                <artifactId>maven-resources-plugin</artifactId>
                <version>2.4.3</version>
                <executions>
                    <execution>
                        <id>copy-main-files</id>
                        <phase>generate-test-resources</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}/test-classes/main-js</outputDirectory>
                            <resources>
                                <resource>
                                    <directory>src/main/webapp/scripts</directory>
                                    <filtering>false</filtering>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                    <execution>
                        <id>copy-test-files</id>
                        <phase>generate-test-resources</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}/test-classes/test-js</outputDirectory>
                            <resources>
                                <resource>
                                    <directory>src/test/webapp/scripts</directory>
                                    <filtering>false</filtering>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.6</version>
                <executions>
                    <execution>
                        <phase>test</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <target>
                                <mkdir dir="${project.build.directory}/js-test-results" />
                                <exec executable="${xvfb.location}">
                                    <arg line="java" />
                                    <arg line="-jar ${project.build.directory}/jstestdriver/jsTestDriver.jar" />
                                    <arg line="--port 4220" />
                                    <arg line="--testOutput ${project.build.directory}/js-test-results" />
                                    <arg line="--config ${project.build.directory}/test-classes/jsTestDriver.conf" />
                                    <arg line="--browser ${firefox.location}" />
                                    <arg line="--tests all" />
                                    <arg line="--verbose" />
                                </exec>
                            </target>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

Possible problems

Although I cannot predict or fix all problems, I can share the one major problem I ran into with Solaris and the script used to fix that. In Solaris (and could happen to other distros) the xvfb-run script was not available and several of the other libraries did not exist. I first had to download the latest X libraries and place them in their appropriate locations on the CI server. Next, I had to re-engineer the xvfb-run script. Here is a copy of my script (NOTE: This is the solution for my server and this may not work for you)

I created a script that contains:

    /usr/openwin/bin/Xvfb :1 screen 0 1280x1024x8 pixdepths 8 24 fbdir /tmp/.X11-vbf &  

From http://www.ensor.cc/2011/08/maven-javascript-unit-test-running-in.html