Making Web UI Testing Great Again With Arquillian, Docker, and Selenium

Introduction to the Problem

But probably at the same time you've been faced with some of the most common problems in functional testing, some related with Web UI testing and others not.

For example one of the major problems people find in functional tests is the preparation of the environment. To run the tests, you need to boot up a server and deploy your application, then install and start the database. Also perhaps the cache system needs to be started, and so on with all the servers, leaving the user to install each service locally. Some errors could happen, like installing the incorrect version of the server used in production, reusing another local installation of the database which might not be the same version, or running something in a different JDK version from the one used in production.

But also there are some other problems that are more specific to Web UI testing, such as browser installation or configuration of WebDriver properties.

Fixing the First Problem

To fix the problems with the functional testing environment, the easiest solution is using Docker containers and of course Docker Compose, since you can define and run multi-container Docker applications. So basically you define in a docker compose file all the servers that you might need to run the tests, so that when you run tests, you make sure to have all of them running. More important you have a fixed version, so you can be sure that the tests are always run against the right version of the servers, JDK, etc., rather than depending on what is installed by the developers or on the CI server machine.

But this approach has one problem. You need to specifically run docker-compose up, docker-compose down before and after the test. Of course, you can automate this in your build script, which will solve the problem on a CI environment, but if a developer wants to execute a test from the IDE—let's say for debugging—then he needs to be aware of the required setup.

And this is what Arquillian Cube solves. Arquillian Cube is an Arquillian extension that uses a docker-compose file to start and configure all the containers defined, execute the tests, and finally shut down the containers. Since Arquillian works with JUnit (and TestNG and Spock), you can run the tests from the IDE without worrying about starting and stopping containers, since the Docker lifecycle is managed by Arquillian Cube.

So the first part of the problem; that is, defining the test environment, is fixed with Arquillian Cube. Let's see how to fix the second one.

Fixing the Second Problem

The Selenium project provides Docker images with Selenium standalone or Selenium node with a browser (Firefox or Chrome) and a VNC server installed.

So, it seems a perfect fit to fix the problem of having to install browsers with a specific version or configuration locally, since you can use a Docker image with a browser configured for the tests.

New Problems When Using Docker for Testing

And that's cool, but it has some problems. The first one is that you need to create a docker-compose file specific for testing purposes. This is not a bad thing per se, but it requires more management on the developer's part to maintain this file and of course to copy it again and again into all the projects where you want to use it. In each place, you must define the browser to use and the VNC client image to get the recording for future inspection.

The second problem is the configuration of the WebDriver. When running WebDriver against a remote browser, you need to set the IP address of the browser and configure the RemoteWebDriver accordingly with the desired capabilities.

So again, you have to repeat the WebDriver configuration in all the tests. You can create a factory class that can be reused in all the projects. This is good, but you still have one problem. Some developers might use Docker machine, so the IP would not be static and might change every time. Another problem might be using native Docker; for example, some phases of the CI pipeline might run the tests against a fully remote environment, like a pre-production environment. Either way, before executing the tests you need to manually specify the IP of the Docker host.

The third problem you'll get is that you need to instruct WebDriver to open a page:
webdriver.get("http://www.google.com"); The problem is that, in this case, the browser is inside the Docker infrastructure, so you need to set the internal IP of the server container. As a result, you don't only need to know the Docker host IP for connecting the remote web driver, but also the internal IP of the server container to open the page in the remote browser using the get method. And again, this might be quite difficult to acquire in an automatic way.

But all these problems are solved when using the new integration between Arquillian Drone and Arquillian Cube.

Fixing the New Problems

Arquillian Drone is an Arquillian extension that integrates the Selenium WebDriver with Arquillian. This extension manages the configuration of the WebDriver so you don't need to repeat it in all your tests. It also manages the lifecycle of the browser.

So as you can see, this pair of extensions is a perfect fit for solving these problems. Drone takes care of configuration, while Cube takes care of setting up the Selenium/VNC containers and starting and stopping them.

Even better, you don't need to worry about creating  docker-compose file for testing purposes. You only need to create the one used for deploying, and Arquillian will take care of the rest.

Example

The first thing to do is create a project with the required dependencies. For this example, we are using Maven, but you can achieve the same using other build tools.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.lordofthejars.helloworld</groupId>
    <artifactId>dronecube</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencyManagement>
        <dependencies>
            <!-- Use BOMs to set same versions in all dependencies -->
            <dependency>
                <groupId>org.jboss.arquillian</groupId>
                <artifactId>arquillian-bom</artifactId>
                <version>1.1.11.Final</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.jboss.arquillian.extension</groupId>
                <artifactId>arquillian-drone-bom</artifactId>
                <version>2.0.0.Final</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.jboss.arquillian.selenium</groupId>
                <artifactId>selenium-bom</artifactId>
                <version>2.53.1</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <!--  Use standalone mode in Arquillian (no @Deployment) -->
        <dependency>
            <groupId>org.jboss.arquillian.junit</groupId>
            <artifactId>arquillian-junit-standalone</artifactId>
            <scope>test</scope>
        </dependency>
        <!-- Cube dependencies -->
        <dependency>
            <groupId>org.arquillian.cube</groupId>
            <artifactId>arquillian-cube-docker</artifactId>
            <version>1.0.0.Alpha13</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.arquillian.cube</groupId>
            <artifactId>arquillian-cube-docker-drone</artifactId>
            <version>1.0.0.Alpha13</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.assertj</groupId>
            <artifactId>assertj-core</artifactId>
            <version>3.5.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <!-- Drone dependencies -->
        <dependency>
            <groupId>org.jboss.arquillian.extension</groupId>
            <artifactId>arquillian-drone-webdriver-depchain</artifactId>
            <type>pom</type>
            <scope>test</scope>
        </dependency>

    </dependencies>

</project>

<?xml version="1.0"?>
<arquillian xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://jboss.org/schema/arquillian"
            xsi:schemaLocation="http://jboss.org/schema/arquillian
    http://jboss.org/schema/arquillian/arquillian_1_0.xsd">

    <extension qualifier="docker">
        <!-- Not required if native docker or only one docker machine installed -->
        <property name="machineName">dev</property>
        <!-- Not required if file is in root of classpath -->
        <property name="dockerContainersFile">docker-compose.yml</property>
    </extension>

</arquillian>

helloworld:
  image: lordofthejars/helloworldgo
  ports:
    - "8080:80"

package org.lordofthejars.cubedrone;

import org.arquillian.cube.CubeIp;
import org.jboss.arquillian.drone.api.annotation.Drone;
import org.jboss.arquillian.junit.Arquillian;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;

import java.net.MalformedURLException;
import java.net.URL;

import static org.assertj.core.api.Assertions.assertThat;

@RunWith(Arquillian.class)
public class HelloWorldTest {

    public static final int EXPOSED_PORT = 80;
    // Enrich with webdriver configured to connect to remote browser
    @Drone
    WebDriver webDriver;

    // Enrich with helloworld container ip
    @CubeIp(containerName = "helloworld")
    String ip;

    @Test
    public void shouldShowHelloWorld() throws MalformedURLException, InterruptedException {
        // Constructs url that browser should connect
        URL url = new URL("http", ip, EXPOSED_PORT, "/");
        // Typical test using WebDriver
        webDriver.get(url.toString());
        final String message = webDriver.findElement(By.tagName("h1")).getText();
        assertThat(message).isEqualTo("Hello World");
    }

}