Automated Acceptance Testing — JBehave & Thucydides Work in Unison

Thucydides is an open source library designed to make it easier to define, implement and report automated acceptance criteria. Until now, Thucydides tests have been implemented using JUnit or easyb. However, the most recent version of Thucydides, Version 0.9.12, now lets you write your acceptance criteria using the popular JBehave framework.

JBehave and Thucydides

JBehave is an open source BDD framework originally written by Dan North, the inventor of BDD. It is strongly integrated into the JVM world, and it is widely used by Java development teams wanting to implement BDD practices in their projects. In this article you will learn how to use JBehave with Thucydides.

In JBehave you write and automate your acceptance criteria by writing test stories and scenarios using the familiar BDD "given-when-then" notation, as shown in the following example:

Scenario: Searching by keyword and category

Given Sally wants to buy some antique stamps for her son
When she looks for ads in the 'Antiques' category containing 'stamps'
Then she should obtain a list of ads related to 'stamps' from the 'Antiques' category

Scenarios like this go in .story files. A story file is designed to contain all the scenarios (acceptence criteria) of a given user story. A story file can also have a narrative section at the top, which gives some background and context about the story being tested:

In order to find the items I am interested in faster
As a buyer
I want to be able to list all the ads with a particular keyword in the description or title.

Scenario: Searching by keyword and category

Given Sally wants to buy some antique stamps for her son
When she looks for ads in the 'Antiques' category containing 'stamps'
Then she should obtain a list of ads related to 'stamps' from the 'Antiques' category

Scenario: Searching by keyword and location

Given Sally wants to buy a puppy for her son
When she looks for ads in the Pets & Animals category containing 'puppy in New South Wales'
Then she should obtain a list of Pets & Animals ads containing the word 'puppy'
  from advertisers in New South Wales

You usually implement a JBehave story using classes and methods written in Java, Groovy or Scala. You implement the story steps using annotated methods to represent the steps in the text scenarios, as shown in the following example:

public class SearchSteps {
    @Given("Sally wants to buy a $gift for her son")
    public void sally_wants_to_buy_a_gift(String gift) {
        // test code
    }

    @When("When she looks for ads in the $category category containing $keyword in $region")
    public void looking_for_an_ad(String category, String keyword, String region){
        // more test code
    }
}

Working with JBehave and Thucydides

Thucydides and JBehave work well together. Thucydides uses simple conventions to make it easier to get started writing and implementing JBehave stories, and it reports on both JBehave and Thucydides steps that can be seamlessly combined in the same class, or placed in separate classes, depending on your preferences.

To get started, you will need to add the Thucydides JBehave plugin to your project. In Maven, just add the following dependencies to your pom.xml file:

<dependency>
    <groupId>net.thucydides</groupId>
    <artifactId>thucydides-core</artifactId>
    <version>0.9.12</version>
</dependency>
<dependency>
    <groupId>net.thucydides</groupId>
    <artifactId>thucydides-jbehave-plugin</artifactId>
    <version>0.9.12</version>
</dependency>

New versions come out regularly, so be sure to check the Maven Central repository (http://search.maven.org) to know the latest version numbers for each dependency.

Setting up your project and organizing your directory structure

JBehave is a highly flexible tool. The downside of this is that, out of the box, JBehave requires quite a bit of bootstrap code to get started. Thucydides tries to simplify this process by using a convention-over-configuration approach. This approach significantly reduces the amount of work needed to get started with your acceptance tests. In fact, you can get away with as little as an empty JUnit test case and a sensibly-organized directory structure for your JBehave stories.

The JUnit test runner

The JBehave tests are run via a JUnit runner. This makes it easier to run the tests both from within an IDE or as part of the build process. You only need to extend the ThucydidesJUnitStories, as shown here:

package net.thucydides.showcase.jbehave;

import net.thucydides.jbehave.ThucydidesJUnitStories;

public class JBehaveTestCase extends ThucydidesJUnitStories {
    public JBehaveTestCase() {}
}

When you run this test, Thucydides will run any JBehave stories that it finds in the default directory location. By convention, it will look for a 'stories' folder on your classpath, so 'src/test/resources/stories' is a good place to put your story files.

Organizing your requirements

Placing all of your JBehave stories in one directory does not scale well; it is generally better to organize them in a directory structure that groups them in some logical way. In addition, if you structure your requirements well, Thucydides will be able to provide much more meaningful reporting on the test results.

By default, Thucydides supports a simple directory-based convention for organizing your requirements. The standard structure uses three levels: capabilities, features and stories. A story is represented by a JBehave .story file so two directory levels underneath the 'stories' directory will do the trick. An example of this structure is shown below:

+ src
  + test
    + resources
      + stories
        + grow_potatoes                     [a capability]
          + grow_organic_potatoes           [a feature]
            - plant_organic_potatoes.story  [a story]
            - dig_up_organic_potatoes.story [another story]
          + grow_sweet_potatoes             [another feature]
          ...

If you prefer another hierarchy, you can use the thucydides.capability.types system property to override the default convention. For example, if you prefer to organize your requirements in a hierachy consisting of epics, theme and stories, you could set the thucydides.capability.types property to epic,theme (the story level is represented by the .story file).

When you start a project, you will typically have a good idea of the capabilities you intend to implement, and probably some of the main features. If you simply store your .story files in the right directory structure, the Thucydides reports will reflect these requirements, even if no tests have yet been specified for them. This is an excellent way to keep track of project progress. At the start of an iteration, the reports will show all of the requirements to be implemented, even those with no tests defined or implemented yet. As the iteration progresses, more and more acceptance criteria will be implemented until acceptance criteria have been defined and implemented for all of the requirements that need to be developed.

An optional but useful feature of the JBehave story format is the narrative section. This section can be placed at the start of a story to help provide more context about that story and the scenarios it contains. This narrative will appear in the Thucydides reports to help give product owners, testers and other team members more information about the background and motivations behind each story. For example, if you are working on an online classifieds website, you might want users to be able to search ads using keywords. You could describe this functionality with a textual description like this one:

Story: Search for ads by keyword
In order to find the items I am interested in faster
As a buyer
I want to be able to list all the ads with a particular keyword
in the description or title.

However, to make the reports more useful still, it is a good idea to document not only the stories, but to also do the same for your higher level requirements. To do this, you can place a narrative.txt file in the directory containing the narrative description for each requirement. These files follow the JBehave/Cucumber convention for writing narratives, with an optional title on the first line, followed by a narrative section started by the keyword `Narrative:`. For example, for a search feature for an online classifieds web site, you might have a description along the following lines:

Search for online ads

Narrative: 
In order to increase sales of advertised articles
As a seller
I want potential buyers to be able to display only the ads for
articles that they might be interested in purchasing.

In Thucydides, you can do this by placing a text file called narrative.txt in each of the requirements directories you want to document (see below). The first line (with the colon notation) is optional — if you only provide a narrative text, Thucydides will derive the name of the requirement from the name of the directory.

When you run these stories (without having implemented any actual tests), you will get a report containing lots of pending tests, but, more interestingly, a list of the requirements that need to be implemented, even if no tests or stories are associated with them yet. This makes it easier to plan an iteration: you will initially have a set of requirements with only a few tests, but as the iteration moves forward, you will typically see the requirements fill with pending and passing acceptance criteria as work progresses.

Implementing the tests

If you want your tests to actually do anything, you will also need classes in which you place your JBehave step implementations. If you place these in any package at or below the package of your main JUnit test, JBehave will find them with no extra configuration.

Thucydides makes no distinction between the JBehave-style @Given, @When and @Then annotations, and the Thucydides-style @Step annotations — both will appear in the test reports. However, you need to start with the @Given, @When and @Then-annotated methods so that JBehave can find the correct methods to call for your stories. A method annotated with @Given, @When or @Then can call Thucydides @Step methods, or call page objects directly (though the extra level of abstraction provided by the @Step methods tends to make the tests more reusable and maintainable on larger projects).

A typical example is shown below. In this implementation of one of the scenarios we saw above, the high-level steps are defined using methods annotated with the JBehave @Given, @When and @Then annotations. These methods in turn use steps that are implemented in the BuyerSteps class, which contains a set of Thucydides @Step methods. The advantage of using this two-leveled approach is that it helps maintain a degree of separation between the definition of what is being done in a test and how it is being implemented. This tends to make the tests easier to understand and easier to maintain.

public class SearchScenarioSteps {
    @Steps
    BuyerSteps buyer;

    @Given("Sally wants to buy a $present for her son")
    public void buyingAPresent(String present) {
        buyer.opens_home_page();
    }

    @When("she looks for ads in the $category category containing $keyword in $region")
    public void adSearchByCategoryAndKeywordInARegion(String category,
                                                      String keyword,
                                                      String region){
        buyer.chooses_region(region);
        buyer.chooses_category_and_keywords(category, keyword);
        buyer.performs_search();
    }

    @Then("she should obtain a list of $category ads containing the word $keyword \
          from advertisers in $region")
    public void resultsForACategoryAndKeywordInARegion(String category,
                                                      String keyword,
                                                      String region){
        buyer.should_only_see_results_with_titles_containing(keyword);
        buyer.should_only_see_results_from_region(region);
        buyer.should_only_see_results_in_category(category);
    }
}

The Thucydides steps can be found in the BuyerSteps class. This class in turn uses Page Objects to interact with the actual web application, as illustrated here:

public class BuyerSteps extends ScenarioSteps {

    HomePage homePage;
    SearchResultsPage searchResultsPage;

    public BuyerSteps(Pages pages) {
        super(pages);
        homePage = getPages().get(HomePage.class);
        searchResultsPage = getPages().get(SearchResultsPage.class);
    }

    @Step
    public void opens_home_page() {
        homePage.open();
    }

    @Step
    public void chooses_region(String region) {
        homePage.chooseRegion(region);
    }

    @Step
    public void chooses_category_and_keywords(String category, String keywords) {
        homePage.chooseCategoryFromDropdown(category);
        homePage.enterKeywords(keywords);
    }

    @Step
    public void performs_search() {
        homePage.performSearch();
    }

    @Step
    public void should_only_see_results_with_titles_containing(String title) {
        searchResultsPage.allTitlesShouldContain(title);
    }
    ...
}

The Page Objects are similar to those you would find in any Thucydides project as well as most WebDriver projects. An example is listed below:

@DefaultUrl("http://www.newsclassifieds.com.au")
public class HomePage extends PageObject {

    @CacheLookup
    @FindBy(name="adFilter.searchTerm")
    WebElement searchTerm;

    @CacheLookup
    @FindBy(css=".keywords button")
    WebElement search;

    public HomePage(WebDriver driver) {
        super(driver);
    }

    public void chooseRegion(String region) {
        findBy("#location-select .arrow").then().click();
        waitFor(500).milliseconds();
        findBy("//ul[@class='dropdown-menu']//a[.='" + region + "']").then().click();
    }

    public void chooseCategoryFromDropdown(String category) {
        getDriver().navigate().refresh();
        findBy("#category-select").then(".arrow").then().click();
        findBy("//span[@id='category-select']//a[contains(.,'" + category + "')]").then()
                                                                                  .click();
    }

    public void enterKeywords(String keywords) {
        element(searchTerm).type(keywords);
    }

    public void performSearch() {
        element(search).click();
    }
}

When these tests are executed, the JBehave steps combine with the Thucydides steps to create a narrative report of the test results:

Note: All the source code for this example is available on Github.

Conclusion

JBehave and Thucydides make a great combination. JBehave’s free-text format of the stories and scenarios makes it easy to get BAs and testers involved, and it helps promote a common understanding of the solution the team is trying to deliver. Thucydides makes it easier to report on progress using these automated scenarios, not just as a reporting tool but also by tracking the relationship between the requirements that have been defined and the corresponding acceptance tests (or lack of them).

You can learn more about Thucydides on the Thucydides website.