Page Objects Refactored: SOLID Steps to the Screenplay/Journey Pattern

PageObjects have been the staple of automated web testing for over seven years. Simon Stewart wrote the original Selenium PageObject wiki entry in 2009. The concept was introduced to help test-developers reduce maintenance issues that, for many, resulted in flaky tests. Some mistook this as issues with Selenium, rather than with their own coding practices. While PageObjects can act as training wheels and be a good place to start, in this article we show you issues inherent to PageObjects and the benefits of refactoring them using SOLID OO principles. We then introduce theScreenplay Pattern, an alternative approach that could save you the trouble.

Why Page Objects

PageObjects were recommended at a time when Selenium and WebDriver were being used increasingly by testing teams, and not necessarily by those with extensive programming skills.

Prior to this, many test-automators would get themselves into trouble creating tests that had repetition and inconsistencies causing ‘flaky’ tests. For some, this gave test automation a bad name as those inexperienced in programming misinterpreted the failings of poor coding for failings of the testing frameworks.

Simon Stewart talked of this in his article “My Selenium Tests aren’t Stable”when he said:

“Firstly, let’s state clearly: Selenium is not unstable, and your Selenium tests don’t need to be flaky. The same applies for your WebDriver tests. […] When your tests are flaky, do some root cause analysis to understand why they’re flaky. It’s very seldom because you’ve uncovered a bug in the test framework.”

Page Objects & Baby Steps

The solution to this problem had to be good enough to prevent test code becoming too riddled with code smells and be accessible to the many testers “scripting” automated tests with little or no OO programming experience. In an ideal world, this would have sparked interest in the concepts behind the design of PageObjects and result in continuous and merciless refactoring — removing the code smells as they emerged.

Unfortunately, as with many examples, PageObjects have remained a wholesale drop-in solution for many teams who have managed to apply some refactoring, but not enough to avoid subsequent maintenance overheads. Some teams claim this hasn’t affected them. However, many teams we’ve worked with, making similar claims, later realised that they had simply become accustomed to the inherent overheads — not realising that life could actually be a lot easier.

The Common Code Smell in Page Objects

“A code smell is a surface indication that usually corresponds to a deeper problem in the system. The term was first coined by Kent Beck while helping me with my Refactoring book.” -Martin Fowler

The most common code-smell we see in PageObjects is ‘Large Class’. Of course, the term ‘Large’ is subjective and we’ve met a variety of people with different views on what ‘Large’ means.

Many well regarded programmers consider a class that is larger than a single screen (or their head) as a “Large Class”. This is the guideline that we subscribe to. A reasonable size by these standards would amount to 50–60 lines of code (including imports, requires, etc.). If you are reading this thinking “yep, I’m with you” then we’re on the same page. If you are thinking “a class that small can’t do anything useful” then I hope we can change your mind.

Let’s start with the de-facto LoginPage example on the Selenium Wiki. Representing just two fields and one button, it requires 45 lines of code (minus the comments), already near the upper limit. Consider a more involved page with tens-of-elements and this number can grow considerably, as in this example with over 200 lines of code. Why is this a problem?

Apart from taking longer to understand what a class does (and how), a large class is an indicator that other good programming principles are absent. For example, they are likely to have multiple responsibilities, violating the Single Responsibility Principle (SRP) making it harder to see where a change to the code should be made. They are also more likely to contain duplication — which can result in a bug being fixed in one place but recurring elsewhere.

One approach to this is to think of each PageObject not as an object representing a page but as objects within a page, as described by Martin Fowler. This means thinking of each object like a widget or page component. Even still, if a component has more than a couple of fields and buttons (as with the LoginPage example) the PageObject can still grow well beyond the 50–60 lines mark. Addressing this is where some SOLID principles can be helpful.

SOLID Principles

Maintainability of code can make or break any project, whether it is to write test code, production code or both. If maintenance overheads increase, the time-to-market increases along with costs. Code smells help us recognise when there is a potential problem. SOLID principles help us recognise when we’ve resolved those problems in an effective way.

SOLID is an acronym coined by Michael Feathers and Bob Martin that encapsulates five good object-oriented programming principles:

Single Responsibility Principle
Open Closed Principle
Liskov Substitution Principle
Interface Segregation Principle
Dependency Inversion Principle

For the purposes of this article, we’ll concentrate on the two that have the most noticeable effect on refactoring of PageObjects — the Single Responsibility Principle (SRP) and the Open Closed Principle (OCP).

SRP

The SRP states that a class should have only one responsibility and therefore only one reason to change. This reduces the risk of us affecting other unrelated behaviours when we make a necessary change.

“If a class has more than one responsibility, then the responsibilities become coupled. Changes to one responsibility may impair or inhibit the class’ ability to meet the others. This kind of coupling leads to fragile designs that break in unexpected ways when changed.” -Robert Martin, Agile Principles, Patterns & Practices

PageObjects commonly have the following responsibilities:

• Provide an abstraction to the location of elements on a page via a meaningful label for what those elements mean in business terms.
• Describe the tasks that can be completed on a page using its elements(often, but not always, expressing navigation in the PageObject returned by a task).

Let’s consider a simple todo list application (e.g. the Todo MVC example app).

The structure of a typical PageObject for the above example might look like this:

As these responsibilities are in a single class, when the way we locate a specific element is altered this class requires change. If the sequence of interactions required to complete a task change, again so must this class. There we have more than one reason for change — violating the SRP.

OCP

The Open Closed Principle (coined by Bertrand Meyer in Object-Oriented Software Construction) states that a class should be open for extension, but closed for modification. This means that it should be possible to extend behaviour by writing a new class without changing existing, working code.

“When a single change to a program results in a cascade of changes to dependent modules, that program exhibits the undesirable attributes that we have come to associate with “bad” design. The program becomes fragile, rigid, unpredictable and unreusable. The open closed principle attacks this in a very straightforward way. It says that you should design modules that never change. When requirements change, you extend the behavior of such modules by adding new code, not by changing old code that already works.” — Robert C. Martin, The Open Closed Principle, C++ Report, Vol. 8, January 1996

In practice, this means:

“Adding a new feature would involve leaving the old code in place and only deploying the new code, perhaps in a new jar or dll or gem.”
 — Robert Martin, The Open Closed Principle

Let’s say you have a TodoListPage and we want to add the ability to sort todo items alphabetically. The most likely approach would be to ‘open’ the TodoListPage class and modify it with new method to handle this behaviour. These steps would likely be repeated for any behaviour added to the page. Furthermore, there are times where a task may span multiple pages. Having to take this approach may cause you to artificially split the task up across two classes to conform to PageObject dogma.

To satisfy OCP it should be possible to simply add a new class that describes how to sort the list.

Refactoring with the SRP & OCP in mind

To satisfy the OCP, a naive approach would be to extract classes into smaller PageObjects where adding a new behaviour would involve adding a new class.

While this satisfies the OCP in this example, it does not satisfy the SRP. In each of the above classes the two responsibilities of a) how to find the elements; and b) how to complete a given task; are still grouped together. Each class has two reasons to change, violating the SRP. Additionally, the task responsibility is limited to the elements of a single page and where tasks share an element the tasks must either duplicate or refer to elements declared in other tasks. This is bad.

To satisfy the SRP, a similarly naive approach might be to extract a class for each obvious responsibility, such as locating elements and performing tasks. While this satisfies the SRP it does not satisfy the OCP (each new task requires editing the Tasks class)

Instead of the above, a less naive approach would be to combine the Extract Class refactoring with the Replace Method with Method Object refactoring. The result satisfies both OCP and SRP. New behaviours do not require you to modify existing classes and each class has only one reason to change — i.e. if the way we locate elements no longer works or, for the tasks, if the way a given task is performed no longer works. The key phrase here in Robert’s explanation of OCP is:

“…you extend the behavior of such modules by adding new code, not by changing old code that already works.”

This is where we begin to move away from PageObjects as we know them and see something closer to the Screenplay Pattern (formerly known as the Journey Pattern) begin to emerge.

Dogmatically, the TodoListPage in this example doesn’t quite satisfy the OCP, however, we believe this is a reasonable compromise where discoverability of cohesive elements is useful (i.e. elements that make sense as a single unit, e.g. a todo list item has a title, a complete button and a delete button that don’t make sense independently). Furthermore, these classes are essentially metadata and carry very low risk when the class is changed.

Domain Strain

Beyond these design issues there is also another fundamental problem with PageObjects. Some developers and testers may think of the product they’re developing/testing in terms of “pages” (although the relevance of this in an era of single-page web-apps is debatable). User stories (done right) steer us to think in terms of something valuable a user will be able to achieve once the resulting capability is implemented.

For this reason, we believe that behaviours are the primary concern of our tests; the implementation — a secondary concern. This is especially true if you’re employing BDD and writing acceptance tests where the user’s ability to complete a goal is more relevant than the eventual solution. For these reasons, we start from a different perspective:

Roles ➭ Who is this for?
..Goals ➭ Why are they here and what outcome do they hope for?
…Tasks ➭ What will they need to do to achieve these goals?
….Actions ➭ How they complete each task through specific interactions?

By starting from this point of view, this changes the way we perceive the domain and therefore how we model it. This thinking takes us away from pages. Instead, we find we have actors with the abilities to play a given role. Each test scenario becomes a narrative describing the tasks and how we expect a story to play out for a given goal.

This perspective, combined with the OO design principles we’ve outlined above, is what steers us away from PageObjects and is what gave us the Screenplay Pattern.

The Screenplay Pattern by Example

The Screenplay Pattern has been around since 2007 and arose independently of PageObjects. We introduce it as a refactoring because, for many, it helps to start somewhere familiar.

To understand it more fully, we’ll take you step-by-step through an example, using the built-in support for this pattern in the Serenity framework. All of these examples can be found on GitHub.

For this example, we’re going to continue using the Todo MVC sample app as inspiration, starting with a simple user-story:

As James (the just-in-time kinda guy)
I want to capture the most important things I need to do
So that I don’t leave so many things until the last minute

We’re going to apply the following thinking to how we implement one of the acceptance tests for this story:

Roles ➭ Who is this for?
..Goals ➭ Why are they here and what outcome do they hope for?
…Tasks ➭ What will they need to do to achieve these goals?
….Actions ➭ How they complete each task through specific interactions?

Which, using the built in Screenplay support in the Serenity framework, results in a scenario that looks like this:

@Test
public void should_be_able_to_add_the_first_todo_item() {
  givenThat(james).wasAbleTo(Start.withAnEmptyTodoList());
  when(james).attemptsTo(AddATodoItem.called("Buy some milk"));
  then(james).should(seeThat(TheItems.displayed(), hasItem("Buy some milk")));
}

Let’s explore the basics of how this works before we delve into more detail…

Roles
In the example scenario, “James” is a persona we are using to understand a specific role. An Actor is used to play this role, performing each Task in the scenario:

Actor james = Actor.named(“James”);

Goals

We see from the scenario title that his goal is to add the first todo item.

@Test 
public void should_be_able_to_add_the_first_todo_item()

Tasks

To achieve this goal, there are two tasks involved: Start.withAnEmptyTodoList(), which in this case opens the app in a browser with an empty todo list; and AddATodoItem.called(“Buy some milk”). To see if this goal has been fulfilled James performs another special kind of task — he asks the question: what are TheItems.displayed()?

While we like to use given/when/then, these statements can be written without these methods, which is how we would write these tasks in a cucumber step definition:

@Given(“James starts with one item in his todo list”)
public void start_with_one_todo(){
  james.attemptsTo(
    Start.withAnEmptyTodoList(),
    AddATodoItem.called(“Buy some milk”)
  );
}
view raw

Note that both attemptsTo() and wasAbleTo() can take a list of tasks to perform.

Actions
Within each task is a performAs() method where ‘instructions’ for the task live. These are the actions required to complete that task. In this case we enter some text:

public <T extends Actor> void performAs(T theActor) {
  theActor.attemptsTo(
    Enter.theValue(thingToDo)
    .into(WHAT_NEEDS_TO_BE_DONE)
    .thenHit(RETURN)
  );
}

Screen Components

Actions, like Enter.theValue( thingToDo ).into( WHAT_NEEDS_TO_BE_DONE ) can interact with elements of a screen component:

public class ToDoList {
  public static Target WHAT_NEEDS_TO_BE_DONE =the(“’What needs to be done?’ field”).locatedBy(“#new-todo”);
  public static Target ITEMS = the(“List of todo items”).locatedBy(“.view label”);
  //…
}

Now, let’s examine how this all works in a little more detail. 

Actors Have Abilities

James is the name that we’ve given to an instance of an Actor who, in this case, has the Ability to BrowseTheWeb. We say who James is and what abilities he has like this (perhaps in a @Before method):

Actor james = Actor.named(“James”);
james.can(BrowseTheWeb.with(hisBrowser));

An Actor is responsible for performing things. This can be anything that is Performable such as a Task or Action. In Serenity, we say Actor.named(“James”) so that we get this information in some nice reporting (more on this another time).

This separation of actors and their respective browsers also makes it easy to have more than one actor when relevant:

@Before
public void ourUsersCanBrowseTheWeb() {
  james.can(BrowseTheWeb.with(hisBrowser));
  jane.can(BrowseTheWeb.with(herBrowser));
}

@Test
public void should_not_affect_todos_belonging_to_another_user() {
  givenThat(james).wasAbleTo(Start.withATodoListContaining(“Walk the dog”, “Put out the garbage”));
  andThat(jane).wasAbleTo(Start.withATodoListContaining(“Walk the dog”, “Feed the cat”));

  when(james).attemptsTo(
    CompleteItem.called(“Walk the dog”),
    Clear.completedItems()
  );

  then(jane).should(seeThat(TheItems.displayed(), contains(“Walk the dog”, “Feed the cat”)));
}

Tasks Involve Actions

The task AddATodoItem in our scenario is created by the line AddATodoItem.called(“Buy some milk”) and is given to our actor who will call the performAs(Actor) method.

If you are familiar with Hamcrest (also found within JUnit) you’ll be comfortable with using static creation methods to instantiate an object that is passed in for deferred execution — i.e. Hamcrest matchers — for example equalTo(…) or hasItems(…). In Hamcrest you pass a matcher to an Assertclass. Here, we pass a Task to an instance of an Actor via its attemptsTo() or wasAbleTo() methods.

One or more actions will be required to complete a task. Because this is a very simple task there happens to be only one Action involved — typing some text into a field followed by hitting the return key. You write these in the performAs() method of the task. The Actor will call this method, passing in a reference to itself.

You might be more familiar with a method like this being one of many methods on a TodoListPage class, however with the Screenplay Pattern — applying the SRP & OCP — this task is literally in a class of its own:

public class AddATodoItem implements Task {
  private final String thingToDo;
  public <T extends Actor> void performAs(T theActor) {
    theActor.attemptsTo(
      Enter.theValue(thingToDo)
      .into(WHAT_NEEDS_TO_BE_DONE)
      .thenHit(RETURN)
    );
  }

  public static AddATodoItem called(String thingToDo) {
    return instrumented(AddATodoItem.class, thingToDo);
  }
  public AddATodoItem(String thingToDo) { this.thingToDo = thingToDo; }
}

Note: the instrumented(…) method happens to be from the Serenity framework, enabling reporting features such as writing each task into the report along with the actions performed.

We find that most of the time we are interested in how the task was performed and much less in how it was created, so we place the most valuable information at the top of the class. Further down is where we put the creation method and constructor where they “cuddle” (i.e. no extra line break) to show how close they are in their relationship to each other.

Actions Abstractions

There are some really good reasons to abstract yourself away from any 3rd party library, even when that is Selenium/WebDriver. The Enter action is built into Serenity, following the same pattern as our tasks, and helps us to remain isolated from Selenium:

public <T extends Actor> void performAs(T theActor) {
  theActor.attemptsTo(
    Enter.theValue(thingToDo)
    .into(WHAT_NEEDS_TO_BE_DONE)
    .thenHit(RETURN)
  );
}

This Action requires the Actor to have the Ability to BrowseTheWeb which was given to the actor earlier. Remember when we said how in a @Beforemethod you might write:

Actor james = Actor.named(“James”);
james.can(BrowseTheWeb.with(hisBrowser));

With the Ability and Action concepts (along with Question and Targetclasses explained below) Selenium is kept very much behind the scenes. This approach continues our use of the SRP and OCP allowing us to add new and interesting ways to leverage Selenium, or whatever framework is behind this abstraction.

In future, there may be numerous jars available where there are many different implementations of how we interact with the product, perhaps using another framework to ‘browse the web’ or ‘use a mobile app’ or ‘use REST APIs’ and so on.

Questions and Consequences

Beyond tasks and the actions within them, the actor needs to understand the expected Consequence of a scenario. In this case the actor should:

seeThat(TheItems.displayed(), hasItem("Buy some milk"));

This is made up of the question TheItems.displayed() and a Hamcrest matcher hasItem(“Buy some milk”). The actor is able to then evaluate that consequence within it’s should method:

then(james).should(seeThat(TheItems.displayed(), 
                   hasItem("Buy some milk")));

This is how the question TheItems.displayed() can be implemented:

public class TheItems implements Question<List<String>> {
  @Override
  public List<String> answeredBy(Actor actor) {
    return Text.of(ToDoList.ITEMS)
               .viewedBy(actor)
               .asList();
  }

  public static Question<List<String>> displayed() { return new TheItems(); }
}

User Interface

The elements WHAT_NEEDS_TO_BE_DONE and ToDoList.ITEMS from the task and question above can be found in a representation of a ‘screen component’ (or widget if you prefer). Each Target within this component gives us the means of locating a given element on the screen. In our example, these can be found in our user_interface package.

package net.serenitybdd.demos.todos.user_interface;

import net.serenitybdd.screenplay.targets.Target;

public class ToDoList {
  public static Target WHAT_NEEDS_TO_BE_DONE = the(“’What needs to be done?’ field”).locatedBy(“#new-todo”);
  public static Target ITEMS = the(“List of todo items”).locatedBy(“.view label”);
  public static Target ITEMS_LEFT = the(“Count of items left”).locatedBy(“#todo-count strong”);
  public static Target TOGGLE_ALL = the(“Complete all items link”).locatedBy(“#toggle-all”);
  public static Target CLEAR_COMPLETED = the(“Clear completed link”).locatedBy(“#clear-completed”);
  public static Target FILTER = the(“filter”).locatedBy(“//*[@id=’filters’]//a[.=’{0}’]”);
  public static Target SELECTED_FILTER = the(“selected filter”).locatedBy(“#filters li .selected”);
}

Each screen component is a cohesive set of elements that belong together. Here is another example, the TodoListItem:

package net.serenitybdd.demos.todos.user_interface;

import net.serenitybdd.screenplay.targets.Target;

public class TodoListItem {
  public static Target COMPLETE_ITEM = the(“Complete item tick box”).locatedBy( “//*[@class=’view’ and contains(.,’{0}’)]//input[@type=’checkbox’]”);
  public static Target DELETE_ITEM = the(“Delete item button”).locatedBy( “//*[@class=’view’ and contains(.,’{0}’)]//button[@class=’destroy’]”);
}

These screen components are essentially data (which you could of course refactor out into, say, a json file) but for convenience and discoverability these have been implemented here as simple classes with constants.

Were we to take the OCP to the extreme you could split these further, adding a new ‘component’ to the user_interface package with each new enhancement. We think that applying the OCP to this extreme would result in greater loss than gains; these screen components help form a cohesive picture and are so simple that opening them for modification is very low risk.

Furthermore, if an element is moved in the application such that it should be grouped with a different set of components in another class, moving each Target remains extremely low-risk when refactored through an IDE. There will be no methods in this class that depend on the Target you want to move (as well as targets that stay where they are) as you might have in ‘traditional’ PageObjects. As a result, this change can be made with ease and confidence that it will not cascade into a massive refactoring of where dependent methods might live.

A Place for Everything

In this example, the result is a folder structure that can be easily browsed to understand what tasks are available and what user interface components there are.

.
├── model
│ ├── ApplicationInformation.java
│ ├── TodoStatus.java
│ └── TodoStatusFilter.java
├── questions
│ ├── ApplicationDetails.java
│ ├── ClearCompletedItemsOptionAvailability.java
│ ├── CurrentFilter.java
│ ├── DisplayedItems.java
│ ├── ElementAvailability.java
│ ├── ItemsLeftCounter.java
│ ├── PlaceholderText.java
│ ├── TheItemStatus.java
│ └── TheItems.java
├── tasks
│ ├── AddATodoItem.java
│ ├── AddTodoItems.java
│ ├── ClearCompletedItems.java
│ ├── CompleteAll.java
│ ├── CompleteItem.java
│ ├── DeleteAnItem.java
│ ├── FilterItems.java
│ └── Start.java
└── user_interface
  ├── ApplicationHomePage.java
  ├── ToDoList.java
  └── TodoListItem.java

Note: Because this is such a simple project, there is only one topic (e.g. Working with Todos). On a larger project, rather than group by types (e.g. Tasks, Questions), we’d group by cohesive topics.

We take great care in choosing the names of each class to ensure it is intuitively discoverable and tells you at a glance what it does from a user’s perspective.

The end result of the Screenplay Pattern, is ease of maintenance through readability, discoverability, consistency and structural simplicity where there is a place for everything — making it easier to put everything in the right place.

In Closing

The thinking and pattern we’ve shown you is an alternative approach to automating web application tests and more. The approach has been around in one form or another since its early conception by Antony Marcano in 2007. While this model arose independently, it is where you might end up following merciless refactoring of your everyday PageObject with SOLID principles in mind.

It was later illustrated in a more mature form with JNarrate and Screenplay4J, significantly through refinements made by Andy Palmer and Antony Marcano together. These ideas were used extensively across numerous organisations with one subsequently open-sourcing their interpretation.

Further implementations followed, integrating the ideas of others including Andrew Parker, Jan Molak, Kostas Mamalis and John Ferguson Smart. Now, thanks to John, the Serenity framework supports (even favours) the approach.

Yet with all these refinements and specific implementations over time, the fundamentals of the original ideas have remained the same — except its name.

The narrative of a user journey is what you see, hence its former name The Journey Pattern. There is, however, so much more to the pattern that this simply doesn’t communicate… The narrative for each scenario is a script; there is a cast of actors playing the roles; each actor has the task of performing action to the best of their ability…

In our search for a better name, it turns out that the right metaphor was always there, staring us in the face and pointing us at the perfect name. The final piece of the puzzle has now fallen into place and our journey of continuous improvement has given us The Screenplay Pattern.