Patterns and Coding Practices for Stable End-to-End GUI Tests

we all know the importance of the test automation pyramid and why it makes sense to align various automation in this way. given that guiding principle, end-to-end gui tests sit at the top, with a considerably small number of tests compared to other types (unit, integration, service tests), and they are useful to verify business workflows. in the book agile testing: a practical guide for testers and agile teams , the authors explain the testing quadrants, the gui tests’ fit in the grand scheme of things, how to rationalize intention, and be smart about overall quality strategy.

image source

the intention of the e-e gui tests is to verify “whether we build the right thing from the business perspective.”

until some other technique evolves, webdriver-based gui tests are the best choice today. when we’re trying to mimic the end-user journey through our application, these tests naturally represent an outside-in perspective. testing from the surface of the application creates a vulnerability to many hidden factors, such as browser stability, browser performance, network speed, latency, sub-systems’ performance underneath the gui, etc. due to the many moving parts with every small action, gui tests have a non-deterministic nature. how do we win the game? the following are the top five coding practices that helped me achieve success.

1. adopt a page object pattern

our tests need to interact with the html elements on the web page. if we mix html element locators within the test case, tests will be brittle whenever the ui changes. html selectors may be scattered and mixed with the logic that interacts with the web page. this will lead to high maintenance and will limit the possibility of reusing test logic. instead, consider separating the test from the logic that interacts with the page. a typical design looks like this:

you’ll want to define responsibility for these layers as well ( this post from martin fowler explains the process in detail ).

in short, the page object should:

• provide an api to perform actions on the application.
• provide access to the state of the underlying page.
• encapsulate and hide the details of ui/html structure from the rest of the world (i.e., tests).

the ui map should:

• serve as a ui elements repository for the page object.
• encourage reusability and create a standard way of accessing ui elements for easier maintenance.
• abstract ui element finder logic from consumers.

image source

2. follow the “tell, don’t ask” pattern

now that tests and page object responsibilities have been defined, what is the protocol between them? where do we leave the logic to perform the action for the behavior defined by the test?

for example, below is the behavior we want to automate:

given the user logged into amazon, when the user searches for a book “mastering ansible,” t hen the system should perform search and display results.

while automating this, searching for a book is a multi-step process:

1. select “books” from the drop-down.
2. provide the search string to the search box.
3. perform search.

we could perform these steps right from the test. however, one of the oop principles recommends that instead of asking the page object for data and then acting on it, we should “tell” the page object what to do. this principle enables us to move the logic (steps one through three above) into the page object and combine data and behavior. this tightly couples data and the logic that operates on the data. this co-location helps developers achieve better understanding and allows for easier maintenance.

image source

3. following the single responsibility principle

now that the page object owns the logic and data, where should we do the assertions? who is responsible for assertions, exception handling, and reporting?

i would suggest keeping this responsibility with the test and let the page object action always return an object with necessary information, either to assert or handle an exception. i prefer assertion-free page objects. for example, the below page object action returns an error object on the exception and expects the test to handle exceptions, report, etc.

4. dealing with asynchronous behavior

one of the common challenges is figuring out how to deal with the asynchronous behavior of an application. webdriver offers implicit and explicit waits where you can wait for “x” seconds before proceeding; however, this is unreliable because we are waiting for that magic number of “x” seconds. while there are ways to handle this in the framework, i would suggest thinking about it from the end-user experience perspective. remember that e-e gui tests’ intention is to verify the app from the business/end-user standpoint. so, how long are we asking our end-user to wait between actions? if an expected element is not visible, clickable, or whatever the case may be after a certain point, bubble an exception back to the test and let the test fail consistently.

5. lint, build, publish and share the logic

since a page object represents composite actions, they can be very handy and reusable for many behaviors.

• depending on the stack, lint the page object code, since it likely contains logic and data handling, etc. good coding practices are critical.
• run sonar analysis and set quality gates on this codebase. i would even suggest writing custom rules to prevent some traditional mistakes such as thread.sleep(), path selectors, etc.
• set up ci build; fail the build on code quality sla violation.
• package and publish reusable page object apis and encourage wider reusability.

overall, treat test code the same way you treat production code. write test code like you’re developing an app, not letting your developers run your tests.