Selenium is not a panacea
The Agile Zone is brought to you in partnership with JetBrains. Learn how Agile Boards in YouTrack are designed to help teams plan, visualize and manage their work in an efficient manner, with support for both Scrum and Kanban processes.
Selenium is a great tool for acceptance testing of web applications: it works with real browsers, and drives them to exercise your application of choice in the same way a real user would do. But it is incredibly fast in performing this real world tests, as the interaction from the fake user (Selenium itself) and the server is instantaneous.
Our application will be the case study for this article. The environment comprehends PHPunit test cases that extend the Selenium one (PHPUnit_Extension_SeleniumTestCase), and run against a sandboxed instance of the application which can be thrown away at the end of the test suite execution. A Selenium RC server is also necessary, as it would expose the HTTP proxy server needed for the client tests to work. I'm writing this article so that I can refer to it in the future when it comes the time to work again with Selenium, and to avoid repeating the same errors and false assumptions all over again.
Let's start with the typical issues that a developer encounters when he starts using Selenium.
For starters, the generated tests created via recording in Selenium IDE are garbage: they use DOM ids and every kind of reference to the elements physical structure that will break every day from here to the eternity. You can reuse their flow (maybe), but not their CSS selectors.
Remember that you need a Selenium RC server to run a real suite, and you must exclude the acceptance test from the build where this harness is not available. If you want to run the whole test suite on a Continuos Integration server that does not have a a graphical interface, follow the related article.
Last but not least, when JS frameworks get in the way, it's not so simple to model the user interaction. They generate a lot of markup, which you will find hard to navigate into. There are alternative to fighting with the DOM.
Accessing the widgets
So navigating the DOM is difficult, and it may break very often since JavasScript frameworks employ generated markup and ids.
There is a trade-off between the approaches of executing code on the browser side or on the test suite side:
- more logic in PHP makes the test code easier to work with, and to wrap with helper methods. It is also possible to dump values for debugging.
Thus we are seeking a combination of the two approaches.
The goal is to build a DSL for acceptance testing: tests will be written with this higher-level API where not only there is a Page Object pattern, but also objects for all the widgets that link to each other, and skip the various DOM nodes and references that get in the way.
DOM events are also not simple to manage, since you are never sure if you're generating the right event on the right element via PHP code.
Remember that clicks are not your only tool: there are many other events that an trigger an action by the Ajax widgets, like mousedown, blur and change. Don't rely on Selenium IDE to discover the right events for you: it will seldomly record everything you produce, and often only the wrong, obvious event that does not work alone when transported in your tests.
$this->type() is available in SeleniumTestCase subclasses for inserting text into widgets. There are many other methods that generate the related events, such as keyPress() and keyDown().
These methods can even be used to produce special characters like \r, \t, to simulate the pressure of the Enter or Tab key, which is quite useful when widgets respond to this event.
Keep an eye on the window object, which is not the default anymore. For example, you'll have to call window.Ext or window.$ when writing code for $this->getEval().
When writing tests for Selenium, work in very little iterations, which are quick if you execute one test at the time. Selenium will perform all the user interaction by responding just after the page has finished loading itself or its components. You'll slowly gain confidence and become able to skip intermediate steps.
You should also always use a mockup, even when working with Acceptance Test-Driven Development. When you have the mocked up test, you can move to the back end to make it work with more than one data set, following the rules of Red-Green-Refactor on your server-side code. However, exploring the DOM in your brain memory is not a good idea for writing tests if you do not have a quick live application available (always in a sandbox) as a reference.
If creating a sandbox is too complicated, work on your Phing/Ant build scripts to automate it. This will pay back in the future, not only for the sake of easy testing but as a mean to quickly build your application.