To Selenese or not to Test? That Seems to be the Question.
The Agile Zone is brought to you in partnership with JetBrains. Learn how Agile Boards in YouTrack are designed to help teams plan, visualize and manage their work in an efficient manner, with support for both Scrum and Kanban processes.
- 58% of respondents are using some variant of Selenium, with Selenium 2/WebDriver garnering the highest percentage of any testing tool. I also included in this percentage those that are using a Selenium wrapper framework such as Tellurium, Geb, or Cucumber.
- 31% of respondents are simply not doing automated browser testing
- 11% of respondents are using "something else," and every category received at least two votes.
These three summary points lead me to three major conclusions about the state of the automated browser testing world.
Selenium is still "King"Three out of five developers are using Selenium in some form or fashion, representing a pretty solid majority. Selenium 2 uptake is moving quickly as well, with the majority of raw Selenium users (57%) already migrating to Selenium 2 while it is still at its alpha release stage. I can think of several reasons why this may be the case:
- Selenium is a mature, battle-tested framework
- It has a great deal of commercial backing (ThoughtWorks, Sauce Labs, etc.)
- Google contributes to and uses it
- It can target a huge number of browsers (Firefox, IE, Safari, Chrome, iPhone, Android, Blackberry, etc..)
- Tests can be written in nearly any target language
We still have a lot of work to do...All that gushing over Selenium aside, no tool garnered a higher percentage of the vote (31%) than "I don't do automated browser testing!" What's amazing to me is that these folks still took the time to click on the poll and then vote. I honestly wouldn't have expected them to be reading this stuff! At any rate, I'm still curious as to why this is the case. No one posted up any relevant comments, so I'm left to speculate.
- No time? I would argue that you don't have time not to automate your browser tests. If you're building an application of any size and you're doing incremental releases, the only professional and responsible thing to do is to test every single feature of the application on every supported browser platform every time you release the software. I don't know about you, but I don't know of anyone that has enough folks sitting around with the time to do that *manually.* Add to this the fact that you really ought to be testing everything following any change to the source code, and you really can't get it done. But that's the only way you're going to minimize the time to discovery (and thus the impact) of bugs in your system.
- Too hard? I would argue that it's never been easier to write automated browser tests. Pick your programming language, pick your style, pick your browser: something has support. Selenium 1 and Watir have freely available recording tools that reduce the barrier to entry. If you're doing automated unit testing, you can do automated browser testing.
The ecosystem is only getting bigger!It seems like new automated browser testing tools/frameworks are popping up every day, and as fast as they are appearing, folks are starting to use them. One very recent arrival, Geb, is a Groovy wrapper around the WebDriver API that uses a very "jQuery-esque" syntax to interact with the DOM. Coupled with Spock, the popular BDD/unit testing framework for Groovy, one can get very close to the idea of executable specifications for web applications. I think the combination of Geb and Spock is quite compelling, and so I've decided to build a session on it for the Rich Web Experience. What was incredibly surprising to me is that Geb garnered NINE VOTES in our poll.
So there you have it. Thanks to everyone that voted!