Should You Use Selenium 3?

Wow. So, Selenium 3.0.0 and then Selenium 3.0.1 in just a few days. Should you use it?

I have a fairly small set of tests that I use on my Selenium WebDriver Online Course and I’ve run these against pretty much every version of Selenium WebDriver and browser version released.

I spent time last week and this week running them against Selenium 2.53.1, Selenium 3.0.0, and Selenium 3.0.1. 

To be honest, I wasn’t expecting Selenium 3 to move out of Beta so fast. I wasn’t expecting this extra work so soon. However, having a stable version is all the better, because now I don’t have to answer questions about the beta software.

The most obvious difference between Selenium 3.0.1 and Selenium 3.0.0 for me, was that HTMLUnit (v 2.23) works in Selenium 3.0.1. So, I wouldn’t use Selenium 3.0.0.

Now, my choice is between 2.53.1 or 3.0.1. 

Getting Started

Creating instructions to help people get started with WebDriver is a painful job when the version keeps changing and drivers keep changing names, so I’m happy for some stability.

I’ve updated my Start Using Selenium WebDriver project to use 3.0.1 in the pom.xml and I've updated the Speedrun install notes. I’ve left instructions for 2.53.1 in there, but I’ll phase those out after a few point releases of Selenium 3.

I recommend that people starting to learn Selenium WebDriver use Selenium 3.0.1 and use ChromeDriver as their default browser because you’ll be up to date and you’re more likely to receive support rather than upgrade to Selenium 3.0.1.

Why ChromeDriver? Because it is much more mature than GeckoDriver, and if you do want to drop back to Selenium 2.53.1, you won’t have to make any changes.

WebDriver in CI Locally and With Cloud Vendors

I have a simple CI project on GitHub here. 

I run this locally on a bunch of browsers through Jenkins, on Travis CI, and against SauceLabs and BrowserStack. I had to make minor code changes when starting Firefox to upgrade to Selenium 3 and I found no obvious issues.

So, I have upgraded my wdci project to use Selenium 3.0.1.

• SauceLabs:
  • Windows 10 – Edge v 14.14393
  • Windows 10 – Chrome v 53.0
  • Windows 10 – IE v 11.103
  • Windows 10 – FIREFOX v 49.0
  • Apple Mac – OS X 10.11, Safari 9.0
• BrowserStack:
  • Windows 10 – Edge v 13
  • Windows 10 – Chrome v 53.0
  • Windows 10 – IE v 11
  • Windows 10 – FIREFOX v 49.0
  • Apple Mac – “El Capitan”, Safari 9.1
• Locally
  • Windows 10 – Edge v 13
  • Windows 10 – Chrome v 53.0
  • Windows 10 – IE v 11
  • Windows 10 – FIREFOX v 49.0
  • Windows 10 – HTMLUnit v 2.23
• Travis CI
  • Linux HTMLUnit v 2.23
  • Linux Firefox v 47.1

All that I had to change was how I started Firefox browser. Now, this isn’t a particularly large set of tests (or even complicated tests). It might not be enough to convince you.

What About Bigger Code Bases?

I haven’t found any issues with Selenium in my main code. My issues tend to be cross-browser or cross-driver related rather than Selenium related. They mainly relate to synchronization – the bane of cross-platform execution.

For example, moving from Selenium 2.53.1 to 3.0.1 made no obvious difference to Edge, IE, Chrome, or Safari. 

For Firefox, clearly, we have to change how we instantiate the driver. GeckoDriver is still a work in progress. However, the legacy Firefox driver in Selenium 3.0.1 works just as well as the one in 2.53.1, so if you’re still using the FirefoxDriver rather than GeckoDriver, I don’t think you’ll notice a difference during execution.

Issue Conflation

The problem I found was my tendency to conflate driver issues with Selenium issues. It is easy to do that since we tend to upgrade driver versions at the same time as Selenium. In the case of HTMLUnit, I had to update the driver when I updated selenium.

However, by moving to Selenium 3, we will hopefully have less of that confusion since the separation between driver and Selenium becomes much cleaner.

Summary

• I have updated all my teaching projects to Selenium 3.0.1.
• It doesn’t take much to upgrade to Selenium 3. Hopefully, for most projects, with a small tweak to the driver abstraction layer, you’ll be up and running quickly.
• You can continue using FirefoxDriver in legacy mode if that is still your default browser and driver.
• Remember not to conflate driver issues with Selenium 3 issues.

Since it is so easy to upgrade and drop back to 2.53.1 if you need to, I think it is worth trying.