Criteria for Selection
E2E or Unit – An end-to-end framework is specialized to run through an entire human interaction on a website. A unit testing framework will give you different capabilities since it’s for testing single functions.
Assertions – Every end-to-end testing framework needs an assertion library. Assertions make sure that our tests return expected results. When choosing a framework, you also need to have a reason for choosing which assertion library you’re going to use and why you’re going to use it.
BDD/TDD Friendly – Some frameworks are not as easy to use with a BDD Girken style language as others, and you will want to stay away from those if your organization is following a certain style. Choose BDD or TDD and pick a framework that makes sense for your team and fits your organizational flow.
Visual Testing – Some frameworks are better suited for visual testing and capturing screenshots than others, and some actually come out of the box with it. This will give you the ability look at visual differences in your regression suite.
Speciality – You may be searching for a framework with a certain specialty. Is it easy to set up and get running out of the box? Is there a clean and simple syntax? Is it compatible with Angular testing or a Selenium binding? Does it have good customization? Is it flexible, and can it fit into other assertion libraries, code coverage tools, and frameworks? These are all considerations you’ll want to assess with your team.
Cloud Support – Pay attention to which frameworks work with cloud providers because not every framework will out of the box. Running tests in the cloud will allow you to increase test coverage and speed while eliminating hassle, cost, and maintenance, so compatibility should be part of your consideration.
Size of the Community – The size of the community matters because it’s where you look for answers and find tutorials and documentation, but this also depends on personal preference. For example, one framework may be good because it’s newer so more people are talking about it, but some might prefer a more mature framework that already has a good community base and documentation.
- Jasmine – Jasmine is a BDD/TDD compatible unit testing framework with an out-of-the-box assertion library, an easy to read syntax, and broad Angular support.
- Should.js – Should.js has a BDD assertion library and often pairs with Mocha.
- Assert.js – Assert.js works with Angular and often is used with Chai to provide a more simple set of assertions.
Unit Testing Frameworks
- Mocha – The self-proclaimed simple, flexible and fun framework is one of the most popular choices. It depends on other frameworks like Chai for assertions, but it is BDD/TDD compatible and includes a runner. Mocha is a very flexible, extendable, and feature-rich configuration that’s also good for asynchronous testing.
- Chai – Chai often goes hand-in-hand with Mocha as the most popular assertion library used alongside it. It is also BDD/TDD compatible.
- Tape – Tape is also a newer framework for unit testing that some might uses as an alternative to Mocha. It uses a traditional syntax, making it very simple to use.
- PhantomJS– Phantom.js deals with headless browsers but has had a drop-off in popularity since Chrome came out with their own and real browsers have been optimized to be comparably fast to headless ones.
- Webdriver.io – Webdriver.io includes visual testing with WebDriver CSS. It also specializes in customization and is a flexible framework where you can fit assertion libraries, code coverage tools, and other frameworks.
- Nightwatch – Nightwatch is an E2E test framework with its own assertion library and runners. It’s an easy, out of the box setup that a simple syntax, which couples together with a configuration file. Nightwatch is also a custom implementation of W3C WebDriver and works with the JSON wire protocol.
- CapserJS – Casper.js also works for headless browser testing with a runner for Mocha, Jasmine, and Chai for end-to-end testing. It has added capabilities like assertions and testing utilities that PhantomJS lacks.
- Karma – Karma is a test runner created by the AngularJS team, making it often the preferred test runner for apps written in Angular. It’s frequently used alongside Jasmine and integrates with popular CI servers as well.
- Cucumber – Cucumber is a popular BDD framework for functional testing using the Gerkin syntax that works with a lot of these tools and runners, like WebDriver.io. It’s being used more and more as more teams use BDD.
- InternJS – Intern is a fairly new framework that’s BDD/TDD compatible. As a test runner, it also integrates with CI tools like Jenkins and TeamCity as well the cloud. Don’t let the immaturity of this framework fool you, it has comprehensive documentation and a strong community.
This article was first published on the CrossBrowserTesting blog.