Often, teams choose an automation tool in a hurry without going into details of its pros and cons. The tool may not be comprehensive enough to satisfy all the testing needs of the application. Even if the best tools are selected, they may not integrate smoothly into the QA process. We have highlighted pros and cons of the best open source testing tools that give more clarity on their suitability.

JMeter

Apache JMeter is a protocol level load testing tool. It can be used to test loading times for static and dynamic elements in a web application. A tester can simulate a heavy load on a server, group of servers, network or object to test their strengths.

Pros of JMeter

• Easy installation: It can be installed on any desktop with Windows, Mac or Linux.
• It has a user-friendly interface or can be used in a command line interface.
• The test IDE allows test recording from browsers or native applications.
• Has the ability to extract data from popular response formats like HTML, JSON, XML or any textual format.
• Readily available plugins, for example, visualization plugin for data analysis.

Cons of JMeter

• Has a high learning curve thus it requires skilled testers.
• It doesn’t support JavaScript and by extension doesn’t automatically support AJAX requests.
• Complex applications that use dynamic content like CSRF tokens, or use JS to alter requests can be difficult to test using JMeter.
• Memory consumption is high in GUI mode which causes it gives out errors for a large number of users.

Capybara

Capybara is popularly used for end-to-end, acceptance and integration testing the Rack applications like Rails, Sinatra and Merb. It runs tests on headless browsers.

Pros of Capybara

• Powerful synchronizing feature - No need to add manual wait for asynchronous processes to complete.
• It has an Intuitive API to simulates real user actions on an application. For example, hidden elements/links are not clicked by a user so they are avoided.
• Agnostic for the driver running the tests - No need to change code when you switch from one driver to another
• Built-in support for Selenium.

Cons of Capybara

• High memory consumption when using multiple drivers for testing.
• It can be slow because it loads entire app stack. Or due to calling many controllers, models or views. Also, it doesn’t run JS (including AJAX calls) by default.
• Tests become fragile due to minor changes in model/controller, text or design.
• Hard to debug. For example, in case of timeouts or JS driver bugs.

Selenium WebDriver

The Selenium WebDriver is the most popular testing tool in the Selenium suite. It has an object-oriented API for testing modern complex web applications. This was developed by Selenium in order to support dynamic web pages (where elements of a page may change without the page itself being reloaded).

Pros of Selenium WebDriver

• Capable of testing across web browsers like Firefox, Chrome, IE etc.
• These browsers can be on platforms like Windows, Mac or Linux.
• Independence of using C#, Java, Perl, PHP, Python, JS(Node) and Ruby as scripting languages.
• Tests for user-like actions on the web application.
• Parallel execution on multiple machines saves time.
• Can be used for more complex testing such as production monitoring and load testing.
• Plenty of documents and a large web community is available.

Cons of Selenium WebDriver

• It requires experienced test automation engineer.
• Test maintenance is difficult, say due to the element waits in applications using AJAX.
• Users need to learn and use different frameworks to standardize the testing process.
• Proper implementation methods, if not followed, will slow down the testing.

CloudQA Complements Selenium to Enable Complete QA

Selenium can be a comprehensive tool that does almost everything you need. But the test maintenance is cumbersome, it requires an expensive infrastructure and highly skilled tester. Even when you overcome these difficulties the productivity can be low leading to low ROI.

CloudQA is built on top of Selenium and strengthens it to address the key pain points of QA automation. Having a single platform for Regression testing, Performance testing, and Load testing it is a complete QA automation solution. Read more on Why CloudQA.