How To Select The Right Test Automation Tool

Every software development team or company will come to the point where a test automation tool is needed to downsize the effort of regression testing. The test automation tool can help the tester and the whole team to concentrate on other important testing tasks that a tool can't handle.

Selecting a tool sounds easy at first. Many people will pick the most popular tool on the market or the one that supports the programming language of the product. Sure, these are two important factors when selecting a tool, but there is much more to consider when searching and selecting the tool.

Different Types of Test Automation Tools

But where to start? When deciding for a tool, it's important to know the different types of test automation tools and how the tools are working on a high-level view. Basically, there are five different types of test automation tools on the market:

1. Image recognition: Tools in this category are taking screenshots of the UI elements in the product. The screenshots will be added to the automation scripts. With the help of the screenshots, the application under test will be automated and verified.
2. Coordinate based recognition: With the help of x/y coordinates, tools in this category are interacting with the application under test to automate and verify the application.
3. OCR/ text recognition: Optical Character Recognition (OCR) or text recognition tools are able to identify elements based on their given text. The tools use the visible texts to drive the automation and to verify the application.
4. Native object recognition: Tools that use the native object recognition detect the control or UI elements on their given element tree. In most cases, this tree is built by CSS, XPATH or XML to identify, automate and verify the elements.
5. Capture & Replay: Tools in this category offer a convenient and easy way to record the automated scripts. Every interaction on the screen, like scrolling, tabbing/clicking, or typing will be recorded and translated into automation steps for the tools. The recorded scripts can be used to replay the activities and to verify the application under test.

Some of the available tools on the market support more than one recognition approach. Sometimes this might be useful to combine the approaches in the automation scripts to get more robust automation scripts.

Two Important Questions to Raise

Now that the different automation tool types are known, every team should answer the following two questions before selecting the automation tool:

1. What should be automated in the application under test?
2. Where it should be automated, in a browser or on a real mobile device?

We all know that 100% test automation coverage is not possible and not worth investing in the team to achieve this goal. However, every team should spend some time to identify the parts of the application that must be automated. These parts might be repetitive or critical parts of the application like the login process or the checkout process.

Once the parts are identified, the team should ask the question where the automated test scripts should be executed. Should the tests run in a web browser on a big stationary screen or on a real mobile device?

Finding the answers to these questions might eliminate already some tools on the market. But there is more to check upfront.

Write Down The Selection Criteria

In the next step, it's time to write down the selection criteria for the own development environment and the product that should be automated.

Note: The selection of criteria will look different from project to project and from product to product but it's important to take the time to write down the requirements.

This article will cover the top 10 most important test automation tool criteria:

1. Does the tool support different programming languages (SDKs)?
2. Which platforms are supported (Web, iOS, Android, Windows)?
3. Is the tool able to re-use test scripts to different platforms?
4. Does the tool provide a test recorder for easy recording functionality?
5. Is the tool easy to integrate into the own CI/CD environment?
6. Is the tool well documented and offers a support (paid or community)?
7. Can the tool be extended with addons?
8. Does the tool support different languages (not only English)?
9. Does the tool offer extensive analytics and test reports?
10. Is the tool following certain/required standards?

Selecting the Test Automation Tool that Fits Your Needs

Once the selection criteria are known and written down, it's recommended to create a table with each criteria and prioritization columns to see how good a test automation tool is covering the criteria. Each prioritization has a range of achievement and given points. For each selection criteria a team can now verify if a certain tool is fulfilling the criteria.

An example table can look like this:

At the end of the evaluation, each tool will get a total number of points. The tool with the highest number of points will fit best to the team and the development environment.

Conclusion

In the first place, selecting a test automation tool sound like an easy task. However, taking a deeper look into the automation approaches and your own needs makes the selection much more intense. The invested time upfront is important to not make the mistake to take the tool that seems to fit in the first place. The preparation phase and the selecting criteria will save project budget in the long run.