5 Problems With Functional Test Automation Today

Search for “automated testing” on Google and it will return About 667,000 results in 0.28 seconds. Undoubtedly, there is a lot of interest, curiosity, notions, and misconceptions about it. Take any software project and you will hear discussion about test automation at least once during its lifetime. There have been long-standing debates about Manual Testing vs Automated Testing. There are ardent followers of automated testing and believe that it is a magic wand which can completely replace manual testing (not true!) and there are groups who swear by manual testing and believe that test automation is just a fad (again, not true!).

Although test automation has come a long way from age-old Record and Playback framework based test automation, we see that there are some fundamental errors, as described below, which are behind unsuccessful test automation projects.

Work in Isolation

Automation teams are the technical people who ‘understand’ technology and that’s all they need to know. You give them the requirement and they will write the code for that. Unfortunately, it is not that simple. Writing good automation scripts require knowledge of the application and knowledge of the domain. Companies generally have separate teams for manual testing and automated testing. The manual testing teams are typically the application experts who have a deep understanding of the application. Automation teams work separately with the help of “test cases” and “specs” which are documented. In our opinion, this is a recipe for failure because, in the absence of complete application knowledge, an automation engineer cannot create a robust and scalable test automation framework which is extensible – even if she is a good test engineer.

Incorrect Expectations

One of the most common misconceptions about automation is that it is required only when you want to save time and money on testing. While saving time and cost are some of the major benefits of test automation, the real benefits go beyond that – automated testing is useful to build effectiveness, efficiency, increase test coverage, have consistency in testing, guarantee accuracy, and help in regression testing. It also helps in improving team morale because, with automation, you create time for the team to work on more challenging tasks. Having the right expectations from automation makes it easier for organizations in accurately measuring the true quality of an application.

Automation as an Afterthought

“Let the application get built first, and we will do automation for version 2.0.” This, unfortunately, is something we hear it quite often. There is a common belief that test automation cannot be started unless the application is fully built and ready. In reality, the true ROI from test automation can be achieved if it starts along with the development of the application – as soon as the wireframes are finalized. Having a test automation design and strategy ready early on in the project lifecycle can help you achieve a faster time to market, provide more test coverage, and give better application stability.

Aim for 100% Automation

There are thousands of combinations because of which it is practically impossible to achieve 100% test automation. Test automation can enable more tests with more data and configurations – but that does not necessarily mean better quality. Instead of aiming for 100% test automation, the goal should be on testing the critical functionality for the application and building a solid automation design and strategy which is sustainable and extensible to accommodate the future versions of the application.

ROI Measurement

To derive a true Return on Investment, first the organization needs to be clear on the objectives of automation and the ROI has to map with the objectives. Reduced costs, faster testing, and reliability are some of the parameters for realizing the Test Automation ROI. One needs to be aware that automation is NOT for improving test processes or it is not a replacement for manual testing. Remember that test automation is a strategic decision and the real ROI is realized in the long run when the same tests are executed multiple times at regular intervals.