DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Building Scalable Real-Time Apps with AstraDB and Vaadin
Register Now

Trending

  • Extending Java APIs: Add Missing Features Without the Hassle
  • What Is Istio Service Mesh?
  • Avoiding Pitfalls With Java Optional: Common Mistakes and How To Fix Them [Video]
  • Effortlessly Streamlining Test-Driven Development and CI Testing for Kafka Developers
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. How to Calculate ROI for Test Automation

How to Calculate ROI for Test Automation

Test Automation is attractive, necessary, and expensive too. The decision in front of the test manager is whether to invest in test automation tools or continue with manual testing. In this article, we will look at a couple of ways of analyzing returns on test automation.

Pratik Satasiya user avatar by
Pratik Satasiya
·
Mar. 11, 16 · Tutorial
Like (9)
Save
Tweet
Share
20.75K Views

Join the DZone community and get the full member experience.

Join For Free

Test Automation is attractive, necessary, and expensive too. The decision in front of the test manager is whether to invest in test automation tools or continue with manual testing. In this article, we will look at a couple of ways of analyzing returns on test automation.

The most basic form of ROI calculation is Simple ROI calculation; however, in this article, we will discuss slightly advanced techniques–Efficiency ROI calculation and Risk Reduction ROI calculation.

Efficiency ROI Calculation

The calculation needs to be done in terms of days since automated tests can be run for 24 hours continuously; as opposed to 8 hours that is considered for manual testing. However, 18 hours seems more reasonable since sometimes the test cases are interrupted and do not run for 24 hours.

The formula, which forms the basis of ROI calculation, is:

Image title

(a) Automated test script development time = (Hourly automation time per test * Number of automated test cases) / 8

(b) Automated test script execution time = (Automated test execution time per test * Number of automated test cases*Period of ROI) / 18

(c)Automated test analysis time = (Test Analysis time * Period of ROI) / 8

(d) Automated test maintenance time = (Maintenance time * Period of ROI) / 8

(e) Manual Execution Time = (Manual test execution time * Number of manual test cases * Period of ROI) / 8

Notes: Period of ROI is the number of weeks for which ROI is to be calculated. Divided by 8 is done wherever manual effort is needed. Divided by 18 is done wherever automation is done.

In this method, time investment gains are considered rather than monetary gains. Once gains and investment costs are calculated, we can insert the values of these variables into the formula and get the total Efficiency ROI.

In this method, the focus is on actual efficiency of automation and not just the money factor. This method does not require any sensitive information such as the hourly rate of tester, etc. However, this method makes several assumptions such as automated test cases fully replace the manual test cases (but, in reality, some cases automated test cases may need manual intervention) and that the manual testing counts only one manual tester.

Risk Reduction ROI

In this method, automation benefits are independently calculated thus addressing the ROI concerns that were not addressed in the earlier approach. When automation is adopted, resources can be freed to do more productive tasks such as time in-depth analysis of application, exclusive random and negative testing, test design/development etc.

In all, when automation is adopted increased coverage can be attained. However, when manual testing is solely relied upon, it can lead to undiscovered application bugs and post delivery bug fixes. Because of this, the quality of testing and the product is reduced, thus putting the organization at a risk of having to pay a penalty or undergo some loss. This loss needs to be considered in the formula so as to calculate the gain in terms of money if a bug were discovered post delivery/implementation.

Thus, investment cost remains the same, but gain is the monetary loss that the company may have to face if automation is not implemented. These values are then to inserted in the same formula which will then give the ROI.The advantage of this method is that the problems in the above method are addressed and it deals with the positive effect of test coverage. However, there is some subjectivity in this method and high level of risk analysis is required along with difficulty in calculating the loss of not doing test automation.

Conclusion

Since there are several factors that affect the ROI calculation, it is advisable to introduce improvements to the overall automation testing by introducing efficient tools, investing more time in the initial phase itself, using better reporting mechanisms, etc.

Also, there is no single accurate mechanism to calculate ROI. It is up to the tester’s/management’s discretion to use the method that suits their needs and situation the best.

Testing Test automation

Published at DZone with permission of Pratik Satasiya. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • Extending Java APIs: Add Missing Features Without the Hassle
  • What Is Istio Service Mesh?
  • Avoiding Pitfalls With Java Optional: Common Mistakes and How To Fix Them [Video]
  • Effortlessly Streamlining Test-Driven Development and CI Testing for Kafka Developers

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: