Executive Insights on Automated Testing

This article is featured in the new DZone Guide to Automated Testing: Your End-to-end Ecosystem. Get your free copy for more insightful articles, industry statistics, and more! 

1. The keys to automated testing are to save time and improve quality while realizing the goals and objectives of the organization. Value drivers are quality and velocity, and they need to be prioritized first. 70% of the time, clients are looking for both and it can take multiple meetings to decide what to prioritize. Automation helps customers with a quality assurance process that runs in minutes and hours versus days and weeks, all while removing errors.

The organization must understand its goals. Planning is important. Many organizations are facing the traditional issue of “we’re too busy to improve, we don’t have time to think about test automation, but we know we are going to fail if we don’t have it in place soon.” They need to understand the end goal of doing this kind of testing.

There are four key factors of success: 1) level of automation (must be 90%+ for CI/CD to succeed); 2) maximize coverage of all flows, all datasets, quality of app and UX; 3) creating an efficient feedback loop with data; and 4) adopting an integrated approach and platform for testing across all devices. Organizations need to be able to do end-to-end DevOps testing throughout the SDLC. Testing needs to shift left at conceptualization.

2. The most significant change to automated testing in the past year has been increased adoption. This is attributed to the maturation of Agile methodologies and adoption of the DevOps culture. Agile-driven teams are no longer separating automation from development. Organizations are realizing as complexity increases, automation is the only way to keep up with all of the changes. Also, the continued evolution of tools and frameworks is easier to work with. Enablement tools integrate automatically, so tests can be built quickly.

3. There are more than a dozen benefits to automated testing being seen in six key industries. Automated testing is invaluable for: 1) saving time by running tests automatically 24/7; 2) reporting with daily insights and accurate feedback; 3) consistency and accuracy; 4) saving money; 5) reducing resources (i.e. manual testers); and 6) achieving full coverage. Manual testing can achieve 6% code coverage, while well-written automated tests can achieve 100% coverage. Automated testing is helping organizations achieve continuous integration (CI)/continuous delivery (CD) and is helping legacy enterprises make the digital transformation with microservices.

The industries most frequently mentioned as benefitting from automated testing are: 1) automotive, especially automated cars; 2) healthcare – pharmaceuticals, medical devices, and patient monitoring; 3) telecommunications; 4) financial services– brokerage and algorithmic trading; 5) e-commerce; and 6) the Federal Government.

4. The ROI of automated testing is three-fold: time saved, fewer defects, and greater customer satisfaction and retention. Organizations can see 20 to 40x cycle time improvements by spreading the work across different machines. Going from weeks to days and days to minutes can yield 20x savings. Conservatively speaking, if you find a defect, it takes 10 times longer to fix in production than earlier in the SDLC. Catching errors earlier and more accurately saves a million dollars per year in developers not having to look for errors. For us, automated testing has had a direct correlation with customer satisfaction. The product is simply running better, and the customers are happier.

5. The most common issues affecting the implementation of automated testing is the corporate culture used to doing manual testing. Where manual processes have been used, people need to be retrained and turned into programmers — but management doesn’t want to ruffle feathers. Companies need to shift left and developers need to learn to write tests. Old style testers are not adapting to, let alone embracing, automated testing and AI. We need one comprehensive, automated, visible delivery process to share the feedback from the different tools.

6. The future of automated testing is the use of AI/ML to improve the process on several fronts. Testing will be designed for AI/ML to build predictable models and patterns. Testing will be a natural as writing code, which will be done by machines. AI/ML will be part of the solution as teams generate more data which will make the models smarter. With AI/ML, testing will be faster, more thorough, and will result in self-healing tests and self-remediating code. You will be able to use AI/ML to run every test imaginable in the least amount of time to ensure your code is always vulnerability and defect-free.

ML will also improve automated security testing, as securityteams are able to leverage historical vulnerability data to trainML models to automate the vulnerability verification process,thereby providing developers accurate vulnerability data in nearreal-time.

7. The three primary skills a developer will need to ensure their code and applications perform well with automated testing are: 1) test scripting skills; 2) understanding the use case of the application; and, 3) move left, beginning testing earlier in the SDLC.

Hone test scripting skills. Write small, simple tests. Recognize the different types of tests you will need to run at different points in the SDLC. Have the ability to write your own unit and regression tests. Know how automated tests are going to be written. Learn to write a model type of code from mentors and previous products.

Think about the use cases first and what the purpose of the code is. Understand users, domains, who they are, and what problem they’re trying to address. Have an overarching view of the application, what it’s doing, and how that impacts APIs and services.

Get rid of simple, recurring problems by architecting the system to be tested from the beginning. Leverage the testing methodology from the planning phase, and build richer code earlier in the SDLC.

Here's who we interviewed:

• Gil Sever, CEO, Applitools

• James Lamberti, Chief Marketing Officer, Applitools

• Shailesh Rao, COO, BrowserStack

• Kalpesh Doshi, Senior Product Manager, BrowserStack

• Aruna Ravichandran, V.P. DevOps Products & Solutions Marketing, CA Technologies

• Pete Chestna, Director of Developer Engagement, CA Veracode

• Julian Dunn, Director of Product Marketing, Chef

• Isa Vilacides, Quality Engineering Manager, CloudBees

• Antony Edwards, CTO, Eggplant

• Anders Wallgren, CTO, Electric Cloud

• Kevin Fealey, Senior Manager Application Security, EY Cybersecurity

• Hameetha Ahamed, Quality Assurance Manager, FileCloud

• Amar Kanagaraj, CMO, FileCloud

• Charles Kendrick, CTO, Isomorphic

• Adam Zimman, VP Product, LaunchDarkly

• Jon Dahl, CEO and Co-founder, Mux

• Matt Ward, Senior Engineer, Mux

• Tom Joyce, CEO, Pensa

• Roi Carmel, Chief Marketing & Corporate Strategy Officer, Perfecto Mobile

• Amit Bareket, CEO and Co-founder, Perimeter 81

• Jeff Keyes, Director of Product Marketing, Plutora

• Bob Davis, Chief Marketing Officer, Plutora

• Christoph Preschern, Managing Director, Ranorex

• Derek Choy, CIO, Rainforest QA

• Lubos Parobek, Vice President of Product, Sauce Labs

• Walter O'Brien, CEO and Founder, Scorpion Computer Services

• Dr. Scott Clark, CEO and Co-founder, SigOpt

• Prashant Mohan, Product Manager, SmartBear

• Sarah Lahav, CEO, SysAid Technologies

• Wayne Ariola, CMO, Tricentis

• Eric Sheridan, Chief Scientist, WhiteHat Security

• Roman Shaposhnik, Co-founder V.P. Product & Strategy, Zededa

This article is featured in the new DZone Guide to Automated Testing: Your End-to-end Ecosystem. Get your free copy for more insightful articles, industry statistics, and more!