What Is the Future of Automated Testing?
Artificial intelligence/machine learning, automation, and CI/CD/DevOps were the themes of our interviews on the future of automated testing.
Join the DZone community and get the full member experience.Join For Free
To gather insights on the state of automated testing today, we spoke with twenty executives who are familiar with automated testing and asked, "What is the future of automated testing?"
Three themes came through in the discussion: 1) artificial intelligence (AI)/machine learning (ML); 2) automation; and, 3) CI/CD/DevOps.
- Leverage AI/ML to automate the automation. Learn from testing and automatically automate the process. Instead of the quality engineer writing script, AI does it faster, cheaper, and better.
- Depends how far out you look. All automated by the system or app being testing with AI/ML. The API world is close to that. Developers are building APIs. Solutions based on Swagger development, definition, and documentation becomes easier to test APIs. UI has a lot of opportunity for consolidation with Selenium and Appium. Interoperability, reuse, and parallelization with the changing landscape of technology. Teams are only 30 to 40% automated with a strong appetite to do more.
- AI and machine learning will lead to more personalized, relevant and engaging CX.
- Run 420,000 tests in 20 minutes versus 500 in six hours. Change rapidly. Huge app to have one single script that can be recorded and played back on all browsers and mobile devices. AI capabilities into test automation.
- Visual Validation Services. Breaking down bigger barriers to entry – codeless automated testing that is accessible and maintainable. Device Farm technology. Artificial intelligence/machine learning based solutions that can identify and proactively check areas known to introduce critical issues.
- Automatic test avoidance. Test parallelization – JUnit tests. Run tests in parallel. The best test is the one you don’t have to run because dynamic and static analysis has become so good. Run relevant tests first to fail fast. You don’t have to be perfect, just good. There are opportunities and payoffs for open source solutions.
- Next generation embedded user testing into code frameworks. For automation to go to the next level we need to think about auto coding. Ultimately neural networks will be self-healing code.
- There are always holes in the integration that require a higher level. developers, more requirements, and more opportunities for errors. Build better APIs. Think automation first. Solve the business problem to be consistent, successful, and have confidence. You cannot leave testing to a human. Automation is more accurate and intelligent.
- We have a fantastic opportunity to continue to improve our Auto-Heal/Bisect coverage. Using metrics like Code Coverage and/or Use Case Coverage we can show incremental improvements around our automation's ability to cover large amounts of code. That coverage, coupled with technical analysis and use case coverage will enable us to move even faster and shorten our release lifecycle even further. This, in turn, enables customers to add new features in their environments that accelerate their own businesses.
- With cloud-based on-demand scalability, we can define a completely automated environment to be scaled up and down to optimize resources while enabling testing to scale. Cloud provides the ability to automate the infrastructure from build-up, deployment, and running tests – plug and play with everything automated. With CI/CD in development, there is more Agile speed to execution and delivery. Cloud and automation will continue to increase the speed with which code is able to be released.
- You increase the effectiveness and efficiency of test automation by providing mechanisms that further reduce the maintenance effort. In the future, the regression tests – the tests that address existing functionality – we will be able to gather them through logs from production use. This will come essentially for free – without any additional efforts. You still will need to make sure your test data is in place. This will be free not only for environments such as SAP but for any other technology as well.
- Orchestrate the test process end-to-end and integrate DevOps CD release cycle. Automate more. Different types of testing: unit, functional, performance, and load. Some are difficult to automate. Automate 99%.
- More shops will employ CI/CD, which is complete automation. It frees up time in the long run and is a best practice. Leveraging tech like machine learning and predictive analytics will likely be incorporated. The sectors that stand to benefit the most from automated testing are e-commerce and banking because they’re complicated. Likewise, AI and machine learning shops will employ automated testing since they’re a newer technology.
- Opportunities lie in strengthening Agile testing practices to its core i.e. TDD/BDD/ATDD. Making sure testing process is leftshift driven. In other words, in today’s technology-driven fast paced environment, organizations innovate rapidly to compete in crowded, evolving digital markets. LeftShift can help in making the transition to a streamlined continuous delivery model, ensuring that business value is realized on-budget in the shortest timeframes. And this can be achieved by having the right automated testing strategy in place.
- In the immediate future, I would expect to see more autonomy. Things like dynamically updating validation criteria from change requests, tests expanding themselves to test new/closed entry points, etc.
- Cloud/grid based testing solutions have a lot of potential. Their main weaknesses right now are: 1) Limited extensibility. Organizations like SauceLabs are going a good job at addressing obvious points of extensibility like integration into CI. Right now, deeper extensibility like capturing and storing custom diagnostics is not really on the radar. Our own test suites have the ability do things like making a request to the server to capture server logs, or capture dumps of the last few network requests and responses, or capture the entire visual component tree. These are extremely important to us, so we can’t yet move to cloud/grid solutions for most of our test suite. 2) Terrible tools for record/playback. Visual test recording makes a lot of sense, but tools like Selenium IDE are borderline worthless without adding a lot of extensions: the visual tests are extremely fragile, timing-dependent, and saved as horrifically bad code that no one would ever want to extend or modify. These problems can be solved and we’ve solved them for our framework, but Selenium HQ and/or cloud/grid providers need to focus on making visual recording a feasible approach, because it would be extremely compelling to just go to a website and start defining an automated test suite by point and click navigation through your application.
- Ability to black box test with white box results to see what happens. Microservices and web applications network need visibility – Accepted Test-Driven Development.
What is the future of automated testing from your perspective?
Here’s who we talked to:
- Murali Palanisamy, EVP and Chief Product Officer, AppViewX
- Yann Guernion, Director of Product Marketing, Automic
- Eric Montagne, Technology PM, Barclaycard
- Greg Luciano, Director of Services and Amit Pal, QA Manager, Built.io
- Donovan Greeff, Head of QA, Currencycloud
- Shahin Pirooz, CTO, DataEndure
- Luke Gordon, Senior Solutions Engineer and Daniel Slatton, QA Manager, Dialexa
- Anders Wallgren, CTO, ElectricCloud
- Charles Kendrick, CTO, Isomorphic
- Bryan Walsh, Principal Engineer, NetApp
- Derek Choy, V.P. of Engineering, Rainforest QA
- Subu Baskaran, Senior Product Manager, Sencha
- Ryan Lloyd, V.P. Products, Testing and Development and Greg Lord, Director of Product Marketing, SmartBear
- Christopher Dean, CEO, Swrve
- Wolfgang Platz, Founder and Chief Product Officer, Tricentis
- Pete Chestna, Director of Developer Engagement, Veracode
- Harry Smith, Technology Evangelist, Zerto
Opinions expressed by DZone contributors are their own.