Today Agile is king, and teams that aren't releasing frequently are left behind. How is it possible to maintain quality when software is changing all the time? There is a growing awareness that existing testing tools and practices are too sluggish to cope with the pace.
Continuous Testing aims to get testing up to speed with development by conducting many types of tests—both automated and manual—continually throughout the software delivery pipeline. This dynamic approach to testing is, at least in theory, supposed to keep up with rapid development iterations.
We say "in theory" because, like all the new things that start with "Continuous," it is easier said than done. Even if you have automated tests to a great extent, you may be far from the vision of Continuous Testing.
Continuous Testing talks about a test suite that encompasses the full functionality of the product from the user's perspective and gives developers and managers full visibility into what is really working, what isn't working, and what is currently not tested, so we don't really know if it's working at all.
This type of visibility allows you to make informed decisions about the level of business risk in a specific release, and whether a release is really ready to go to production.
Just like everything in software development these days, the "how" is all about tools. Tools can help with two main things that facilitate continuous testing:
Enabling faster and more comprehensive testing within an Agile development cycle.
Providing visibility into the business risks associated with each software release.
The following five tools can help companies transition to and achieve the aims of Continuous Testing. Our review is based in part on the Forrester Continuous Testing Service Providers report.
1. IBM Rational Test Workbench
IBM's Rational Test Workbench supports Continuous Testing by enabling easier automation of many types of tests. It includes API testing, functional UI testing, and performance testing, and makes it possible to shift left and run these tests much earlier in the development cycle, allowing testers to act in tandem with developers.
Regression tests are vital in ensuring that software quality remains intact in the wake of changes. However, regression tests are difficult to write and time-consuming to execute. RTB helps create regression tests automatically, to enable testing the effect of changes without a constant manual effort.
Parasoft's Continuous Testing offering supports continuous delivery environments by allowing organizations to quickly validate functionality, reliability, performance, and security requirements of each software iteration.
Parasoft's platform has a Service Virtualization feature that helps Continuous Testing by giving companies access to realistic test environments at any time. This feature allows meaningful testing to happen earlier, with more complete tests, by virtualizing or mocking out all relevant parts of the environment.
Another feature of Parasoft is API testing, which automates end-to-end test scenarios across multiple endpoints, such as APIs, databases, and Web UIs. The product also offers automated continuous regression testing with sophisticated validations for deeper regression coverage.
3. Tricentis Tosca
The Tricentis Tosca Continuous Testing platform aims to bring testing up to Agile speed by achieving 90 percent+ test automation. The goal is to provide fast and continuous feedback on software quality, beyond unit testing.
The Tosca solution automates tests across many layers of enterprise architecture, including APIs, app UIs, business intelligence, and data warehouses.
Tosca uses a model-based automation approach, in an attempt to relieve the maintenance burden that can undermine test automation efforts. The model is an abstraction of the software functionality; teams can simply update the model, and all automated tests automatically synchronize.
SmartBear TestComplete is a test automation platform built with Continuous Testing in mind, which makes it easy to maintain and execute automated tests across web applications, mobile, and desktop apps.
Testers can run tests in parallel both on-premise and in the cloud, to prevent infrastructure bottlenecks that can slow down the testing cycle. The suite also automates regression tests and reduces test maintenance using a GUI Object Recognition algorithm, which lets you execute the same recorded tests across all major browsers without any modifications to the tests.
The SeaLights Continuous Testing platform focuses on providing agile test metrics and visibility into business risk. SeaLights measures the test coverage of all types of testing within the Agile organization: regression and unit tests, functional tests and UI/browser automation, integration and acceptance tests, and even manual testing. Uniquely, it is able to combine data from all these types of tests into a single measure of software test coverage.
Sealights tracks Quality Holes, which are untested, high-risk code changes, across both unit and functional test environments. This enables teams to quickly redirect testing efforts to those areas, without relying on manual reporting.
Sealights provides a holistic view of software quality, bringing together QA, development, and DevOps. It provides a centralized view of critical test metrics per software build, reflecting actual business-level software quality.
Continuous Testing is crucial in a modern development environment because it helps bring testing up to speed with the pace of Agile development.
However, implementing Continuous Testing is a complex task, especially given that modern software comprises not only UI and source code, but also APIs, third-party app connections, microservices, and a plethora of mobile and desktop devices to support.
According to Research from CA, 79 percent of companies strongly agree that tools are key to achieving Continuous Testing. This was only a very brief review of some leading tools, there are more, and more to learn about them.
We encourage you to dig deeper and find the tools that will help your testers surf, instead of drown in, the Agile software delivery wave.