Top 6 Continuous Testing Questions at Gartner AADI

Parasoft has been showcasing our Continuous Testing platform at Gartner AADI this week. Since there's been so much interest in Continuous Testing, we thought we'd take this opportunity to share the most frequently-asked questions...and our responses to them.

What is Continuous Testing?

According to the Continuous Testing Wikipedia page, the definition of Continuous Testing is:

"Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. For Continuous Testing, the scope of testing extends from validating bottom-up requirements or user stories to assessing the system requirements associated with overarching business goals."

How is Service Virtualization different than Server Virtualization?

Service Virtualization provides a simulated response from message traffic, "virtualizing" only a fraction of the overall system behavior. Server Virtualization is much more extensive. As a result, it requires you to have access to the entire system — and also to know how to configure and maintain the virtualized system components. Moreover, it's not feasible to leverage server virtualization for every dependent application (e.g., mainframes, third-party systems, ERPs). Service Virtualization fills the gap by providing access to the specific behavior that's needed to complete development and testing tasks.

How is Continuous Testing Impacted By Container Strategies?

It's somewhat common to talk about container strategies and microservices in the same breath. Container strategies, on their own, can ease the constraints of Continuous Testing by allowing for more ubiquitous access of specific applications via flexible containers. Although container strategies do not necessarily reduce configuration complexity, they're certainly more compartmentalized for broader access across teams.

By encapsulating more granular components of an application, microservices could also reduce test constraints. However, when you're using microservices, complexity increases because there are so many more components to manage. Whereas microservices might introduce more flexibility for a developer who is changing code that touches relatively few additional microservice components, a QA tester exercising a broader end-to-end transaction could be faced with a test environment management headache of having to corral a much more complex set of endpoints.

How Does Service Virtualization Impact DevOps Initiatives?

Increased collaboration among stakeholders is a primary objective of DevOps initiatives. Yet, the complex, distributed nature of today's systems — paired with the high rate and pace of application and code changes — inevitably creates tension for the development teams who need to validate their work against a complete system. Service Virtualization helps mitigate this tension and foster cooperation among development teams, test teams, and release automation teams by providing on-demand access to a simulated test environment that allows any single entity to validate their code changes anytime, anywhere. 

Although the fax machine still exists, it lost popularity with the advent of email. Today, it's almost impossible to understand how business was conducted with fax machines as a primary form of communication. In the same vein, Service Virtualization, like email, is a transformative technology that allows organizations to achieve unprecedented levels of productivity across their software development teams.

Why Do I Need to Automate API Testing?

During the SOA era, which primarily promoted the adoption of internally-created services, organizations were familiar with the service producer and could reasonably assume that certain standards or "policies" were met. Essentially, they could adopt a "trust but verify" strategy. However, now that architectures have become increasingly distributed and more partner or 3rd party services are being consumed as part of an organization's business-critical transactions, the need for automation has reached an inflection point.

After all, if your application fails to deliver the expected business results, your customers and partners won't care if that failure stems from code you developed internally or from an external API that you've integrated. If you consume it, you own it. Using test automation and monitoring to ensure the integrity of all the APIs driving your business-critical transactions is imperative for protecting your brand — and will continue to grow in importance as business-critical APIs keep expanding throughout the organization.

Is Service Virtualization Required for a Continuous Release Strategy?

Bluntly, Service Virtualization is not required for anything. Whether it makes sense for your organization truly depends on the level of business risk that you're willing to accept upon release. If your software is empowering business-critical transactions and your tolerance for failure is low, then I would say that Service Virtualization is critical. Without it, having the necessary level of access to a complete test environment is virtually impossible, given the complexity of today's systems and the difficulty associated with staging complete test environments. You'd have to either limit the scope of your testing, or risk performing at least some of your testing in production. 

Service Virtualization gives organizations on-demand access to simulated test environments — including all the distributed components required to validate an end-to-end transaction. This ubiquitous access to complete test environments is critical for continuous release strategies. Having a simulated test environment significantly reduces the risk of application failure by enabling the organization to execute a broader, more complete suite of tests. 

At the end of the day, if you're unable to continuously execute a suite of tests that helps you understand business risks, testing becomes a huge waste of time. Service Virtualization eliminates the constraints associated with test execution, enabling you to get feedback faster and more consistently.