Continuous Testing: What Exactly is It?

The aptly named DevOps practice of Continuous Testing synchronizes Testing/QA with Dev and Ops processes optimized to achieve business and development goals.

A few years ago when Agile was all the rage, lightning-fast Dev sprints left the Ops and QA personnel playing catch-up due to inadequate and sluggish testing practices with minimal coverage across frequent builds.

The widespread culture of Agile development has accelerated development while software testing still lags behind, forcing organizations to cut corners in QA or slow down Dev processes entirely.

Instead of speeding up the conveyor belt beyond control or slowing down development to kill efficiencies, Continuous Testing in DevOps encourages a systematic approach toward process improvement.

Continuous Testing goes beyond automation and encompasses all practices — including tooling and cultural change — that help mitigate risks before progressing to subsequent Software Development Lifecycle stages.

Continuous testing and Automation

The approach to Continuous Testing can vary and follow diverse pathways to ensure the best user experience is delivered, free of defects.

It’s barely viable to repeat all tests every time a new feature is added, so the Continuous Testing strategy fosters a company-wide cultural change to achieve four capabilities: Test early, test faster, test often and automate.

End-to-end test automation practices are intended to integrate QA into existing fast-paced Dev and Ops processes as a means to create continuity while maintaining faster development cycles.

It’s important to understand the role of automation in Continuous Testing. Automation alone does not enable continuity in testing, but it helps provide a qualitative assessment of risk and practice actionable tasks to mitigate these risks throughout the SDLC.

In essence, automated testing constitutes the detection process for software issues and defect prevention, whereas Continuous Testing addresses the wider challenge of improving the effectiveness of these detection sensors.

What does Continuous Testing Look Like

This example demonstrates how collaboration, automation and tooling enable end-to-end testing that’s performed faster, more often and better aligned with Agile Dev and Ops practices.

Continuous Testing starts within development processes as Devs use open source tools such as Selenium to test the functionality of their code.

Tools such as GitHub or other internal repositories can then be used to store tests and version together with the software code. DevOps teams can use the same tests to integrate frequent builds of the software with the Load Impact API.

Once the code reaches preproduction, dedicated QA teams can use these tests by modifying dynamic test parameters as appropriate. In the production environment, Ops can reuse these tests for acceptance testing and ongoing post production monitoring.

Once the application goes live, integration with existing APM tools help keep tabs on server-side app metrics.

Continuous Testing in this example is broken up into at least four environments: Development environment, Continuous Integration environment, QA Pre-production and Performance testing environment, and the Staging environment. Additional post-live monitoring further enable end-to-end continuous testing.

Key Elements of Continuous Testing

Parasoft, a global software solutions company, is one of the great thought leaders in DevOps and Continuous Testing. The company has laid out some key elements of the process in a fantastic ebook, and they're definitely worth sharing here:

Risk Assessment: Covers risk mitigation tasks, technical debt, quality assessment and test coverage optimization to ensure the build is ready to progress toward next SDLC stage.

Policy Analysis: Ensure all processes align with the organization’s evolving business and compliance demands. Primary objectives include:

• Identifying trends associated with injection of dangerous patterns within the code
• Augmenting defect prevention practices for high risk areas
• Isolating risk in target areas

Requirements Traceability: Ensure true requirements are met and rework is not required. An object assessment to identify which requirements are at risk, working as expected or require further validation.

Advanced Analysis: Using automation in areas such as static code analysis, change impact analysis and scope assessment/prioritization to prevent defects in the first place and accomplishing more within each iteration.

Test Optimization: Ensure tests yield accurate outcomes and provide actionable findings. Aspects include Test Data Management, Test Optimization Management and Test Maintenance.

Service Virtualization: Early, frequent and everywhere testing requires access to real-world testing environments. Service visualization enables access to the virtual form of the required testing stages, cutting the waste time to test environment setup and availability.

An effective Continuous Testing strategy incorporates all of these elements and forces an organization-wide cultural change to synchronize Dev, Ops and QA/Testing as part of the true DevOps philosophy.