Doing Agile Testing Properly: How Metrics Can Help

The Agile methodology for software development focuses on releasing high-quality software regularly. Thorough software testing forms an important part of achieving that aim, in particular, by ensuring software is high-quality.

However, there must be a way to measure testing so that it can be improved. The measurement is where metrics come in—metrics provide a way to assess the efficiency of a testing effort in Agile. Without tracking test metrics, your Agile team can't get feedback or insight about any problems related to its testing processes.

The following article details how testing fits into the Agile methodology. You'll find out how the role of testing has changed over time, some best practices to follow that ensure proper software testing in Agile, and examples of the types of test metrics your team should track. See SeaLights' Agile testing metrics learning section for a wider list of recommended metrics.

Where Does Testing Fit Within Agile?

Agile testing simply means testing software while following the principles of Agile software development. Agile principles include flexibility, responsiveness to feedback, and collaboration.

In the old waterfall software development model, a separate team tested software away from development. The separation of developers and testers in waterfall led to a divide that hampered software production. Developers strove to create perfect code, while the QA team tried to break the developers' code by finding as many bugs as possible.

The Agile approach is cross-functional, meaning there are no longer any separate QA teams—the developers and testers unite, comparing notes daily. In Agile, developers think more like testers, which means errors are found earlier, and testers think more like developers by engaging more with the creative development process through collaboration. In other words, you test as you build.

Agile Testing — How to Get it Right

The below best practices teach you how to get testing right in Agile by aligning how you test software with the overall aims of Agile development.

Automate Tests Where It Makes Sense 

Testing early and testing often are important concepts in Agile testing. Teams must attempt to automate tests that make sense to get quick feedback on the status of a release. Manual testing should still be a cornerstone of your approach to QA but should be shifted left where possible.

Since Agile teams operate on a collaborative level, automation is the responsibility of both testers and developers. In particular, focus on automating regression tests to ensure continuing system stability with each change and shorter development cycles. 

Understand where your limitations are with automation and engage with a service that can scale the manual testing component of your development cycles.

Consult with Business Analysts

Good testing relies on setting testable requirements from the outset. Communication with Business Analysts should be encouraged to remove ambiguity from user stories. Every user story in agile should include clear acceptance criteria.

Avoid Functional Complexity 

The more complex a feature or function is, the more difficult and laborious it is to thoroughly test that function. The whole team should discuss the desired functionality and determine whether test cases can be written to cover all functions in every way possible. The software must be tested thoroughly before release to ensure it delivers what the user expects.

Instill Team Value 

The only way Agile testing can work is if software quality becomes a combined team responsibility rather than falling at the feet of QA testers. The whole team must agree upon testing strategies, test cases, and defect prioritization. Proper Agile testing, therefore, is reliant upon team value as much as specific testing processes.

Enter Test Metrics

Within Agile teams, test metrics provide feedback on testing efforts, areas that need improvement, and whether the testing processes achieve the vital Agile aim of high-quality software with frequent releases.

Some good examples of Agile test metrics that can provide value are:

• Defect cycle time — Quick software release times are important in a fast-paced Agile team. Measuring defect cycle time indicates how quickly defects are resolved by the team. Shorter defect cycle times help achieve faster software releases.
• Defects found in production — Early software testing aims to find defects when they are cheaper to fix. Counting the defects in production is a measure of the team's testing efficacy. Agile teams can capture this metric per sprint, release, or unit of time to find out specific testing or development issues.
• Code complexity — Metrics such as cyclomatic complexity can measure the risk inherent in any build by determining how complex the code is. Simple readable code results in reduced defect counts, which in turn means higher quality software. Static code analysis can also test if code adheres to established industry standards.
• Cumulative flow — Cumulative flow diagram shows work-in-progress, completed tasks, testing, velocity, and the current backlog on an area graph. Agile teams can use this metric to measure software testing progression and analyze whether testing is a bottleneck in the development cycle.

Getting testing right in Agile requires a multi-faceted approach that ensures testing practices align with Agile aims. Automate possible tests, especially regression tests, and instill a team ethic that ensures equal responsibility for software quality.

Don't neglect to measure test metrics in Agile teams. All metrics must be useful in an Agile context, meaning older individually focused metrics aren't sufficient to provide value.

Last, always bear in mind that no metric provides all the answers for judging software quality or testing efforts. A holistic approach is needed that collates relevant measurements and provides a unified perspective on software quality, reflecting the combined efforts of Agile team members.

For further information on Agile testing, we have written the Ultimate Guide to Agile Testing that takes you through the principles, methodology, benefits and potential pitfalls of Agile testing.