A COO's Guide to Test Strategy Creation

A test strategy is a set of high-level definitions that determine how software testing processes are going to be performed, driving our decisions on how to invest our testing efforts. Creating a well-thought test strategy is crucial to understanding the overall scope of your project, and what testing approaches, tools and skills are required in order to develop a successful product that provides a great user experience.

However, creating a test strategy is a complex task. Every piece of software is unique, and as such, objectives, user expectations, budgets and timelines differ greatly between projects. In this webinar recording and in this article, I'll share the foundational bases for creating and revising successful test strategies that you can adapt to your software projects regardless of their specific requirements.

Here’s a rundown of the main topics mentioned in the webinar, we hope they help you improve your test strategy.

Key Considerations of a Test Strategy

All aspects of a testing strategy are important, but some of them are crucial to meeting customer requirements, improving testing processes and achieving high quality software. Here are some of the main considerations to keep in mind when creating and revisiting your test strategy:

• Shift-left testing: This approach proposes to integrate testing practices early in the software development process, and is the key to identifying and solving bugs at earlier stages rather than later in the production phase, when fixing defects is much more expensive and time consuming. Also, shift-right testing it’s important, testing in production, getting information from monitoring, access logs, user reviews, etc. Any piece of information that can help us catch errors and reduce risks better in the future is very valuable. 
• Required technical skills: Each software project has distinct requirements, therefore different testing tools and frameworks need to be implemented in each case. Putting together a testing team with the right technical knowledge and expertise is critical in order to achieve a successful product (or training them on what’s missing). Identifying key project requirements is a great first step to understand what skills your testing team needs to manage the project and any challenge that may arise along the way successfully. 
• Required communication skills: Efficient communication is essential for team growth, as well as achieving high quality code. Having frequent and effective communication allows teams to identify defects early and solve them quickly, and improve the understanding of risks and challenges across teams and stakeholders. Defining the communication skills needed for your test team is crucial, and it will depend on the different teams they need to interact with (different providers, vendors, internal teams, etc.), on the type of communications they will need to manage (are they going to report to management? Will they need to prepare documentation for a specific compliance with a standard? etc). 
• Keep an eye on lead time: This is an important metric that measures the time it takes from the definition of a new requirement (new feature or change) until it’s delivered to the final users. Keeping track of this metric provides teams with an overarching picture of the project, and allows them to improve the test strategy to optimize the development pipeline by identifying inefficiencies. Once unnecessary delays have been recognized it’s possible to increase the speed of deployment by choosing better testing approaches and tools that directly support the project.
• Don’t forget the “ilities”: There are certain quality attributes that every now and then are forgotten and these are: portability, compatibility, usability, reliability, security, maintainability and performancibility (just kidding, performance). We should also consider these characteristics when defining our testing strategy, deciding when we are going to test each, how, who, how frequent, with which tools, etc. 

Quality: A Shared Responsibility

Although test strategies are designed to ultimately improve the quality of software, produce deliverables on time, reduce risks and optimize costs, this doesn’t mean the quality of a product relies solely on the effectiveness of a test strategy or the work of software testers. As we have mentioned before in our blog, quality is not a tester’s responsibility, it is a team’s responsibility and a shared effort across teams. On top of this, the concept of quality is a subjective one, and for those reasons, it’s essential to have a shared understanding of what quality means for your team and your customers.

Don’t Fall Into This Trap

One of the biggest mistakes is to underestimate the importance of a test strategy, either by believing that only large companies need them, or that they require too many resources to create. Implementing short-term methodologies that don’t serve a long-term test strategy, and using trending testing practices that aren’t actually needed are not beneficial for your project. Effective test strategies are always necessary, and revising them often is also key to successfully achieving the results we seek in our software products.

I hope you find this guide useful. Let me know in the comments below