Testing: What It Is, What It Can Be
Testing: What It Is, What It Can Be
A preview from our upcoming Research Guide to Code Quality and Software Agility, out now!
Join the DZone community and get the full member experience.Join For Free
What do you think of when you hear the term “software testing?” Many people I’ve encountered say it’s a group of people refining the software just before release, doing their darndest to break the beautiful software that was handed over to them, cackling with glee each time a new defect is found. Other people have an answer that involves someone as a gatekeeper, akin to the bridge keeper in Monty Python and the Holy Grail, subjecting each release to questions for which a wrong answer can mean death (at least for the predetermined release date chosen back at the beginning of the iteration).
While there certainly are testers who fit these stereotypes, the reality of testing is undergoing a bit of a renaissance. Test teams are in the process of adapting their work from the slow, documentation-heavy methods of the past into more flexible and rapid approaches, enabling them to better keep pace with software development. At the core of this adaptation is a realization that testing isn’t about finding bugs. If it were, test teams would “fail” more and more as the overall team improved since there would be fewer bugs to find. Instead, modern testing focuses more on providing information at the time it’s needed. Bugs are just one piece of that information.
Testing As It Is
In a progression similar to the Technology Adoption Lifecycle from Geoffrey Moore’s Crossing the Chasm (Figure 1), different organizations are at various points in their responses to testing changes in the software industry. At the back of the pack, some organizations function much as test organizations have for decades. Their test cycles are often measured in months. These organizations are likely feeling pressure to reduce their delivery cycle while still finding as many bugs as possible.
Moving left along the curve, we find the majority of testing organizations. These teams have begun to adapt, at least in part, to increased time pressure from consumers. Some have brought exploratory methods into their testing, leaving behind rigid scripting and allowing the tester autonomy to immediately incorporate information gathered during testing. Some have also built automation into their efforts for well-defined, repeatable testing work, leveraging tools to perform those tasks rather than humans. Some may have even brought testing work forward in their development process, mobilizing the entire team to catch problems earlier and collaborate on fixes. Teams may not explicitly focus on being information providers and, as such, may be missing opportunities to reach their full potential. However, the testing team is probably achieving some success in keeping up with the rest of their organization, depending on how they interact with the developers.
The early adopters portion of the curve is where we begin to really see testing innovations, achieving results beyond the old norms. Testers in these organizations routinely analyze systems they are testing from multiple viewpoints or “lenses" . They go beyond tests that confirm software functionality and use risk-based test design to think about specific failure cases. At this level, testers move away from vague failure scenarios and instead draw on skills in experimental design and risk analysis, as they craft tests to reveal if certain imagined failures canactually be triggered. Here we see testing becoming a vibrant and exciting career path.
Testers in early adopter organizations often draw on concepts from psychology, both to recognize their own limitations (such as confirmation bias and inattentional blindness ) and to better understand how others might use their software. These testers are embedded in the overall team, working closely with analysts, developers, and designers to build software the entireteam can be proud of releasing.
In teams within these organizations, testers can serve as headlights, illuminating things to come so that the team canreact accordingly . They can investigate areas of unanticipated feature interaction or operating conditions such as input data, limited resources (i.e., CPU or memory), and user behavior. These might be things that the larger team hasn’t considered.
Finally, there are innovators: the vanguards who are defining the new vision of software testing for the rest of the industry. Testing jobs in these organizations may not look anything like traditional testing jobs. They may involve skills like data analysis—digging into massive sets of data to provide their team with detailed insights around their system and users. These testers invent new ways of visualizing the information they gather, and communicate it to their teams effectively and efficiently.
Some teams on the cutting edge of testing work closely with their operations team, exploring the space of DevOps. These test teams may be better-equipped to leverage automation by increasing its sophistication, using it as a tool to support testing work, rather than replacing it. These organizations are continually refining their work flows, discarding those that no longer provide valuable information, tweaking others to keep them relevant, and introducing new tasks to answer questions the team didn’t know they had. Testing in this type of organization is a rewarding challenge, a critical role to keep the team moving forward, and a far step from the perceived drudgery of more traditional testing.
That’s Great! How Do I Get There?
Most organizations fall into the middle portion of the curve. Moving towards the front of the curve takes effort, but it is achievable. To start, you can:
Analyze your current practices. Effective testing provides information that the team needs. If a task only results in information the team and stakeholders don’t value, it maybe time to stop the task or change it to make it more useful.
Analyze “release day” emotions. If the team has an uncomfortable mood on release day, it can be a red flag indicating that the team doesn’t have all the information needed to have confidence in the release.
Pick one unanswered question that the team has at release. Don’t try to make changes all at once. It can be difficult to formulate clear questions, so the team needs time to gain experience. By identifying just one question to answer, you can begin building experience within your culture while making just a small number of changes.
Break down walls. Teams build software best when they function as one team. Break down the walls isolating developers, testers, and analysts. Foster communication between groups that doesn’t go through the bug tracker.
Look to the cutting edge. The practices described here areonly a small subset of what test teams can do, but they are good starting points. If you are uncertain where to start, seek help—either online, from the broader testingcommunity, or by bringing in someone external with the expertise to meet your needs.
Testing is a critical function in software development, andwhen utilized effectively, it provides a team with a steady f lowof information. This allows the team to make quick, confidentdecisions, and to avoid repetitious and ineffective work whileensuring you deliver high quality software.
For best practices on writing, testing, and monitoring quality code, get your free copy of the DZone Guide to Code Quality and Software Agility!
Opinions expressed by DZone contributors are their own.