Defining Failure in Software Testing

In most contexts, the idea of failure is pretty black and white. An objective is set by an individual or group, and if this goal is not reached, within the defined parameters, the effort is considered a failure. This win-or-lose mentality is widespread, but things are a bit different in the world of software testing.

Since the idea of failure takes on a whole new definition in the test environment, test managers and front-line staff members need to rethink the concept in its entirety. Here’s a brief guide that will explore the idea of failure in the context of software testing and put teams and managers on the same page.

Also, if you are going to Agile 2016 in Atlanta, be sure to check out Eric Jacobson’s presentation on Talk the Walk: Using Language to Improve Testing.  Eric recently gave this presentation to our Atlanta Testing Group sponsored by QASymphony this past year.

Unique Terminology

Defining failure in the test environment is mostly a matter of perspective. Who encounters an issue, where and when this happens will all play a role in choosing the correct diagnosis. An ISTQB Exam Certification website offered a more in-depth breakdown of these words and their meanings:

• Errors: Mistakes made by programmers due to confusion, miscalculation or misinterpretation.
• Defects: Bugs in code that enter the test environment and are addressed by testers.
• Failures: Defects that are executed during the testing phase and cause the software to fail.

The source also noted that failures are not always the result of undetected defects or unresolved bugs – faults in hardware or firmware can be caused by environmental conditions or an error involving an incorrect input value entered in the test phase. In other words, the circumstances surrounding failure may vary.   

Learning from Mistakes

While the vocabulary of developers and testers may seem oriented in a negative way, it’s important to remember that there is a practical reason behind this terminology: to improve the software in a fast and accurate manner, at minimal risk. The ISTQB Exam Certification resource emphasized that these terms are used with the best interests of the stakeholders in mind.

“These defects or bugs are reported not to blame the developers or any people, but to judge the quality of the product,” explained the source. “The quality of product is of utmost importance. To gain the confidence of the customers it’s very important to deliver the quality product on time.”

In fact, terms such as “bug” are so commonly used because they are thought to remove some of the blame associated with defects and errors, according to QA Test Lab. Phrased in this manner, bugs seem like a natural and necessary part of the testing process, and a stepping stone toward an overall better final product.

Tools of the Trade

Just as software testers bring a distinct set of terminology to the table in their discussions of failure and success, they can work with specialized tools that help them navigate the process with greater fluidity and better results. Today’s managers can leverage enterprise software testing solutions to help streamline these processes and ensure that software is as resistant to failure as possible.