Testing for Developers, Part 1: What Is Testing?
In order to make developers more adept at the testing they'll do in DevOps, this series starts with some introductory terminology.
Join the DZone community and get the full member experience.Join For Free
In the series, we will outline the basic things that every developer needs to know about testing. First, we will start with defining testing and discussing the basic terms. The purpose is to give all team members a shared understanding of the fundamental terminology of quality assurance and all related processes. Later this will improve communication and reviews quality. It will further increase the testing capabilities of each member.
As part of the professional services we provide at Bellatrix, we consult companies and help them to improve their QA process and set up an automated testing infrastructure. After the initial review process and giving improvement recommendations for some companies we need to hire new talents that can help the company to scale-up the solutions we provided. This was part of the training we did for a company we consulted so that we educate all of their developers.
Software quality definitions found in IEEE Standard Glossary Of Software Engineering Terminology:
- Software Quality: The degree to which a system, component, or process meets specified requirements.
- Software Quality: The degree to which a system, component, or process meets customer or user needs or expectations.
Testing is a process rather than a single activity. It includes all life cycle activities such as static, dynamic testing, planning, preparation, evaluation of software products and related work products.
Main Objectives for Testing
- Identification: Identify defects in the software
- Confidence: Enable stakeholders to gain confidence in the system's level of quality
- Decision-making: Provide information for decision-making
- Prevention: Prevent future bugs by improving the development process and standards
- Help meeting standards
- Contractual or legal requirements
- Industry-specific standards
Cost of Failure
To correct a problem at the requirements stage may cost $1. To correct the problem post-implementation may cost thousands of dollars. In extreme cases, a software or systems failure may cost lives.
- Anomaly: Any condition that deviates from expectation based on requirements specifications, design documents, user documents, standards, other specifications, or from someone's perception or experience. Anomalies may be found during, but not limited to, reviewing, testing, analysis, compilation, or use of software products or applicable documentation. [IEEE1044]
- Bug/Defect/Fault/Problem: A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system.
- Failure: Actual deviation of the component or system from its expected delivery, service or result. Also called an "external fault."
- Defect/Fault Masking: An occurrence in which one defect prevents the detection of another.
How Much Testing Is Enough?
Too much testing can delay the product release and increase the product price. Insufficient testing hides risks of errors in the final product. We need a test approach which provides the right amount of testing for the project. We can vary the testing effort based on the level of risk (technical and business risks) in different areas.
Seven Testing Principles
Testing Shows Presence of Defects
Testing can show that defects are present, but cannot prove that there are no defects. Appropriate testing reduces the probability of undiscovered defects remaining in the software.
Exhaustive Testing Is Impossible
Testing everything (all combinations of inputs and preconditions) is not feasible except in trivial cases. Risks analysis and priorities should be used to focus testing efforts.
Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives. The later a bug is found, the more it costs!
Testing efforts should be focused proportionally to the expected and later observed defect density of modules. A small number of modules contain most of the defects discovered during pre-release testing or show the most operational failures
The same tests repeated over and over again tend to lose their effectiveness. Previously undetected defects remain undiscovered. The test cases need to be regularly reviewed and revised. New and different tests need to be written.
Testing Is Context Dependent
Testing is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site.
The acts of finding and fixing defects themselves do not help if the system built is unusable and does not fulfill the users' needs and expectations.
Published at DZone with permission of Anton Angelov, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.