5 Reasons to Model During QA, Part 1/5: “Shift Left” QA Uproots Design Defects

DZone 's Guide to

5 Reasons to Model During QA, Part 1/5: “Shift Left” QA Uproots Design Defects

Shift-left testing and model-based techniques help find defects in design before they reach production.

· DevOps Zone ·
Free Resource

Model-Based Testing (MBT) itself is not new, but Model-Based Test Automation is experiencing a resurgence in adoption. Model-Based Testing is the automation technique with the greatest current business interest according to the 2018 World Quality Report, with 61% of respondents stating that they can foresee their organization adopting it in the coming year. [1]

Technologies like The VIP Test Modeler have significantly reduced the time and technical knowledge needed to model complex systems. Organizations can now enjoy all the benefits of Model-Based techniques within short iterations, whereas previously modeling had been reserved for only the most high-stake projects, such as lengthy Waterfall projects in aerospace and defense.

This five-part blog series considers the benefits of introducing Model-Based Techniques to QA. It considers five reasons for adopting modeling, the theme of an upcoming Curiosity webinar. Today’s article begins with the very beginning of the delivery lifecycle, setting out how modeling can uproot defects earlier, where they require far less time and resources to fix.

The Case for “Shift Left” QA

Historic data from both industry and academic research has consistently shown that the errors introduced earliest in the software delivery lifecycle account for the greatest number of defects:

  1. In 2001, research from The IT University, Denmark, found that 59% of defects are traceable to the design phase; [2]
  2. In 2009, Bender RBT suggest that 59% stem from requirements; [3]
  3. By 2012, the Hyderabad Business School found that 64% of bugs emerge in the requirements analysis and design phase. [4]

The same research has established that it is far more costly and time-consuming to remedy these defects after the design phase. In fact, requirements defects can account for as much as 82% of total defect remediation efforts, [5] while a bug identified during testing costs on average 15 times more to fix than one discovered during the design phase. [6]

This is the textbook argument for beginning QA earlier: it allows you to detect the majority of defects as they emerge, rather than after they have been perpetuated throughout the code. Detecting defects during the requirements analysis and design phase, therefore, gives you a greater chance of delivering quality software on time and within budget.

This raises an important question: what does it mean to perform QA during the design phase? To understand this, we must first consider why so many defects are introduced during requirements analysis and system design.

Low-Quality Requirements: The Target of Shift-Left Testing

The quality of the requirements themselves remains the major cause of defects. The formats and methods used to capture software designs are ill-matched to reflect complex system logic. They tend to be ambiguous and incomplete, leading to miscommunications. These misunderstandings then require rectification through costly and time-consuming rework.

Organizations still frequently capture user and business needs in a range of disparate formats. These include written user stories, written documents, and behavior-driven specifications, as well as email or chat requests and business process diagrams. Such requirements are rarely connected formally and rely heavily on natural language.

Natural language is naturally ambiguous and is far removed from the precise system logic that needs to be converted into code. Written requirements therefore frequently allow for multiple interpretations of how the system should function, risking a drift in the vision held by BAs, developers, and testers.

Ambiguous Requirements in Gherkin
Figure 1: A Gherkin Feature File: the natural language is ambiguous and the feature file focuses only on a handful of possible logical scenarios. A multitude of these feature files are needed to document a system. However, the files are rarely formally connected to one another.

The same requirements also tend to be incomplete. Unsystematically piling up unconnected requirements inevitably leave holes in the design of vastly complex systems, and requirements often focus near-exclusively on a limited number of expected scenarios. Developers must fill the remaining holes with their imagination, and defects arise when their vision deviates from the desired system.

Finally, the unconnected and informal nature of requirements formats prevents formal dependency mapping. Developers must then imagine how the mass of components in a complex system should integrate, again increasing the likelihood of bugs. Numerous high-profile outages that have followed supposedly routine updates point to the real danger this presents.

Shift-Left Testing With Models

Requirements are a prime place to improve the speed and reliability of software delivery, given the prevalence of design defects. Formally modeling requirements offers one way to achieve this “shift left” QA. It is now furthermore now well-suited to short delivery lifecycles.

Flowchart modeling offers one such way to formally model a system’s logic. The blocks of a flowchart break a system down into its core cause-and-effect, with the process and decision blocks forming a directed series of “if this, then this” statements. These are consolidated clearly and graphically, mapping the logical journeys through a system that could be taken, and the data required to traverse that path:

“Shift Left” QA with Visual Flowchart Modelling
Figure 2: A visual flowchart clearly depicts the validation rules of a new user registration page.

The act of modeling new or existing requirements works to eradicate ambiguity. If there are multiple ways of parsing a requirement, the singular and correct understanding must be established to build the model. Testers can thereby act as critical modelers, working with BAs from day one to crystallize the desired system logic. This “shift left” QA roots out ambiguity in the design phase.

Modeling further works to eradicate incompleteness. Any missing logic is far easier to spot in consolidated visual models when compared to unconnected disparate written requirements and diagrams.

Formal Modeling in Short Iterations

Flowchart modeling offers the significant advantage that it is already frequently used by the BAs who gather requirements, many of whom already create business process. Flowcharts, therefore, enable testers to work collaboratively with BAs, to eradicate ambiguity and incompleteness from day one.

The VIP Test Modeler further provides a range of connectors for building flowcharts rapidly. This includes importers for existing requirements and test cases, as well as for recordings and scans of already built systems. This accelerates the modeling process, introducing all the advantages of modeling to short iterations. The choice between clear and complete specifications and rapid delivery is no necessary.

“Shift Left” QA using a Gherkin Importer
Figure 3: The Gherkin importer converts specification files automatically to unambiguous flowchart models in The VIP Test Modeller.

Not only does explicitly modelling requirements to eradicate defects before they enter code, but the same requirements models can be used to generate the functional and performance tests needed to validate the code once it has been developed. This enables greater testing agility, and is the subject of the next article in this series.


[1] Capgemini, Micro Focus, Sogeti (2019), World Quality Report, 27-8. Retrieved from https://www.sogeti.com/globalassets/global/wqr-201819/wqr-2018-19_secured.pdf on 30-May-2019.

[2] Soren Lausen and Otto Vinter (2001), “Preventing  Requirement  Defects:  An  Experiment  in  ProcessImprovement”, in Requirements Engineering (2001, 6:37-50), 38. Retrieved from http://www.itu.dk/people/slauesen/Papers/PrevDefectsREJ.pdfon 30-May-2019.

[3] Bender RBT (2009), Requirements Based Testing Process Overview, 2, 16. Retrieved from http://benderrbt.com/Bender-Requirements%20Based%20Testing%20Process%20Overview.pdf on 30-May-2019.

[4] P Mohan, A Udaya Shankar, K JayaSriDevi (2012), “Quality Flaws: Issues and Challenges in Software Development”, in Computer Engineering and Intelligent Systems (2012, 3:12:40-48), 44. Retrieved from www.iiste.org/Journals/index.php/CEIS/article/viewFile/3533/3581 on 30-May-2019.

[5] James Martin, cited from Bender RBT, Requirements Based Testing Process Overview 2.

[6] P Mohan, A Udaya Shankar, K JayaSriDevi (2012), “Quality Flaws: Issues and Challenges in Software Development”, 44.

[Image: Pixabay]

devops, model, model-based testing, requirements, requirements analysis, requirements gathering, shift left, shift left testing, test automation, testing

Published at DZone with permission of Thomas Pryce . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}