Many organizations today use requirements management or test case management tools to capture and store their requirements, including user stories and associated test cases. But these tools usually lack the capability to model requirements in visual and dynamic flowcharts and they don’t have the capability to optimize tests for maximum coverage.
The key idea behind requirements grooming is having the capability to create requirements directly and organically or model the environment in a supported lifecycle management tool and import it. Support for common tools like Microsoft Visio and Business Process Models via BPMN and XML Process Definition Language (XPDL)-compliant formats becomes a must, as well as the ability to import test assets such as user stories and existing test cases from a range of tools, including JIRA, VersionOne, MicroFocus/HPE ALM, and CA Agile Central. That is important, as those testing assets can be used as the starting point for grooming a requirement when there is nothing currently documented. While there are a variety of options for requirements grooming, the key point here is that once youhave captured your requirements in a flowchart, all the remaining artifacts (specifically, test cases and test scripts) are automatically generated from the flowchart.
Automatic Requirements Modeling
Once you start defining your requirements as models, you are effectively going into the realm of model-based testing. In this case, your requirements and your testing model become one and the same.
Here’s an example to show why this approach has advantages. In the traditional testing process, you usually define your requirements by hand in some text-based format. Your testing team then creates test cases based on them. When these requirements change, your testers will need to scour through the change request, analyze the impact of those changes on your existing test cases, and painstakingly carry out those changes across potentially hundreds or thousands of test cases by hand. This slow and error-prone approach simply goes away when you choose to take the model-based testing route.
A model-based testing approach delivers real ROI, but building your initial model can take time and effort. One way to break this barrier to model-based testing adoption is using technology to automatically create models for you. Truly game-changing automatic requirements modeling accelerates model creation, allowing you to quickly and easily generate your initial flow. That is accomplished by capturing the user actions (steps, assertions, data used, etc.) during your regular regression testing cycle. The data captured from those actions is then parsed into a flowchart. With the flowchart, this automatic requirements modeling capability allows you to deduplicate the steps, turning them into one concise model. As you can imagine, this allows you to radically simplify and accelerate the creation of your initial models, opposed to manually creating them by hand. Then, you just inspect the model for accuracy and add to it what you might have missed.
Test Case Design and Optimization
The right test case design and optimization approach greatly reduces the number of tests by only generatingthe minimum number of tests needed to achieve the desired coverage — driving testing efficiencies in the process.
In practice, there are different variations of test coverage. These include (in order of most coverage to least): all possible paths; all pairs (that is, every pair of edges); all in/out edges (every pair of edges where one can follow directly from the other); all edges; all nodes; and probabilistic. But oftentimes, the balance between your testing needs and how much you’re willing to compromise forcost determine your desired coverage level. In any case, test case design and optimization can provide metrics that show the amount of coverage you currently have. You can also see if you’re over-testing by showing how much each part of your model is being tested more than once. On the flip side, you can see if you’re under-testing by showing how much exposure you’re potentially dealing when your test coverage is lacking.
Test Automation Script Generation
Once you have determined your desired coverage and generated your manual test cases from the model, test automation script generation allows you to generate a suite of automated test scripts. For this, you must define a configuration file for your application, primarily consisting of standard and custom functions for the language you intend to use in your test automation execution engine (Selenium, Appium, VBScript, C#, Java, Ruby, etc.). Each of these functions are available to be referenced from blocks in your model. With each block referencing the functions in your configuration file, as each path through the model is defined based on the coverage technique you selected, the series of function calls concatenated from each block will form a fully executable test automation script that your test automation execution engine will pick up and execute.
You can run these withintest automation frameworks such as Selenium, CA Application Test, Ranorex, etc. Automation makes running and, more importantly, rerunning tests much faster and more efficient.
Change automation can single-handedly make testing much more cost-efficient by simply having the ability for test cases to be updated automatically when requirements change. If you’re still stuck doing the more traditional testing approaches, how do you make changes to the actual test cases when changes in requirements occur? The conventional way is to ignore existing tests in your suite and recreate them all by hand. Some say it’s easier that way. Yes, you can do this, of course. But doing so is highly inefficient and potentially costly. Just imagine if a given change only affects 8% of test cases. You’re effectively wasting the other 92% of the test cases, which are now being discarded and recreated just as they were. This is utterly unnecessary and on an enterprise-class project involving a large number of tests, the cost of recreating these tests can be huge.
Change automation becomes even more valuable to the business, as it provides the following capabilities in an automated manner:
- Analyze the impact of a change. This means that test cases that are affected by the changes in requirements are automatically detected.
- Heal your test suite. This means that test cases that were detected to have been impacted by a change can be fixed automatically, so they adapt to the new requirements. Or even better, when new tests are generated, they will be confined to paths that haven’t already been covered by existing tests.
What if those test cases haven’t been automated? If you’re part of an agile team, analyzing the impact of that change to the automated scripts — in addition to the manual test cases — would represent a considerable additional effort that won’t fit in your current sprint.
The power of change automation is magnified when you’re working in an agile environment. Those impacted automated scripts, in this example, would’ve been automatically healed or regenerated without the need to manually inspect and update scripts. How’s that for ROI?
Integrating With Test Data and Virtual Services
Going beyond requirements and test case design, test design automation delivers greater business value when it is further associated with test data and virtual services for testing.
Integration with test data management allows you to create and link the test data needed for testing, so when you run a test, relevant test data is generated automatically based on the appropriate parameters.
Integration with virtualized services allows you to access test data beyond the control of the development team. For example, if youare working in a B2B environment, you will need to test against data that is provided by your business partners, simulating this and other sources of data for testing purposes. You’re not necessarily testing your business partners’ systems; you’re testing your own application, which needs data from those partners. That way, you’re breaking the dependency on those external teams and not slowing down your development and testing activities.
Integrating With Broader DevOps Environment
The right test design automation technology integrates with the majority of testing frameworks. This is particularly important since organizations involved in software development looks for technology that enhances their existing testing practices and tools. This means tests (either manual tests or test scripts) can be exported into CA Agile Central, Micro Focus/HPE ALM, and Atlassian JIRA, among others. These integrations go beyond just passing information from one to the other. For example, synchronization with JIRA supports both sync up and sync down — so, once synced, any change in either tools will update the other automatically.
Going beyond testing framework integrations, your test design automation technology must have the capacity to import your existing test cases in a variety of formats — from basic spreadsheets to test cases written in XPDL, as well as tests output by Bender and SoapUI.
The experience must be smooth and simple so you can easily import your existing tests, reformat the resulting requirements model, and produce a suite of optimized test cases before exporting them back into your test case management tool of choice. Moreover, once this has been done once, there’s no need to import a second time. You can simply keep refining your requirements and generating test cases.