How To Build a Business Case for Test Data Management
How To Build a Business Case for Test Data Management
Presenting a complete picture of test data management ROI is key to stakeholder buy-in.
Join the DZone community and get the full member experience.Join For Free
In today’s increasingly complex and fast-paced applications development environment, budgets are no less important than before; in fact, they may be even more important as the technology teams look to keep pace with business growth. Moves into Agile and DevOps are made with the promise of doing more faster to keep up with business demands. Adopting those requires increased investments in people and tooling/infrastructure.
While often overlooked and viewed as a reluctant necessity, testing, if done poorly, can end up being a much larger share of your budget than expected. Often those unexpected costs emanate from the lack of realistic and safe test data that can easily be accessed, reset and adjusted by developers and testers alike.
Savvy executives responsible for application development and/or quality assurance are finding that an investment in a Test Data Management (TDM) tool suite reaps big rewards in time and productivity and results in across the board savings for their organization. This frees up their budget for investments in other development tooling and staff. However, many leaders struggle to build a compelling business case that gains traction across the varied stakeholders required for approval to move forward.
An investment in TDM is compelling when the complete story is told, and in actuality the business case is pretty straight-forward. Nonetheless, some organizations struggle with this, so I’ve set forth the following points to keep in mind when building your case for a TDM investment.
1. Focus Your Case From The Start
TDM tool suites can bring benefits broadly across an organization. A business case can be structured to position a TDM investment as an enabler for an application development organization to support a move to DevOps. It could be an efficiency play for the QA group. The CISO could make a case based upon reducing insider threats. An enterprise-wide business case would consider all of those. Your case and specific components would vary based upon the specific use case. Focus your case from the start to achieve buy-in from key stakeholders across the organization.
2. Identify Your Approach and Scope
You have determined your case, but now what is your approach? It’s important to have a clear path forward. Regardless of the case, you have at least a couple of paths you can take: limited pilot or full roll-out. You probably understand your organization's tolerance for change and risk. Your strategy for going forward should take that into consideration. If a "pilot – evaluate – invest" model is an accepted method, start there and make the determination if that will work for your case as well. Keep in mind, if your rollout plans are taking a pilot-first approach, some organizations require that the business case reflect the enterprise-wide rollout in terms of investment and projected savings.
3. Identify Your Stakeholders – There Are Probably Many
Your business case needs to include and be addressed to all the players that will have a hand in approving the decision to move forward with the TDM program. It is important to note that even if you chose a more tactical use case, like QA department efficiency, other parts of the organization may be affected by a TDM implementation and as such, should be given the opportunity to weigh in on the case. In addition to the areas noted above, it may be worthwhile to include architecture, risk management, and compliance. In addition, don’t forget the business side of your organization. TDM can also enable the business, whether it is helping to improve time to market or enabling risk-free partner implementations, don’t overlook letting the business team have a part in the discussion.
4. Identify Expected Cost Savings and Productivity Gains
In order to win the approval of the investment, you will need to be able to demonstrate the quantifiable benefits. These are, of course, dependent upon the case you are making, but usually fall into one of two categories: cost savings/avoidance and productivity gains. It goes without saying that your model of these benefits needs to be as accurate as possible.
- Reducing Infrastructure Costs. If your organization lacks a TDM tool today, chances are you are making copies of the production environment and taking some rudimentary approach to de-identifying data. It’s not uncommon to find organizations who have replicated production environments three or four times. That means infrastructure costs are three or four times the production costs. A TDM tool that includes a powerful subsetting engine allows you to create referentially intact subsets of data that are a fraction of the size of the production version. This will allow you to dramatically decrease the footprint of the lower environments and the resources required to support them.
- Improving Tester Productivity. The provisioning of test data is a challenge to most organizations with a TDM solution. Testers are often left waiting while the data or operations team extracts and loads (re-loads) the data into the desired environment. As a result, QA teams must staff up in order to complete the testing without extending the timeline. TDM, when done right, eliminates that waiting and thus reduces the resources needed to complete the testing. Additionally, a poorly-executed test data strategy can negatively impact the testing itself. Bad test data means time spent chasing down bugs only to find it was really just bad data. Improving the quality of the test data increases QA productivity and can allow you to reduce QA headcount.
- Improving Developer Productivity. Much like your testers, developers can often find themselves waiting on test data to be provisioned. As you seek to "shift to the left," this becomes more of a consideration. A TDM tool that can provision data on demand so that data is available for developers when they are ready to unit test can help shorten the overall delivery cycle. Quality data also means that developers don’t waste time troubleshooting issues only to find it’s a data problem not a code problem.
- Reducing Support and Reworking Costs. Your production support costs increase or decrease based upon the quality of the application. By improving the quality of the application that goes into production, you will expect fewer defects which will reduce remediation (triage, assess, fix), retesting and redeployments. As the quality of the product goes up the costs to support it in production are reduced.
- Avoiding Data Breach Penalties and Costs: Without a good TDM solution, many organizations resort to testing with their production data and may do little to no de-identification of those records. According to the 2018 “Cost of a Data Breach Study” by Ponemon, the probability of suffering a breach of 10,000 records or more over 24 months was a staggering 28%. At an average cost of $148 per record (much higher for certain industries, such as healthcare) this can be a significant cost that can be mitigated by shrinking the threat surface area by de-identifying data used for testing.
5. Document and Defend
We just laid out some of the benefits of TDM that can be quantified. The next step is to translate those into actual savings and productivity gains that take into account the realities of your organization. Unless you create an actual model that demonstrates how you calculated these benefits, you will never be able to defend it under scrutiny. One thing that has worked well for me in the past is using already existing KPIs or metrics that are familiar across your organization. In addition, it is a good practice to identify and clearly call out assumptions that you may have made along the way. Usually, assumptions are used when key information may not be readily available. The can also be used to prepare sensitivity analysis to further ground your case.
Important metrics might include:
- How many test environments do you maintain?
- How frequently are you able to refresh the data in your test environments?}
- How many releases are done each quarter?
- What is the average length of time spent provisioning test data for each release?
- How many issues are raised in each testing cycle? What percentage of issues raised during a cycle are data issues and not code issues?
- How many production issues occur in a typical month/quarter/year? What percentage of those are related to the lack of good test data?
Once you have this information you can start to model the savings or productivity gains that a TDM can actually make (e.g. how many tester hours does a 90% decrease in manual effort to provision test data make available?).
6. Understand the Complete Investment
The flip side of your case is, of course, what is the cost of a TDM solution? Given the range of solutions on the market, the actual license and maintenance costs can vary dramatically depending upon the solution and approach you take. TDM historically was only within reach of organizations that could make an seven-figure investment in enterprise classes of tools. Fortunately, newer entrants into the market offer powerful features without an enterprise price tag. It is very important to understand that implementing a TDM solution goes beyond the licensing and annual software maintenance costs. Depending on the chosen solution, the implementation costs can be quite large. In addition, some solutions may require specialized skillsets that could require outside hires that are difficult to find in some markets. Another cost that is sometimes overlooked is the required infrastructure to support your solution. Getting a handle on these expenses upfront allows everyone to understand the complete picture from the start. Again, many of the new entrants in the market offer compelling alternatives that don’t require million-dollar consulting engagements for implementation.
Testing Your Way to Success
Keeping the above points in mind will help you put together a comprehensive business case that provides the rationale and justification for a TDM investment. As previously stated, you need to know your organization and right-size your business case such that it matches your ask.
Creating a winning business case and getting the green light to move forward with a TDM investment is just the first step in bringing improved efficiency, quality, security and cost savings to your organization.
Opinions expressed by DZone contributors are their own.