DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. Five Test Data Management Practices You Should Improve

Five Test Data Management Practices You Should Improve

Testing is more complicated than it used to be, but you can make it easier by improving your test data management with these tips.

Denise Kalm user avatar by
Denise Kalm
·
Aug. 17, 18 · Opinion
Like (2)
Save
Tweet
Share
4.10K Views

Join the DZone community and get the full member experience.

Join For Free

Five Test Data Management Practices You Should Improve

Application testing used to be simpler — in the '80s. You had one mainframe, a limited set of datasets, all data in-house and little awareness of security and privacy. Today, there are many more factors you have to consider for test data management.

If you overlook or do a poor job in any one area, your testing results will be suspect, exposing you to risks in production as well as compliance fines and lawsuits. In short, getting your test data right is vital to your company. Which means it is job one for the testing team.

It helps to define a clear process so you can ensure you have checked every box before you call a test complete. Because careful preparation ensures better results, you should be considering these five steps when managing test data:

1. Data Extraction/Generation

The data most applications use will often span databases, LPARs, and systems. Some may even be external data. You have to define what you need clearly; this can take a lot of time when you consider:

  • What types of data your test teams need
  • Where the data is located
  • How you access data, knowing production data may not be easy for test teams to access without the skills needed to understand different databases and schemas
  • When or how frequently you need to refresh your data extraction to keep it current

2. Protecting Sensitive Data

In the vast data stores, companies maintain a huge amount of customer data in their production systems. Even prior to the enactment of GDPR, laws, and regulations made compliance with customer privacy a critical requirement. To comply with increasingly stringent data privacy regulations, you need to consider masking, scrambling, and encryption, as well as ensuring access to production data is managed. The rules surrounding this are even more important if you outsource.

3. Planning

You can't use all the data, so you need to select a representative subset that mimics production in all needed ways. Ideally, planning also includes understanding ways to standardize testing across groups to reduce the overhead of managing the test environment.

4. Maintenance

All this test data must be stored, maintained and updated regularly. Cost of storage maintenance can be significant, so you want to find a way to manage the effort cost-effectively.

5. Auditing

The test data needs to be audited to ensure it accurately represents the workloads that will run in production. You have to maintain referential integrity.

Learn more about the importance of these and other test data management best practices in our webcast with TCF Bank, "The Importance of Data for DevOps: How TCF Bank Meets Test Data Challenges."

Advance Your Test Data Processes

If you find yourself daunted by the challenge of accomplishing these steps, you have to be wondering if there isn't a better way to get good test data into the hands of your testers. In a DevOps world, there's no time for the process above if you try to do them manually.

Automation can make these steps much more manageable. Smart people need smart tools, and in the case of working with test data, they need a well-designed solution that provides:

  • Visualization of data relationships. It is essential to understand the relationship between data objects, but that's an impossible task if you do it manually. Instead, a modern DevOps feature like Relationship Visualizer in Topaz for Enterprise Data will show you these relationships automatically.
  • Visualization of data-related extracts. Topaz for Enterprise Data's Extract Visualizer will help you understand the extracts you have from a variety of data sources so that you can develop quality test data.
  • Evaluation of differences between files on different LPARS. Understanding how files differ can make the difference between having a great representation of production data or failing to accurately test.
  • One editor for all data types. Few applications rely only on a single data source and most include a variety, including Db2, IMS, VSAM and even non-mainframe data. If you have to describe your test data on a variety of tools, you can easily make mistakes. By using a common editor, you get a better result.
  • Ensured protection of sensitive data. With the implementation of GDPR and bound-to-be subsequent regulations across the world, companies have to improve their protection of customer data. Topaz for Enterprise Data empowers you to create your rules and share them in a repository, demonstrate compliance and exploit a variety of disguise rules to protect the data you have to use.

DevOps is a "hurry-up" world where each team member has to find a way to expedite their work while still ensuring that the job is done well. The demands of higher quality, velocity, and efficiency on testing are no different than those on development.

With Topaz for Enterprise Data, you can more rapidly prepare test data, ensure it is a good representation of production and protect customers' privacy. The quality of your test data is essential to ensuring great results. Why not make this part of your testing job a lot easier?

Test data Data management

Published at DZone with permission of Denise Kalm, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Cloud Native London Meetup: 3 Pitfalls Everyone Should Avoid With Cloud Data
  • Handling Automatic ID Generation in PostgreSQL With Node.js and Sequelize
  • A Beginner's Guide to Back-End Development
  • Top 10 Secure Coding Practices Every Developer Should Know

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: