You may have read recently about how we’ve spent the last month exploring the theme of data classification in Foundry, Redgate’s R&D division. SQL Atlas is our imagined solution to the challenges you’ve been telling us about during this research.
So, What’s the Problem?
All companies and organizations that exist collect data in some way, shape, or form. We rely on this data to be able to provide our products and services to our customers. Crucially, we’ve also realized it’s got additional value.
Business intelligence gives us the opportunity to become more efficient or spot new commercial opportunities. Using real data in development and QA means we can launch better, more stable new features for our products. We can improve the customer support experience if customer service representatives can access the details to give them the context for the problem.
The problem is that all these business activities generate data sprawl: copies of data strewn across your organization. As we become more aware of our obligations to treat particular pieces of sensitive data carefully, the free access/use of data is less attractive (especially when the risk of hefty HIPAA, SOX, or GDPR fines could have dire consequences). And so we revoke access and build complex request systems for the consumers of data within our organization, trying to manage the flow of data centrally — yet also creating a bottleneck and choking the business.
How Might SQL Atlas help?
What if you could devolve and distribute the responsibility of accessing, preparing, and using sensitive data appropriately? Here, we’ve imagined how SQL Atlas might be able to make that work.
Step 1: Auto-Classification
In the first step, SQL Atlas does the hard work for you, traversing your SQL Server to help you answer the question, What data have we got and how dangerous is it?
Figure 1: The results of a data scan showing the amount of sensitive data on production, and its classification.
Step 2: Treatment Policy Management
SQL Atlas uses classification categories – or you can use your own – from which to create a set of corresponding rules for handling that type of data.
Figure 2: Tuning the rules that apply the classification categories to each data type,
along with their corresponding treatment policies.
The policies that these rules correspond to describe how the data must be handled according to the regulation(s) that your organization is subject to.
Figure 3: The collection of your policies, defined by the regulation that describes how to deal with different pieces of sensitive data.
Step 3: Using the Data Catalog
The result of a scan is a single repository, the catalog of all the data held within SQL Server. For each piece of data, the catalog holds basic information like:
- A description: A description of the data
- The owner: Who is responsible for the collection/management of this data
- Access conditions: Who’s allowed access to this data (and in what form; raw, treated, etc)
- Possible uses: A description of what the data is allowed to be used for
But crucially, it also now holds new information about the sensitivity of the data, including:
- The classification category: What, if anything, makes this piece of data sensitive
- The appropriate treatment policy: This is the policy that describes how this piece of data must be handled, according to its classification for BAU activities like BI or development
This means that SQL Atlas has created a single catalog of the data held within the SQL Servers in your organization. Now, when someone within your organization needs to consume this data, they’re able to query the catalog.
Figure 4: Users can select the data they want to use from the data catalog.
When the consumer selects the data they’re interested in, SQL Atlas presents them with the documentation you prepared in Step 2 that describes how they must prepare and handle that data in order to be able to use it.
Figure 5: SQL Atlas presents the corresponding handling policy for each piece of data
that the user searching the catalog is interested in.
We’ve created an interactive demo so you can try out the tool for yourself!