The Data Disconnect With DevOps and Digital Transformation
The Data Disconnect With DevOps and Digital Transformation
Can database do better when adhering to DevOps? See how taking advantage of DevOps concepts to boost efficiency and facilitate digital transformation.
Join the DZone community and get the full member experience.Join For Free
Rapidly provision TLS certificates from any certificate authority within your DevOps CI/CD pipeline. Speed up deployments with automated TLS certificate provisioning using your existing DevOps toolchain.
Industry surveys indicate growing momentum of DevOps with widespread adoption and experimentation. But, a disconnect looms for most organizations due to inadequate access to relational data. Industry surveys highlight this disconnect:
- Puppet reports in the State of DevOps 2017, “high performers measure feature branch life and integration in hours (page 41).”
- Puppet reports in the same survey, that “high performers include systems of record (i.e. systems with relational database back-ends (page 45).”
- Dell reports in research conducted by Unisphere in State and Adoption of DevOps that “over 80% of enterprises refresh database environments for Dev/Test 2x monthly or less (page 19).
- RightScale reports in the State of the Cloud 2017 that Docker containers have risen to the #1 enterprise tool chain for DevOps and software delivery strategies (page 25).
While .NET and Java containers are provisioned in seconds, it takes days or weeks for most to refresh database environments. Without dramatic changes in delivery of relational data, many DevOps and Digital Transformation initiatives will simply fail.
Let’s take a look at the most promising strategies for alleviating this disconnect. Full disclosure: I am a principal at Windocks, which is discussed in this article.
Database Copies From Flash-Based Storage
The Public Cloud delivers Virtual Machines on demand, and Docker containers are provisioned with Java and .NET workloads in seconds. In contrast, databases average hundreds of GBs or more, and high performance flash based SANs can be used to improve data delivery.
An all flash SAN with a fast network can deliver 2 GB/second, enabling delivery of a VM with 500 GB of data in 10 minutes or less. This is a huge step forward for most organizations, but the cost for flash storage begins at $100,000 or more. Use is also complicated by the lack of data masking, which becomes another step in the workflow.
Database Clones and Snapshots
Storage Array Networks (SANs) also support fast, writable snapshots. Snapshots are writable, provisioned in seconds, and consume minimal storage and offer excellent support for delivery of relational data. But, these capabilities go largely unused due to complex scripting that is required to provision snapshots, LUNs, and mount points.
Fortunately, a new generation of storage vendors including Cohesity, Rubrik, and others, are delivering storage systems with data access via restful APIs. Customer feedback is positive, and these systems are a big step forward for improved access to relational data. Cost of ownership continues to be a challenge for organizations with budget constraints, and data masking remains as an extra step for sensitive data.
The primary drawback to storage centered strategies is their reliance on a UNIX operating system. Most storage systems run on Solaris UNIX, with the ZFS file system for fast snapshot support. Dedicated storage administrators come as part of the package.
Windows Database Cloning and Containers
A software based approach for database cloning is offered by Windocks on standard Windows Servers. Windows database clones deliver writable database environments in seconds, with minimal storage. Windocks is growing rapidly, and provides data delivery for a fraction of the cost of storage systems. As a Windows based solution, Windocks runs wherever Windows Servers are supported, including public clouds or on premise, and the cloning tools are designed for use by SQL Server admins.
Windocks is also an independent port of Docker’s source to Windows, supporting .NET, and all editions of SQL Server 2008 onward with containers. The combination of database clones with containers allows terabyte class databases to be delivered in seconds, with multi-tier application environments. Teams work on a shared server with containers realizing an average reduction in VMs used of 5:1 or more.
Windocks also includes a SAN ready container that simplifies use of SAN hosted snapshots. Customers extend the useful life of their SANs, and organizations with a mix of storage systems acquired through mergers and acquisitions also benefit from this approach.
Data Governance and Regulatory Compliance
Relational data strategies face growing Regulatory Compliance needs. Here, a software defined strategy that manages files rather than storage blocks offers further advantages. Windocks delivers fast access to relational data with data images that are immutable, versioned, and auditable in an enterprise data repository.
Windocks images can support scores of databases. Data masking, user permissions, and encryption are applied during the image build, delivering an environment that complies with privacy and security policies. These images “shift left” security concerns, incorporating security and privacy by default and by design. Implementing a similar workflow on storage systems involves a multi-step process involving mounting a snapshot, implementing privacy and security, and unmounting the prepared data volume.
Call to Action on Database Cloning and Containers
It’s time for CIOs and Cloud Architects to incorporate delivery of relational data as part of the larger DevOps strategy. Industry surveys indicate that Docker containers are becoming the preferred tooling for DevOps, and database cloning should fit into a container oriented strategy. Many will choose to invest in storage systems, but others will prefer a software defined strategy with Windows or Linux database cloning. For organizations that want a software solution on Windows, that combines containers with database cloning, Windocks presents an option.
Where leaders choose to invest in secondary storage systems, Windocks simplifies delivery of snapshots with SQL Server containers. A final advantage of software based solutions is their ease of evaluation, with a software download. Explore how software defined database cloning can enhance your relational database delivery, with the free Windocks Community Edition available here.
Opinions expressed by DZone contributors are their own.