A New Approach to Container Volumes
A New Approach to Container Volumes
Integrated storage virtualization delivers a solution to container volumes for organizations with large databases who are relying on storage primitives.
Join the DZone community and get the full member experience.Join For Free
Read why times series is the fastest growing database category.
Windocks is an independent port of Docker’s open source to Windows which supports the full Windows OS family and all editions of SQL Server 2008 onward, and .NET and varied open source projects. Windocks was released a year ago, and we just shipped the 2.0 release, featuring integrated storage virtualization and web UI. In full disclosure, I am a principal at Windocks.
Windocks supports scores of customers who rely on network or host attached volumes, with a combination of Dockerfiles and PowerShell to use snapshots and cloning available in NetApp, EqualLogic, and others. This approach works, but requests for simpler, more affordable solutions have led us to re-examine how to deliver data environments for Dev and Test. Let’s take a look at Docker Volumes and the new integrated approach provided in Windocks 2.0.
Docker Volumes Revisited
Docker supports volumes within a container or on the local host, or with networked attached volumes with Docker Volume plugins. Docker Volume commands support storage primitives, such as create, name, and mount volumes to a container. When working with storage arrays, Docker Volume plugins can support the creation and mounting of virtualized or cloned data volumes. This is the defacto standard today, and is effective in support of large data sets.
Why should an application level service like Docker ask developers to work with storage primitives? It seems incongruous for a cloud native platform to rely on provisioning infrastructure as part of a software delivery strategy. It works, but requires on-going maintenance. For every container removed, there is clean-up of mount points and volumes. There must be another way.
Data Governance and Compliance
Another challenge with Docker volumes is the lack of a data image registry. Dockerfiles and scripts are controlled by developers or DBAs, and there is no good way to ensure data provisioned conforms with corporate policies and regulations.
A New Approach With Integrated Storage Virtualization
Windocks was released in April of 2016, and is popular for delivery of SQL Server environments for development and testing. Initial persistent storage support was limited to in-container and mounted files. Our design improved on Docker-specific volumes by utilizing Windows mount points, but still required cleanup of mount points when a container is removed. Working with large data sets has been limited to customers with access to NetApp, EqualLogic, and other storage arrays that support cloning. Organizations that don’t have these tools, typically create “baseline” images of production data with reduced file size. The feedback from customers has been consistent: “deliver a better solution for large databases.”
This past month we released Windocks 2.0; the first container engine with integrated database cloning (storage virtualization) support. The design is based on Windows Virtual Hard Disks (VHD), which are “cloned” with differencing disks. The workflow begins with a snapshot or backups, which are restored or moved to a VHD. The VHD supports the creation of differencing disks (clones) which can be mounted to a SQL Server container. The VHD image is a full byte copy of the database, but a 1 Terabyte environment can be cloned and delivered to users in just 30 seconds, and only requires an incremental 40 MB of storage.
The creation and management of clonable images and clones is integrated into the Windocks daemon, with the use of new Dockerfile commands, such as “SETUPCLONING FULL” which specifies the creation of a clonable image based on a full backup. Other commands are used to create images with Differential backups or raw files. In every case, the image can incorporate scripts to perform data masking or other data preparation.
Creating the VHD image takes time for large data sets, as the image is a full byte copy. Once created, however, clones of the image are created in seconds, and only require 40 MB of storage. Clones are writable, as differencing disks capture writes. Images can be updated as needed with differential backups, enabling a team to work for a week or more at a time with a single full backup.
Windocks 2.0 data virtualization supports any infrastructure, including AWS, Azure, VMware, or bare metal. Developers and DBAs no longer need to worry about volumes, mount points, and clean up. Windocks creates a one-to-one relationship between a VHD and an associated image, and a one-to-one relationship between each clone and container. Maintenance is simplified, with the removal of a container the associated clones and mount points are removed. The removal of an image cleans up the associated VHD.
Clones for Containers and Other SQL Server Instances
In an ironic twist, we’re now expanding use of the database clones to support other (non-container) SQL Server instances. Each clone is created with a unique path and mount point, and the Windocks 2.0 web UI will make it simple to use the path and mount point in a SQL script to mount the clones to a SQL Server instance.
Have It Your Way: SQL Server Environments for Everyone
Windocks is now the first container engine with integrated database cloning. Windocks is also a comprehensive solution for delivery of database environments for any SQL Server developer or tester, by providing clones anyone can use.
The free downloadable Windocks Community Edition is available here.
Opinions expressed by DZone contributors are their own.