DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Building Scalable Real-Time Apps with AstraDB and Vaadin
Register Now

Trending

  • How To Manage Vulnerabilities in Modern Cloud-Native Applications
  • What Is Envoy Proxy?
  • What to Pay Attention to as Automation Upends the Developer Experience
  • Front-End: Cache Strategies You Should Know

Trending

  • How To Manage Vulnerabilities in Modern Cloud-Native Applications
  • What Is Envoy Proxy?
  • What to Pay Attention to as Automation Upends the Developer Experience
  • Front-End: Cache Strategies You Should Know
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Deployment
  4. Git-like Functionality for Docker Data Volumes

Git-like Functionality for Docker Data Volumes

FlockerHub and Fli increase effectiveness of testing by providing DevOps teams with data management capabilities across entire application lifecycle.

Tom Smith user avatar by
Tom Smith
CORE ·
Nov. 03, 16 · News
Like (3)
Save
Tweet
Share
4.87K Views

Join the DZone community and get the full member experience.

Join For Free

ClusterHQ™, today announced that on November 8, two new container data management products–FlockerHub™ and Fli™–will be available through its open beta program.

FlockerHub is like GitHub® for data. With FlockerHub, teams can store and share any Docker data volume with access-controlled users or servers.

 Fli, a Git-like command line interface that runs on any Linux server or laptop, lets developers snapshot, clone, push and pull data volumes to FlockerHub. With FlockerHub and Fli, DevOps teams can seamlessly move data between laptops, test environments, data centers and clouds, all with version and access controls. This flexibility to have data where you need it, when you need it, no matter the underlying infrastructure, has an immediate and positive impact on DevOps teams doing continuous integration, continuous delivery and automated testing. In addition, FlockerHub and Fli in conjunction with Flocker have the potential to reinvent how disaster recovery, backup and cross-cloud workload migration are done in the enterprise.

“As a leading provider of a Continuous Security-as-a-Service Platform for application delivery, Cybric deeply understands that data management needs start in development and continue all the way into production,” said Mike D. Kail, Chief Innovation Officer at Cybric and former CIO at Yahoo. “The platform that ClusterHQ is building to enable entire DevOps teams to effectively manage and distribute their volumes across the entire application lifecycle will do for data management what Git did for code.”

“We recently surveyed more than 300 IT professionals and discovered that more than 40 percent of app developers spend unconscionable amounts of time debugging errors found in production. This is time wasted that should be redirected to developing useful new innovations,” said Mark Davis, CEO, ClusterHQ. “Not only will FlockerHub and Fli reduce the number of bugs that make it into production by enabling better testing, but they also have the potential to dramatically improve how databases are managed across the entire application lifecycle, just as GitHub and Git have made an indelible impact on how software development teams deal with code.”

FlockerHub: GitHub for data

FlockerHub, available initially as a ClusterHQ hosted service, is a version-control repository for Docker data volumes, similar to how GitHub is a version-control repository for software source code. FlockerHub allows DevOps teams to keep track of and distribute their data volumes to any host, all with access controls that fulfill data governance responsibilities.

Fli: Git for data

Fli—pronounced “fly”, as in making your application data fly—is the command line interface (CLI) to FlockerHub. Fli allows DevOps teams to version-control Docker data volumes in the same way they use Git to version-control code. Fli can take incremental snapshots of any database or data volume that runs on Linux, and push those snapshots to FlockerHub where they can be pulled by any access-controlled user. Because these snapshots are incremental, each additional push or pull to FlockerHub sends only part of the data, which speeds up transfer times and reduces bandwidth consumption. Additionally, just like you can use Git to organize code into repos and branches, Fli provides the ability to organize data into volume sets and branches, making it simple to operate large-scale test fixture libraries.

Data, where it needs to be, when it needs to be there

In today’s world of agile software development, it’s essentially impossible to develop microservices-based applications without automated testing and continuous integration. Yet, even with an automated application development pipeline paired with technologies like Docker, there are still unsolved challenges for effective data management across complete application lifecycles.

FlockerHub and Fli improve processes for managing and distributing data across containerized environments. Some specific use cases include:

  • Realistic test data for CI environments:
    • The best tests include realistic test data. But getting data into continuous integration environments can be cumbersome since it usually involves asking operations teams for access. Not only does this manual request take time, slowing down testing, but itis also expensive because multiple copies of the data are needed for parallel testing. With Fli, Ops teams can automatically snapshot any data volume and send it to FlockerHub, where it can be shared with only access-controlled users or servers. Additionally,
    • Fli can clone the data multiple times without increasing storage consumption, so multiple tests can run in parallel without additional cost. When combined with popular CI software like Jenkins, the data management capabilities provided by FlockerHub and Fli can accelerate incremental and parallel builds, speeding up testing dramatically.
  • On-demand staging for each commit:
    • As developers work on feature branches, they often preview their code in a staging environment. Docker Compose is an easy way to quickly spin up test environments for each commit, but without realistic test data, bugs can be easily missed, increasing the likelihood of errors in production. Through an integration with Docker Compose, FlockerHub volumes can be referenced directly in the Docker Compose YAML file, allowing the Dockerized application to be spun up along with its data, providing a truly representative staging environment.
  • Commit and share error state for simplified debugging:
    • The recent Application Testing survey conducted by ClusterHQ uncovered that 11 percent of application developers report encountering bugs discovered in production every single day, while 25 percent encounter new bugs multiple times per week. Typically when a bug is reported, a developer wastes several hours trying to reproduce the problem before it can be fixed. With Fli, database state can be regularly snapshotted and pushed to FlockerHub, so that any time an application needs to be debugged, the accompanying data can be included in the debugging environment. Any developer who needs to work on an issue can pull the error state, along with the correct code version and Dockerized application environment, and have everything needed to isolate the bug.  

“Data lives at the heart of every application. With the additions of FlockerHub and Fli to the ClusterHQ platform, we provide a holistic solution for managing this precious resource at every stage of application development,” said Mohit Bhatnagar, VP of products, ClusterHQ. “Today, FlockerHub and Fli dramatically improve DevOps teams’ ability to test their code before moving into production. Tomorrow, these capabilities will change the way enterprises do disaster recovery, backup and multi-cloud operations.”

The ClusterHQ platform, which now consists of FlockerHub, Fli and the Flocker volume orchestrator, brings all the benefits of containers–namely portability, flexibility and agility–to an organization’s precious data resources.  

Test data Docker (software) application Continuous Integration/Deployment agile Software development Data management Version control Git Command-line interface

Opinions expressed by DZone contributors are their own.

Trending

  • How To Manage Vulnerabilities in Modern Cloud-Native Applications
  • What Is Envoy Proxy?
  • What to Pay Attention to as Automation Upends the Developer Experience
  • Front-End: Cache Strategies You Should Know

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: