Docker, DevOps, and Cloud Storage on AWS

DZone 's Guide to

Docker, DevOps, and Cloud Storage on AWS

The worlds of containers, DevOps, and cloud storage all combine within AWS. Learn how to manage them and what problems to keep in mind along the way.

· Cloud Zone ·
Free Resource

This post will focus on Docker, DevOps, cloud storage and how they apply to Amazon Web Services (AWS). I'll also talk a bit about the common Docker storage challenges that people see on a daily basis.

Surveys show that about 80% of organizations are adopting DevOps in some way. If you haven't adopted DevOps yet, you're probably in the process of researching DevOps and how it can impact you, either at an organizational level or at a business unit level, etc. And this will only become more and more popular as time progresses because it's the natural order of things. Companies want to be as efficient as possible. They want the life cycle of applications to be controlled and have every component of IT working together as efficiently as possible.

Container Storage Options

Image title

As the figure above shows, there are a couple of options for container storage. You can currently spin up a Docker image and you can throw your application in there. The great thing about Docker storage is that it's simple and shared by containers on the same host. But, it's only temporary storage.

You can also leverage network storage. This is where we start getting into creating mappings, exports, NFS exports, and iSCSI block LUNs that we can attach to machines. Basically, you're just offloading storage requirements somewhere else. There are definitely benefits as we start getting into some manageability and scale aspects that make network storage something that's much friendlier toward the enterprise. It becomes much more tenable as far as management and DevOps are concerned. 

Well, now we can bring cloud storage into the mix. There's a proliferation of different block devices depending upon our cloud provider — AWS, Microsoft Azure, etc. As it relates to object storage, we've got several different flavors of object storage that we can now bring to bear and start leveraging within DevOps, within our Docker containers, again based upon your cloud provider. These are good because, obviously, we're going to start leveraging the infrastructure of someone else in the data center that's in the business of making things highly available for us. That can only be a good thing for us as developers, as people who want to get applications out to our community or consumers as fast as possible.

Image title

Why Should DevOps Care About Cloud Storage?

There are a lot of storage options we can integrate into our DevOps platform that we can leverage with Docker. But why should you care? It's important when you start developing, creating, and deploying containers that you have the ability to configure these environments one time. You want to run anything you want specifically wherever you want and maintain the consistency and the efficiency and the repeatability that DevOps bring to the table. And most importantly be able to maintain the persistent data storage. 

You want to be able to control security and Docker brings some challenges that we need to take into consideration with regards to locking the environment down, locking the underlying storage down so someone doesn't get outside of that container and start taking control of objects they're not necessarily supposed to. So we need to start taking all of those existing considerations and now applying them to our different file storage options and seeing how we can make life better for everyone involved.

Docker Storage Challenges

Image title

Now we're going to talk about some of the Docker storage challenges in the public cloud. Specifically around persistent storage, container security, support, and the lack of expertise/resources available.

Image title

As it relates to persistent storage challenges, there are some challenges when maintaining a dedicated data layer within VMs. The challenges start making themselves apparent within the networking layer, within the security that's applied to that, etc. Our goal here is to allow you to create this data layer very efficiently, without having much experience in abstracting from most of the manual process that’s normally associated with this. Most importantly, we can share snapshots of our data to S3 from EBS.

Image title

We also have those container security challenges that are kind of ever persistent, so if we ever have someone take control of the container, they can essentially use those privileges to go outside the container and take control of the underlying machine. We still have those challenges. SoftNAS isn't going to remove that problem, but what we're going to do is we’re going to give you the ability to manage, at least from that data layer, those exports in a much more friendly fashion. Within the exports tab within SoftNAS Cloud NAS, some of the things I couldn't go over because of time was how we can manage the IDs associated with those users or devices that are connecting to these containers or these NFS exports. 

Now, one thing I recommend as you're going along and you're starting to leverage Docker or DevOps more heavily is that you evaluate your support options in SLAs. Because even cloud providers are susceptible to outages.

One of the things you can do is use a product that provides a guaranteed 99.999% uptime. One good way to augment processes or policies you have in place with regards to supporting SLA is to give evaluate support offerings and look for a 99.999% uptime guarantee.

As far as resources and expertise challenges are concerned, storage is definitely a science that a lot of people aren’t really interested in. They definitely know they need it, but there aren't a lot of people who can go down to the bits and bytes level of it and then can manage it at every different facet of a solution. Going from provisioning that first bucket within Amazon S3 to lighting up that highly available solution can be quite daunting when done or configured by hand. Look into using a product that has a GUI to make your life much easier. What we're doing is we're assuming that even if you have those abilities, your time's probably better spent elsewhere, so we'd like to make something much more temporal, much more easy to use just in case you don't have those types of resources.

We hope you found this post helpful as you move to the cloud. Leave a comment and let us know what we should write about next!

This post was cross-posted on SoftNAS.

cloud storage, cloud storage infrastructure, container deployment, container environments, container management, devops, devops adoption, docker, persistent data

Published at DZone with permission of Taran Soodan , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}