This blog will explain how to run a Couchbase Docker container using Amazon EC2 Container Service (Amazon ECS).
Many thanks to @moviolone for helping understand the concepts and getting this setup running.
What is Amazon ECS?
Amazon ECS is a container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. Amazon ECS integrates well with rest of the AWS infrastructure and eliminates the need to operate your own cluster or configuration management systems.
One obvious question to wonder is how is this different from other container orchestration frameworks like Docker Swarm, Kubernetes, or Mesos. The first big difference is that each of these frameworks is open source. Amazon uses a proprietary orchestration framework at this time.
A big advantage of ECS is that just like rest of the AWS infrastructure, this is a managed service. And so you only need to worry about deploying your containers without worrying about the infrastructure.
A better comparison of ECS is with Docker for AWS/Azure (backed by newly introduced Swarm Mode in Docker), Google Container Engine (backed by Kubernetes), DC/OS (backed by Mesos) as they are managed services as well.
An advantage point of ECS is that it seamlessly integrates with AWS infrastructure such as deploying container instances using CloudFormation templates, scaling containers using Autoscaling Group, port mapping using Security Groups, manage incoming container traffic using Elastic Load Balancer, viewing logs using CloudWatch and others.
If you are already bought in the Amazon infrastructure, then ECS sounds like a good fit. Docker for AWS, announced at DockerCon, is also a similar offering in this space.
However, there are a couple of cons that you need to be aware of as well:
- Portability – Application designed Docker Swarm, Kubernetes, and Mesos can run on a variety of platforms, such as Amazon, Azure, GCE, OpenStack, on-prem, VMWare, bare metal data centers, etc. But ECS is tied to Amazon only. Do you consider that as a vendor lock-in?
Amazon may release their orchestration platform or scheduler as a standalone product, but that's not very typical.
- Container format – ECS service is focused on Docker containers only. For all practical purposes, at least today, this may be perfectly fine. I've not heard or seen any deployments of Rkt or any other container formats. However, this may change once OCI-compliant runtimes start showing up in the future.
One last thing before we dig in, the concepts and code, there is no additional charge for Amazon EC2 Container Service. You pay for AWS resources (e.g. EC2 instances or EBS volumes) you create to store and run your application.
Amazon ECS Concepts
Here is an overview of the key concepts in ECS:
- Container Instance: An AMI instance that is primed for running containers. By default, each Amazon instance uses Amazon ECS-Optimized Linux AMI. This is the recommended image to run ECS container service. The key components of this base image are:
- Amazon Linux AMI.
- Amazon ECS Container Agent – It manages containers lifecycle on behalf of ECS and allows them to connect to the cluster.
- Docker Engine (as of this writing, this is version 1.11.1).
- Other images like CoreOS, Suse or Ubuntu can be configured to meet Container Instance AMI specification. This can be done because ECS Agent code is available in open source.
- Task: A task is defined as a JSON file and describes an application that contains one or more container definitions. This usually points to Docker images from a registry, port/volume mapping, etc.
- Service: ECS maintains the "desired state" of your application. This is achieved by creating a service. A service specifies the number of instances of a task definition that needs to run at a given time. If the task in a service becomes unhealthy or stop running, then the service scheduler will bounce the task. It ensures that the desired and actual states are matched. This is what provides resilience in ECS. New tasks within a service are balanced across Availability Zones in your cluster. The service scheduler figures out which container instances can meet the needs of a service and schedules it on a valid container instance in an optimal Availability Zone (one with the fewest number of tasks running).
Getting Started with Amazon EC2 Container Service
Click on the
Get started button to define your application.
Create ECS Task
In ECS, Docker workloads are defined as tasks. A task can contain multiple containers. All containers for a task are co-located on the same machine.
Enter the values as shown:
Few items specified in this step:
- Task definition is the description of an application that contains one or more container definitions.
- Container name is the name that will be given to the container started as part of this task.
- Image allows us to specify one or more images that need to be started as containers as part of this application. The image specified here uses couchbase:latest as the base image and uses Couchbase REST API to configure the server. Dockerfile for this image provides more details about how this image is prepared.
- Maximum memory is the memory that needs to be allocated for the container (equivalent to
-mDocker CLI switch). Couchbase needs 1GB for running in dev and so that is specified here.
- And finally, the port mappings (
-pon Docker CLI). Port 8091 is needed for Couchbase administration.
More details about these are available in Task Definition Parameters.
Create ECS Service
Next step to configure a service.
Give the service a name. The desired state can be specified here. For now, we'll keep it simple and launch a single-node Couchbase container. And since the desired state is run a single container, no ELB is required.
More details about these are available in Service Definition Parameters.
Create ECS Cluster
Tasks run on a container instance, and these instances need to register in a cluster. This allows us to scale the cluster up/down later to accommodate for running more containers.
Next step to configure the cluster.
In this image:
- Take the default cluster name
- A homogenous cluster of container instances is created. m3.medium is a good size to run Couchbase node
- Choose a previously created security key. This will allow you to open an SSH connection to the container instance
- A new IAM role will be created to allow ECS agent to communicate with ECS service
Container instances in a cluster can span multiple availability zones and be balanced with ELB.
Review all the specified options:
Click on the
Launch instance & run service button to start the service.
The following status is shown after the service is created:
The output shows that the cluster, service, and task definitions are created. It takes a few minutes for the instances to be provisioned and initialized and for tasks to run on them.
View ECS Service and Task
Click on the
View Service button to see the newly created service.
A few things in this image:
- The service shows the task definition
couchbase:6. Each service is assigned a task definition, and multiple versions are indicated by the trailing number at the end. In this case, a few versions were created earlier but otherwise the version number starts from
- Desired and Running count is shown as 1.
Minimum healthy percentand
Maximum percentare used if a new version of task definition needs to be deployed. With 100% and 200% corresponding values, a new version of the task will be deployed first and then the older versions will be terminated. We’ll play with these numbers in a subsequent blog.
- Running task is shown toward the bottom of the screen. Click on the UUID to learn more about the running task.
Task definition shows EC2 instance where it is running, current status, port mapping, and several other useful information. The critical piece that we need to look at is the
External Link. This URL is where our Couchbase Web Console will be accessible.
Couchbase Web Console
Clicking on this link will open a new tab with Couchbase Web Console:
Enter the login as
Administrator and password as
password. These are configured in arungupta/couchbase image.
And here you see Couchbase Web Console in full glory!
This blog explained how to run a Couchbase Docker container using Amazon ECS.
Future blogs will show:
- Setting up a Couchbase cluster using ECS.
- Deploying a multi-container application using Docker Compose (v2 is now supported).
- Setting up an ECS cluster using CLI.