Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Clusters of Docker Containers in AWS With GitLab

DZone's Guide to

Clusters of Docker Containers in AWS With GitLab

If you're just starting off with Docker deployment in AWS, read on to learn how to use GitLab to deal with clusters of Docker containers in AWS.

· Cloud Zone
Free Resource

Are you joining the containers revolution? Start leveraging container management using Platform9's ultimate guide to Kubernetes deployment.

This is meant to be a quick how-to reference for people who are new to Docker deployment in AWS.

Before starting, have a certificate that allows admin access to AWS. This is the *.pem that the web browser offers to download when a key-pair in EC2 is created.

Familiarity with the concepts of AWS/ECS, Docker, and Pipelines is assumed.

It's also assumed that the app is already in a Docker image.

Without further ado, let’s dive right in.

AWS: Create a Load Balancer

The load balancer will provide a single URL to reach the cluster where containers can be added or removed. This is a manual step that is only necessary to be done once.

Navigate to Services > EC2 > Target Groups

Image title

Image title

Click on Create Target Group

All containers deployed under this Load Balancer will be associated directly with this Target Group.

Fill in this form below. The important thing to note here is the health check settings section: the health check endpoint must actually exist, otherwise, the container will be considered unhealthy and ECS will keep killing and creating it forever.

Image title

Create the actual Load Balancer

Navigate to Services > EC2 > Load Balancers.

Image title

Fill in the name and port where the Load Balancer will listen to and...

Image title

...select the availability zones.

Image title

On the second step of the wizard, there will be a warning about the lack of HTTPS. Just skip it.

Select the default security group.

Image title

In Target Group, select Existing Target Group and in Name, type MyTargetGroup, which was created above.

Image title

Go to the end of the wizard and confirm/save the new Load Balancer.

The address to access the application is under DNS name.

Image title

AWS: ECS Initial State

Navigate to Services > EC2 Container Service.

Image title

The page should look either this…

Image title

…or this:

Image titleAt this point, it is expected to have no containers running.

GitLab: Configure the Pipelines

The docker-compose.yml referred to in the .gitlab-ci.yml below:

version: '2'

services:
  api:
    image: "registry.gitlab.com/my_username/my_project_name/api:latest"
    ports:
      - "80:8080"

Create the .gitlab-ci.yml File

The environment variables seen below can be configured in Settings > CI/CD Pipelines.

image: docker:latest
services:
  - docker:dind

stages:
  - build
  - dockerise
  - deploy

api-build:
  stage: build
  image: maven:3-jdk-8-alpine
  script:
    - cd api
    - mvn clean verify
##### 'artifacts' is the way artifacts can be passed around to the next pipelines #####
  artifacts:
    paths:
      - api/target/api.jar

##### This simply builds and pushes the Docker image #####
api-containerisation:
  stage: dockerise
  script:
    - docker version
    - docker build -t registry.gitlab.com/my_username/my_project_name/api:$CI_JOB_ID api/
    - docker build -t registry.gitlab.com/my_username/my_project_name/api:latest api/
##### $DOCKER_LOGIN_KEY can be obtained in GitLab as a Personal Access Token #####    
    - docker login -u my_username -p $DOCKER_LOGIN_KEY registry.gitlab.com
    - docker images
    - docker push registry.gitlab.com/my_username/my_project_name/api:$CI_JOB_ID
    - docker push registry.gitlab.com/my_username/my_project_name/api:latest

CI_deploy:
  stage: deploy
  image: python:3-alpine
  variables:
    AWS_DEFAULT_REGION: "us-east-2"
  before_script:
##### Install the AWS ECS-CLI #####
    - apk add --update curl
    - curl -o /usr/local/bin/ecs-cli https://s3.amazonaws.com/amazon-ecs-cli/ecs-cli-linux-amd64-latest
    - chmod +x /usr/local/bin/ecs-cli
  script:
##### Configure ECS-CLI, run the container and scale to 2 #####
    - ecs-cli configure --region $AWS_DEFAULT_REGION --access-key $AWS_ACCESS_KEY_ID --secret-key $AWS_SECRET_ACCESS_KEY --cluster CI-MyCluster-API
    - ecs-cli up --keypair $AWS_KEY_PAIR --capability-iam --size 2 --instance-type t2.micro --vpc vpc-xxxxxxx --subnets subnet-123abc,subnet-321cba
##### This docker-compose.yml is the one described above #####
    - ecs-cli compose --file api/docker-compose.yml service up --target-group-arn $PROD_TARGET_GROUP_ARN --container-name api --container-port 8080 --role ecsServiceRole
    - ecs-cli compose --file api/docker-compose.yml service scale 2
  environment:
    name: ci
##### This is the URL seen under 'DNS name' when the LB was created #####
    url: $PROD_LOAD_BALANCER_URL
  only:
    - master

When the file above is committed to master in GitLab, the pipelines should be automatically triggered:
Image title…and by clicking on the pipeline, the console should show the logs. The example below is the CI_deploy logs.

Image title

Notice the last message from the CLI: "ECS Service has reached a stable state." In the .gitlab-ci.yml, the requested count was 2.

AWS: ECS Final State

The dashboard will now show some stats about the new cluster.

Image title

Testing the Cluster

Notice the request is distributed evenly across the available containers in the cluster (in this case 2) by hitting the Load Balancer URL several times.

Image title

Image title

Using Containers? Read our Kubernetes Comparison eBook to learn the positives and negatives of Kubernetes, Mesos, Docker Swarm and EC2 Container Services.

Topics:
cloud ,tutorial ,aws ,docker ,gitlab ,clusters

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}