Clusters of Docker Containers in AWS With GitLab
Clusters of Docker Containers in AWS With GitLab
If you're just starting off with Docker deployment in AWS, read on to learn how to use GitLab to deal with clusters of Docker containers in AWS.
Join the DZone community and get the full member experience.Join For Free
See why enterprise app developers love Cloud Foundry. Download the 2018 User Survey for a snapshot of Cloud Foundry users’ deployments and productivity.
This is meant to be a quick how-to reference for people who are new to Docker deployment in AWS.
Before starting, have a certificate that allows admin access to AWS. This is the
*.pem that the web browser offers to download when a key-pair in EC2 is created.
Familiarity with the concepts of AWS/ECS, Docker, and Pipelines is assumed.
It's also assumed that the app is already in a Docker image.
Without further ado, let’s dive right in.
AWS: Create a Load Balancer
The load balancer will provide a single URL to reach the cluster where containers can be added or removed. This is a manual step that is only necessary to be done once.
Navigate to Services > EC2 > Target Groups
Click on Create Target Group
All containers deployed under this Load Balancer will be associated directly with this Target Group.
Fill in this form below. The important thing to note here is the health check settings section: the health check endpoint must actually exist, otherwise, the container will be considered unhealthy and ECS will keep killing and creating it forever.
Create the actual Load Balancer
Navigate to Services > EC2 > Load Balancers.
Fill in the name and port where the Load Balancer will listen to and...
...select the availability zones.
On the second step of the wizard, there will be a warning about the lack of HTTPS. Just skip it.
Select the default security group.
In Target Group, select Existing Target Group and in Name, type MyTargetGroup, which was created above.
Go to the end of the wizard and confirm/save the new Load Balancer.
The address to access the application is under DNS name.
AWS: ECS Initial State
Navigate to Services > EC2 Container Service.
The page should look either this…
At this point, it is expected to have no containers running.
GitLab: Configure the Pipelines
docker-compose.yml referred to in the
version: '2' services: api: image: "registry.gitlab.com/my_username/my_project_name/api:latest" ports: - "80:8080"
Create the .gitlab-ci.yml File
The environment variables seen below can be configured in Settings > CI/CD Pipelines.
image: docker:latest services: - docker:dind stages: - build - dockerise - deploy api-build: stage: build image: maven:3-jdk-8-alpine script: - cd api - mvn clean verify ##### 'artifacts' is the way artifacts can be passed around to the next pipelines ##### artifacts: paths: - api/target/api.jar ##### This simply builds and pushes the Docker image ##### api-containerisation: stage: dockerise script: - docker version - docker build -t registry.gitlab.com/my_username/my_project_name/api:$CI_JOB_ID api/ - docker build -t registry.gitlab.com/my_username/my_project_name/api:latest api/ ##### $DOCKER_LOGIN_KEY can be obtained in GitLab as a Personal Access Token ##### - docker login -u my_username -p $DOCKER_LOGIN_KEY registry.gitlab.com - docker images - docker push registry.gitlab.com/my_username/my_project_name/api:$CI_JOB_ID - docker push registry.gitlab.com/my_username/my_project_name/api:latest CI_deploy: stage: deploy image: python:3-alpine variables: AWS_DEFAULT_REGION: "us-east-2" before_script: ##### Install the AWS ECS-CLI ##### - apk add --update curl - curl -o /usr/local/bin/ecs-cli https://s3.amazonaws.com/amazon-ecs-cli/ecs-cli-linux-amd64-latest - chmod +x /usr/local/bin/ecs-cli script: ##### Configure ECS-CLI, run the container and scale to 2 ##### - ecs-cli configure --region $AWS_DEFAULT_REGION --access-key $AWS_ACCESS_KEY_ID --secret-key $AWS_SECRET_ACCESS_KEY --cluster CI-MyCluster-API - ecs-cli up --keypair $AWS_KEY_PAIR --capability-iam --size 2 --instance-type t2.micro --vpc vpc-xxxxxxx --subnets subnet-123abc,subnet-321cba ##### This docker-compose.yml is the one described above ##### - ecs-cli compose --file api/docker-compose.yml service up --target-group-arn $PROD_TARGET_GROUP_ARN --container-name api --container-port 8080 --role ecsServiceRole - ecs-cli compose --file api/docker-compose.yml service scale 2 environment: name: ci ##### This is the URL seen under 'DNS name' when the LB was created ##### url: $PROD_LOAD_BALANCER_URL only: - master
When the file above is committed to master in GitLab, the pipelines should be automatically triggered:
…and by clicking on the pipeline, the console should show the logs. The example below is the CI_deploy logs.
Notice the last message from the CLI: "ECS Service has reached a stable state." In the
.gitlab-ci.yml, the requested count was 2.
AWS: ECS Final State
The dashboard will now show some stats about the new cluster.
Testing the Cluster
Notice the request is distributed evenly across the available containers in the cluster (in this case 2) by hitting the Load Balancer URL several times.
Opinions expressed by DZone contributors are their own.