AWS/Terraform Workshop #6: EC2 Container Service and AWS Lambda

DZone 's Guide to

AWS/Terraform Workshop #6: EC2 Container Service and AWS Lambda

In this workshop, get an up close look at EC2, how to use AWS Lambda to help configure setups, and how Terraform helps provision your infrastructure along the way.

· Cloud Zone ·
Free Resource

This post is part of our AWS/Terraform Workshops series that explores our vision for Service Oriented Architecture (SOA), and closely examines AWS Simple Storage Service, Terraform Remote State, and Identity Access Management.


AWS EC2 Container Service (Amazon ECS)

Amazon ECS is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. It lets you launch and stop container-based applications with simple API calls, allows you to get the state of your cluster from a centralized service, and gives you access to many familiar Amazon EC2 features.

  • Cluster: A logical grouping of container instances on which you can place tasks.
  • Container instance: EC2 instance that is running the Amazon ECS agent and has been registered into a cluster.
  • Task definition: A description of an application that contains one or more container definitions.
  • Task: An instantiation of a task definition that is running on a container instance.
  • Service: ECS service allows you to run a specified number of tasks simultaneously. If any of the tasks fails or stops for any reason (e.g. OOM), ECS service scheduler re-launches the failed task.

It’s possible to attach Elastic Load Balancer (ELB) to ECS service to distribute traffic across the multiple instances of the service.

You can update a running service to change the number of tasks (desired count) and/or definition for the tasks in ECS service.

Read more:

AWS Lambda for Synchronization of Auto Scale Group and ECS

In this workshop, we will set up an AWS Lambda function that will sync a desired count of tasks in ECS depending on the number of EC2 instances in the Auto Scaling Group. This is now natively supported by AWS, therefore, mentioning the Lambda function may be used for learning purposes.

Note: We recommend using AWS ECS service Auto Scaling in production environments instead of the Lambda function presented in this workshop.

Hands On

Note: There are no intentional mistakes or confusions at this stage, but there's information you'll need to change in the following sections.

  • Go to the w6/workshop_part_1 directory in the cloned Smartling/aws-terraform-workshops git repository.
  • Configure Terraform to create an IAM role, security group, and EC2 instance.
  • Make sure the user data for EC2 instance contains your SSH key.
  • Apply terraform configuration:
$ terraform plan

$ terraform apply

  • Log into the newly created EC2 instance via SSH.
  • Check out the git repository with the Terraform workshops.
$ git clone https://github.com/Smartling/aws-terraform-workshops

$ cd w6/workshop_part_2

Create an ASG and ECS Cluster With the EC2 Instance Registered in It

The ECS cluster is a logical grouping of container instances on which you can place tasks.

Container instance: The EC2 instance that is running the Amazon ECS agent and has been registered into a cluster. Instances of different types can be included in a cluster. A container instance can be a member of one cluster at a time. The type of container instance determines the resources available within the cluster.

The ECS agent makes API calls to the ECS service on your behalf. Therefore, container instances must be created in an instance profile that opens the required permissions.

Note: Credentials for Terraform are not required for these steps as “terraform apply” will be executed on the EC2 instance, which has full access to AWS.

  • Go to the w6/workshop_part_2 directory at the newly created EC2 instance and configure Terraform to create the ECS cluster. You may use mcedit, vim, or nano command line text editors.
  • Apply the Terraform configuration:
$ terraform plan

$ terraform apply

Note: At this step, you should run Terraform in the w6/workshop_part_2 directory at the newly created EC2 instance.

  • Go to the AWS ECS console to see the newly created ECS cluster.
  • Configure Terraform to create the Auto Scaling Group and instances registered in this ECS cluster.
  • Check the AWS ECS web console to see the new container instance.

Create the ECS Service With ELB Attached

The ECS service allows you to simultaneously run and maintain a specified number of instances of a task definition simultaneously. If any of the tasks fails or stops for any reason (e.g. crash, OOM), the ECS service scheduler launches another instance of task definition to replace it and maintain the desired count of tasks in the service.  The number of tasks running in the ECS service is specified in its parameter desired_count.

If attached to the ECS service Elastic Load Balancer (ELB), it distributes traffic across the tasks that are associated with said ECS service. ECS and ELB take care of adding and removing container instances into/from balancing, without downtime for the application running in containers. ECS can have one ELB attached.

In order to attach the ELB to ECS, you must specify the IAM role that will allow the ECS service to communicate with ELB on your behalf.

After the ECS service is created, the container port, container name, and ELB specified in the ECS service configuration cannot be changed. In order to change them, you must re-create ECS service.

    “family”: “myservice”,
            “name”: “myservice - container - one”,
            “image”: “myservice - image: latest”,
            “cpu”: 100,
            “memory”: 512,
                    “containerPort”: 8080,
                    “hostPort”: 80,
                    “protocol”: “tcp”
                    “name”: “variable1”,
                    “value”: “value1”

ignore_changes is meta-parameter in Terraform that customizes how diffs are evaluated for resources, allowing individual attributes to be ignored through changes. For example, terraform will not apply desired_count specified in ECS service configuration if ECS service already exists.

  • Change the Docker image name from anosulchik/workshop-sample-application:v1.0 to anosulchik/workshop-sample-application:v2.0 in containers.txt, which defines containers in ECS task.
  • Apply Terraform configuration.
$ terraform plan

$ terraform apply

  • Use the ELB DNS name to open a sample service in your browser: You should see the new version v2.0 of the sample application here.

Create a Lambda Function to Sync Capacity With Desired Count

  • Configure Terraform to bind the SNS topic used by ASG to send notifications to trigger the Lambda function.
  • Apply the Terraform configuration.
$ terraform plan

$ terraform apply

  • Uncomment the resource in lambda.tf and apply the Terraform configuration.
  • Change min_size = max_size = 2 in the ASG configuration.
  • Check the Lambda function’s logs and the ECS service's desired count value in the web console (you should see desired_count = 2 here applied by the Lambda).
  • Change min_size = max_size = 1 in the ASG configuration.

Destroy the AWS Resources Created in This Workshop

Note: Order is extremely important here — run Terraform destroy at EC2 instance first, then locally.

  • Run the “terraform destroy” command on the EC2 instance where you logged in via SSH, in the w6/workshop_part_1 directory.
  • Run “terraform destroy” on your computer in the w6/workshop_part_2 directory.
amazon ecs ,aws lambda ,cloud ,task management ,terraform ,tutorial

Published at DZone with permission of Artem Nosulchik . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}