Automating Your Enterprise Infrastructure: Part 2: Cloud Infrastructure as Code in Practice (AWS Cloud Formation Example)
In this second installment of the infrastructure automation guide, you will get hands-on experience with building infrastructure as code scripts.
Join the DZone community and get the full member experience.Join For Free
Given that you went through Part 1 of the Infrastructure automation guide, and you already know basic Infrastructure as Code and AWS Cloud Formation concepts, we can proceed with getting some hands-on experience!
Note that in this article, we’ll build Infrastructure as code scripts for the infrastructure described by Michal Kapiczynski in the series of mini-articles.
HINT before we begin:
If you’re building your Cloud Formation scripts from scratch, we highly recommend starting with spinning the infrastructure manually from the AWS console, and later on, using the AWS CLI tool to get a ‘description’ of the resource. The output will show you the parameters and their values that were used to create the resource.
aws ec2 describe-instancesto obtain properties for EC2 instances.
Let’s recall what our target state is below:
As already mentioned in the first part of the automation guide, we’ve split the infrastructure setup into two Templates (scripts). Let’s start with the first one, which is called infra-stack, as it contains the following architecture scaffolding resources:
- Internet gateway
- Elastic IP
- Route Tables
Note: All of the Cloud Formation scripts are presented below, and even more are publicly accessible in this GitHub repository.
The backbone: Virtual private cloud. In fact, is a network that hosts all of our resources. The Cloud Formation definition for this one is a simple one. See below:
Just a few lines of code. The first line defines the Amazon resource name; we’ll use this name later on to reference the VPC. Type specifies whether this is VPC, Subnet, EC2 VM, etc. The Properties section contains a set of configuration key-value pairs fixed for a particular resource. The only required property that we define here is CidrBlock of our VPC. Note that the network mask is (256.256.252.0). Additionally, we can specify a Name Tag that might help us to quickly find our VPC amid the VPC list in the AWS console.
As stated above, we’ll need 4 subnets. Specifically, one public and one private network subnet in Availability Zone A. The same goes for AZ B. Let’s see public subnet A definition below:
When specifying AvailabilityZone, we can use the !Sub function to substitute the Region script parameter variable name with the actual value and, at the same time, concatenate it with the ‘a’ suffix. This is to have an actual AWS Region name. So, e.g. taking the Region default value, the actual value for AvailabilityZone shown in Fig … is “eu-central-1a“.
Next, we have to specify the CidrBock of the subnet. This one is easy, although, note that the subnet cidr should be ‘within’ the VPC cidr block.
Last but not least, is VpcId. At the time we write the script, we don’t know the actual VPC identifier, which is why we have to reference (!Ref) VPC by its name (UserManagementVpc).
Both of the functions – !Sub and !Ref are so-called intrinsic function references built into the cloud formation service. More on that can be found here.
We won’t go through the rest of the Subnet definitions because they are basically the same; the only thing that changes is AvailabilityZone suffix and CirdBlock. You can find these definitions in the Github repository.
This one seems to be a simple one:
The only required field is Type. Not so fast though. As we already know, IGW should be attached to a specific VPC, but there is no VPC reference here! Here comes the other resource called VpcGatewayAttachment:\
As we can clearly see, this one is responsible for the association between IGW and VPC. Same as in Subnet definition, we can reference these by name using !Ref.
Now, let’s take care of the prerequisites for NAT setup. We should set up Elastic IP that NAT can reference later on. We need two of these for each AZ:
Note that the ‘a’ suffix indicates target AZ for the EIP.
NAT (Network Address Translation) Gateway
Since we have prerequisites provisioned, we can now set up two NAT Gateway instances in our public subnets like so:
As you, the careful reader, noted, to obtain the value for AllocationId, we used yet another intrinsic function reference, Fn::GetAtt. This use facilitates obtaining Elastic IP attribute: AllocationId. Next, we reference the target SubnetId. As always, we have to remember to spin up the twin NAT in b AZ.
Things get a little bit messy here. First, we’ll create our Main Route table that will hold the rules for our public subnets:
This is where our CloudFormation IoC script turns out to be more complicated than a simple setup through Amazon console.
It turns out that rules specification is yet another resource:
The essence of this is the DestinationCidrBlock configuration. As you see, we’ve set it to 0.0.0.0/0, which means that we allow for unrestricted access to all IPv4 addresses. Also, we need to reference our Internet gateway and instruct our Route resource to attach itself to the MainRT.
Unfortunately, Route Table configuration doesn’t end here. Additionally, we have to associate RouteTable with the subnet. As we mentioned, we’ll associate the MainRT with our public subnets. See below:
Remember to do the same for the public subnet b!
For private subnets, the story repeats all over again. We need yet another route table, SubnetRouteTableAssociation, and Route definitions. But, in this case, we will enforce all outgoing traffic to be routed through NAT Gateways.
Note: In production environments, it’s considered good practice to disable internet access in private networks!
Besides actual resources, the script also defines the outputs section. The section defines what stack information may be exposed for other stacks. This mechanism will allow us to, later on, reference VPC and Subnet identifiers in the second stack.
EC2, Database, and Load Balancer Stack
Next in line, vm-and-db-stack, it contains declarative definitions of the following:
- AWS KeyPair – prerequisite
- Server and client VMs
- EC2 Security Groups
- Multi-AZ Database setup
- DB Subnet Group
- DB Security Group
- DB Instance
- Load Balancer – AWS Application Elastic Load balancer
- Target Groups
- ELB Security Group
- ELB Listeners
- Load Balancer itself
The script accepts three parameters (don't worry – the default values are included):
- NetworkStackName – the name of the infrastructure stack that we created in the previous step.
- DBPass – self-explanatory.
AvailabilityZone – target AWS Availability Zone for the stack. Note that the value has to be coherent with the AZ parameter value specified when running the infrastructure stack script.
AWS KeyPair – Prerequisite
Before we proceed with this stack, there is one resource that you, as an account owner, have to provision manually; the AWS KeyPair. Long story short, it’s AWS is equivalent to private and public asymmetric cryptographic keys. We’ll need these to access Virtual Machines running in the cloud!
You can do it either through AWS console or use the aws cli tool:
$ aws ec2 create-key-pair --key-name=YourKeyPairName \
--query ‘KeyMaterial’ --output text > MySecretKey.pem
Remember the key name, since we’ll reference it later.
Eventually, we need some VM to run our application! Let’s look at an example configuration for our EC2 running in a private subnet in AZ a:
This one is a little bit longer. First, as mentioned, we reference our KeyPair name (KeyName parameter) that we’ve created as a prerequisite.
Then, there comes the persistence storage configuration – BlockDeviceMappings. We state that we’re going to need 8 GB of storage, attached to the /dev/sda1 partition.
Next, we choose the operating system – ImageId. I’ve used Amazon Linux OS, but you can use whatever AMI you need.
In the networking section (NetworkInterfaces), we’ll link our EC2 instance with the subnet. The SubnetId sub-section uses another intrinsic function – Fn::ImportValue. We use it to capture the output exported by the infrastructure stack (Outputs section). By combining it with Fn::Sub, we can easily reference the private subnet ‘a’.
NetworkInterfaces property also contains a list named GroupSet, although the name might not indicate this so, this is a list containing Security Group references that should be attached to our EC2. We’ll follow up with the Security Group resource in the next section.
Remember to follow this pattern to create client-facing EC2 VMs in public subnets. These are pretty much the same; the only notable difference is security groups. For client-facing machines, we’ll reference ClientSecurityGroup.
The example above shows the Security Group configuration for the backend server. We apply 2 main rules for incoming traffic (SecurityGroupIngress). First of all, we open port 22; this one is to be able to ssh to the machine. Note that the best practice in production environments nowadays would be to use AWS systems manager instead. Another ingress rule allows traffic coming from LoadBalancerSecurityGroup, which we configure in the last section of this guide. The restriction also states that only port 8080 can receive traffic from the LoadBalancer. For client-facing machines, on the other hand, we’ll expose port 5000.
The only rule in the SecurityGroupEgress section states that we allow for any outgoing traffic hitting the internet. Note that this is not recommended for production configuration!
Multi-AZ Database Setup
Database Security Group
Same as for EC2 machines, databases need to be secured. For this reason, we’ll set up a Security Group for our MySQL AWS RDS. For instance, see below:
Ingress traffic is only allowed from Server machines, and the only port that we can hit is 3306, which is the default MySQL port. This is the same for the server security group for production deployments; we strongly revive to allow outgoing internet access.
Database Subnet Group
The AWS::RDS::DBSubnetGroup resource simply gathers a set of subnets that the DB is going to reside in. Notably, it is required that these subnets reside in different Availability zones. The motivation behind this resource is to inform the database in which subnet (AZ) can be replicated. So, having this resource in place is a highway to achieving database High Availability!
Data persistence is the cornerstone of our systems. If the data is not there, then there is no point in having the system at all. So, let’s take a minute to look into it below:
First of all, let’s make sure that we have enough storage. Depending on the use, 20GB that we configured in the example above, may or may not be enough, although that’s a good starting point. Actually, we don’t really have to care if this is enough since we also configured the MaxAllocatedStorage property, which enables storage autoscaling for us!
We’ll choose db.t2.micro as DBIstanceClass because this is the only one that is free tier eligible.
Next, we set the database password by referencing our DBPass script parameter. Remember not to hardcode your passwords in the code!
According to the plan, we set the value for the MultiAZ property to true. We can do that thanks to our SubnetGroup!
Elastic Load Balancer
There are two main goals for the Target Group resource. The first one is to group EC2 machines handling the same type of traffic. In our case, we’ll create one Target Group for our server and the other for machines running the client application.
The latter is to achieve reliable and resilient application deployments through the health check definition for our applications. Let’s see how it goes below:
Health check configuration is pretty straightforward. For the sample application used throughout this guide, we need the /users endpoint to return the 200 HTTP code to consider an application as healthy. Underneath, we reference our EC2 instances running in a and b private subnets. Naturally, the target port is 8080.
Load Balancer Security Groups
We went through the Security Group configuration before, so we won’t go into details. The most important thing to remember is that we need to allow the traffic coming to LB only for two ports; The 8080 (server port) and the 5000 (UI application port).
Load Balancer Listeners
This resource is a glue connecting the load balancer with the target groups. We’ll have to create two of these, one for the server Target group and one for the client target group.
The key setting here is TargetGroupArn and Action Type. In our case, we just want to forward the request to the ClientTG target group.
Load Balancer Itself
The last component in this guide will help us with balancing the traffic between our EC2 instances.
We expect it to be an internet-facing load balancer by exposing the IPv4 address. Further, we restrict access to the LB by referencing LoadBalancerSecurityGroup, thus allowing clients to exclusively hit ports 5000 and 8080. Last, we’re required to associate LB with target subnets.
Booting Up the Stacks
Now that we have everything in place, let’s instruct AWS to build our infrastructure! You can do it in a few ways. The fastest one is to use bash scripts we’ve prepared, by issuing:
./create-infra.sh && ./create-vm-and-db.sh in your terminal.
Alternatively, if you want to customize script parameters, you can issue the aws cli command by yourself. This is a good start:
aws cloudformation create-stack \ --template-body=file://./infra-stack.yml \ --stack-name=infrastructure
aws cloudformation create-stack --template-body=file://./vm-and-db-stack.yml --stack-name=vm-and-db
Note that infrastructure stack is a foundation for vm-and-db-stack, therefore you have to run the commands sequentially.
The third way is to just enter Cloud Formation Stacks UI and upload the script from the console by clicking on “Create stack” and then “With new resources (standard)”. The AWS console will guide you through the procedure
After you have successfully issued the cloud formation scripts to the Cloud Formation service, you can see the script progressing in the AWS console like so:
You may find Events and Resource tabs useful while you follow the resource creation procedure.
Once all infrastructure components are up and running, you’ll see your stack status marked as
In case your infrastructure definition contained any errors, you will be able to see them in the Cloud Formation console events tab. The status reason column will contain an error message from Cloud Formation or a specific resource service. For example:
For more information on troubleshooting CloudFormation, visit the AWS documentation page.
Summary: Cloud Infrastructure as Code in Practice
If you’re reading this, congrats! You’ve reached the end of this tutorial. We went through the basics of what Infrastructure as code is, how it works, and when to use it. Furthermore, we got hands-on experience with Cloud Formation.
As a next step, we strongly encourage you to take a deep dive into the AWS Cloud Formation documentation. It will help you adjust the infrastructure to your specific needs and make it even more bulletproof. Eventually, now with all of your infrastructure scripted, you can shout out loud: look ma, no hands!
Supplement: AKA Cloud Formation Tips and Tricks
When you’re done playing around with your CF Stacks, remember to delete them! Otherwise, AWS will charge you!
Cloud Formation does not warn you if your updated stack definition might cause infrastructure downtime (resource replacement needed). However, there are two ways to validate that before you deploy. The first one (manual) is to double-check specific resource documentation, especially if the updated property description contains Update requires: Replacement clause. See this example for CidrBlock VPC property:
The second way is to use the Change Sets mechanism provided by Cloud Formation. This one would automatically validate the template and tell you how these changes might impact your infrastructure. See these docs.
Cloud Formation does not watch over your resources after they’re created. Therefore, if you make any manual modifications to the resource that is maintained by CF Stack, the stack itself won’t be updated. A situation where the actual infrastructure state is different from its definition (CF script) is called configuration drift. CF comes in handy and lets you see the actual drift for the stack in the console. See this.
If you create your own Cloud Formation script and are looking for more examples, the CF registry might come in handy.
Published at DZone with permission of Maciej Jozefczyk. See the original article here.
Opinions expressed by DZone contributors are their own.