DevOps Automation and IaC
Take a look at how you can experience the optimization of automation using infrastructure-as-code with Terraform.
Join the DZone community and get the full member experience.Join For Free
In my previous blog, I talked about DevOps and compared it to the manufacturing of physical goods. I also mentioned the four key tenets of DevOps (CAMS) and stressed the importance of the automation component.
With DevOps, automation manifests itself in two forms, automation of the
- Underlying infrastructure – commonly referred to as IaC (Infrastructure-as-Code).
- Application Development and Delivery – using a Continuous Integration and Continuous Delivery (CI/CD) pipeline.
In this blog, I will discuss IaC and show you how to set up your own AWS infrastructure using the Terraform IaC tool. In a future blog, we will move on to discuss CI/CD and I will show you how to set up a CI/CD pipeline using the same underlying platform we build in this blog.
So let us get started.
Let’s face it – IT is extremely complex. To accomplish simple infrastructure tasks requires the involvement of many departments. These handoffs result in frequent configuration errors, difficult-to-troubleshoot errors, inconsistent configurations, and outdated documentation. In almost all cases, to address IT problems, we are constantly implementing new processes and purchasing or building new tools. The processes create more work and tools will address some issues, but also cause other issues and increase our technical debt.
It is a vicious cycle.
Infrastructure-as-Code (IaC) is the ability to programmatically manage and provision an entire technology stack through software or code, rather than using manual and complex processes and outdated documentation to configure hardware devices and operating system components. IaC can address this problem by introducing a high level of automation. With IaC tools, you can provision infrastructure in the most efficient way possible. Infrastructure changes become repeatable processes with less errors or defects and minimal manual user interventions.
With IaC tools, you can deliver pre-configured servers ready to host your applications and scale computing dynamically. IaC exposes workflows to the entire team creating transparency and allowing quicker iteration, reduced bottlenecks and increased flow. It is possible to easily define and clone configurations and make changes seamlessly and consistently. Documentation is always current because it is in the code and stored in a central repository. Training new staff is also easier.
Terraform is an IaC tool that supports multiple private providers, such as VMware, Nutanix, and Vrealize, and public cloud providers, such as AWS, Azure, GCP, and Rackspace. Both AWS and Microsoft provide IaC tools that are specific to their own platforms: CloudFormation for AWS and Azure Resource Manager. You can use these IaC tools to codify, deploy and maintain the underlying infrastructure supporting your application.
- Setup AWS VPC, Network ACLs, Security Groups
- EC2 instances to host your CI/CD infrastructure (Jenkins, Maven, Selenium, Puppet, etc.)
- Application servers (Web, Application and Database servers)
- Bastion servers
- And any other infrastructure required for your applications
Using Terraform to manage the infrastructure can significantly eliminate handoffs, thereby eliminating bottlenecks and improving flow.
Example Using an IaC Tool — Terraform
So, let us see how IaC works by provisioning a test environment shown below on AWS, which we will use in a future blog to build a CI/CD pipeline using a sample application.
Before we can start setting up our test environment on AWS using Terraform, you will need to set up the following key components.
- AWS Platform – The test CI/CD platform will require an infrastructure, which we will build using the Terraform IaC tool on the AWS cloud platform. You can use an existing account or a new account. Information on how to create an account on AWS free tier platform is available here.
- GIT – With CI/CD, a central code repository is an absolute requirement. Click here to learn how to create a free GIT account. After creating your account, you can download all my code and the sample Hode.js application (available here) and use it create your very own GIT repository, which you can customize further.
- Terraform – While you do not need a Terraform account, you will need to download and install it. Click here for more details.
Note: The AWS test platform and the terraform script is a customized version of Mohamed Labouardy’s excellent tutorial, "Manage AWS VPC as Infrastructure-as-Code with Terraform."
After completing the prerequisites above you can use the instructions below to begin the setup process.
- Setup your AWS credentials (some nice instructions available here).
- Create a new folder for your Terraform scripts and supporting files
- Change to the folder
- Initialize and sync GIT repository
terraform plan to test your script
If all is good, run
terraform apply to kick off the script using the
terraform apply command.
Confirm by typing yes, and sit back and watch your environment come alive. After a few minutes, you should see a message similar to above, indicating that your environment provisioned successfully. If everything goes well, the Terraform infrastructure code scripts will automatically create your AWS environment, Jenkins servers on EC2 instances, and an EC2 instance running Tomcat.
So, let us review what happened.
First, as shown below, we created a new VPC called DevOpsOne-VPC.
- A public subnet for the Web Server
- A private subnet for the Jenkins, database and Docker server.
- Security groups, Routing table, Internet and NAT Gateway for the instances on the public subnet to access the internet, as they will need to access external software repos.
AWS EC2 Instances
Shown below, four EC2 instances created.
The Tomcat web server will be our test app server. If you connect to the webserver, on port 8080, you will see a fresh copy of Tomcat.
- Obtain the external IP address and your favorite terminal program to connect (I really like MobaXterm).
- Configure your terminal session with your private key (.pem file)
- Login with the EC2-User account – see below
4. After connecting to the web server, you can then ssh to the other servers.
To setup the Web, Jenkins and the Docker host, a simple shell script did the trick. See the Terraform snippet below.
Docker Host — To setup the Docker host, which we will use this server in future blog), I used a similar script to install the Docker components and set the necessary permissions.
Destroying Your Test Environment
After you have completed testing or at any point, you can run
terraform destroy to delete your test environments and avoid unnecessary charges from AWS.
We are now ready to set up our CI/CD pipeline to deploy our sample node.js application.
I hope you were able to follow along, and use Terraform IaC tool to provision your test environment on AWS, and see for yourself the value of such a tool. As more companies adopt a DevOps approach for software delivery and to provision the underlying application infrastructure. The infrastructure will become invisible similar to our utilities or manufacturing infrastructure. We will focus more on products, services, minimizing risks, and improving the user experience.
In my next blog, I will show you how to set up our CI/CD pipeline on AWS and set up the trigger between Jenkins and Git so changes will automatically trigger a build.
Until next time…
Opinions expressed by DZone contributors are their own.