Getting Started With IaC

In its strictest sense, Infrastructure as code (IaC) means that you use code to define and manage infrastructure rather than using manual processes. More broadly, and perhaps more importantly, IaC is about bringing software engineering principles and approaches to cloud infrastructure.  

IaC is the latest step in the evolving process of defining and managing infrastructure. Before IaC, infrastructure was (and, in some cases, still is) provisioned by methods such as pointing and clicking in a user interface, batch scripts, and configuration management tools that weren’t designed for the modern cloud.   

It’s also true that much of what’s called IaC today is closer to “infrastructure as text.” As infrastructure that is written as structured text, it’s repeatable and can be versioned, but it does not support the software engineering practices that are used with application code. For example, there’s no support for standard development tools, testing frameworks, or package management.  

Truly modern approaches to IaC use platforms that are designed for infrastructure in the cloud. These platforms allow you to apply standard software engineering practices and tools to your cloud infrastructure. 

IaC matters for three reasons. One is the transition to the cloud. More and more workloads are being moved from on-premises data centers to cloud environments. Nothing suggests that this trend is going to stop. However, cloud computing alone isn’t a panacea for maintaining scalable and reliable infrastructure. It’s just as possible to have an inconsistent, poorly documented set of scripts for cloud infrastructure as it is for a physical datacenter. IaC, because it enforces proven engineering practices, is how you make order out of the chaos. 

The second reason is a greater sophistication in how people use the cloud. Companies are changing architectures, patterns, and ways of working to optimize the benefits they can get. It's no longer simply CapEx versus OpEx. It's about how to incorporate all the practices that make up the engineering lifecycle, such as versioning and testing to unlock all the value that the cloud can provide. It’s about using engineering practices to take advantage of the cloud’s potential and innovate faster to drive your business. 

The third reason is that the burden of managing infrastructure in the cloud is increasing. The number of cloud services available is growing every year and more companies are adopting modern cloud architectures (like containers or serverless), which often have many loosely-coupled and interdependent components. The result is that the number of cloud resources that people must manage is going up at a tremendous pace. This is certainly a good thing, because it means companies are getting more value from the cloud to drive their business forward, but the consequence is an increase in complexity and scale.  

For example, one way to get more value from the cloud is to take advantage of the ever-growing number of services that cloud vendors are providing. Those services can speed innovation and accelerate velocity but remember that with every new service comes new APIs. Each new service adds complexity to the infrastructure. 

Increased scale and complexity demand a modern approach to IaC to help you build, deploy, and manage your infrastructure. If you’re managing between 1 and 10 resources, point and click probably works fine. When you're managing between 10 and 100 resources, then “infrastructure as text” or legacy IaC tools might still suffice. But what happens when you have hundreds or thousands of resources, which is not at all uncommon today? On top of that, those thousands of resources change not once a month but multiple times a day. A great way to manage all this is to put in place the same software engineering practices and tools that you use for application code. 

Ask yourself: 

• How can I make sure my infrastructure scales, changes, and evolves rapidly enough to support the business and create a competitive advantage? 
• How can I maintain visibility into my cloud infrastructure and any changes to it?  
• How can I put in place the policies, security, and guardrails that will ensure safety and reliability? 
• How can I best empower my teams to build, deploy, and manage infrastructure through better collaboration and processes? 

A modern approach to IaC is needed to address these questions. It is the critical tool needed to harness the modern cloud through tried-and-true software engineering practices applied to infrastructure. IaC is how we can harness the cloud’s potential.  

The IaC platform you choose is critical. If your goal is to use standard software engineering tools and practices that are already in place, then look for the following qualities when you evaluate your choices. 

Standard Languages 

Support for standard languages means that your developers can define and configure infrastructure using the same languages used to write application code. For example, common languages like TypeScript, Go, Python, and C#. Many older IaC tools have their own domain-specific language (DSL), and these can be problematic. Developers often find that common programming constructs are missing.  

The platform you choose should allow engineers to easily create strongly typed, structured configurations and to use features they’ve always relied on such as loops, constants, and functions. Another advantage to using standard languages is, of course, that the developers already know it. They can begin coding right away. Learning the idiosyncrasies and limitations of a DSL can be time-consuming and frustrating. 

Standard Development Tools 

Using standard programming languages means that you can also use standard development tools such as IDEs. One advantage is, again, familiarity. Developers can work in an environment they already understand. The other is that developers can work in environments designed to help them easily author, debug, test, and deploy code. 

Testing Frameworks

It’s important that infrastructure is tested thoroughly, just as applications are. A modern IaC platform should support standard testing frameworks and it should also help your teams to expand the types of tests they perform.  

Standard ops testing focuses on acceptance tests. That means the ops team spins up infrastructure in the cloud and they then test that infrastructure to see if it’s correct. Of course, if it wasn’t spun up correctly, the team needs to destroy and redeploy it. That’s not an optimal approach because, potentially, something that shouldn’t have happened already has, depending on how quickly the team reacts. A modern IaC platform should help your teams “shift risk left” through frequent testing before and during deployment. If they’re not already performing them, here are the types of tests your teams should be able to perform with a modern IaC platform. 

Unit Tests 

Unit tests evaluate the behavior of your infrastructure in isolation. External dependencies, such as databases, are replaced by mocks to check your resource configuration and responses. It’s possible to use mocks because responses from cloud providers are well known and tested. You already know how, given some parameters, the provider will respond.  

Unit tests run in memory without any out-of-process calls, which makes them very fast. Use them for fast feedback loops during development. Unit tests really help you solve problems early in the lifecycle of your infrastructure.  

Integration Tests 

Integration testing (also known as black-box testing) comes after unit testing, and it takes a different approach. Integration tests deploy cloud resources and validate their actual behavior — but in an ephemeral environment. An ephemeral environment is a short-lived environment that mimics a production environment. It’s often simpler and only includes the first-level dependencies of the code you’re testing. Once the integration tests are finished, you can destroy the ephemeral infrastructure. 

Security Tests 

Too often, security tests are left until the last minute, or code that’s considered “finished” gets thrown over the wall to a security team, who’ve been left out of the entire development process. The phrase “courting disaster” comes to mind when considering this approach.  

First, a modern IaC platform should encrypt sensitive configuration data. It should also make it easy to follow standard security practices such as key rotation. Check to see if the platform you’re evaluating encrypts state metadata and ensures that secret values are never exposed in plain text. The platform should also integrate easily with security services offered by the cloud providers. 

In addition, as with other types of tests, the IaC platform should help you include security tests that you write yourself into your workflow. Just as you start testing your code early with unit tests, so should you start testing early to find security problems. Those tests belong in your CI/CD pipeline, so the infrastructure is thoroughly tested for vulnerabilities before it’s released. 

Creating Reusable Components  

Reusable components mean you build higher level resources out of individual ones. With them, you can create useful abstractions that can be reused in other places. These components can be written with your company’s best practices built in, tested, and shared within the company and with the community. Using reusable components helps to create repeatable, reliable infrastructure. Look to see if the platform you’re considering helps you create these components easily.  

Standard Package Managers 

If you want to create reusable components, you’ll need a way to package them so you can share them easily. Along with using standard tools, you’ll want support for standard package managers. For example, you might want to put your component into a GitHub repo and publish it through NPM. Your IaC platform should make that a simple task. 

Creating Visibility 

Central visibility across all infrastructure resources, with an historical view of past changes, is important both for accountability and collaboration. Your platform should give you visibility across your infrastructure by supporting audit logs and the ability to see diffs when cloud resources change (similarly to how teams use collaborative tools such as Git). Additionally, the platform should allow you to set fine-grained controls so you can control who can access and change your infrastructure.   

Support for Multiple Cloud Vendors

Not every company wants to use multiple cloud vendors but it’s something you should consider. Do you want to leave that option open? If so, look for an IaC platform that won’t lock you into a single provider. 

Policy as Code

Another too-often ignored facet of IaC is policy as code. A modern IaC platform should allow you to apply software engineering principles and approaches to your policies, just as it does with infrastructure. The benefits for policy as code are much the same as they are for infrastructure. Policies continuously enforce your organization's cloud governance in terms of security, compliance, and cost controls. Policies are unambiguous, they can be written with standard languages and tools, they can be versioned, tested, and finally integrated into the CI/CD pipeline so all infrastructure follows the company’s best practices.  

This section will review the various IaC tools that you can use to get started with your infrastructure-as-code journey; we will explore the basics using Terraform, along with common commands, and then dive into creating reusable modules. 

Infrastructure as Code Tools 

There are many open-source IaC tools that you can use to automate your resource provisioning, deployment, and management. Choosing the right infrastructure automation tool that fits your use case is the key. Let us look at some of the popular IaC categories and tools: 

Terraform is an open-source platform-agnostic tool that allows you to codify your infrastructure as declarative configuration files. Terraform supports many providers and enables you to provision resources in major cloud platforms like AWS, Google Cloud, Azure, and Oracle. 

With Terraform, you can quickly scale the provisioning of infrastructure resources. Building automation into your deployment process improves the productivity of the development teams in your organization, who can now safely deploy infrastructure changes with confidence. It helps reduce the dependency on a centralized infrastructure team and empowers dev teams to move faster, reducing the cycle time of business features. 

To provision resources using Terraform, use the following commands: 

Creating Reusable Infrastructure With Terraform Modules 

The concept of Terraform modules is straightforward — you can write your code inside a module and reuse it in multiple places throughout the codebase. With Terraform modules, you can build infrastructure quickly with a few lines of code. As your infrastructure grows and you need to deploy similar resources in different environments like dev and staging, you don’t want to copy-paste the same code multiple times.  

Terraform modules are much easier to read. They enforce best practices wherein you are not hardcoding values in the Terraform file. To make the module reusable by different teams and cater to various use cases, you need to make it configurable. You should be able to pass additional parameters to multiple resources for environments. Terraform is highly reliable due to its rigorously tested and documented centralized modules.  

As a best practice, start thinking about infrastructure as reusable modules. Terraform modules promote the reuse of code, avoid duplication, and help share modules within your organization. This allows you to invest more time in improving the quality of the centralized reusable modules.  

Sample Code 

In this section, we will look at the steps required to create an AWS S3 bucket in various environments using Terraform modules. Let's get started by using AWS to interact with the required resources. The code below configures the AWS provider: 

terraform { 

  required_providers { 

    aws = { 

      source  = "hashicorp/aws" 

      version = "~> 4.9" 

    } 

  } 

} 

provider "aws" { 

  region = "us-east-1" 

} 

Let us now create a Terraform module for provisioning an S3 bucket resource: 

resource "aws_s3_bucket" "s3-bucket" { 

  bucket = var.bucket 

  policy = var.policy != null ? var.policy : null 

  tags   = merge(var.tags, { Name = "${var.bucket}-bucket" }) 

  server_side_encryption_configuration { 

    rule { 

      apply_server_side_encryption_by_default { 

        sse_algorithm = "AES256" 

      } 

    } 

  } 

  dynamic "lifecycle_rule" { 

    for_each = var.expire-days > 0 ? [var.expire-days] : [] 

    content { 

      id      = "expire" 

      enabled = true 

      expiration { 

        days = var.expire-days 

      } 

    } 

  } 

}

The module supports various arguments like bucket, policy, expire-days, and tags: 

variable "bucket" { 

  description = "S3 Bucket Name" 

  type        = string 

} 

variable "policy" { 

  description = "Optional S3 bucket policy to apply. Should be a valid JSON string" 

  type        = string 

  default     = null 

} 

variable "expire-days" { 

  description = "If set to positive number, lifecycle policy for expiring the objects after specified number of days will be attached to the bucket" 

  type        = number 

  default     = 0 

} 

​

variable "tags" { 

  description = "Common tags to be applied to all resources" 

  type        = map(any) 

} 

Since the reusable S3 module is already created, we can now call the module from various environments like dev or live and pass in the required variables. 

module "dev-dzone-bucket" { 

  source      = "../modules/s3-bucket" 

  bucket      = "dev-dzone-iac-bucket" 

  policy      = null 

  expire-days = 7 

  tags        = local.tags 

} 

​

module "live-dzone-bucket" { 

  source      = "../modules/s3-bucket" 

  bucket      = "live-dzone-iac-bucket" 

  policy      = null 

  expire-days = 14 

  tags        = local.tags 

} 

The file layout in the Terraform project can look something like Figure 2 below, with separate folders for development and production environments that contain AWS resources under “terraform-modules.” 

Bringing a modern IaC platform into a startup or a company with many greenfield applications may not be difficult. For most companies, however, it’s not so straightforward. Many companies, both large and small, have a lot of infrastructure that was created by pointing and clicking in the console of a cloud provider. That’s how many new projects get started. Then, one day, an ops engineer wakes up and realizes that the new project is now production infrastructure. To make it more “official,” the team writes a run book or a wiki that describes what buttons to click when someone wants to perform a common task. Another common situation is that there are Bash or PowerShell scripts floating around that only one or two people know about. What do you do if that’s your situation?  

Stay Calm 

Remember that change can be scary. Many people feel paralyzed when they think about touching their infrastructure. It's too complicated and they don't understand how it works. Take the time to build up your confidence. 

Define What Is Good 

The first step, perhaps even before you begin to evaluate tools and approaches, is to define what “good” looks like to your company. Achieving that ideal depends on understanding what assumptions will remain true regardless of which tools you use.  A team made up of all the stakeholders is one way to define what your company wants to achieve with its cloud infrastructure. 

Pick a Few Tools to Evaluate

After thinking about the critical points listed above, narrow your search for the perfect platform down to a few candidates to evaluate. You might want to design a small project whose only purpose is to test the platform and see how well it helps you reach your goals. 

Import Existing Infrastructure 

Once you’ve selected a tool, try importing some existing infrastructure. If you’re working with the right platform, this should be straightforward.  

Integrate With Existing Engineering Practices 

Assuming your infrastructure code is integrated with your continuous delivery pipeline, you can start instituting the same best practices you use with your application code.  

Start Small 

Start with a new service or non-critical service—something that won’t disrupt your business if it fails. Pick a project where you’ll start seeing value early and then iterate.  

A modern approach to IaC is a great way to reduce cloud complexity, unlock the potential of the modern cloud, and achieve faster innovation. With a modern IaC approach, you apply standard software engineering practices and tools to infrastructure, usually with an IaC platform that supports these practices. Briefly, here is a summary of the high-level benefits that you can expect. 

Increase Innovation, Velocity, and Agility 

With a modern IaC approach, teams can apply the same practices, testing rigor, and automation of modern software development to cloud infrastructure. This increases the rate and reliability of releases so that companies can react to customer feedback and iterate quickly. 

Decrease Infrastructure Risks 

Because developers can use standard testing frameworks, IaC “shifts risk left”. Early, frequent, and thorough testing can be a part of the authoring process and CI/CD pipeline. Since policy and security requirements are also written as code, compliance and safety are automatically tested with every deployment.  

Foster Closer Collaboration 

Modern IaC platforms use standard tools and languages, which can break down silos between infrastructure, application development, and security teams. Using shared practices and tools increases collaboration between different teams.