{{announcement.body}}
{{announcement.title}}
refcard cover
Refcard #356

Getting Started With IaC

Infrastructure as code (IaC) means that you use code to define and manage infrastructure rather than using manual processes. More broadly, and perhaps more importantly, IaC is about bringing software engineering principles and approaches to cloud infrastructure. In this Refcard, explore the fundamentals of IaC and how to get started setting up your environment.

Free PDF for Easy Reference

Brought to You By

Pulumi
refcard cover

Written By

author avatar George Huang
Director of Product Marketing, Pulumi
Section 1

Why Does IaC Matter?

 IaC matters for three reasons. One is the transition to the cloud. More and more workloads are being moved from on-premises data centers to cloud environments. Nothing suggests that this trend is going to stop. However, cloud computing alone isn’t a panacea for maintaining scalable and reliable infrastructure. It’s just as possible to have an inconsistent, poorly documented set of scripts for cloud infrastructure as it is for a physical datacenter. IaC, because it enforces proven engineering practices, is how you make order out of the chaos. 

 The second reason is a greater sophistication in how people use the cloud. Companies are changing architectures, patterns, and ways of working to optimize the benefits they can get. It's no longer simply CapEx versus OpEx. It's about how to incorporate all the practices that make up the engineering lifecycle, such as versioning and testing to unlock all the value that the cloud can provide. It’s about using engineering practices to take advantage of the cloud’s potential and innovate faster to drive your business. 

The third reason is that the burden of managing infrastructure in the cloud is increasing. The number of cloud services available is growing every year and more companies are adopting modern cloud architectures (like containers or serverless), which often have many loosely-coupled and interdependent components. The result is that the number of cloud resources that people must manage is going up at a tremendous pace. This is certainly a good thing because it means companies are getting more value from the cloud to drive their business forward, but the consequence is an increase in complexity and scale.  

For example, one way to get more value from the cloud is to take advantage of the ever-growing number of services that cloud vendors are providing. Those services can speed innovation and accelerate velocity but remember that with every new service comes new APIs. Each new service adds complexity to the infrastructure. 

Increased scale and complexity demand a modern approach to IaC to help you build, deploy, and manage your infrastructure. If you’re managing between 1 and 10 resources, point-and-click probably works fine. When you're managing between 10 and 100 resources, then “infrastructure as text” or legacy IaC tools might still suffice. But what happens when you have hundreds or thousands of resources, which is not at all uncommon today? On top of that, those thousands of resources change not once a month but multiple times a day. A great way to manage all this is to put in place the same software engineering practices and tools that you use for application code. 

Ask yourself: 

  • How can I make sure my infrastructure scales, changes, and evolves rapidly enough to support the business and create a competitive advantage? 
  • How can I maintain visibility into my cloud infrastructure and any changes to it?  
  • How can I put in place the policies, security, and guardrails that will ensure safety and reliability? 
  • How can I best empower my teams to build, deploy, and manage infrastructure through better collaboration and processes? 

A modern approach to IaC is needed to address these questions. It is the critical tool needed to harness the modern cloud through tried-and-true software engineering practices applied to infrastructure. IaC is how we can harness the cloud’s potential.  

Section 2

Important Considerations

 The IaC platform you choose is critical. If your goal is to use standard software engineering tools and practices that are already in place, then look for the following qualities when you evaluate your choices. 

Standard Languages 

Support for standard languages means that your developers can define and configure infrastructure using the same languages used to write application code. For example, common languages like TypeScript, Go, Python, and C#. Many older IaC tools have their own domain-specific language (DSL), and these can be problematic. Developers often find that common programming constructs are missing. The platform you choose should allow engineers to easily create strongly typed, structured configurations and to use features they’ve always relied on such as loops, constants, and functions. Another advantage to using standard languages is, of course, that the developers already know it. They can begin coding right away. Learning the idiosyncrasies and limitations of a DSL can be time-consuming and frustrating. 

Standard Development Tools

Using standard programming languages means that you can also use standard development tools such as IDEs. One advantage is, again, familiarity. Developers can work in an environment they already understand. The other is that developers can work in environments designed to help them easily author, debug, test, and deploy code. 

Testing Frameworks

It’s important that infrastructure is tested thoroughly, just as applications are. A modern IaC platform should support standard testing frameworks and it should also help your teams to expand the types of tests they perform.  

Standard ops testing focuses on acceptance tests. That means the ops team spins up infrastructure in the cloud and they then test that infrastructure to see if it’s correct. Of course, if it wasn’t spun up correctly, the team needs to destroy and redeploy it. That’s not an optimal approach because, potentially, something that shouldn’t have happened already has, depending on how quickly the team reacts. A modern IaC platform should help your teams “shift risk left” through frequent testing before and during deployment. If they’re not already performing them, here are the types of tests your teams should be able to perform with a modern IaC platform. 

Unit Tests 

Unit tests evaluate the behavior of your infrastructure in isolation. External dependencies, such as databases, are replaced by mocks to check your resource configuration and responses. It’s possible to use mocks because responses from cloud providers are well known and tested. You already know how, given some parameters, the provider will respond.  

Unit tests run in memory without any out-of-process calls, which makes them very fast. Use them for fast feedback loops during development. Unit tests really help you solve problems early in the lifecycle of your infrastructure.  

Integration Tests 

Integration testing (also known as black-box testing) comes after unit testing, and it takes a different approach. Integration tests deploy cloud resources and validate their actual behavior — but in an ephemeral environment. An ephemeral environment is a short-lived environment that mimics a production environment. It’s often simpler and only includes the first-level dependencies of the code you’re testing. Once the integration tests are finished, you can destroy the ephemeral infrastructure. 

Security Tests

Too often, security tests are left until the last minute, or code that’s considered “finished” gets thrown over the wall to a security team, who’ve been left out of the entire development process. The phrase “courting disaster” comes to mind when considering this approach.  

First, a modern IaC platform should encrypt sensitive configuration data. It should also make it easy to follow standard security practices such as key rotation. Check to see if the platform you’re evaluating encrypts state metadata and ensures that secret values are never exposed in plain text. The platform should also integrate easily with security services offered by the cloud providers. 

In addition, as with other types of tests, the IaC platform should help you include security tests that you write yourself into your workflow. Just as you start testing your code early with unit tests, so should you start testing early to find security problems. Those tests belong in your CI/CD pipeline, so the infrastructure is thoroughly tested for vulnerabilities before it’s released. 

Creating Reusable Components 

Reusable components mean you build higher-level resources out of individual ones. With them, you can create useful abstractions that can be reused in other places. These components can be written with your company’s best practices built-in, tested, and shared within the company and with the community. Using reusable components helps to create repeatable, reliable infrastructure. Look to see if the platform you’re considering helps you create these components easily.  

Standard Package Managers

If you want to create reusable components, you’ll need a way to package them so you can share them easily. Along with using standard tools, you’ll want support for standard package managers. For example, you might want to put your component into a GitHub repo and publish it through NPM. Your IaC platform should make that a simple task. 

Creating Visibility 

Central visibility across all infrastructure resources, with a historical view of past changes, is important both for accountability and collaboration. Your platform should give you visibility across your infrastructure by supporting audit logs and the ability to see diffs when cloud resources change (similarly to how teams use collaborative tools such as Git). Additionally, the platform should allow you to set fine-grained controls so you can control who can access and change your infrastructure.  

Support for Multiple Cloud Vendors

Not every company wants to use multiple cloud vendors but it’s something you should consider. Do you want to leave that option open? If so, look for an IaC platform that won’t lock you into a single provider. 

Policy as Code 

Another too-often ignored facet of IaC is policy as code. A modern IaC platform should allow you to apply software engineering principles and approaches to your policies, just as it does with infrastructure. The benefits for policy as code are much the same as they are for infrastructure. Policies continuously enforce your organization's cloud governance in terms of security, compliance, and cost controls. Policies are unambiguous, they can be written with standard languages and tools, they can be versioned, tested, and finally integrated into the CI/CD pipeline so all infrastructure follows the company’s best practices.  

Section 3

Getting Started

Bringing a modern IaC platform into a startup or a company with many greenfield applications may not be difficult. For most companies, however, it’s not so straightforward. Many companies, both large and small, have a lot of infrastructure that was created by pointing and clicking in the console of a cloud provider. That’s how many new projects get started. Then, one day, an ops engineer wakes up and realizes that the new project is now production infrastructure. To make it more “official,” the team writes a run book or a wiki that describes what buttons to click when someone wants to perform a common task. Another common situation is that there are Bash or PowerShell scripts floating around that only one or two people know about. What do you do if that’s your situation?  

Stay Calm 

Remember that change can be scary. Many people feel paralyzed when they think about touching their infrastructure. It's too complicated and they don't understand how it works. Take the time to build up your confidence. 

Define What Is Good  

The first step, perhaps even before you begin to evaluate tools and approaches, is to define what “good” looks like to your company. Achieving that ideal depends on understanding what assumptions will remain true regardless of which tools you use.  A team made up of all the stakeholders is one way to define what your company wants to achieve with its cloud infrastructure.  

Pick a Few Tools to Evaluate

After thinking about the critical points listed above, narrow your search for the perfect platform down to a few candidates to evaluate. You might want to design a small project whose only purpose is to test the platform and see how well it helps you reach your goals. 

Import Existing Infrastructure 

Once you’ve selected a tool, try importing some existing infrastructure. If you’re working with the right platform, this should be straightforward. 

Integrate With Existing Engineering Practices

Assuming your infrastructure code is integrated with your continuous delivery pipeline, you can start instituting the same best practices you use with your application code.  

Start Small 

Start with a new service or non-critical service—something that won’t disrupt your business if it fails. Pick a project where you’ll start seeing value early and then iterate.  

Section 4

Best Practices

The next sections show examples of how an IaC platform can help you implement software engineering best practices. We’re using Pulumi as our IaC platform and TypeScript as the programming language.  

Create Reusable Infrastructure Components  

Reusable components are when you create a logical grouping of cloud resources that creates a larger, higher-level abstraction that encapsulates its implementation details. 

Here’s an example of how to create a reusable component for an s3 bucket.

1. Create a file called s3folder.js. 

Java
 
const aws = require("@pulumi/aws"); 
const pulumi = require("@pulumi/pulumi"); 


// Define a component for serving a static website on S3 
class S3Folder extends pulumi.ComponentResource { 
 
    constructor(bucketName, path, opts) { 
        // Register this component with name examples:S3Folder 
        super("examples:S3Folder", bucketName, {}, opts); 
        console.log(`Path where files would be uploaded: ${path}`); 
 
        // Create a bucket and expose a website index document 
        let siteBucket = new aws.s3.Bucket(bucketName, {}, 
            { parent: this } ); // specify resource parent 

        // Create a property for the bucket name that was created 
        this.bucketName = siteBucket.bucket, 
 
        // Register that we are done constructing the component 
        this.registerOutputs(); 
    } 
}

module.exports.S3Folder = S3Folder; 


2. Then, in index.js use this code (the same way you would use any other Node module):  

Java
 
const s3folder = require("./s3folder.js"); 

// Create an instance of the S3Folder component 
let folder = new s3folder.S3Folder("s3-website-bucket", "./www"); 

// Export output property of `folder` as a stack output 
exports.bucketName = folder.bucketName; 


Implement Testing 

 Testing is a large subject. Here we’ll give an example of a unit test that makes sure instances have a Name tag, they don’t use an inline userData script but a virtual machine image, and that they don’t have SSH open to the Internet. This example uses the Mocha testing platform.  

Here is the code to be tested:  

Java
 
import * as aws from "@pulumi/aws"; 

export const group = new aws.ec2.SecurityGroup("web-secgrp", { 
    ingress: [ 
        { protocol: "tcp", fromPort: 22, toPort: 22, cidrBlocks: ["0.0.0.0/0"] }, 
        { protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] }, 
    ], 
}); 

const userData = `#!/bin/bash echo "Hello, World!" > index.html nohup python -m SimpleHTTPServer 80 &`; 
 
export const server = new aws.ec2.Instance("web-server-www", { 
    instanceType: "t2.micro", 
    securityGroups: [ group.name ], // reference the group object above 
    ami: "ami-c55673a0",            // AMI for us-east-2 (Ohio) 
    userData: userData,             // start a simple webserver 
}); 


This program violates all three conditions. 

First, create the mocks.  

Java
 
import * as pulumi from "@pulumi/pulumi"; 

pulumi.runtime.setMocks({ 
    newResource: function(args: pulumi.runtime.MockResourceArgs): {id: string, state: any} { 
        return { 
            id: args.inputs.name + "_id", 
            state: args.inputs, 
        }; 
    }, 
    call: function(args: MockCallArgs) { 
        return args.inputs; 
    }, 
}); 


Next, create the test.  

Java
 
import * as pulumi from "@pulumi/pulumi"; 
import "mocha"; 

pulumi.runtime.setMocks({ 
    // ... mocks as shown above 
}); 
 
describe("Infrastructure", function() { 
    let infra: typeof import("../index"); 
 
    before(async function() { 
        // It's important to import the program _after_ the mocks are defined. 
        infra = await import("../index"); 
    }) 
      
    describe("#server", function() { 
        // TODO(check 1): Instances have a Name tag. 
        // TODO(check 2): Instances must not use an inline userData script. 
    }); 
  
    describe("#group", function() { 
        // TODO(check 3): Instances must not have SSH open to the Internet. 
    }); 

 

Next, implement the first test, which checks that instances have Name tags. 

Java
 
// check 1: Instances have a Name tag. 
it("must have a name tag", function(done) { 
    pulumi.all([infra.server.urn, infra.server.tags]).apply(([urn, tags]) => { 
        if (!tags || !tags["Name"]) { 
            done(new Error(`Missing a name tag on server ${urn}`)); 
        } else { 
            done(); 
        } 
    }); 
}); 

 

Following this pattern, you can write the other two tests yourself.  

Implement Policy as Code 

Decide on the policies and security requirements that should hold true for the entire organization. Those policies should also be a part of your CI/CD pipeline, just as your other tests are.  

The following example is a policy that prevents access to data in S3. 

Java
 
import * as aws from "@pulumi/aws"; 
import { PolicyPack, ReportViolation, validateResourceOfType } from "@pulumi/policy"; 

new PolicyPack("policy-pack-typescript", { 
    policies: [{ 
        name: "s3-no-public-read", 
        description: "Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.", 
        enforcementLevel: "mandatory", 
        validateResource: validateResourceOfType(aws.s3.Bucket, (bucket, args, reportViolation) => { 
            if (bucket.acl === "public-read" || bucket.acl === "public-read-write") { 
                reportViolation( 
                    "You cannot set public-read or public-read-write on an S3 bucket. " + 
                    "Read more about ACLs here: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html"); 
            } 
        }), 
    }], 
}); 
 
        }, 
    }], 
}); 

interface AwsTagsPolicyConfig { 
    requiredTags?: string[]; 
} 


Section 5

Conclusion

A modern approach to IaC is a great way to reduce cloud complexity, unlock the potential of the modern cloud, and achieve faster innovation. With a modern IaC approach, you apply standard software engineering practices and tools to infrastructure, usually with an IaC platform that supports these practices. Briefly, here is a summary of the high-level benefits that you can expect. 

Increase Innovation, Velocity, and Agility 

With a modern IaC approach, teams can apply the same practices, testing rigor, and automation of modern software development to cloud infrastructure. This increases the rate and reliability of releases so that companies can react to customer feedback and iterate quickly. 

Decrease Infrastructure Risks 

Because developers can use standard testing frameworks, IaC “shifts risk left”. Early, frequent, and thorough testing can be a part of the authoring process and CI/CD pipeline. Since policy and security requirements are also written as code, compliance and safety are automatically tested with every deployment.   

Foster Closer Collaboration 

Modern IaC platforms use standard tools and languages, which can break down silos between infrastructure, application development, and security teams. Using shared practices and tools increases collaboration between different teams. 

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}