Pulumi Is Real Infrastructure-as-Code

This is a small outline of my first impressions of Pulumi along with some code to create VPCs and subnets across 4 different AWS regions. The punchline is that Pulumi is good. The only thing that comes close is GeoEngineer but Pulumi is still a few miles ahead in terms of capabilities.

One of the first things I try to do with any infrastructure management tool is figure out how hard/easy it is to create a VPC and subnets in several regions. The fewer hoops I have to jump through to accomplish this, the better. Previously, the best tool for the job was Terraform, but it required heroic workarounds. You had to generate the infrastructure graph using a real programming language because Terraform itself can't target more than one AWS region in a single run. I'm happy to say that Pulumi doesn't have this restriction and it uses a real programming language to generate the graph so there is no need for a two-step "macro" expansion process. The graph is spelled out with the help of one of their SDKs and it uses constructs native to the language of the SDK. I used TypeScript for my experiments so I could spell everything out with classes, functions, interfaces, etc.

I'll first outline the general structure and then fill in the blanks.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Common options we create subnets
interface SubnetArgs<B extends boolean> extends aws.ec2.SubnetArgs {
    mapPublicIpOnLaunch: B;
    tags: { Name: string; };
}

// Given the relevant pieces of information create a public subnet
function createSubnet<B extends boolean>(p: aws.Provider, vpc: aws.ec2.Vpc, az: string, ipAssignment: B, cidr: string) {
  // ...
}

// Create the subnets for each availability zone for the given regional provider
async function createSubnets(p: aws.Provider, vpc: aws.ec2.Vpc, regionAzs: aws.GetAvailabilityZonesResult) {
  // ...
}

// Create 10.0.0.0/16 VPC
function createVpc(p: aws.Provider, vpcName: string): aws.ec2.Vpc {
    // ...
}

// Where we put everything together for creating VPCs, subnets, security groups, routing tables, etc.
async function main() {
    // ...
}

main();

Hopefully, the outline is pretty clear. We have basic functions for creating VPCs and subnets and we glue everything together with the classes that the Pulumi SDK gives us. I'll start with main and work upwards

// Where we put everything together for creating VPCs, subnets, security groups, routing tables, etc.
async function main() {
    // Regions for the VPCs
    const regions: aws.Region[] = ['us-east-1', 'us-east-2', 'us-west-1', 'us-west-2'];
    // Iterate over each region and create the VPC and associate public/private subnets
    const networkConfig = regions.map(async (r: aws.Region) => {
        const vpcName = `${r}-vpc`;
        const providerName = `${r}-provider`;
        // Provider for the region
        const p = new aws.Provider(providerName, { region: r });
        // Availability zones for each region. Used when creating the subnets
        const azs = await aws.getAvailabilityZones(undefined, { provider: p });
        // Creat the VPC
        const vpc = createVpc(p, vpcName);
        // Private and public subnets for each availability zone
        const subnets = await createSubnets(p, vpc, azs);
        return { region: r, vpc: vpc, subnets: subnets };
    });
    return networkConfig;
}

main();

Pulumi gives us a few helper methods for programmatically querying our chosen cloud provider as we generate the infrastructure graph. In the above code I'm creating VPCs in 4 regions (us-east-{1,2}, us-west-{1,2}) and for each region, I also query for the availability zones so that I can create subnets in each zone.

// Create 10.0.0.0/16 VPC
function createVpc(p: aws.Provider, vpcName: string): aws.ec2.Vpc {
    // VPC for the region
    const vpcArguments: aws.ec2.VpcArgs = {
        cidrBlock: '10.0.0.0/16',
        assignGeneratedIpv6CidrBlock: true,
        enableDnsHostnames: true,
        enableDnsSupport: true,
        tags: { Name: vpcName }
    };
    const vpc = new aws.ec2.Vpc(vpcName, vpcArguments, { provider: p });
    return vpc;
}

Creating the VPC in the given region is pretty simple. We just pass in the relevant arguments to the Pulumi VPC constructor and the Pulumi runtime does the rest. Note that all we are doing right now is creating the dependency graph. Nothing is created until we run the code to generate the graph and pass it to the Pulumi runtime. This process is a lot like constructing an abstract syntax tree that will then be interpreted by some runtime.

// Create the subnets for each availability zone for the given regional provider
async function createSubnets(p: aws.Provider, vpc: aws.ec2.Vpc, regionAzs: aws.GetAvailabilityZonesResult) {
    // Subnets in different availability zones must have different CIDR blocks.
    // We use this counter as an index for generating /24 subnets
    let counter = -1;
    const subnets = regionAzs.names.map((az) => {
        // 10.0.{index}.0/24. This is used below to create the CIDR
        const privateIndex = ++counter;
        const publicIndex = ++counter;
        // Private gets even index, public gets odd index.
        const privateCidr = `10.0.${privateIndex}.0/24`;
        const publicCidr = `10.0.${publicIndex}.0/24`;
        // Create the private subnet
        const privateSubnet = createSubnet(p, vpc, az, false, privateCidr)
        // Create the public subnet
        const publicSubnet = createSubnet(p, vpc, az, true, publicCidr)
        // Return the results upstream in case we need to use it
        return { az: az, privateSubnet: privateSubnet, publicSubnet: publicSubnet };
    });

    return subnets;
}

The above does some index juggling and creates the resources for the private and public subnets in the given availability zones for the region. Again, nothing fancy is going on other than populating the options that the Pulumi SDK will use at runtime to create the subnets.

// Given the relevant pieces of information create a public subnet
function createSubnet<B extends boolean>(p: aws.Provider, vpc: aws.ec2.Vpc, az: string, ipAssignment: B, cidr: string) {
    const name = ipAssignment ? `public-subnet-${az}` : `private-subnet-${az}`;
    const subnetArgs: SubnetArgs<B> = {
        cidrBlock: cidr, vpcId: vpc.id, mapPublicIpOnLaunch: ipAssignment,
        tags: { Name: name }
    };
    const subnet = new aws.ec2.Subnet(name, subnetArgs, { provider: p });
    return subnet;
}

And that's it. That's all of the code for generating a VPC in 4 different AWS regions along with associated public and private subnets. This was all after only a few hours of playing around. So if you're in the market for a better infrastructure management tool then Pulumi is currently the state of the art and you should definitely give it a try.